# [PATCH] Teach filter-branch about subdirectory filtering 7 messages from 2007-06-08 to 2007-06-08. Participants: Johannes Schindelin, Junio C Hamano, Johannes Sixt, Alex Riesen. Thread: https://gitlist.dev/t/8495 ## Johannes Schindelin, 2007-06-08 00:30 Subject: [PATCH] Teach filter-branch about subdirectory filtering Message-ID: URL: https://gitlist.dev/e/Pine.LNX.4.64.0706080130000.4046%40racer.site ``` With git-filter-branch --subdirectory-filter you can get at the history, as seen by a certain subdirectory. The history of the rewritten branch will only contain commits that touched that subdirectory, and the subdirectory will be rewritten to be the new project root. Signed-off-by: Johannes Schindelin --- git-filter-branch.sh | 33 ++++++++++++++++++++++++++++++--- t/t7003-filter-branch.sh | 24 ++++++++++++++++++++++++ 2 files changed, 54 insertions(+), 3 deletions(-) diff --git a/git-filter-branch.sh b/git-filter-branch.sh index e6ed7b9..4990729 100755 --- a/git-filter-branch.sh +++ b/git-filter-branch.sh @@ -122,6 +122,10 @@ # attached, the rewritten tag won't have it. Sorry. (It is by # definition impossible to preserve signatures at any rate, though.) # +# --subdirectory-filter DIRECTORY:: Only regard the history, as seen by +# the given subdirectory. The result will contain that directory as +# its project root. +# # EXAMPLE USAGE # ------------- # Suppose you want to remove a file (containing confidential information @@ -227,7 +231,13 @@ set_ident () { # list all parent's object names for a given commit get_parents () { - git-rev-list -1 --parents "$1" | sed "s/^[0-9a-f]*//" + case "$filter_subdir" in + "") + git-rev-list -1 --parents "$1" + ;; + *) + git-rev-list -1 --parents "$1" -- "$filter_subdir" + esac | sed "s/^[0-9a-f]*//" } tempdir=.git-rewrite @@ -238,6 +248,7 @@ filter_parent= filter_msg=cat filter_commit='git-commit-tree "$@"' filter_tag_name= +filter_subdir= while case "$#" in 0) usage;; esac do case "$1" in @@ -283,6 +294,9 @@ do --tag-name-filter) filter_tag_name="$OPTARG" ;; + --subdirectory-filter) + filter_subdir="$OPTARG" + ;; *) usage ;; @@ -316,7 +330,14 @@ ret=0 mkdir ../map # map old->new commit ids for rewriting parents -git-rev-list --reverse --topo-order --default HEAD "$@" >../revs +case "$filter_subdir" in +"") + git-rev-list --reverse --topo-order --default HEAD "$@" + ;; +*) + git-rev-list --reverse --topo-order --default HEAD "$@" \ + -- "$filter_subdir" +esac > ../revs commits=$(cat ../revs | wc -l | tr -d " ") test $commits -eq 0 && die "Found nothing to rewrite" @@ -326,7 +347,13 @@ while read commit; do i=$(($i+1)) printf "\rRewriting commits... ($i/$commits)" - git-read-tree -i -m $commit + case "$filter_subdir" in + "") + git-read-tree -i -m $commit + ;; + *) + git-read-tree -i -m $commit:"$filter_subdir" + esac export GIT_COMMIT=$commit git-cat-file commit "$commit" >../commit diff --git a/t/t7003-filter-branch.sh b/t/t7003-filter-branch.sh index 3739cb1..292b837 100755 --- a/t/t7003-filter-branch.sh +++ b/t/t7003-filter-branch.sh @@ -54,4 +54,28 @@ test_expect_success 'common ancestor is still common (unchanged)' ' test "$(git-merge-base modD D)" = "$(git-rev-parse B)" ' +test_expect_success 'filter subdirectory only' ' + mkdir subdir && + touch subdir/new && + git add subdir/new && + test_tick && + git commit -m "subdir" && + echo H > a && + test_tick && + git commit -m "not subdir" a && + echo A > subdir/new && + test_tick && + git commit -m "again subdir" subdir/new && + git rm a && + test_tick && + git commit -m "again not subdir" && + git-filter-branch --subdirectory-filter subdir sub +' + +test_expect_success 'subdirectory filter result looks okay' ' + test 2 = $(git-rev-list sub | wc -l) && + git show sub:new && + ! git show sub:subdir +' + test_done -- 1.5.2.1.2689.gaf768-dirty ``` ## Junio C Hamano, 2007-06-08 00:57 Subject: Re: [PATCH] Teach filter-branch about subdirectory filtering Message-ID: <7vzm3bmfeq.fsf@assigned-by-dhcp.cox.net> URL: https://gitlist.dev/e/7vzm3bmfeq.fsf%40assigned-by-dhcp.cox.net In-Reply-To: ``` Good thing you folded git-split into this, which is an obviously right thing to do. But it makes me wonder, if it is common to want to split a combined project into two (or more) sets, taking more than one directories at a time. Contents of directory A and B (without hoisting them up) are filtered into one history and remaining bits (again without hoisting them up) are filtered into another. I do not think that is common, and you can easily do that with the index filter, I think, so it probably is pointless to make this new filter to try supporting that mode of operation. We might however want to make sure that we detect an attempt to give more than one --subdirectory-filter parameter. ``` ## Johannes Schindelin, 2007-06-08 05:04 Subject: Re: [PATCH] Teach filter-branch about subdirectory filtering Message-ID: URL: https://gitlist.dev/e/Pine.LNX.4.64.0706080603520.4059%40racer.site In-Reply-To: <7vzm3bmfeq.fsf@assigned-by-dhcp.cox.net> ``` Hi, On Thu, 7 Jun 2007, Junio C Hamano wrote: > Good thing you folded git-split into this, which is an obviously right > thing to do. So my secritt evvil plann did not stay secritt that long. > But it makes me wonder, if it is common to want to split a combined > project into two (or more) sets, taking more than one directories at a > time. It's just a guess, but now that we come really, really close to having a concise implementation of git-subproject which will probably soon propagate to master, and then maint, I gather that more and more people come and want to split their projects (which they maintained as one big project) into several subprojects (which they should have been from the start, but the tool did not easily allow for that). > We might however want to make sure that we detect an attempt to give > more than one --subdirectory-filter parameter. Yeah. But then, I think that rev-list actually does not like to be passed a file/directory which does not exist currently, so that is probably a sensible safeguard against that usage. Unless you think -- as I fear you do -- that people could think git-filter-branch (its name not withstanding) filters _multiple_ branches _at once_. That case I did not foresee, and you're right, I should output a big fat error for the case when somebody says "filter-branch --subdirectory-filter a/ --subdirectory-filter b/". It does not make any sense. But humans are known for their unique feeling that things should work, even if they make no sense at all, and their brain should have warned them (that it is unlikely, at best, to work), but failed to do so. Ciao, Dscho ``` ## Johannes Sixt, 2007-06-08 07:19 Subject: Re: [PATCH] Teach filter-branch about subdirectory filtering Message-ID: <46690315.8244CAF7@eudaptics.com> URL: https://gitlist.dev/e/46690315.8244CAF7%40eudaptics.com In-Reply-To: ``` Johannes Schindelin wrote: > It's just a guess, but now that we come really, really close to having a > concise implementation of git-subproject which will probably soon > propagate to master, and then maint, I gather that more and more people > come and want to split their projects (which they maintained as one big > project) into several subprojects (which they should have been from the > start, but the tool did not easily allow for that). I think that --subdirectry-filter needs to become a bit smarter to be really useful for splitting a big project into sub-projects plus a super-project. The reason is that once you have extracted the sub-project(s), you have a hard time to find out which commits to gitlink into the super-project. I don't have a plan how to make it smarter, though. -- Hannes ``` ## Johannes Schindelin, 2007-06-08 13:03 Subject: Re: [PATCH] Teach filter-branch about subdirectory filtering Message-ID: URL: https://gitlist.dev/e/Pine.LNX.4.64.0706081358060.4059%40racer.site In-Reply-To: <46690315.8244CAF7@eudaptics.com> ``` Hi, On Fri, 8 Jun 2007, Johannes Sixt wrote: > Johannes Schindelin wrote: > > It's just a guess, but now that we come really, really close to having > > a concise implementation of git-subproject which will probably soon > > propagate to master, and then maint, I gather that more and more > > people come and want to split their projects (which they maintained as > > one big project) into several subprojects (which they should have been > > from the start, but the tool did not easily allow for that). > > I think that --subdirectry-filter needs to become a bit smarter to be > really useful for splitting a big project into sub-projects plus a > super-project. The reason is that once you have extracted the > sub-project(s), you have a hard time to find out which commits to > gitlink into the super-project. I don't have a plan how to make it > smarter, though. Yes, it seems a good direction to follow. I will have to think about it, but I guess that with yet another set of files in ../map/ (probably prefixed by the subdirectory name), we should even be able to smartly filter a huge project into a super-project together with its subprojects. The command line option would be something like "--split-superproject dir1,dir2,dir3", where dir are the subdirectories that are to become the subprojects. Ciao, Dscho ``` ## Johannes Sixt, 2007-06-08 13:19 Subject: Re: [PATCH] Teach filter-branch about subdirectory filtering Message-ID: <46695763.D65CDC30@eudaptics.com> URL: https://gitlist.dev/e/46695763.D65CDC30%40eudaptics.com In-Reply-To: ``` Johannes Schindelin wrote: > # list all parent's object names for a given commit > get_parents () { > - git-rev-list -1 --parents "$1" | sed "s/^[0-9a-f]*//" > + case "$filter_subdir" in > + "") > + git-rev-list -1 --parents "$1" > + ;; > + *) > + git-rev-list -1 --parents "$1" -- "$filter_subdir" > + esac | sed "s/^[0-9a-f]*//" > } I think --full-history in the subdir case would make a lot of sense. -- Hannes ``` ## Alex Riesen, 2007-06-08 21:05 Subject: Re: [PATCH] Teach filter-branch about subdirectory filtering Message-ID: <20070608210505.GA3025@steel.home> URL: https://gitlist.dev/e/20070608210505.GA3025%40steel.home In-Reply-To: ``` Johannes Schindelin, Fri, Jun 08, 2007 02:30:35 +0200: > > With git-filter-branch --subdirectory-filter you can > get at the history, as seen by a certain subdirectory. The history > of the rewritten branch will only contain commits that touched that > subdirectory, and the subdirectory will be rewritten to be the new > project root. > Maybe it could be made more general --subdirectory-filter [--subdirectory-filter-to ] meaning root by default, and allow multiple filters to be set? You would have to deal with file name conflicts in case some filters refer to the same target directory, though. It has already been asked how to split a project into many projects based on directory boundaries. That'd help... ```