{"thread":{"id":"8495","subject":"[PATCH] Teach filter-branch about subdirectory filtering","startedAt":"2007-06-08T00:30:35Z","lastAt":"2007-06-08T21:05:05Z","messageCount":7,"participants":["Johannes Schindelin","Junio C Hamano","Johannes Sixt","Alex Riesen"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"44295","messageId":"Pine.LNX.4.64.0706080130000.4046@racer.site","threadId":"8495","inReplyTo":null,"subject":"[PATCH] Teach filter-branch about subdirectory filtering","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2007-06-08T00:30:35Z","receivedAt":"2007-06-08T00:30:35Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"\nWith git-filter-branch --subdirectory-filter <subdirectory> you can\nget at the history, as seen by a certain subdirectory. The history\nof the rewritten branch will only contain commits that touched that\nsubdirectory, and the subdirectory will be rewritten to be the new\nproject root.\n\nSigned-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>\n---\n git-filter-branch.sh     |   33 ++++++++++++++++++++++++++++++---\n t/t7003-filter-branch.sh |   24 ++++++++++++++++++++++++\n 2 files changed, 54 insertions(+), 3 deletions(-)\n\ndiff --git a/git-filter-branch.sh b/git-filter-branch.sh\nindex e6ed7b9..4990729 100755\n--- a/git-filter-branch.sh\n+++ b/git-filter-branch.sh\n@@ -122,6 +122,10 @@\n #\tattached, the rewritten tag won't have it. Sorry. (It is by\n #\tdefinition impossible to preserve signatures at any rate, though.)\n #\n+# --subdirectory-filter DIRECTORY:: Only regard the history, as seen by\n+#\tthe given subdirectory. The result will contain that directory as\n+#\tits project root.\n+#\n # EXAMPLE USAGE\n # -------------\n # Suppose you want to remove a file (containing confidential information\n@@ -227,7 +231,13 @@ set_ident () {\n \n # list all parent's object names for a given commit\n get_parents () {\n-\tgit-rev-list -1 --parents \"$1\" | sed \"s/^[0-9a-f]*//\"\n+\tcase \"$filter_subdir\" in\n+\t\"\")\n+\t\tgit-rev-list -1 --parents \"$1\"\n+\t\t;;\n+\t*)\n+\t\tgit-rev-list -1 --parents \"$1\" -- \"$filter_subdir\"\n+\tesac | sed \"s/^[0-9a-f]*//\"\n }\n \n tempdir=.git-rewrite\n@@ -238,6 +248,7 @@ filter_parent=\n filter_msg=cat\n filter_commit='git-commit-tree \"$@\"'\n filter_tag_name=\n+filter_subdir=\n while case \"$#\" in 0) usage;; esac\n do\n \tcase \"$1\" in\n@@ -283,6 +294,9 @@ do\n \t--tag-name-filter)\n \t\tfilter_tag_name=\"$OPTARG\"\n \t\t;;\n+\t--subdirectory-filter)\n+\t\tfilter_subdir=\"$OPTARG\"\n+\t\t;;\n \t*)\n \t\tusage\n \t\t;;\n@@ -316,7 +330,14 @@ ret=0\n \n mkdir ../map # map old->new commit ids for rewriting parents\n \n-git-rev-list --reverse --topo-order --default HEAD \"$@\" >../revs\n+case \"$filter_subdir\" in\n+\"\")\n+\tgit-rev-list --reverse --topo-order --default HEAD \"$@\"\n+\t;;\n+*)\n+\tgit-rev-list --reverse --topo-order --default HEAD \"$@\" \\\n+\t\t-- \"$filter_subdir\"\n+esac > ../revs\n commits=$(cat ../revs | wc -l | tr -d \" \")\n \n test $commits -eq 0 && die \"Found nothing to rewrite\"\n@@ -326,7 +347,13 @@ while read commit; do\n \ti=$(($i+1))\n \tprintf \"\\rRewriting commits... ($i/$commits)\"\n \n-\tgit-read-tree -i -m $commit\n+\tcase \"$filter_subdir\" in\n+\t\"\")\n+\t\tgit-read-tree -i -m $commit\n+\t\t;;\n+\t*)\n+\t\tgit-read-tree -i -m $commit:\"$filter_subdir\"\n+\tesac\n \n \texport GIT_COMMIT=$commit\n \tgit-cat-file commit \"$commit\" >../commit\ndiff --git a/t/t7003-filter-branch.sh b/t/t7003-filter-branch.sh\nindex 3739cb1..292b837 100755\n--- a/t/t7003-filter-branch.sh\n+++ b/t/t7003-filter-branch.sh\n@@ -54,4 +54,28 @@ test_expect_success 'common ancestor is still common (unchanged)' '\n \ttest \"$(git-merge-base modD D)\" = \"$(git-rev-parse B)\"\n '\n \n+test_expect_success 'filter subdirectory only' '\n+\tmkdir subdir &&\n+\ttouch subdir/new &&\n+\tgit add subdir/new &&\n+\ttest_tick &&\n+\tgit commit -m \"subdir\" &&\n+\techo H > a &&\n+\ttest_tick &&\n+\tgit commit -m \"not subdir\" a &&\n+\techo A > subdir/new &&\n+\ttest_tick &&\n+\tgit commit -m \"again subdir\" subdir/new &&\n+\tgit rm a &&\n+\ttest_tick &&\n+\tgit commit -m \"again not subdir\" &&\n+\tgit-filter-branch --subdirectory-filter subdir sub\n+'\n+\n+test_expect_success 'subdirectory filter result looks okay' '\n+\ttest 2 = $(git-rev-list sub | wc -l) &&\n+\tgit show sub:new &&\n+\t! git show sub:subdir\n+'\n+\n test_done\n-- \n1.5.2.1.2689.gaf768-dirty\n"},{"id":"44297","messageId":"7vzm3bmfeq.fsf@assigned-by-dhcp.cox.net","threadId":"8495","inReplyTo":"Pine.LNX.4.64.0706080130000.4046@racer.site","subject":"Re: [PATCH] Teach filter-branch about subdirectory filtering","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2007-06-08T00:57:33Z","receivedAt":"2007-06-08T00:57:33Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Good thing you folded git-split into this, which is an obviously\nright thing to do.\n\nBut it makes me wonder, if it is common to want to split a\ncombined project into two (or more) sets, taking more than one\ndirectories at a time.  Contents of directory A and B (without\nhoisting them up) are filtered into one history and remaining\nbits (again without hoisting them up) are filtered into another.\n\nI do not think that is common, and you can easily do that with\nthe index filter, I think, so it probably is pointless to make\nthis new filter to try supporting that mode of operation.\n\nWe might however want to make sure that we detect an attempt to\ngive more than one --subdirectory-filter parameter.\n"},{"id":"44305","messageId":"Pine.LNX.4.64.0706080603520.4059@racer.site","threadId":"8495","inReplyTo":"7vzm3bmfeq.fsf@assigned-by-dhcp.cox.net","subject":"Re: [PATCH] Teach filter-branch about subdirectory filtering","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2007-06-08T05:04:58Z","receivedAt":"2007-06-08T05:04:58Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Thu, 7 Jun 2007, Junio C Hamano wrote:\n\n> Good thing you folded git-split into this, which is an obviously right \n> thing to do.\n\nSo my secritt evvil plann did not stay secritt that long.\n\n> But it makes me wonder, if it is common to want to split a combined \n> project into two (or more) sets, taking more than one directories at a \n> time.\n\nIt's just a guess, but now that we come really, really close to having a \nconcise implementation of git-subproject which will probably soon \npropagate to master, and then maint, I gather that more and more people \ncome and want to split their projects (which they maintained as one big \nproject) into several subprojects (which they should have been from the \nstart, but the tool did not easily allow for that).\n\n> We might however want to make sure that we detect an attempt to give \n> more than one --subdirectory-filter parameter.\n\nYeah. But then, I think that rev-list actually does not like to be passed \na file/directory which does not exist currently, so that is probably a \nsensible safeguard against that usage.\n\nUnless you think -- as I fear you do -- that people could think \ngit-filter-branch (its name not withstanding) filters _multiple_ branches \n_at once_.\n\nThat case I did not foresee, and you're right, I should output a big fat \nerror for the case when somebody says \"filter-branch --subdirectory-filter \na/ --subdirectory-filter b/\". It does not make any sense. But humans are \nknown for their unique feeling that things should work, even if they make \nno sense at all, and their brain should have warned them (that it is \nunlikely, at best, to work), but failed to do so.\n\nCiao,\nDscho\n"},{"id":"44320","messageId":"46690315.8244CAF7@eudaptics.com","threadId":"8495","inReplyTo":"Pine.LNX.4.64.0706080603520.4059@racer.site","subject":"Re: [PATCH] Teach filter-branch about subdirectory filtering","fromName":"Johannes Sixt","fromEmail":"j.sixt@eudaptics.com","sentAt":"2007-06-08T07:19:49Z","receivedAt":"2007-06-08T07:19:49Z","isPatch":true,"sender":{"key":"j6t@kdbg.org","avatar":"https://avatars.githubusercontent.com/u/14810926?v=4"},"body":"Johannes Schindelin wrote:\n> It's just a guess, but now that we come really, really close to having a\n> concise implementation of git-subproject which will probably soon\n> propagate to master, and then maint, I gather that more and more people\n> come and want to split their projects (which they maintained as one big\n> project) into several subprojects (which they should have been from the\n> start, but the tool did not easily allow for that).\n\nI think that --subdirectry-filter needs to become a bit smarter to be\nreally useful for splitting a big project into sub-projects plus a\nsuper-project. The reason is that once you have extracted the\nsub-project(s), you have a hard time to find out which commits to\ngitlink into the super-project. I don't have a plan how to make it\nsmarter, though.\n\n-- Hannes\n"},{"id":"44347","messageId":"Pine.LNX.4.64.0706081358060.4059@racer.site","threadId":"8495","inReplyTo":"46690315.8244CAF7@eudaptics.com","subject":"Re: [PATCH] Teach filter-branch about subdirectory filtering","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2007-06-08T13:03:06Z","receivedAt":"2007-06-08T13:03:06Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Fri, 8 Jun 2007, Johannes Sixt wrote:\n\n> Johannes Schindelin wrote:\n> > It's just a guess, but now that we come really, really close to having \n> > a concise implementation of git-subproject which will probably soon \n> > propagate to master, and then maint, I gather that more and more \n> > people come and want to split their projects (which they maintained as \n> > one big project) into several subprojects (which they should have been \n> > from the start, but the tool did not easily allow for that).\n> \n> I think that --subdirectry-filter needs to become a bit smarter to be \n> really useful for splitting a big project into sub-projects plus a \n> super-project. The reason is that once you have extracted the \n> sub-project(s), you have a hard time to find out which commits to \n> gitlink into the super-project. I don't have a plan how to make it \n> smarter, though.\n\nYes, it seems a good direction to follow.\n\nI will have to think about it, but I guess that with yet another set of \nfiles in ../map/ (probably prefixed by the subdirectory name), we should \neven be able to smartly filter a huge project into a super-project \ntogether with its subprojects.\n\nThe command line option would be something like \"--split-superproject \ndir1,dir2,dir3\", where dir<n> are the subdirectories that are to become \nthe subprojects.\n\nCiao,\nDscho\n"},{"id":"44348","messageId":"46695763.D65CDC30@eudaptics.com","threadId":"8495","inReplyTo":"Pine.LNX.4.64.0706080130000.4046@racer.site","subject":"Re: [PATCH] Teach filter-branch about subdirectory filtering","fromName":"Johannes Sixt","fromEmail":"j.sixt@eudaptics.com","sentAt":"2007-06-08T13:19:31Z","receivedAt":"2007-06-08T13:19:31Z","isPatch":true,"sender":{"key":"j6t@kdbg.org","avatar":"https://avatars.githubusercontent.com/u/14810926?v=4"},"body":"Johannes Schindelin wrote:\n>  # list all parent's object names for a given commit\n>  get_parents () {\n> -       git-rev-list -1 --parents \"$1\" | sed \"s/^[0-9a-f]*//\"\n> +       case \"$filter_subdir\" in\n> +       \"\")\n> +               git-rev-list -1 --parents \"$1\"\n> +               ;;\n> +       *)\n> +               git-rev-list -1 --parents \"$1\" -- \"$filter_subdir\"\n> +       esac | sed \"s/^[0-9a-f]*//\"\n>  }\n\nI think --full-history in the subdir case would make a lot of sense.\n\n-- Hannes\n"},{"id":"44375","messageId":"20070608210505.GA3025@steel.home","threadId":"8495","inReplyTo":"Pine.LNX.4.64.0706080130000.4046@racer.site","subject":"Re: [PATCH] Teach filter-branch about subdirectory filtering","fromName":"Alex Riesen","fromEmail":"raa.lkml@gmail.com","sentAt":"2007-06-08T21:05:05Z","receivedAt":"2007-06-08T21:05:05Z","isPatch":true,"sender":{"key":"raa.lkml@gmail.com","avatar":"https://avatars.githubusercontent.com/u/324101?v=4"},"body":"Johannes Schindelin, Fri, Jun 08, 2007 02:30:35 +0200:\n> \n> With git-filter-branch --subdirectory-filter <subdirectory> you can\n> get at the history, as seen by a certain subdirectory. The history\n> of the rewritten branch will only contain commits that touched that\n> subdirectory, and the subdirectory will be rewritten to be the new\n> project root.\n> \n\nMaybe it could be made more general\n\n    --subdirectory-filter <from> [--subdirectory-filter-to <to>]\n\n<to> meaning root by default, and allow multiple filters to be set?\nYou would have to deal with file name conflicts in case some filters\nrefer to the same target directory, though.\n\nIt has already been asked how to split a project into many projects\nbased on directory boundaries. That'd help...\n"}]}