{"thread":{"id":"14862","subject":"git filter-branch --subdirectory-filter, still a mistery","startedAt":"2008-08-06T13:39:50Z","lastAt":"2008-09-14T16:29:59Z","messageCount":35,"participants":["Jan Wielemaker","Thomas Rast","Johannes Schindelin","Junio C Hamano","Petr Baudis","Felipe Contreras"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"86342","messageId":"200808061539.50268.J.Wielemaker@uva.nl","threadId":"14862","inReplyTo":null,"subject":"git filter-branch --subdirectory-filter, still a mistery","fromName":"Jan Wielemaker","fromEmail":"j.wielemaker@uva.nl","sentAt":"2008-08-06T13:39:50Z","receivedAt":"2008-08-06T13:39:50Z","isPatch":false,"sender":{"key":"j.wielemaker@uva.nl","avatar":null},"body":"Hi,\n\nI've been puzzling most of today to do something that must be simple.\nI've got a big repo which contains a project with several nicely related\nsubprojects in directories. Only now, we want to share some of these\nsubprojects with another project. I.e. they must start to live there own\nlife. Of course, I would like to keep the history. So, I did (git --version:\n1.5.6.GIT):\n\n\t% git clone /home/git/pl.git\n\t% cd pl\n\t% git filter-branch --subdirectory-filter packages/chr HEAD\n\nThis indeed creates a nice directory holding only the contents of\npackages/chr.  But, starting qgit I see that all commits, also those\nthat had absolutely nothing to do with this dir are still there.  Also,\nall tags are still there with exactly the same SHA1 as the original.\nI'd expect the tags to be rewritten such that their SHA1 refers to the\nstate of this single directory and its contents!?  Of course, these\ntags give me access to everything, so the repository doesn't shrink\nmuch too.\n\nI must be missing something important ...  I found similar complaints,\nbut few decent answers and the few answer I did find appeared outdated.\nThe one at http://use.perl.org/~rjbs/journal/34411 comes closest, although\nthe reset --hard is no longer needed and the copying and gc-ing doesn't\nhelp much anymore.\n\nShould I write a tree-filter that removes all but the directory I want \nto keep?  I.e. something like this?  Feels like and overkill and I fear\nI'll have a lot of empty commits left.\n\n\t'mv packages/chr .. && rm -r * && mv ../chr/* . && rmdir ../chr'\n\nI'll be grateful for a clue!\n\n\tCheers --- Jan\n"},{"id":"86393","messageId":"200808070913.55212.J.Wielemaker@uva.nl","threadId":"14862","inReplyTo":"200808061539.50268.J.Wielemaker@uva.nl","subject":"Re: git filter-branch --subdirectory-filter, still a mistery","fromName":"Jan Wielemaker","fromEmail":"j.wielemaker@uva.nl","sentAt":"2008-08-07T07:13:55Z","receivedAt":"2008-08-07T07:13:55Z","isPatch":false,"sender":{"key":"j.wielemaker@uva.nl","avatar":null},"body":"On Wednesday 06 August 2008 15:39:50 you wrote:\n> Hi,\n>\n> I've been puzzling most of today to do something that must be simple.\n> I've got a big repo which contains a project with several nicely related\n> subprojects in directories. Only now, we want to share some of these\n> subprojects with another project. I.e. they must start to live there own\n> life. Of course, I would like to keep the history. So, I did (git\n> --version: 1.5.6.GIT):\n>\n>       % git clone /home/git/pl.git\n>       % cd pl\n>       % git filter-branch --subdirectory-filter packages/chr HEAD\n>\n> This indeed creates a nice directory holding only the contents of\n> packages/chr.  But, starting qgit I see that all commits, also those\n> that had absolutely nothing to do with this dir are still there.  Also,\n> all tags are still there with exactly the same SHA1 as the original.\n> I'd expect the tags to be rewritten such that their SHA1 refers to the\n> state of this single directory and its contents!?  Of course, these\n> tags give me access to everything, so the repository doesn't shrink\n> much too.\n>\n> I must be missing something important ...  I found similar complaints,\n> but few decent answers and the few answer I did find appeared outdated.\n> The one at http://use.perl.org/~rjbs/journal/34411 comes closest, although\n> the reset --hard is no longer needed and the copying and gc-ing doesn't\n> help much anymore.\n>\n> Should I write a tree-filter that removes all but the directory I want\n> to keep?  I.e. something like this?  Feels like and overkill and I fear\n> I'll have a lot of empty commits left.\n>\n>       'mv packages/chr .. && rm -r * && mv ../chr/* . && rmdir ../chr'\n>\n> I'll be grateful for a clue!\n\nWeirdness goes on.  I tried this:\n\n    git filter-branch --tree-filter '/home/jan/nobackup/tmp2/keep \npackages/chr'\n\nwhere `keep' is a shell-script:\n\n----------------------------------------------------------------\ntmp=/home/jan/nobackup/tmp2\ndir=\"$1\"\n\nif [ -d \"$dir\" ]; then\n  b=`basename $dir`\n  mv \"$dir\" $tmp/$b\n  rm -rf *\n  mv $tmp/$b/* .\n  mv $tmp/$b/.??* .\n  rmdir $tmp/$b\nelse\n  rm -rf *\nfi\n----------------------------------------------------------------\n\nThis kind of works. I.e. I end up (after 3 hours) with a tree that only\ncontains files from packages/chr. Using qgit it no longer shows the\nother files in the `tree' view. Only, it has *all* commits of the\noriginal project, most of which of course do not change this directory,\nbut now at least their diff is empty. I'd assume there is a command to\nremove these (which?)\n\nSpace wise this isn't ok.  The original project GIT is 140M, after this\naction and a git gc, it is 63M: *much* too big.\n\nWhats more weird: all tags still have the same sha1. I copied using git\nclone --no-hardlinks pl chr, deleted all refs/tags from packed-refs and\ngave a \"git gc --prune\", to end up with 1.1 GIGABYTE repository!?\n\nI'm starting to feel a bit stupid that I can't get this done ...\n\n        Clues?  --- Jan\n"},{"id":"86398","messageId":"200808070950.23754.trast@student.ethz.ch","threadId":"14862","inReplyTo":"200808061539.50268.J.Wielemaker@uva.nl","subject":"Re: git filter-branch --subdirectory-filter, still a mistery","fromName":"Thomas Rast","fromEmail":"trast@student.ethz.ch","sentAt":"2008-08-07T07:50:03Z","receivedAt":"2008-08-07T07:50:03Z","isPatch":false,"sender":{"key":"tr@thomasrast.ch","avatar":"https://avatars.githubusercontent.com/u/153510?v=4"},"body":"Jan Wielemaker wrote:\n[...]\n> \t% git filter-branch --subdirectory-filter packages/chr HEAD\n> \n> This indeed creates a nice directory holding only the contents of\n> packages/chr.  But, starting qgit I see that all commits, also those\n> that had absolutely nothing to do with this dir are still there.\n\nThe trick is to rewrite all refs, not just HEAD.  I usually proceed as\nfollows:\n\n  cp -a repo repo.old  # just to keep a backup\n  cd repo\n  git filter-branch --subdirectory-filter somedir -- --all\n\nThe --all tells it to rewrite as many refs as possible.  Note that the\n-- is required.  Also note that refs/original/* will still point to\nthe old commits, so they won't \"just vanish\".  You may want to clone\nthe repository or delete them manually once you are sure the\nfilter-branch did the right thing.\n\n- Thomas\n\n"},{"id":"86408","messageId":"200808071214.25399.J.Wielemaker@uva.nl","threadId":"14862","inReplyTo":"200808070950.23754.trast@student.ethz.ch","subject":"Re: git filter-branch --subdirectory-filter, still a mistery","fromName":"Jan Wielemaker","fromEmail":"j.wielemaker@uva.nl","sentAt":"2008-08-07T10:14:25Z","receivedAt":"2008-08-07T10:14:25Z","isPatch":false,"sender":{"key":"j.wielemaker@uva.nl","avatar":null},"body":"Hi Thomas,\n\nOn Thursday 07 August 2008 09:50:03 am Thomas Rast wrote:\n> Jan Wielemaker wrote:\n> [...]\n>\n> > \t% git filter-branch --subdirectory-filter packages/chr HEAD\n> >\n> > This indeed creates a nice directory holding only the contents of\n> > packages/chr.  But, starting qgit I see that all commits, also those\n> > that had absolutely nothing to do with this dir are still there.\n>\n> The trick is to rewrite all refs, not just HEAD.  I usually proceed as\n> follows:\n>\n>   cp -a repo repo.old  # just to keep a backup\n>   cd repo\n>   git filter-branch --subdirectory-filter somedir -- --all\n>\n> The --all tells it to rewrite as many refs as possible.  Note that the\n> -- is required.  Also note that refs/original/* will still point to\n> the old commits, so they won't \"just vanish\".  You may want to clone\n> the repository or delete them manually once you are sure the\n> filter-branch did the right thing.\n\nThanks. That is moving in the right direction! There are some, possibly\nrelated, problems left (using 1.5.6.GIT).  According to git fsck,\nmy repo is clean.  I got:\n\n  Ref 'refs/tags/V5.6.50' was rewritten\n  error: Ref refs/tags/V5.6.50 is at 8678b32f71178019c06aefa40e2d3fb9a2e8ef25 \nbut\n\texpected 2e8aef64e2fed088720a19ac2ffa2481e5bc7806\n  fatal: Cannot lock the ref 'refs/tags/V5.6.50'.\n  Could not rewrite refs/tags/V5.6.50\n\nNow, if I look in .git/packed-refs, I see this (i.e. a second line with\na ^) for all refs that cause problems:\n\n274ec8ac671542206ba3567ff5d72b3e54c5603c refs/tags/V5.6.59\n^28920c3c0a184698d9cd15a65cd643367200bbf5\nfaf203f9d9e350d84b6b38b7746e710b6232fc97 refs/tags/V5.6.58\n^1edb1adedcc47ec15c3242234cc6b7ede94bbfba\n48488c871227beabcb3ba167b737d6e33ced65bc refs/tags/V5.6.57\n^766587b09e3d2f09c87b03ad0d7faf3529c9dcff\n\nAfter a bit of puzzling I discovered the the SHA1 after the ^ refers to\nthe actual commit and I changed all these to `lightweight' tags by\nputting the SHA1 behind ^ before the tag itself.  I wrote a little\nsh/awk script to automate this (attached).\n\nNow it runs to the end.  Unfortunagtely the history is completely\nscrewed up :-(:\n\n\t* There are a lot of commits that are not related to the dir\n\t* Commits start long before the directory came into existence,\n\tLooks like it just shows the whole project at this place.\n\nI think the problem is related to the fact that the directory I want to\nfilter didn't exists at the start of the project. Looking at\ngit-rev-list, I found --remove-empty, so I added that after the --all,\nbut that doesn't appear to help.  I must admit I don't really know what\nI'm doing (though I still think the result I want it well defined and\nits hard to imagine I'm the only person who wants this).\n\nIf someone wants to help: clone git://gollem.science.uva.nl/home/git/pl.git\nand try to filter the dir packages/chr.  You can browse the git at\nhttp://gollem.science.uva.nl/git/pl.git\n\n\tThanks --- Jan\n\n"},{"id":"86414","messageId":"1218117841-27398-1-git-send-email-trast@student.ethz.ch","threadId":"14862","inReplyTo":"200808070950.23754.trast@student.ethz.ch","subject":"[PATCH] Documentation: filter-branch: document how to filter all refs","fromName":"Thomas Rast","fromEmail":"trast@student.ethz.ch","sentAt":"2008-08-07T14:04:01Z","receivedAt":"2008-08-07T14:04:01Z","isPatch":true,"sender":{"key":"tr@thomasrast.ch","avatar":"https://avatars.githubusercontent.com/u/153510?v=4"},"body":"Document the '--' option that can be used to pass rev-list options\n(not just arguments), and give an example usage of '-- --all'.\n\nSigned-off-by: Thomas Rast <trast@student.ethz.ch>\n---\n\n[This went out to Jan and Junio already, but I forgot to CC the list.\nSorry.]\n\nSomehow I'm imagining this is a FAQ.  Either way, I remember figuring\nout this exact example by accident when I first needed it.\n\n Documentation/git-filter-branch.txt |   13 ++++++++++++-\n 1 files changed, 12 insertions(+), 1 deletions(-)\n\ndiff --git a/Documentation/git-filter-branch.txt b/Documentation/git-filter-branch.txt\nindex a518ba6..1f0fcec 100644\n--- a/Documentation/git-filter-branch.txt\n+++ b/Documentation/git-filter-branch.txt\n@@ -13,7 +13,7 @@ SYNOPSIS\n \t[--msg-filter <command>] [--commit-filter <command>]\n \t[--tag-name-filter <command>] [--subdirectory-filter <directory>]\n \t[--original <namespace>] [-d <directory>] [-f | --force]\n-\t[<rev-list options>...]\n+\t[--] [<rev-list options>...]\n \n DESCRIPTION\n -----------\n@@ -196,6 +196,17 @@ git filter-branch --index-filter 'git rm --cached filename' HEAD\n \n Now, you will get the rewritten history saved in HEAD.\n \n+To rewrite the repository to look as if 'foodir/' had been its project\n+root, and discard all other history:\n+\n+-------------------------------------------------------\n+git filter-branch --subdirectory-filter foodir -- --all\n+-------------------------------------------------------\n+\n+Thus you can, e.g., turn a library subdirectory into a repository of\n+its own.  Note the '--' that separates 'filter-branch' options from\n+revision options, and the '--all' to rewrite all branches and tags.\n+\n To set a commit (which typically is at the tip of another\n history) to be the parent of the current initial commit, in\n order to paste the other history behind the current history:\n-- \n1.6.0.rc1.106.g98a7\n"},{"id":"86416","messageId":"1218118563-28579-1-git-send-email-trast@student.ethz.ch","threadId":"14862","inReplyTo":"1218117841-27398-1-git-send-email-trast@student.ethz.ch","subject":"[PATCH v2] Documentation: filter-branch: document how to filter all refs","fromName":"Thomas Rast","fromEmail":"trast@student.ethz.ch","sentAt":"2008-08-07T14:16:03Z","receivedAt":"2008-08-07T14:16:03Z","isPatch":true,"sender":{"key":"tr@thomasrast.ch","avatar":"https://avatars.githubusercontent.com/u/153510?v=4"},"body":"Document the '--' option that can be used to pass rev-list options\n(not just arguments), and give an example usage of '-- --all'.  Remove\nreference to \"the new branch name\"; filter-branch takes arbitrary\narguments to rev-list since dfd05e3.\n\nSigned-off-by: Thomas Rast <trast@student.ethz.ch>\n---\n\nAt second glance, it turned out the documentation was actually older\nthan the code.  So rewrite the documentation of <rev-list options>.\n\n Documentation/git-filter-branch.txt |   21 ++++++++++++++++-----\n 1 files changed, 16 insertions(+), 5 deletions(-)\n\ndiff --git a/Documentation/git-filter-branch.txt b/Documentation/git-filter-branch.txt\nindex a518ba6..31d3cae 100644\n--- a/Documentation/git-filter-branch.txt\n+++ b/Documentation/git-filter-branch.txt\n@@ -13,7 +13,7 @@ SYNOPSIS\n \t[--msg-filter <command>] [--commit-filter <command>]\n \t[--tag-name-filter <command>] [--subdirectory-filter <directory>]\n \t[--original <namespace>] [-d <directory>] [-f | --force]\n-\t[<rev-list options>...]\n+\t[--] [<rev-list options>...]\n \n DESCRIPTION\n -----------\n@@ -168,10 +168,10 @@ to other tags will be rewritten to point to the underlying commit.\n \t'refs/original/', unless forced.\n \n <rev-list options>...::\n-\tWhen options are given after the new branch name, they will\n-\tbe passed to 'git-rev-list'.  Only commits in the resulting\n-\toutput will be filtered, although the filtered commits can still\n-\treference parents which are outside of that set.\n+\tArguments for 'git-rev-list'.  All positive refs included by\n+\tthese options are rewritten.  You may also specify options\n+\tsuch as '--all', but you must use '--' to separate them from\n+\tthe 'git-filter-branch' options.\n \n \n Examples\n@@ -196,6 +196,17 @@ git filter-branch --index-filter 'git rm --cached filename' HEAD\n \n Now, you will get the rewritten history saved in HEAD.\n \n+To rewrite the repository to look as if 'foodir/' had been its project\n+root, and discard all other history:\n+\n+-------------------------------------------------------\n+git filter-branch --subdirectory-filter foodir -- --all\n+-------------------------------------------------------\n+\n+Thus you can, e.g., turn a library subdirectory into a repository of\n+its own.  Note the '--' that separates 'filter-branch' options from\n+revision options, and the '--all' to rewrite all branches and tags.\n+\n To set a commit (which typically is at the tip of another\n history) to be the parent of the current initial commit, in\n order to paste the other history behind the current history:\n-- \n1.6.0.rc1.106.g98a7\n"},{"id":"86443","messageId":"200808080148.27384.trast@student.ethz.ch","threadId":"14862","inReplyTo":"200808071214.25399.J.Wielemaker@uva.nl","subject":"Re: git filter-branch --subdirectory-filter, still a mistery","fromName":"Thomas Rast","fromEmail":"trast@student.ethz.ch","sentAt":"2008-08-07T23:48:05Z","receivedAt":"2008-08-07T23:48:05Z","isPatch":false,"sender":{"key":"tr@thomasrast.ch","avatar":"https://avatars.githubusercontent.com/u/153510?v=4"},"body":"Jan Wielemaker wrote:\n>   Ref 'refs/tags/V5.6.50' was rewritten\n>   error: Ref refs/tags/V5.6.50 is at 8678b32f71178019c06aefa40e2d3fb9a2e8ef25 \n> but\n> \texpected 2e8aef64e2fed088720a19ac2ffa2481e5bc7806\n>   fatal: Cannot lock the ref 'refs/tags/V5.6.50'.\n>   Could not rewrite refs/tags/V5.6.50\n[...]\n> Now, if I look in .git/packed-refs [...] and I changed all these to\n> `lightweight' tags\n\nThis appears to be a bug.  I've whipped up a patch that will follow\nand should fix the bug.  It has nothing to do with packed-refs; the\ncurrent filter-branch chokes on annotated tags during\n--subdirectory-filter, even though there is support for tag rewriting.\n\nHowever, to enable tag rewriting, you need to say --tag-name-filter\ncat.\n\n> Now it runs to the end.  Unfortunagtely the history is completely\n> screwed up :-(:\n> \n> \t* There are a lot of commits that are not related to the dir\n> \t* Commits start long before the directory came into existence,\n> \tLooks like it just shows the whole project at this place.\n\nFor some reason the ancestor detection does not work right.  I'm also\nfollowing up with an RFH patch that significantly improves the success\nrate (in terms of branches and tags successfully mapped to a rewritten\ncommit) in the case of your repository.  I doubt more staring at the\ncode would yield any more ideas at this hour, so ideas would be\nappreciated.\n\nThe rest is just the other commits/tags showing a lot of the history.\nI don't know of any built-in way to prune the branches and tags that\naren't part of the new master, but\n\n  git branch -a --no-merged master\n\ncan tell you which branches aren't ancestors of master.\n\n- Thomas\n\n-- \nThomas Rast\ntrast@student.ethz.ch\n\n"},{"id":"86444","messageId":"1218153031-18443-1-git-send-email-trast@student.ethz.ch","threadId":"14862","inReplyTo":"200808080148.27384.trast@student.ethz.ch","subject":"[PATCH] filter-branch: be more helpful when an annotated tag changes","fromName":"Thomas Rast","fromEmail":"trast@student.ethz.ch","sentAt":"2008-08-07T23:50:31Z","receivedAt":"2008-08-07T23:50:31Z","isPatch":true,"sender":{"key":"tr@thomasrast.ch","avatar":"https://avatars.githubusercontent.com/u/153510?v=4"},"body":"Previously, git-filter-branch failed if it attempted to update an\nannotated tag.  Now we ignore this condition if --tag-name-filter is\ngiven, so that we can later rewrite the tag.  If no such option was\nprovided, we warn the user that he might want to run with\n--tag-name-filter cat to achieve the intended effect.\n\nSigned-off-by: Thomas Rast <trast@student.ethz.ch>\n---\n git-filter-branch.sh |   14 +++++++++++---\n 1 files changed, 11 insertions(+), 3 deletions(-)\n\ndiff --git a/git-filter-branch.sh b/git-filter-branch.sh\nindex 182822a..a324cf0 100755\n--- a/git-filter-branch.sh\n+++ b/git-filter-branch.sh\n@@ -361,9 +361,17 @@ do\n \t;;\n \t$_x40)\n \t\techo \"Ref '$ref' was rewritten\"\n-\t\tgit update-ref -m \"filter-branch: rewrite\" \\\n-\t\t\t\t\"$ref\" $rewritten $sha1 ||\n-\t\t\tdie \"Could not rewrite $ref\"\n+\t\tif ! git update-ref -m \"filter-branch: rewrite\" \\\n+\t\t\t\t\t\"$ref\" $rewritten $sha1 2>/dev/null; then\n+\t\t\tif test $(git cat-file -t \"$ref\") = tag; then\n+\t\t\t\tif test -z \"$filter_tag_name\"; then\n+\t\t\t\t\twarn \"WARNING: You said to rewrite tagged commits, but not the corresponding tag.\"\n+\t\t\t\t\twarn \"WARNING: Perhaps use '--tag-name-filter cat' to rewrite the tag.\"\n+\t\t\t\tfi\n+\t\t\telse\n+\t\t\t\tdie \"Could not rewrite $ref\"\n+\t\t\tfi\n+\t\tfi\n \t;;\n \t*)\n \t\t# NEEDSWORK: possibly add -Werror, making this an error\n-- \n1.6.0.rc2.19.g3c9ba\n"},{"id":"86445","messageId":"1218153242-18837-1-git-send-email-trast@student.ethz.ch","threadId":"14862","inReplyTo":"200808080148.27384.trast@student.ethz.ch","subject":"[RFH] filter-branch: ancestor detection weirdness","fromName":"Thomas Rast","fromEmail":"trast@student.ethz.ch","sentAt":"2008-08-07T23:54:02Z","receivedAt":"2008-08-07T23:54:02Z","isPatch":false,"sender":{"key":"tr@thomasrast.ch","avatar":"https://avatars.githubusercontent.com/u/153510?v=4"},"body":"THIS WILL VERY LIKELY NOT WORK IN ALL CASES.\n\nUse git rev-list -1 -- <subdir> to discover a random ancestor, instead\nof more correct boundary detection.  Oddly enough, this _increases_\nsuccess rate with Jan's repository and --all.  May break randomly with\nmore complicated args.\n---\n\nMaybe someone understands what's going on and can fix the underlying\nbug...\n\n git-filter-branch.sh |   12 +++---------\n 1 files changed, 3 insertions(+), 9 deletions(-)\n\ndiff --git a/git-filter-branch.sh b/git-filter-branch.sh\nindex 182822a..52b2bdf 100755\n--- a/git-filter-branch.sh\n+++ b/git-filter-branch.sh\n@@ -325,15 +325,9 @@ while read ref\n do\n \tsha1=$(git rev-parse \"$ref\"^0)\n \ttest -f \"$workdir\"/../map/$sha1 && continue\n-\t# Assign the boundarie(s) in the set of rewritten commits\n-\t# as the replacement commit(s).\n-\t# (This would look a bit nicer if --not --stdin worked.)\n-\tfor p in $( (cd \"$workdir\"/../map; ls | sed \"s/^/^/\") |\n-\t\tgit rev-list $ref --boundary --stdin |\n-\t\tsed -n \"s/^-//p\")\n-\tdo\n-\t\tmap $p >> \"$workdir\"/../map/$sha1\n-\tdone\n+\t# Assign the first commit not pruned as the replacement.\n+\tcandidate=$(git rev-list $ref -1 -- \"$filter_subdir\")\n+\ttest \"$candidate\" && map \"$candidate\" > \"$workdir\"/../map/$sha1\n done < \"$tempdir\"/heads\n \n # Finally update the refs\n-- \n1.6.0.rc2.19.g3c9ba\n"},{"id":"86460","messageId":"200808080944.23048.J.Wielemaker@uva.nl","threadId":"14862","inReplyTo":"200808080148.27384.trast@student.ethz.ch","subject":"Re: git filter-branch --subdirectory-filter, still a mistery","fromName":"Jan Wielemaker","fromEmail":"j.wielemaker@uva.nl","sentAt":"2008-08-08T07:44:22Z","receivedAt":"2008-08-08T07:44:22Z","isPatch":false,"sender":{"key":"j.wielemaker@uva.nl","avatar":null},"body":"Hi Thomas,\n\nThanks for looking into this!\n\nOn Friday 08 August 2008 01:48:05 Thomas Rast wrote:\n> Jan Wielemaker wrote:\n> >   Ref 'refs/tags/V5.6.50' was rewritten\n> >   error: Ref refs/tags/V5.6.50 is at\n> > 8678b32f71178019c06aefa40e2d3fb9a2e8ef25 but\n> > \texpected 2e8aef64e2fed088720a19ac2ffa2481e5bc7806\n> >   fatal: Cannot lock the ref 'refs/tags/V5.6.50'.\n> >   Could not rewrite refs/tags/V5.6.50\n>\n> [...]\n>\n> > Now, if I look in .git/packed-refs [...] and I changed all these to\n> > `lightweight' tags\n>\n> This appears to be a bug.  I've whipped up a patch that will follow\n> and should fix the bug.  It has nothing to do with packed-refs; the\n> current filter-branch chokes on annotated tags during\n> --subdirectory-filter, even though there is support for tag rewriting.\n>\n> However, to enable tag rewriting, you need to say --tag-name-filter\n> cat.\n\nGreat.  I knew a more fundamental approach was asked for, but I bet my\nsimple-minded work-around gives the same result, no?\n\n> > Now it runs to the end.  Unfortunagtely the history is completely\n> > screwed up :-(:\n> >\n> > \t* There are a lot of commits that are not related to the dir\n> > \t* Commits start long before the directory came into existence,\n> > \tLooks like it just shows the whole project at this place.\n>\n> For some reason the ancestor detection does not work right.  I'm also\n> following up with an RFH patch that significantly improves the success\n> rate (in terms of branches and tags successfully mapped to a rewritten\n> commit) in the case of your repository.  I doubt more staring at the\n> code would yield any more ideas at this hour, so ideas would be\n> appreciated.\n\nThanks. As I'm using the GIT version anyway, I'll apply these patches\nand see what happens. The trouble is related to tags and possibly to\nbranches. I get completely correct result if I delete all branches and\ntags before filtering.  That at least helps for this particular subproject\n(though some of the tags are useful).\n\nI didn't further investigate branches (I think the packages/chr\ndirectory is not involved in any branch; if you are interested, the boot\ndirectory should show traces of the V57X branch).\n\nI did see that (all/some?) tags that involve changes to the packages/chr\ndirectory nicely end up in its history, but others do not appear on the\nfiltered master branch and give access to the complete project. See for\nexample V5.6.59 (the latest release tag). Try (in the filtered branch)\n\n\tgit diff V5.6.59..\n\nThat should only show some small changes, but it diffs the entire project\nagainst the subdir ...\n\n> The rest is just the other commits/tags showing a lot of the history.\n> I don't know of any built-in way to prune the branches and tags that\n> aren't part of the new master, but\n>\n>   git branch -a --no-merged master\n>\n> can tell you which branches aren't ancestors of master.\n\nThanks for the tip.\n\n\tCheers --- Jan\n"},{"id":"86489","messageId":"200808081325.29241.J.Wielemaker@uva.nl","threadId":"14862","inReplyTo":"200808080148.27384.trast@student.ethz.ch","subject":"Re: git filter-branch --subdirectory-filter, still a mistery","fromName":"Jan Wielemaker","fromEmail":"j.wielemaker@uva.nl","sentAt":"2008-08-08T11:25:29Z","receivedAt":"2008-08-08T11:25:29Z","isPatch":false,"sender":{"key":"j.wielemaker@uva.nl","avatar":null},"body":"Hi Thomas,\n\nOn Friday 08 August 2008 01:48:05 Thomas Rast wrote:\n> This appears to be a bug.  I've whipped up a patch that will follow\n> and should fix the bug.  It has nothing to do with packed-refs; the\n> current filter-branch chokes on annotated tags during\n> --subdirectory-filter, even though there is support for tag rewriting.\n>\n> However, to enable tag rewriting, you need to say --tag-name-filter\n> cat.\n\nThat works!\n\n> > Now it runs to the end.  Unfortunagtely the history is completely\n> > screwed up :-(:\n> >\n> > \t* There are a lot of commits that are not related to the dir\n> > \t* Commits start long before the directory came into existence,\n> > \tLooks like it just shows the whole project at this place.\n>\n> For some reason the ancestor detection does not work right.  I'm also\n> following up with an RFH patch that significantly improves the success\n> rate (in terms of branches and tags successfully mapped to a rewritten\n> commit) in the case of your repository.  I doubt more staring at the\n> code would yield any more ideas at this hour, so ideas would be\n> appreciated.\n>\n> The rest is just the other commits/tags showing a lot of the history.\n> I don't know of any built-in way to prune the branches and tags that\n> aren't part of the new master, but\n>\n>   git branch -a --no-merged master\n>\n> can tell you which branches aren't ancestors of master.\n\nI retried with your two patches. That looks a *lot* better. After using\nthe above and deleting the reported branches there are still some\nbranches left, but at least switching to them doesn't bring the complete\nproject back.\n\nNow there are a few weird tags left, some of these may well be the\nresult of weird things in the repository. The repository was on CVS\nuntil about a year ago and was converted (using SVN as intermediate).\n\nThe big problem is anything that relates to the days before the filtered\ndirectory was part of the project. There are lots of tags there and\nswitching to them brings back the old project. I'd guess the correct\nbehaviour is that either all these tags refer to an empty tree or (which\nI would prefer) all such tags are deleted.\n\nIs this a bug?  Is there a trick here?  git clone --depth doesn't\nseem appropriate.\n\n\tCheers --- Jan\n"},{"id":"86490","messageId":"alpine.DEB.1.00.0808081341170.9611@pacific.mpi-cbg.de.mpi-cbg.de","threadId":"14862","inReplyTo":"1218153242-18837-1-git-send-email-trast@student.ethz.ch","subject":"Re: [RFH] filter-branch: ancestor detection weirdness","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2008-08-08T11:42:32Z","receivedAt":"2008-08-08T11:42:32Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Fri, 8 Aug 2008, Thomas Rast wrote:\n\n> diff --git a/git-filter-branch.sh b/git-filter-branch.sh\n> index 182822a..52b2bdf 100755\n> --- a/git-filter-branch.sh\n> +++ b/git-filter-branch.sh\n> @@ -325,15 +325,9 @@ while read ref\n>  do\n>  \tsha1=$(git rev-parse \"$ref\"^0)\n>  \ttest -f \"$workdir\"/../map/$sha1 && continue\n> -\t# Assign the boundarie(s) in the set of rewritten commits\n> -\t# as the replacement commit(s).\n> -\t# (This would look a bit nicer if --not --stdin worked.)\n> -\tfor p in $( (cd \"$workdir\"/../map; ls | sed \"s/^/^/\") |\n> -\t\tgit rev-list $ref --boundary --stdin |\n> -\t\tsed -n \"s/^-//p\")\n> -\tdo\n> -\t\tmap $p >> \"$workdir\"/../map/$sha1\n> -\tdone\n> +\t# Assign the first commit not pruned as the replacement.\n> +\tcandidate=$(git rev-list $ref -1 -- \"$filter_subdir\")\n\nIs it not just a question of adding '-- \"$filter_subdir\"' to the rev-list \ncall you removed?\n\nCiao,\nDscho\n"},{"id":"86499","messageId":"200808081614.44422.trast@student.ethz.ch","threadId":"14862","inReplyTo":"alpine.DEB.1.00.0808081341170.9611@pacific.mpi-cbg.de.mpi-cbg.de","subject":"Re: [RFH] filter-branch: ancestor detection weirdness","fromName":"Thomas Rast","fromEmail":"trast@student.ethz.ch","sentAt":"2008-08-08T14:14:41Z","receivedAt":"2008-08-08T14:14:41Z","isPatch":false,"sender":{"key":"tr@thomasrast.ch","avatar":"https://avatars.githubusercontent.com/u/153510?v=4"},"body":"Johannes Schindelin wrote:\n> \n> On Fri, 8 Aug 2008, Thomas Rast wrote:\n> \n> > diff --git a/git-filter-branch.sh b/git-filter-branch.sh\n> > index 182822a..52b2bdf 100755\n> > --- a/git-filter-branch.sh\n> > +++ b/git-filter-branch.sh\n> > @@ -325,15 +325,9 @@ while read ref\n> >  do\n> >  \tsha1=$(git rev-parse \"$ref\"^0)\n> >  \ttest -f \"$workdir\"/../map/$sha1 && continue\n> > -\t# Assign the boundarie(s) in the set of rewritten commits\n> > -\t# as the replacement commit(s).\n> > -\t# (This would look a bit nicer if --not --stdin worked.)\n> > -\tfor p in $( (cd \"$workdir\"/../map; ls | sed \"s/^/^/\") |\n> > -\t\tgit rev-list $ref --boundary --stdin |\n> > -\t\tsed -n \"s/^-//p\")\n> > -\tdo\n> > -\t\tmap $p >> \"$workdir\"/../map/$sha1\n> > -\tdone\n> > +\t# Assign the first commit not pruned as the replacement.\n> > +\tcandidate=$(git rev-list $ref -1 -- \"$filter_subdir\")\n\nI think I see the actual problem.  I made a small testing repository\nwith history that looks like this:\n\n*   a6f2213... (refs/heads/master) Merge branch 'side'\n|\\\n| * 311f888... (refs/heads/side) outside\n| * 472893d... inside dir\n* | 9bd52bc... (refs/heads/stale) outside\n* | d1b451a... inside dir\n|/\n* 1c48eea... initial\n\nIt is available at\n\n  git://persephone.dnsalias.net/git/filtertest.git\n\nif you want to try.  All commits labelled 'inside dir' do something in\ndir/; the others don't.  (You can disregard the 'other' branch for\nnow; I wanted to test the behaviour on completely disconnected history\ntoo, since that's the case with Jan's repo.)\n\nLet's depict this as the following for now, where capitals stand for\n\"interesting\" commits under the subdirectory filter:\n\n   i -- A -- b(stale) -- M(master)\n    \\                   /\n     \\- C -- d(side) --/\n\nWhen saying\n\n  $ git filter-branch --subdirectory-filter dir -- --all'\n\nI would expect the history to look like:\n\n   A(stale) -- M(master)\n              /\n   C(side) --/\n\nI think treating it this way makes a lot of sense; you get the last\nstate that your subdirectory had on the corresponding branch or tag.\n(Similarly, a leaf branch that does not affect 'dir' should be backed\nup until it hits an ancestor that survives the filter.)\n\nNow the problem with the above ancestor detection is the following.\nConsider that at this point, the 'map' directory contains the\n(unfiltered) SHA1 for every commit that was rewritten during the\nfiltering process, i.e.\n\n  $ g rev-list --all -- dir | git name-rev --stdin\n  093c591b3d751ce778b4a6e5c2a0906b097b5868 (other~1)\n  a6f22134f8ab8bcc762949df53f674e3410f7fc3 (master)\n  d1b451a4b0657ea894fd772fc609f7863b7dfd15 (stale~1)\n  472893d579383f56f006ff42c563dcbb730bc5b8 (side~1)\n\nSo 'map' has the values for M, A, and C.  Now if you expand the call\n\n  (cd \"$workdir\"/../map; ls | sed \"s/^/^/\") |\n          git rev-list $ref --boundary --stdin\n\nyou'll find that during ref=refs/heads/side, it is equivalent to\n\n  $ git rev-list side --boundary ^master ^side~1 ^stale~1 ^other~1\n  [no output!]\n\nOops, it seems that wasn't what we wanted.  The '^master', which\nreaches 'side' already, precludes all output.\n\nSo now that I've finally understood what is going on, I think a more\ncareful use of rev-list -1 is actually a correct and easy way to\nfigure out an ancestor.  Patch follows.\n\n- Thomas\n\n-- \nThomas Rast\ntrast@student.ethz.ch\n\n"},{"id":"86500","messageId":"1218204962-17900-1-git-send-email-trast@student.ethz.ch","threadId":"14862","inReplyTo":"200808081614.44422.trast@student.ethz.ch","subject":"[PATCH] filter-branch: fix ancestor discovery for --subdirectory-filter","fromName":"Thomas Rast","fromEmail":"trast@student.ethz.ch","sentAt":"2008-08-08T14:16:02Z","receivedAt":"2008-08-08T14:16:02Z","isPatch":true,"sender":{"key":"tr@thomasrast.ch","avatar":"https://avatars.githubusercontent.com/u/153510?v=4"},"body":"The previous code failed on any refs that are (pre-rewrite) ancestors\nof commits marked for rewriting.  This means that in a situation\n\n   A -- B(topic) -- C(master)\n\nwhere B is dropped by --subdirectory-filter pruning, the 'topic' is\nnot moved up to A as intended, but left unrewritten.\n\nFix this by using a more stupid approach: we let 'rev-list -1' figure\nout a nearby ancestor, which handles the pruning automatically.\n\nSigned-off-by: Thomas Rast <trast@student.ethz.ch>\n---\n\nI guarded it with a $filter_subdir check to not cause any unintended\nharm.  It might be useful in some border cases of rev-list arguments\ngiven to filter-branch too, but I can't figure out a safe way to\nhandle that.\n\nEither way, this fixes the problem.\n\n- Thomas\n\n git-filter-branch.sh |   27 +++++++++++----------------\n 1 files changed, 11 insertions(+), 16 deletions(-)\n\ndiff --git a/git-filter-branch.sh b/git-filter-branch.sh\nindex a324cf0..7924aa1 100755\n--- a/git-filter-branch.sh\n+++ b/git-filter-branch.sh\n@@ -317,24 +317,19 @@ done <../revs\n \n # In case of a subdirectory filter, it is possible that a specified head\n # is not in the set of rewritten commits, because it was pruned by the\n-# revision walker.  Fix it by mapping these heads to the next rewritten\n-# ancestor(s), i.e. the boundaries in the set of rewritten commits.\n+# revision walker.  Fix it by mapping these heads to a (random!) nearby\n+# ancestor that survived the pruning.\n \n-# NEEDSWORK: we should sort the unmapped refs topologically first\n-while read ref\n-do\n-\tsha1=$(git rev-parse \"$ref\"^0)\n-\ttest -f \"$workdir\"/../map/$sha1 && continue\n-\t# Assign the boundarie(s) in the set of rewritten commits\n-\t# as the replacement commit(s).\n-\t# (This would look a bit nicer if --not --stdin worked.)\n-\tfor p in $( (cd \"$workdir\"/../map; ls | sed \"s/^/^/\") |\n-\t\tgit rev-list $ref --boundary --stdin |\n-\t\tsed -n \"s/^-//p\")\n+if test \"$filter_subdir\"\n+then\n+\twhile read ref\n \tdo\n-\t\tmap $p >> \"$workdir\"/../map/$sha1\n-\tdone\n-done < \"$tempdir\"/heads\n+\t\tsha1=$(git rev-parse \"$ref\"^0)\n+\t\ttest -f \"$workdir\"/../map/$sha1 && continue\n+\t\tancestor=$(git rev-list -1 $ref -- \"$filter_subdir\")\n+\t\ttest \"$ancestor\" && echo $(map $ancestor) >> \"$workdir\"/../map/$sha1\n+\tdone < \"$tempdir\"/heads\n+fi\n \n # Finally update the refs\n \n-- \n1.6.0.rc2.22.g7d28.dirty\n"},{"id":"86502","messageId":"alpine.DEB.1.00.0808081632580.24820@pacific.mpi-cbg.de.mpi-cbg.de","threadId":"14862","inReplyTo":"200808081614.44422.trast@student.ethz.ch","subject":"Re: [RFH] filter-branch: ancestor detection weirdness","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2008-08-08T14:39:00Z","receivedAt":"2008-08-08T14:39:00Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Fri, 8 Aug 2008, Thomas Rast wrote:\n\n> I think a more careful use of rev-list -1 is actually a correct and easy \n> way to figure out an ancestor.\n\nI have not looked at your patch closely, or at your explanation, but I am \nreally certain that every attempt to replace the --boundary with a -1 must \nfail.\n\nLet me show you why I think that.  Just look at this history:\n\nA - B - C\n  /\nD\n\nWhere all commits except B touch the inside directory.  Two options:\n\n- you make C a merge (that's what I tried with --boundary), or\n\n- you record B, and C as a commit that does not introduce changes, which \n  is obviously wrong, or\n\n- you record B as a merge, with identical content as A and D, which is \n  pretty tricky (which is why I avoided it).\n\nAnyway, I am really swamped in work, and will not have time to review big \nchanges or explanations.  Besides, filter-branch is no fun.  \nrewrite-commits would have been, but Sven chickened out.\n\nCiao,\nDscho\n"},{"id":"86537","messageId":"200808082037.49918.trast@student.ethz.ch","threadId":"14862","inReplyTo":"alpine.DEB.1.00.0808081632580.24820@pacific.mpi-cbg.de.mpi-cbg.de","subject":"Re: [RFH] filter-branch: ancestor detection weirdness","fromName":"Thomas Rast","fromEmail":"trast@student.ethz.ch","sentAt":"2008-08-08T18:37:45Z","receivedAt":"2008-08-08T18:37:45Z","isPatch":false,"sender":{"key":"tr@thomasrast.ch","avatar":"https://avatars.githubusercontent.com/u/153510?v=4"},"body":"Johannes Schindelin wrote:\n> On Fri, 8 Aug 2008, Thomas Rast wrote:\n> \n> > I think a more careful use of rev-list -1 is actually a correct and easy \n> > way to figure out an ancestor.\n> \n> I have not looked at your patch closely, or at your explanation, but I am \n> really certain that every attempt to replace the --boundary with a -1 must \n> fail.\n> \n> Let me show you why I think that.  Just look at this history:\n> \n> A - B - C\n>   /\n> D\n> \n> Where all commits except B touch the inside directory.  Two options:\n\n'rev-list' \"solves\" this problem for us.  At the point where we are\nrewriting the branch pointers, commits have already been rewritten to\nwhatever 'git rev-list --parents -- $subdir' told us to make them.  I\nthink there are only two cases for its output:\n\n(a) Both A and D bring the same subdirectory contents.  'rev-list\n    --parents -- $subdir' drops one side of the merge during pruning.\n    It does not look past the merge to see whether the contents were\n    arrived at via different changesets.  Thus the history becomes\n\n      A' -- C'\n\n      D'\n\n    and even that only if D was reachable by a different ref,\n    otherwise D' is simply dropped.\n\n(b) A and D bring different $subdir contents.  Then the merge is\n    interesting and remains.  History is now\n\n      A' -- B' -- C'\n           /\n      D' -/\n\nNeither of those cases is a problem for the -1 strategy.  A branch\n'topic' pointing to B will be rewritten to (a) A' and (b) B'.\n\nIOW, either the merge remains and there is no problem, or the side\nbranches vanish too and there is no problem.  rev-list never \"forward\nsimplifies\" merges; it merely tries to prune away commits on the\nincoming side of the merge until all its parents are interesting.\n\nEither that, or I missed something obvious.  I think I'll have to come\nup with a better commit message...\n\n- Thomas\n\n-- \nThomas Rast\ntrast@student.ethz.ch\n\n"},{"id":"86538","messageId":"1218220749-8469-1-git-send-email-trast@student.ethz.ch","threadId":"14862","inReplyTo":"200808082037.49918.trast@student.ethz.ch","subject":"[PATCH v2] filter-branch: fix ref rewriting with --subdirectory-filter","fromName":"Thomas Rast","fromEmail":"trast@student.ethz.ch","sentAt":"2008-08-08T18:39:09Z","receivedAt":"2008-08-08T18:39:09Z","isPatch":true,"sender":{"key":"tr@thomasrast.ch","avatar":"https://avatars.githubusercontent.com/u/153510?v=4"},"body":"The previous ancestor discovery code failed on any refs that are\n(pre-rewrite) ancestors of commits marked for rewriting.  This means\nthat in a situation\n\n   A -- B(topic) -- C(master)\n\nwhere B is dropped by --subdirectory-filter pruning, the 'topic' was\nnot moved up to A as intended, but left unrewritten because we asked\nabout 'git rev-list ^master topic', which does not return anything.\n\nInstead, we use the straightforward\n\n   git rev-list -1 $ref -- $filter_subdir\n\nto find the right ancestor.  To justify this, note that the nearest\nancestor is unique: We use the output of\n\n  git rev-list --parents -- $filter_subdir\n\nto rewrite commits in the first pass, before any ref rewriting.  If B\nis a non-merge commit, the only candidate is its parent.  If it is a\nmerge, there are two cases:\n\n- All sides of the merge bring the same subdirectory contents.  Then\n  rev-list already pruned away the merge in favour for just one of its\n  parents, so there is only one candidate.\n\n- Some merge sides, or the merge outcome, differ.  Then the merge is\n  not pruned and can be rewritten directly.\n\nSo it is always safe to use rev-list -1.\n\nSigned-off-by: Thomas Rast <trast@student.ethz.ch>\n---\n\nOnly comments and commit message changed since v1, to update the\njustification.\n\n git-filter-branch.sh |   27 +++++++++++----------------\n 1 files changed, 11 insertions(+), 16 deletions(-)\n\ndiff --git a/git-filter-branch.sh b/git-filter-branch.sh\nindex a324cf0..a140337 100755\n--- a/git-filter-branch.sh\n+++ b/git-filter-branch.sh\n@@ -317,24 +317,19 @@ done <../revs\n \n # In case of a subdirectory filter, it is possible that a specified head\n # is not in the set of rewritten commits, because it was pruned by the\n-# revision walker.  Fix it by mapping these heads to the next rewritten\n-# ancestor(s), i.e. the boundaries in the set of rewritten commits.\n+# revision walker.  Fix it by mapping these heads to the unique nearest\n+# ancestor that survived the pruning.\n \n-# NEEDSWORK: we should sort the unmapped refs topologically first\n-while read ref\n-do\n-\tsha1=$(git rev-parse \"$ref\"^0)\n-\ttest -f \"$workdir\"/../map/$sha1 && continue\n-\t# Assign the boundarie(s) in the set of rewritten commits\n-\t# as the replacement commit(s).\n-\t# (This would look a bit nicer if --not --stdin worked.)\n-\tfor p in $( (cd \"$workdir\"/../map; ls | sed \"s/^/^/\") |\n-\t\tgit rev-list $ref --boundary --stdin |\n-\t\tsed -n \"s/^-//p\")\n+if test \"$filter_subdir\"\n+then\n+\twhile read ref\n \tdo\n-\t\tmap $p >> \"$workdir\"/../map/$sha1\n-\tdone\n-done < \"$tempdir\"/heads\n+\t\tsha1=$(git rev-parse \"$ref\"^0)\n+\t\ttest -f \"$workdir\"/../map/$sha1 && continue\n+\t\tancestor=$(git rev-list -1 $ref -- \"$filter_subdir\")\n+\t\ttest \"$ancestor\" && echo $(map $ancestor) >> \"$workdir\"/../map/$sha1\n+\tdone < \"$tempdir\"/heads\n+fi\n \n # Finally update the refs\n \n-- \n1.6.0.rc2.23.ge69de8\n"},{"id":"86540","messageId":"1218226224-25273-1-git-send-email-trast@student.ethz.ch","threadId":"14862","inReplyTo":"1218153031-18443-1-git-send-email-trast@student.ethz.ch","subject":"[TOY PATCH] filter-branch: add option --delete-unchanged","fromName":"Thomas Rast","fromEmail":"trast@student.ethz.ch","sentAt":"2008-08-08T20:10:24Z","receivedAt":"2008-08-08T20:10:24Z","isPatch":true,"sender":{"key":"tr@thomasrast.ch","avatar":"https://avatars.githubusercontent.com/u/153510?v=4"},"body":"With --delete-unchanged, we nuke refs whose targets did not change\nduring rewriting.  It is intended to be used along with\n--subdirectory-filter to clean out old refs from before the first\ncommit to the filtered subdirectory.  (They would otherwise keep the\nold history alive.)\n\nObviously this is a rather dangerous mode of operation.\n\nNote the \"sort -u\" is required: Without it, --all includes\n'origin/master' twice (from 'origin/master' and via 'origin/HEAD'),\nand the second pass concludes it is unchanged and nukes the ref.\n\nSigned-off-by: Thomas Rast <trast@student.ethz.ch>\n---\n\nThis applies on top of \"filter-branch: be more helpful when an\nannotated tag changes\".\n\nI'm not really sure if this should go in, but it might have solved\nJan's problem.\n\n git-filter-branch.sh |   33 +++++++++++++++++++++++----------\n 1 files changed, 23 insertions(+), 10 deletions(-)\n\ndiff --git a/git-filter-branch.sh b/git-filter-branch.sh\nindex a140337..539b2e6 100755\n--- a/git-filter-branch.sh\n+++ b/git-filter-branch.sh\n@@ -114,6 +114,7 @@ filter_tag_name=\n filter_subdir=\n orig_namespace=refs/original/\n force=\n+delete_unchanged=\n while :\n do\n \tcase \"$1\" in\n@@ -126,6 +127,11 @@ do\n \t\tforce=t\n \t\tcontinue\n \t\t;;\n+\t--delete-unchanged-refs)\n+\t\tshift\n+\t\tdelete_unchanged=t\n+\t\tcontinue\n+\t\t;;\n \t-*)\n \t\t;;\n \t*)\n@@ -215,6 +221,7 @@ export GIT_DIR GIT_WORK_TREE\n \n # The refs should be updated if their heads were rewritten\n git rev-parse --no-flags --revs-only --symbolic-full-name --default HEAD \"$@\" |\n+sort -u |\n sed -e '/^^/d' >\"$tempdir\"/heads\n \n test -s \"$tempdir\"/heads ||\n@@ -344,7 +351,7 @@ do\n \tsha1=$(git rev-parse \"$ref\"^0)\n \trewritten=$(map $sha1)\n \n-\ttest $sha1 = \"$rewritten\" &&\n+\ttest $sha1 = \"$rewritten\" -a -z \"$delete_unchanged\" &&\n \t\twarn \"WARNING: Ref '$ref' is unchanged\" &&\n \t\tcontinue\n \n@@ -355,16 +362,22 @@ do\n \t\t\tdie \"Could not delete $ref\"\n \t;;\n \t$_x40)\n-\t\techo \"Ref '$ref' was rewritten\"\n-\t\tif ! git update-ref -m \"filter-branch: rewrite\" \\\n-\t\t\t\t\t\"$ref\" $rewritten $sha1 2>/dev/null; then\n-\t\t\tif test $(git cat-file -t \"$ref\") = tag; then\n-\t\t\t\tif test -z \"$filter_tag_name\"; then\n-\t\t\t\t\twarn \"WARNING: You said to rewrite tagged commits, but not the corresponding tag.\"\n-\t\t\t\t\twarn \"WARNING: Perhaps use '--tag-name-filter cat' to rewrite the tag.\"\n+\t\tif test \"$delete_unchanged\" -a $sha1 = \"$rewritten\"; then\n+\t\t\techo \"Ref '$ref' was deleted because it is unchanged\"\n+\t\t\tgit update-ref -m \"filter-branch: delete\" -d \"$ref\" $sha1 ||\n+\t\t\t\tdie \"Could not delete $ref\"\n+\t\telse\n+\t\t\techo \"Ref '$ref' was rewritten\"\n+\t\t\tif ! git update-ref -m \"filter-branch: rewrite\" \\\n+\t\t\t   \"$ref\" $rewritten $sha1 2>/dev/null; then\n+\t\t\t\tif test $(git cat-file -t \"$ref\") = tag; then\n+\t\t\t\t\tif test -z \"$filter_tag_name\"; then\n+\t\t\t\t\t\twarn \"WARNING: You said to rewrite tagged commits, but not the corresponding tag.\"\n+\t\t\t\t\t\twarn \"WARNING: Perhaps use '--tag-name-filter cat' to rewrite the tag.\"\n+\t\t\t\t\tfi\n+\t\t\t\telse\n+\t\t\t\t\tdie \"Could not rewrite $ref\"\n \t\t\t\tfi\n-\t\t\telse\n-\t\t\t\tdie \"Could not rewrite $ref\"\n \t\t\tfi\n \t\tfi\n \t;;\n-- \n1.6.0.rc2.24.gf1dd.dirty\n"},{"id":"86562","messageId":"alpine.DEB.1.00.0808090212060.24820@pacific.mpi-cbg.de.mpi-cbg.de","threadId":"14862","inReplyTo":"200808082037.49918.trast@student.ethz.ch","subject":"Re: [RFH] filter-branch: ancestor detection weirdness","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2008-08-09T00:16:22Z","receivedAt":"2008-08-09T00:16:22Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Fri, 8 Aug 2008, Thomas Rast wrote:\n\n> Johannes Schindelin wrote:\n> > On Fri, 8 Aug 2008, Thomas Rast wrote:\n> > \n> > > I think a more careful use of rev-list -1 is actually a correct and \n> > > easy way to figure out an ancestor.\n> > \n> > I have not looked at your patch closely, or at your explanation, but I \n> > am really certain that every attempt to replace the --boundary with a \n> > -1 must fail.\n> > \n> > Let me show you why I think that.  Just look at this history:\n> > \n> > A - B - C\n> >   /\n> > D\n> >\n> > > > Where all commits except B touch the inside directory.  Two \n> > > > options:\n> \n> 'rev-list' \"solves\" this problem for us.  At the point where we are \n> rewriting the branch pointers, commits have already been rewritten to \n> whatever 'git rev-list --parents -- $subdir' told us to make them.  I \n> think there are only two cases for its output:\n> \n> (a) Both A and D bring the same subdirectory contents.  'rev-list\n>     --parents -- $subdir' drops one side of the merge during pruning. It \n>     does not look past the merge to see whether the contents were \n>     arrived at via different changesets.  Thus the history becomes\n> \n>       A' -- C'\n> \n>       D'\n> \n>     and even that only if D was reachable by a different ref,\n>     otherwise D' is simply dropped.\n\nAnd this is what I call wrong.  Simply dropping one side of the equation \nis not what I call \"sane\".\n\nIf you drop information, you are disagreeing with \"content is king\".\n\nBut hey, if other people agree with you, and this kind of thinking ends \nup in Git proper, I can still resort to other DVCSes.\n\nCiao,\nDscho\n"},{"id":"86564","messageId":"alpine.DEB.1.00.0808090235130.24820@pacific.mpi-cbg.de.mpi-cbg.de","threadId":"14862","inReplyTo":"1218226224-25273-1-git-send-email-trast@student.ethz.ch","subject":"Re: [TOY PATCH] filter-branch: add option --delete-unchanged","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2008-08-09T00:35:46Z","receivedAt":"2008-08-09T00:35:46Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Fri, 8 Aug 2008, Thomas Rast wrote:\n\n> With --delete-unchanged, we nuke refs whose targets did not change\n> during rewriting.\n\nFrankly, I do not see any value in this.  Not even with your explanation.\n\nCiao,\nDscho\n"},{"id":"86571","messageId":"7viqub9dzi.fsf@gitster.siamese.dyndns.org","threadId":"14862","inReplyTo":"alpine.DEB.1.00.0808090212060.24820@pacific.mpi-cbg.de.mpi-cbg.de","subject":"Re: [RFH] filter-branch: ancestor detection weirdness","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2008-08-09T01:25:05Z","receivedAt":"2008-08-09T01:25:05Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:\n\n>> (a) Both A and D bring the same subdirectory contents.  'rev-list\n>>     --parents -- $subdir' drops one side of the merge during pruning. It \n>>     does not look past the merge to see whether the contents were \n>>     arrived at via different changesets.  Thus the history becomes\n>> \n>>       A' -- C'\n>> \n>>       D'\n>> \n>>     and even that only if D was reachable by a different ref,\n>>     otherwise D' is simply dropped.\n>\n> And this is what I call wrong.  Simply dropping one side of the equation \n> is not what I call \"sane\".\n>\n> If you drop information, you are disagreeing with \"content is king\".\n\nI think the aggressive merge simplification that gives \"one simplest\nexplanation for the contents of the paths specified\" is a wrong mode of\noperation to use when you are filtering branches.  It might be a good\nthing to support as an option, but I agree with you that it should not be\nthe default.\n\nPerhaps --full-history is needed to the rev-list call (and the recent\ninvention --simplify-merges that will hopefully appear sometime after\n1.6.0)?  See recent discussion of --full-history and the default merge\nsimplification between Linus and Roman Zippel.  I suspect that back when\nthe original cg-rewritehistory was written, not many people understood the\nissues explained in that thread.\n"},{"id":"86589","messageId":"200808091125.48897.trast@student.ethz.ch","threadId":"14862","inReplyTo":"7viqub9dzi.fsf@gitster.siamese.dyndns.org","subject":"Re: [RFH] filter-branch: ancestor detection weirdness","fromName":"Thomas Rast","fromEmail":"trast@student.ethz.ch","sentAt":"2008-08-09T09:25:42Z","receivedAt":"2008-08-09T09:25:42Z","isPatch":false,"sender":{"key":"tr@thomasrast.ch","avatar":"https://avatars.githubusercontent.com/u/153510?v=4"},"body":"Junio C Hamano wrote:\n> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:\n> \n> >> (a) Both A and D bring the same subdirectory contents.  'rev-list\n> >>     --parents -- $subdir' drops one side of the merge during pruning. It \n> >>     does not look past the merge to see whether the contents were \n> >>     arrived at via different changesets.  Thus the history becomes\n> >> \n> >>       A' -- C'\n> >> \n> >>       D'\n> >> \n> >>     and even that only if D was reachable by a different ref,\n> >>     otherwise D' is simply dropped.\n> >\n> > And this is what I call wrong.  Simply dropping one side of the equation \n> > is not what I call \"sane\".\n> >\n> > If you drop information, you are disagreeing with \"content is king\".\n\nI wonder why I have to be the devil's advocate here.\n\nLet me emphasise: _This is how filter-branch currently works._  It is\nnot some obscure feature coming with my patch.  The user _asks_ for\nthis simplification by using --subdirectory-filter.  It is also\n_happening long before branch rewriting_, and we are discussing a\npatch to said branch rewriting.\n\nJunio has a point:\n\n> I think the aggressive merge simplification that gives \"one simplest\n> explanation for the contents of the paths specified\" is a wrong mode of\n> operation to use when you are filtering branches.  It might be a good\n> thing to support as an option, but I agree with you that it should not be\n> the default.\n> \n> Perhaps --full-history is needed to the rev-list call (and the recent\n\nBut --full-history cannot solve this problem; it would entirely defeat\nthe point of --subdirectory-filter.  (I haven't looked into what\n--simplify-merges does yet.)\n\nThe only thing my patch changes is the behaviour with branches _that\nthe user asked us to rewrite to the subdirectory history_ but that\ndon't point to a precise commit that survived the simplification.  Why\nwould rewriting the branch pointer approriately be bad when the user\nspecifically asked for it?\n\nAnd your _existing_ branch rewriting code had the same thing in mind:\nmove back to an ancestor that roughly fits the ticket.  You just\nmissed the problem with 'rev-list ^master ancestor' that has a high\nchance to break the mechanism with --all.\n\nAnd broke in Jan's case, which is why we're having this discussion,\nremember?\n\n- Thomas\n\n-- \nThomas Rast\ntrast@student.ethz.ch\n\n"},{"id":"86591","messageId":"200808091135.28249.trast@student.ethz.ch","threadId":"14862","inReplyTo":"200808091125.48897.trast@student.ethz.ch","subject":"Re: [RFH] filter-branch: ancestor detection weirdness","fromName":"Thomas Rast","fromEmail":"trast@student.ethz.ch","sentAt":"2008-08-09T09:35:25Z","receivedAt":"2008-08-09T09:35:25Z","isPatch":false,"sender":{"key":"tr@thomasrast.ch","avatar":"https://avatars.githubusercontent.com/u/153510?v=4"},"body":"Thomas Rast wrote:\n> Junio C Hamano wrote:\n> > \n> > Perhaps --full-history is needed to the rev-list call (and the recent\n> \n> But --full-history cannot solve this problem; it would entirely defeat\n> the point of --subdirectory-filter.  (I haven't looked into what\n> --simplify-merges does yet.)\n\nActually, on this point I stand corrected, in some tests it has a good\neffect.  I'll look into it.\n\n- Thomas\n\n-- \nThomas Rast\ntrast@student.ethz.ch\n\n\n"},{"id":"86593","messageId":"200808091200.21634.trast@student.ethz.ch","threadId":"14862","inReplyTo":"alpine.DEB.1.00.0808090212060.24820@pacific.mpi-cbg.de.mpi-cbg.de","subject":"Re: [RFH] filter-branch: ancestor detection weirdness","fromName":"Thomas Rast","fromEmail":"trast@student.ethz.ch","sentAt":"2008-08-09T10:00:19Z","receivedAt":"2008-08-09T10:00:19Z","isPatch":false,"sender":{"key":"tr@thomasrast.ch","avatar":"https://avatars.githubusercontent.com/u/153510?v=4"},"body":"Johannes Schindelin wrote:\n> But hey, if other people agree with you, and this kind of thinking ends \n> up in Git proper, I can still resort to other DVCSes.\n\nBTW, the following is fairly ironic.  (It was later rewritten in\n813b473 to the current one-shot 'rev-list --parents' form.)\n\ncommit 685ef546b62d063c72b401cd38b83a879301aac4\nAuthor: Johannes Schindelin <Johannes.Schindelin@gmx.de>\nDate:   Fri Jun 8 01:30:35 2007 +0100\n\n    Teach filter-branch about subdirectory filtering\n    \n    With git-filter-branch --subdirectory-filter <subdirectory> you can\n    get at the history, as seen by a certain subdirectory. The history\n    of the rewritten branch will only contain commits that touched that\n    subdirectory, and the subdirectory will be rewritten to be the new\n    project root.\n    \n    Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>\n    Signed-off-by: Junio C Hamano <gitster@pobox.com>\n\ndiff --git a/git-filter-branch.sh b/git-filter-branch.sh\n[...snip...]\n@@ -224,7 +228,13 @@ set_ident () {\n \n # list all parent's object names for a given commit\n get_parents () {\n-\tgit-rev-list -1 --parents \"$1\" | sed \"s/^[0-9a-f]*//\"\n+\tcase \"$filter_subdir\" in\n+\t\"\")\n+\t\tgit-rev-list -1 --parents \"$1\"\n+\t\t;;\n+\t*)\n+\t\tgit-rev-list -1 --parents \"$1\" -- \"$filter_subdir\"\n+\tesac | sed \"s/^[0-9a-f]*//\"\n }\n \n tempdir=.git-rewrite\n[...snip...]\n\n-- \nThomas Rast\ntrast@student.ethz.ch\n\n\n"},{"id":"86672","messageId":"1218376960-6406-1-git-send-email-trast@student.ethz.ch","threadId":"14862","inReplyTo":"7viqub9dzi.fsf@gitster.siamese.dyndns.org","subject":"[PATCH] filter-branch: use --simplify-merges","fromName":"Thomas Rast","fromEmail":"trast@student.ethz.ch","sentAt":"2008-08-10T14:02:40Z","receivedAt":"2008-08-10T14:02:40Z","isPatch":true,"sender":{"key":"tr@thomasrast.ch","avatar":"https://avatars.githubusercontent.com/u/153510?v=4"},"body":"Use rev-list --simplify-merges everywhere.  This changes the behaviour\nof --subdirectory-filter in cases such as\n\n  O -- A -\\\n   \\       \\\n    \\- B -- M\n\nwhere A and B bring the same changes to the subdirectory: It now keeps\nboth sides of the merge.  Previously, the history would have been\nsimplified to 'O -- A'.  Merges of unrelated side histories that never\ntouch the subdirectory are still removed.\n\nSigned-off-by: Thomas Rast <trast@student.ethz.ch>\n---\n\nThis obviously depends on --simplify-merges which is only in 'next'.\n\nJunio C Hamano wrote:\n>\n> Perhaps --full-history is needed to the rev-list call (and the recent\n> invention --simplify-merges that will hopefully appear sometime after\n> 1.6.0)?  See recent discussion of --full-history and the default merge\n> simplification between Linus and Roman Zippel.\n\nFollowing history pointers, it turns out the discussion surrounding\na17171b4 (Revert \"filter-branch: subdirectory filter needs\n--full-history\") actually mentions that a simplification step on top\nof --full-history is needed:\n\nJunio C Hamano wrote:\n[http://kerneltrap.org/mailarchive/git/2007/6/13/249107]\n> In short,\n> you will end up with something like this:\n> \n>              .---. (side branch)\n>             /     \\\n>         ---A---B---C (merge)\n> \n> The \"merge clean-up\" would conceptually be a simple operation.\n> Whenever you see a merge C, you look at its parents A and B, and\n> cull the ones that are reachable from other parents.  You notice\n> that A is an ancestor of B, drop A from the parents of C, and\n> simplify the above down to:\n> \n>         ---A---B---C (not-a-merge)\n\nWell, turns out that's what you did with --simplify-merges, so let's\nuse it.\n\n git-filter-branch.sh |    7 ++++---\n 1 files changed, 4 insertions(+), 3 deletions(-)\n\ndiff --git a/git-filter-branch.sh b/git-filter-branch.sh\nindex 539b2e6..60f64ac 100755\n--- a/git-filter-branch.sh\n+++ b/git-filter-branch.sh\n@@ -239,11 +239,11 @@ mkdir ../map || die \"Could not create map/ directory\"\n case \"$filter_subdir\" in\n \"\")\n \tgit rev-list --reverse --topo-order --default HEAD \\\n-\t\t--parents \"$@\"\n+\t\t--parents --simplify-merges \"$@\"\n \t;;\n *)\n \tgit rev-list --reverse --topo-order --default HEAD \\\n-\t\t--parents \"$@\" -- \"$filter_subdir\"\n+\t\t--parents --simplify-merges \"$@\" -- \"$filter_subdir\"\n esac > ../revs || die \"Could not get the commits\"\n commits=$(wc -l <../revs | tr -d \" \")\n \n@@ -333,7 +333,8 @@ then\n \tdo\n \t\tsha1=$(git rev-parse \"$ref\"^0)\n \t\ttest -f \"$workdir\"/../map/$sha1 && continue\n-\t\tancestor=$(git rev-list -1 $ref -- \"$filter_subdir\")\n+\t\tancestor=$(git rev-list --simplify-merges -1 \\\n+\t\t\t\t$ref -- \"$filter_subdir\")\n \t\ttest \"$ancestor\" && echo $(map $ancestor) >> \"$workdir\"/../map/$sha1\n \tdone < \"$tempdir\"/heads\n fi\n-- \n1.6.0.rc2.29.g7ec81\n"},{"id":"86762","messageId":"200808111243.06466.J.Wielemaker@uva.nl","threadId":"14862","inReplyTo":"1218226224-25273-1-git-send-email-trast@student.ethz.ch","subject":"Re: [TOY PATCH] filter-branch: add option --delete-unchanged","fromName":"Jan Wielemaker","fromEmail":"j.wielemaker@uva.nl","sentAt":"2008-08-11T10:43:06Z","receivedAt":"2008-08-11T10:43:06Z","isPatch":true,"sender":{"key":"j.wielemaker@uva.nl","avatar":null},"body":"Hi Thomas,\n\nOn Friday 08 August 2008 10:10:24 pm Thomas Rast wrote:\n> With --delete-unchanged, we nuke refs whose targets did not change\n> during rewriting.  It is intended to be used along with\n> --subdirectory-filter to clean out old refs from before the first\n> commit to the filtered subdirectory.  (They would otherwise keep the\n> old history alive.)\n>\n> Obviously this is a rather dangerous mode of operation.\n>\n> Note the \"sort -u\" is required: Without it, --all includes\n> 'origin/master' twice (from 'origin/master' and via 'origin/HEAD'),\n> and the second pass concludes it is unchanged and nukes the ref.\n>\n> Signed-off-by: Thomas Rast <trast@student.ethz.ch>\n> ---\n>\n> This applies on top of \"filter-branch: be more helpful when an\n> annotated tag changes\".\n>\n> I'm not really sure if this should go in, but it might have solved\n> Jan's problem.\n\nI may hope it isn't just `my problem' :-)  I tested with this patch,\nand I can confirm the following produces precisily what I want:\n\ngit clone /home/git/pl.git/\ncd pl\ngit remote rm origin\ngit filter-branch --subdirectory-filter packages/chr --tag-name-filter \ncat --delete-unchanged-refs -- --all\nrm -r .git/refs/original\ncd ..\ngit clone file://pl chr\n\nchr is now a nice clean 2 MB repository, starting in 2004, the epoch of\nthis directory rather than 1992 (the overall project epoch). \n\nB.t.w. Pretending a remote clone was the only way to get a nice 2MB\nrepo.  The initial is 140MB.  After the filtering it is 62 MB.  Funny:\nafter a git gc it grows to 1.1 Gb!?\n\nAnyway, thanks a lot and I hope this makes it into the next git release!\n\n\tCheers --- Jan\n\n>  git-filter-branch.sh |   33 +++++++++++++++++++++++----------\n>  1 files changed, 23 insertions(+), 10 deletions(-)\n>\n> diff --git a/git-filter-branch.sh b/git-filter-branch.sh\n> index a140337..539b2e6 100755\n> --- a/git-filter-branch.sh\n> +++ b/git-filter-branch.sh\n> @@ -114,6 +114,7 @@ filter_tag_name=\n>  filter_subdir=\n>  orig_namespace=refs/original/\n>  force=\n> +delete_unchanged=\n>  while :\n>  do\n>  \tcase \"$1\" in\n> @@ -126,6 +127,11 @@ do\n>  \t\tforce=t\n>  \t\tcontinue\n>  \t\t;;\n> +\t--delete-unchanged-refs)\n> +\t\tshift\n> +\t\tdelete_unchanged=t\n> +\t\tcontinue\n> +\t\t;;\n>  \t-*)\n>  \t\t;;\n>  \t*)\n> @@ -215,6 +221,7 @@ export GIT_DIR GIT_WORK_TREE\n>\n>  # The refs should be updated if their heads were rewritten\n>  git rev-parse --no-flags --revs-only --symbolic-full-name --default HEAD\n> \"$@\" | +sort -u |\n>  sed -e '/^^/d' >\"$tempdir\"/heads\n>\n>  test -s \"$tempdir\"/heads ||\n> @@ -344,7 +351,7 @@ do\n>  \tsha1=$(git rev-parse \"$ref\"^0)\n>  \trewritten=$(map $sha1)\n>\n> -\ttest $sha1 = \"$rewritten\" &&\n> +\ttest $sha1 = \"$rewritten\" -a -z \"$delete_unchanged\" &&\n>  \t\twarn \"WARNING: Ref '$ref' is unchanged\" &&\n>  \t\tcontinue\n>\n> @@ -355,16 +362,22 @@ do\n>  \t\t\tdie \"Could not delete $ref\"\n>  \t;;\n>  \t$_x40)\n> -\t\techo \"Ref '$ref' was rewritten\"\n> -\t\tif ! git update-ref -m \"filter-branch: rewrite\" \\\n> -\t\t\t\t\t\"$ref\" $rewritten $sha1 2>/dev/null; then\n> -\t\t\tif test $(git cat-file -t \"$ref\") = tag; then\n> -\t\t\t\tif test -z \"$filter_tag_name\"; then\n> -\t\t\t\t\twarn \"WARNING: You said to rewrite tagged commits, but not the\n> corresponding tag.\" -\t\t\t\t\twarn \"WARNING: Perhaps use '--tag-name-filter\n> cat' to rewrite the tag.\" +\t\tif test \"$delete_unchanged\" -a $sha1 =\n> \"$rewritten\"; then\n> +\t\t\techo \"Ref '$ref' was deleted because it is unchanged\"\n> +\t\t\tgit update-ref -m \"filter-branch: delete\" -d \"$ref\" $sha1 ||\n> +\t\t\t\tdie \"Could not delete $ref\"\n> +\t\telse\n> +\t\t\techo \"Ref '$ref' was rewritten\"\n> +\t\t\tif ! git update-ref -m \"filter-branch: rewrite\" \\\n> +\t\t\t   \"$ref\" $rewritten $sha1 2>/dev/null; then\n> +\t\t\t\tif test $(git cat-file -t \"$ref\") = tag; then\n> +\t\t\t\t\tif test -z \"$filter_tag_name\"; then\n> +\t\t\t\t\t\twarn \"WARNING: You said to rewrite tagged commits, but not the\n> corresponding tag.\" +\t\t\t\t\t\twarn \"WARNING: Perhaps use '--tag-name-filter\n> cat' to rewrite the tag.\" +\t\t\t\t\tfi\n> +\t\t\t\telse\n> +\t\t\t\t\tdie \"Could not rewrite $ref\"\n>  \t\t\t\tfi\n> -\t\t\telse\n> -\t\t\t\tdie \"Could not rewrite $ref\"\n>  \t\t\tfi\n>  \t\tfi\n>  \t;;\n"},{"id":"86891","messageId":"7vljz3t2ts.fsf@gitster.siamese.dyndns.org","threadId":"14862","inReplyTo":"1218376960-6406-1-git-send-email-trast@student.ethz.ch","subject":"Re: [PATCH] filter-branch: use --simplify-merges","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2008-08-12T01:54:55Z","receivedAt":"2008-08-12T01:54:55Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Thomas Rast <trast@student.ethz.ch> writes:\n\n> @@ -333,7 +333,8 @@ then\n>  \tdo\n>  \t\tsha1=$(git rev-parse \"$ref\"^0)\n>  \t\ttest -f \"$workdir\"/../map/$sha1 && continue\n> -\t\tancestor=$(git rev-list -1 $ref -- \"$filter_subdir\")\n> +\t\tancestor=$(git rev-list --simplify-merges -1 \\\n> +\t\t\t\t$ref -- \"$filter_subdir\")\n>  \t\ttest \"$ancestor\" && echo $(map $ancestor) >> \"$workdir\"/../map/$sha1\n>  \tdone < \"$tempdir\"/heads\n>  fi\n\nHmm, where does this preimage come from?\n"},{"id":"86893","messageId":"7v7iant1yx.fsf@gitster.siamese.dyndns.org","threadId":"14862","inReplyTo":"7vljz3t2ts.fsf@gitster.siamese.dyndns.org","subject":"Re: [PATCH] filter-branch: use --simplify-merges","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2008-08-12T02:13:26Z","receivedAt":"2008-08-12T02:13:26Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Junio C Hamano <gitster@pobox.com> writes:\n\n> Thomas Rast <trast@student.ethz.ch> writes:\n>\n>> @@ -333,7 +333,8 @@ then\n>>  \tdo\n>>  \t\tsha1=$(git rev-parse \"$ref\"^0)\n>>  \t\ttest -f \"$workdir\"/../map/$sha1 && continue\n>> -\t\tancestor=$(git rev-list -1 $ref -- \"$filter_subdir\")\n>> +\t\tancestor=$(git rev-list --simplify-merges -1 \\\n>> +\t\t\t\t$ref -- \"$filter_subdir\")\n>>  \t\ttest \"$ancestor\" && echo $(map $ancestor) >> \"$workdir\"/../map/$sha1\n>>  \tdone < \"$tempdir\"/heads\n>>  fi\n>\n> Hmm, where does this preimage come from?\n\nNevermind.  You based this on top of the \"fix ancestor discovery\" patch.\n\nI'll squash these two and queue them in 'pu' for now.\n"},{"id":"86901","messageId":"200808120747.22228.trast@student.ethz.ch","threadId":"14862","inReplyTo":"7v7iant1yx.fsf@gitster.siamese.dyndns.org","subject":"Re: [PATCH] filter-branch: use --simplify-merges","fromName":"Thomas Rast","fromEmail":"trast@student.ethz.ch","sentAt":"2008-08-12T05:47:19Z","receivedAt":"2008-08-12T05:47:19Z","isPatch":true,"sender":{"key":"tr@thomasrast.ch","avatar":"https://avatars.githubusercontent.com/u/153510?v=4"},"body":"Junio C Hamano wrote:\n> > Thomas Rast <trast@student.ethz.ch> writes:\n> >\n> >> -\t\tancestor=$(git rev-list -1 $ref -- \"$filter_subdir\")\n> >> +\t\tancestor=$(git rev-list --simplify-merges -1 \\\n> >> +\t\t\t\t$ref -- \"$filter_subdir\")\n> >\n> > Hmm, where does this preimage come from?\n> \n> Nevermind.  You based this on top of the \"fix ancestor discovery\" patch.\n> \n> I'll squash these two and queue them in 'pu' for now.\n\nPlease don't.  I'm still convinced the \"fix ancestor discovery\" is a\nfix to current code that works independent of --simplify-merges.  If\nyou squash them, it cannot go into a release before --simplify-merges\neven if I manage to convince Dscho of this.\n\nThanks.\n\n- Thomas\n\n-- \nThomas Rast\ntrast@student.ethz.ch\n\n"},{"id":"86905","messageId":"7vsktara5n.fsf@gitster.siamese.dyndns.org","threadId":"14862","inReplyTo":"200808120747.22228.trast@student.ethz.ch","subject":"Re: [PATCH] filter-branch: use --simplify-merges","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2008-08-12T06:59:32Z","receivedAt":"2008-08-12T06:59:32Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Thomas Rast <trast@student.ethz.ch> writes:\n\n> Junio C Hamano wrote:\n>>  ... \n>> Nevermind.  You based this on top of the \"fix ancestor discovery\" patch.\n>>\n>> I'll squash these two and queue them in 'pu' for now.\n>\n> Please don't.  I'm still convinced the \"fix ancestor discovery\" is a\n> fix to current code that works independent of --simplify-merges.  If\n> you squash them, it cannot go into a release before --simplify-merges\n> even if I manage to convince Dscho of this.\n\nAnything parked in 'pu' is a fair game for replacement later, so please\nsend a replacement series and tell me to drop the previous ones from 'pu'.\n"},{"id":"86908","messageId":"20080812081851.GK32184@machine.or.cz","threadId":"14862","inReplyTo":"7viqub9dzi.fsf@gitster.siamese.dyndns.org","subject":"Re: [RFH] filter-branch: ancestor detection weirdness","fromName":"Petr Baudis","fromEmail":"pasky@suse.cz","sentAt":"2008-08-12T08:18:51Z","receivedAt":"2008-08-12T08:18:51Z","isPatch":false,"sender":{"key":"pasky@ucw.cz","avatar":"https://avatars.githubusercontent.com/u/18439?v=4"},"body":"On Fri, Aug 08, 2008 at 06:25:05PM -0700, Junio C Hamano wrote:\n> Perhaps --full-history is needed to the rev-list call (and the recent\n> invention --simplify-merges that will hopefully appear sometime after\n> 1.6.0)?  See recent discussion of --full-history and the default merge\n> simplification between Linus and Roman Zippel.  I suspect that back when\n> the original cg-rewritehistory was written, not many people understood the\n> issues explained in that thread.\n\nJust as a historical note, --subdirectory-filter was actually not part\nof cg-admin-rewritehist.\n\n\t\t\t\tPetr \"Pasky\" Baudis\n"},{"id":"86961","messageId":"7vod3yqe19.fsf@gitster.siamese.dyndns.org","threadId":"14862","inReplyTo":"20080812081851.GK32184@machine.or.cz","subject":"Re: [RFH] filter-branch: ancestor detection weirdness","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2008-08-12T18:33:22Z","receivedAt":"2008-08-12T18:33:22Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Petr Baudis <pasky@suse.cz> writes:\n\n> On Fri, Aug 08, 2008 at 06:25:05PM -0700, Junio C Hamano wrote:\n>> Perhaps --full-history is needed to the rev-list call (and the recent\n>> invention --simplify-merges that will hopefully appear sometime after\n>> 1.6.0)?  See recent discussion of --full-history and the default merge\n>> simplification between Linus and Roman Zippel.  I suspect that back when\n>> the original cg-rewritehistory was written, not many people understood the\n>> issues explained in that thread.\n>\n> Just as a historical note, --subdirectory-filter was actually not part\n> of cg-admin-rewritehist.\n\nOk, that sounds more plausible.  Recent addition whose wrinkles have not\nbeen ironed out.\n"},{"id":"86975","messageId":"7v7ialrk9a.fsf@gitster.siamese.dyndns.org","threadId":"14862","inReplyTo":"200808091200.21634.trast@student.ethz.ch","subject":"Re: [RFH] filter-branch: ancestor detection weirdness","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2008-08-12T21:33:37Z","receivedAt":"2008-08-12T21:33:37Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Thomas Rast <trast@student.ethz.ch> writes:\n\n> Johannes Schindelin wrote:\n>> But hey, if other people agree with you, and this kind of thinking ends \n>> up in Git proper, I can still resort to other DVCSes.\n>\n> BTW, the following is fairly ironic.  (It was later rewritten in\n> 813b473 to the current one-shot 'rev-list --parents' form.)\n\nHmm, Dscho, perhaps we should take Thomas's patch as a \"revert to 685ef54\nto fix breakage introduced by 813b473\", and demonstrate the breakage with\none of the new tests in his series?\n\nI think it is Ok to use the \"view --parents for all branches, instead of\nlooping with -1\" approach when there is no path limiter, and that might be\nfaster, but if it complicates the logic too much, it probably is not worth\nit.  I also _suspect_ that if you use --simplify-merges, the optimization\nmade by 813b473 would still be usable even with path limiter.\n\nBy the way, I am not sure if using --simplify-merges unconditionally is\nnecessarily a good thing to do.\n\nThe user who filters the branches may be interested in a full history\n(where using --simplify-merges is the right thing to do), or may be\ninterested in getting one simplest possible explanation of the end result,\nsimilar to what you get from rev-list without the option.\n"},{"id":"86979","messageId":"200808130016.13948.trast@student.ethz.ch","threadId":"14862","inReplyTo":"7v7ialrk9a.fsf@gitster.siamese.dyndns.org","subject":"Re: [RFH] filter-branch: ancestor detection weirdness","fromName":"Thomas Rast","fromEmail":"trast@student.ethz.ch","sentAt":"2008-08-12T22:15:51Z","receivedAt":"2008-08-12T22:15:51Z","isPatch":false,"sender":{"key":"tr@thomasrast.ch","avatar":"https://avatars.githubusercontent.com/u/153510?v=4"},"body":"Junio C Hamano wrote:\n> Hmm, Dscho, perhaps we should take Thomas's patch as a \"revert to 685ef54\n> to fix breakage introduced by 813b473\", and demonstrate the breakage with\n> one of the new tests in his series?\n\nNow you've lost me.\n\nIf you're saying 813b473 is at fault: it is not.  The code I'm trying\nto fix came about in dfd05e38.\n\nTo see that the change in 813b473 is ok, you can simply run the\nfollowing in git.git:\n\n  diff -u <(git rev-list --reverse --parents --topo-order HEAD -- gitk) \\\n    <(git rev-list --reverse --topo-order HEAD -- gitk | while read commit\n      do echo $(git rev-list -1 --parents $commit -- gitk); done)\n\nThe one thing that breaks down is (04c6e9e:git-filter-branch.sh:331)\n\n        for p in $( (cd \"$workdir\"/../map; ls | sed \"s/^/^/\") |\n                git rev-list $ref --boundary --stdin |\n                sed -n \"s/^-//p\")\n\n> I also _suspect_ that if you use --simplify-merges, the optimization\n> made by 813b473 would still be usable even with path limiter.\n\nIt is always usable, if we are careful enough to use the same limiting\narguments in all rev-lists involved.\n\n> By the way, I am not sure if using --simplify-merges unconditionally is\n> necessarily a good thing to do.\n\nI think filter-branch would need a generic mechanism to pass arguments\nthat affect commit selection.  Passing '-- -- file' or '-- ^commit' to\nfilter-branch --subdirectory-filter will probably break a few things,\nso it either needs to recognize those arguments itself or have a\nmechanism to specify them, if we want to support it.  This also goes\nfor the simplification mode.\n\n- Thomas\n\n-- \nThomas Rast\ntrast@student.ethz.ch\n\n"},{"id":"90678","messageId":"94a0d4530809140929s1728a7aevb9f4b0a0469eba0c@mail.gmail.com","threadId":"14862","inReplyTo":"1218226224-25273-1-git-send-email-trast@student.ethz.ch","subject":"Re: [TOY PATCH] filter-branch: add option --delete-unchanged","fromName":"Felipe Contreras","fromEmail":"felipe.contreras@gmail.com","sentAt":"2008-09-14T16:29:59Z","receivedAt":"2008-09-14T16:29:59Z","isPatch":true,"sender":{"key":"felipe.contreras@gmail.com","avatar":"https://avatars.githubusercontent.com/u/8358?v=4"},"body":"On Fri, Aug 8, 2008 at 11:10 PM, Thomas Rast <trast@student.ethz.ch> wrote:\n> With --delete-unchanged, we nuke refs whose targets did not change\n> during rewriting.  It is intended to be used along with\n> --subdirectory-filter to clean out old refs from before the first\n> commit to the filtered subdirectory.  (They would otherwise keep the\n> old history alive.)\n>\n> Obviously this is a rather dangerous mode of operation.\n>\n> Note the \"sort -u\" is required: Without it, --all includes\n> 'origin/master' twice (from 'origin/master' and via 'origin/HEAD'),\n> and the second pass concludes it is unchanged and nukes the ref.\n\nThis is really useful, why isn't it merged?\n\nPersonally I use filter-branch to, duh, filter a branch, so I don't\nwant the commit objects that are not filtered, nor the refs to them.\n\n-- \nFelipe Contreras\n"}]}