{"thread":{"id":"17716","subject":"[PATCH] Add new testcase to show that fast-export can squash merge commits","startedAt":"2009-02-11T06:03:52Z","lastAt":"2009-02-11T17:53:54Z","messageCount":10,"participants":["newren@gmail.com","Johannes Schindelin","Mike Ralphson","Jeff King","Elijah Newren"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"104146","messageId":"1234332233-10017-1-git-send-email-newren@gmail.com","threadId":"17716","inReplyTo":null,"subject":"[PATCH] Add new testcase to show that fast-export can squash merge commits","fromName":"","fromEmail":"newren@gmail.com","sentAt":"2009-02-11T06:03:52Z","receivedAt":"2009-02-11T06:03:52Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"From: Elijah Newren \n\nSigned-off-by: Elijah Newren \n---\nI'm not certain this testcase will actually trigger the bug for everyone,\nsince I don't know the order of commits used by git rev-list. It does\ntrigger it for me. What you need is a history that looks like\n B--D\n / /\n A--C\nwith the master branch pointing at D, and have \"git rev-list --all\" show\nthese commits in the order\n C\n D\n B\n A\nIf you have such a repository, \"git fast-export --all\" will give\ninstructions to create a master branch that only contains D, B, and A.\n t/t9301-fast-export.sh | 29 +++++++++++++++++++++++++++++\n 1 files changed, 29 insertions(+), 0 deletions(-)\n\ndiff --git a/t/t9301-fast-export.sh b/t/t9301-fast-export.sh\nindex 9985721..a1ee400 100755\n--- a/t/t9301-fast-export.sh\n+++ b/t/t9301-fast-export.sh\n@@ -259,4 +259,33 @@ test_expect_success 'cope with tagger-less tags' '\n \n '\n \n+test_expect_success 'avoid squashing merges' '\n+\n+\tmkdir repo &&\n+\tcd repo &&\n+\tgit init &&\n+\techo fee-fi-fo-fum > giant &&\n+\tgit add giant &&\n+\tgit commit -m \"Initial commit\" &&\n+\tgit branch alternate_root &&\n+\techo hello > world &&\n+\tgit add world &&\n+\tgit commit -m \"Commit on master\" &&\n+\tgit checkout alternate_root &&\n+\techo foo > bar &&\n+\tgit add bar &&\n+\tgit commit -m \"Commit on alternate_root\" &&\n+\tgit checkout master &&\n+\tgit merge alternate_root &&\n+\tMASTER=$(git rev-parse --verify refs/heads/master) &&\n+\trm -rf ../new &&\n+\tmkdir ../new &&\n+\tgit --git-dir=../new/.git init &&\n+\tgit fast-export --all |\n+\t(cd ../new &&\n+\t git fast-import &&\n+\t test $MASTER = $(git rev-parse --verify refs/heads/master))\n+\n+'\n+\n test_done\n-- \n1.6.0.6\n"},{"id":"104147","messageId":"1234332233-10017-2-git-send-email-newren@gmail.com","threadId":"17716","inReplyTo":"1234332233-10017-1-git-send-email-newren@gmail.com","subject":"[PATCH] fast-export: ensure we traverse commits in topological order","fromName":"","fromEmail":"newren@gmail.com","sentAt":"2009-02-11T06:03:53Z","receivedAt":"2009-02-11T06:03:53Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"From: Elijah Newren \n\nfast-export will only list as parents those commits which have already\nbeen traversed (making it appear as if merges have been squashed if not\nall parents have been traversed). To avoid this silent squashing of\nmerge commits, we request commits in topological order.\n\nSigned-off-by: Elijah Newren \n---\n builtin-fast-export.c | 1 +\n 1 files changed, 1 insertions(+), 0 deletions(-)\n\ndiff --git a/builtin-fast-export.c b/builtin-fast-export.c\nindex e9ee2c7..fdf4ae9 100644\n--- a/builtin-fast-export.c\n+++ b/builtin-fast-export.c\n@@ -514,6 +514,7 @@ int cmd_fast_export(int argc, const char **argv, const char *prefix)\n \n \tget_tags_and_duplicates(&revs.pending, &extra_refs);\n \n+\trevs.topo_order = 1;\n \tif (prepare_revision_walk(&revs))\n \t\tdie(\"revision walk setup failed\");\n \trevs.diffopt.format_callback = show_filemodify;\n-- \n1.6.0.6\n"},{"id":"104168","messageId":"alpine.DEB.1.00.0902111125410.10279@pacific.mpi-cbg.de","threadId":"17716","inReplyTo":"1234332233-10017-2-git-send-email-newren@gmail.com","subject":"Re: [PATCH] fast-export: ensure we traverse commits in topological order","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2009-02-11T10:25:53Z","receivedAt":"2009-02-11T10:25:53Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Tue, 10 Feb 2009, newren@gmail.com wrote:\n\n> From: Elijah Newren \n> \n> fast-export will only list as parents those commits which have already\n> been traversed (making it appear as if merges have been squashed if not\n> all parents have been traversed). To avoid this silent squashing of\n> merge commits, we request commits in topological order.\n> \n> Signed-off-by: Elijah Newren \n> ---\n> builtin-fast-export.c | 1 +\n> 1 files changed, 1 insertions(+), 0 deletions(-)\n> \n> diff --git a/builtin-fast-export.c b/builtin-fast-export.c\n> index e9ee2c7..fdf4ae9 100644\n> --- a/builtin-fast-export.c\n> +++ b/builtin-fast-export.c\n> @@ -514,6 +514,7 @@ int cmd_fast_export(int argc, const char **argv, const char *prefix)\n> \n> \tget_tags_and_duplicates(&revs.pending, &extra_refs);\n> \n> +\trevs.topo_order = 1;\n\nACK.\n\nCiao,\nDscho\n"},{"id":"104170","messageId":"e2b179460902110248m8055b3amdebcfc550438cff2@mail.gmail.com","threadId":"17716","inReplyTo":"alpine.DEB.1.00.0902111125410.10279@pacific.mpi-cbg.de","subject":"Re: [PATCH] fast-export: ensure we traverse commits in topological order","fromName":"Mike Ralphson","fromEmail":"mike.ralphson@gmail.com","sentAt":"2009-02-11T10:48:18Z","receivedAt":"2009-02-11T10:48:18Z","isPatch":true,"sender":{"key":"mike.ralphson@gmail.com","avatar":"https://avatars.githubusercontent.com/u/21603?v=4"},"body":"> On Tue, 10 Feb 2009, newren@gmail.com wrote:\n> fast-export will only list as parents those commits which have already\n> been traversed (making it appear as if merges have been squashed if not\n> all parents have been traversed). To avoid this silent squashing of\n> merge commits, we request commits in topological order.\n\nAny comparative timings? We don't need to rename this to 'git\nreasonably-speedy-export'? 8-)\n\nMike\n"},{"id":"104173","messageId":"alpine.DEB.1.00.0902111158500.10279@pacific.mpi-cbg.de","threadId":"17716","inReplyTo":"e2b179460902110248m8055b3amdebcfc550438cff2@mail.gmail.com","subject":"Re: [PATCH] fast-export: ensure we traverse commits in topological order","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2009-02-11T10:59:45Z","receivedAt":"2009-02-11T10:59:45Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Wed, 11 Feb 2009, Mike Ralphson wrote:\n\n> > On Tue, 10 Feb 2009, newren@gmail.com wrote:\n> > fast-export will only list as parents those commits which have already\n> > been traversed (making it appear as if merges have been squashed if not\n> > all parents have been traversed). To avoid this silent squashing of\n> > merge commits, we request commits in topological order.\n> \n> Any comparative timings? We don't need to rename this to 'git\n> reasonably-speedy-export'? 8-)\n\nWhat we _could_ do is add proper error handling: fast-export should \nrecognize that it did not see the commit yet, and error out suggesting \nusing --topo-order with fast-export.\n\nCiao,\nDscho\n"},{"id":"104183","messageId":"20090211135640.GA19600@coredump.intra.peff.net","threadId":"17716","inReplyTo":"e2b179460902110248m8055b3amdebcfc550438cff2@mail.gmail.com","subject":"Re: [PATCH] fast-export: ensure we traverse commits in topological order","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2009-02-11T13:56:40Z","receivedAt":"2009-02-11T13:56:40Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Wed, Feb 11, 2009 at 10:48:18AM +0000, Mike Ralphson wrote:\n\n> > On Tue, 10 Feb 2009, newren@gmail.com wrote:\n> > fast-export will only list as parents those commits which have already\n> > been traversed (making it appear as if merges have been squashed if not\n> > all parents have been traversed). To avoid this silent squashing of\n> > merge commits, we request commits in topological order.\n> \n> Any comparative timings? We don't need to rename this to 'git\n> reasonably-speedy-export'? 8-)\n\nHmm.\n\nIn git.git:\n\n $ time git fast-export --all --signed-tags=strip >/dev/null\n real 1m6.013s\n user 1m3.840s\n sys 0m2.140s\n\n $ time git fast-export --all --signed-tags=strip --topo-order >/dev/null\n real 0m49.018s\n user 0m47.987s\n sys 0m0.888s\n\nI certainly didn't expect it to be _faster_. More efficient use of the\ndelta cache, maybe?\n\n-Peff\n"},{"id":"104191","messageId":"alpine.DEB.1.00.0902111618120.13279@intel-tinevez-2-302","threadId":"17716","inReplyTo":"20090211135640.GA19600@coredump.intra.peff.net","subject":"Re: [PATCH] fast-export: ensure we traverse commits in topological order","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2009-02-11T15:18:28Z","receivedAt":"2009-02-11T15:18:28Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Wed, 11 Feb 2009, Jeff King wrote:\n\n> On Wed, Feb 11, 2009 at 10:48:18AM +0000, Mike Ralphson wrote:\n> \n> > > On Tue, 10 Feb 2009, newren@gmail.com wrote:\n> > > fast-export will only list as parents those commits which have already\n> > > been traversed (making it appear as if merges have been squashed if not\n> > > all parents have been traversed). To avoid this silent squashing of\n> > > merge commits, we request commits in topological order.\n> > \n> > Any comparative timings? We don't need to rename this to 'git\n> > reasonably-speedy-export'? 8-)\n> \n> Hmm.\n> \n> In git.git:\n> \n> $ time git fast-export --all --signed-tags=strip >/dev/null\n> real 1m6.013s\n> user 1m3.840s\n> sys 0m2.140s\n> \n> $ time git fast-export --all --signed-tags=strip --topo-order >/dev/null\n> real 0m49.018s\n> user 0m47.987s\n> sys 0m0.888s\n> \n> I certainly didn't expect it to be _faster_. More efficient use of the\n> delta cache, maybe?\n\nOr a warm against a cold cache?\n\nHides,\nDscho\n"},{"id":"104192","messageId":"51419b2c0902110723y75d0c4a4r34cf92cf371313ea@mail.gmail.com","threadId":"17716","inReplyTo":"20090211135640.GA19600@coredump.intra.peff.net","subject":"Re: [PATCH] fast-export: ensure we traverse commits in topological order","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2009-02-11T15:23:22Z","receivedAt":"2009-02-11T15:23:22Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"On Wed, Feb 11, 2009 at 6:56 AM, Jeff King wrote:\n> On Wed, Feb 11, 2009 at 10:48:18AM +0000, Mike Ralphson wrote:\n>\n>> > On Tue, 10 Feb 2009, newren@gmail.com wrote:\n>> > fast-export will only list as parents those commits which have already\n>> > been traversed (making it appear as if merges have been squashed if not\n>> > all parents have been traversed). To avoid this silent squashing of\n>> > merge commits, we request commits in topological order.\n>>\n>> Any comparative timings? We don't need to rename this to 'git\n>> reasonably-speedy-export'? 8-)\n>\n> Hmm.\n>\n> In git.git:\n>\n> $ time git fast-export --all --signed-tags=strip >/dev/null\n> real 1m6.013s\n> user 1m3.840s\n> sys 0m2.140s\n>\n> $ time git fast-export --all --signed-tags=strip --topo-order >/dev/null\n> real 0m49.018s\n> user 0m47.987s\n> sys 0m0.888s\n>\n> I certainly didn't expect it to be _faster_. More efficient use of the\n> delta cache, maybe?\n\nYeah, I also saw a speed-up, though the difference was less pronounced\n-- 19.7 seconds versus 19.5 (that was the average for 5 runs, after\nfirst doing a run that I ignored to make sure I was comparing\nhot-cache to hot-cache).\n\nI'm a little surprised why my numbers were so much lower than yours,\nthough. I don't think of my hard drive and CPU as being all that\nquick -- is this a difference in packing, by chance, with your\nrepository not having recently been repacked?\n"},{"id":"104222","messageId":"20090211173717.GA19749@coredump.intra.peff.net","threadId":"17716","inReplyTo":"alpine.DEB.1.00.0902111618120.13279@intel-tinevez-2-302","subject":"Re: [PATCH] fast-export: ensure we traverse commits in topological order","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2009-02-11T17:37:17Z","receivedAt":"2009-02-11T17:37:17Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Wed, Feb 11, 2009 at 04:18:28PM +0100, Johannes Schindelin wrote:\n\n> > $ time git fast-export --all --signed-tags=strip >/dev/null\n> > real 1m6.013s\n> > user 1m3.840s\n> > sys 0m2.140s\n> \n> Or a warm against a cold cache?\n\nNope. See how the CPU is more or less pegged in the numbers above? Cold\ncache looks like this:\n\n $ echo 3 >/proc/sys/vm/drop_caches\n $ time git fast-export --all --signed-tags=strip >/dev/null\n real 1m11.148s\n user 1m3.952s\n sys 0m0.760s\n\nSo only 7 extra seconds paging in. Hooray for packing. :)\n\n-Peff\n"},{"id":"104226","messageId":"20090211175354.GB19749@coredump.intra.peff.net","threadId":"17716","inReplyTo":"51419b2c0902110723y75d0c4a4r34cf92cf371313ea@mail.gmail.com","subject":"Re: [PATCH] fast-export: ensure we traverse commits in topological order","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2009-02-11T17:53:54Z","receivedAt":"2009-02-11T17:53:54Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Wed, Feb 11, 2009 at 08:23:22AM -0700, Elijah Newren wrote:\n\n> I'm a little surprised why my numbers were so much lower than yours,\n> though. I don't think of my hard drive and CPU as being all that\n> quick -- is this a difference in packing, by chance, with your\n> repository not having recently been repacked?\n\nI don't think the hard drive matters much at all; the CPU is pegged the\nwhole time. I'm not fully packed, but almost so. I suspect it is a\ncombination of:\n\n 1. I compile git with \"-g\" and no optimizations to make debugging\n easier.\n\n 2. My system sucks. It is a 1.8GHz dual Athlon from around 2003.\n fast-export should be pretty memory intensive, which I'm sure is\n saturating my totally rocking 266MHz FSB.\n\nJust for fun, I tried:\n\n - fully repacking; no change\n\n - compiling with -O2; shaved off about 2s. Not too surprising;\n previous profiling has shown that git spends a lot of time in zlib,\n and I am dynamically linking against an optimized version.\n\nSo I think it is mainly (2) above, assuming your machine sucks a lot\nless than mine.\n\n-Peff\n"}]}