{"thread":{"id":"23771","subject":"git gc taking forever to reflog expire --all","startedAt":"2010-05-10T17:58:11Z","lastAt":"2010-05-11T15:30:49Z","messageCount":5,"participants":["Geert Bosch","Jeff King"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"141424","messageId":"0C5B1A89-551A-4EC8-862A-C306A286FAF4@adacore.com","threadId":"23771","inReplyTo":null,"subject":"git gc taking forever to reflog expire --all","fromName":"Geert Bosch","fromEmail":"bosch@adacore.com","sentAt":"2010-05-10T17:58:11Z","receivedAt":"2010-05-10T17:58:11Z","isPatch":false,"sender":{"key":"bosch@adacore.com","avatar":null},"body":"On my recent MacBook Pro, I gave up after reflog expire --all\nhad chewed up 30 minutes of perfectly fine CPU time.\n\nBasically, I have a SVN import of a single branch of a repository\nwith just a single directory with 3K files or so (this SVN import\nis part of a bigger repository). There are about 117K commits\nin the GIT history. I have a few local branches of just a couple\ncommits each. The total .git size is 450MB, with about 3K loose\nobjects.\n\nThis seems something that should be \"git gc\"-ed in about a\nminute or less, so I'm puzzled what is going on in this\ngit reflog expire. From a cursory look it seems that\nwe're in a loop finding a merge base for each commit, or\nsimilar. I'm including a log of a random interruption\n(using Ubuntu here instead of MacOS for better debugging).\n\n -Geert\n\n#0 0x080b1c8c in clear_commit_marks (commit=0x87b2d60, mark=983040)\n at commit.c:403\n#1 0x080b2d44 in get_merge_bases_many (one=0x90a9120, n=1, twos=0xbffff344, \n cleanup=1) at commit.c:655\n#2 0x080b2e67 in get_merge_bases (one=0x90a9120, two=0x819f720, cleanup=1)\n at commit.c:704\n#3 0x080b2e9b in in_merge_bases (commit=0x90a9120, reference=0xbffff51c, \n num=1) at commit.c:728\n#4 0x08092ffa in unreachable (cb=0xbffff514, commit=0x5, \n sha1=0xbffff484 \"\\264\\252\\244\\232\\320=Q\\213\\243Ҋ\\323\\070%[T\\232\\360,\\032\\214=\\226\\340\\001\\253\\335tZ \", ) at builtin/reflog.c:233\n#5 0x080930f5 in expire_reflog_ent (\n osha1=0xbffff498 \"\\232\\360,\\032\\214=\\226\\340\\001\\253\\335tZ \", , \n nsha1=0xbffff484 \"\\264\\252\\244\\232\\320=Q\\213\\243Ҋ\\323\\070%[T\\232\\360,\\032\\214=\\226\\340\\001\\253\\335tZ \", , \n email=0x81c0ada \"Geert Bosch \", timestamp=1265908541, \n tz=0, message=0x81c0b08 \"r4741\\n\", cb_data=0xbffff514)\n at builtin/reflog.c:298\n#6 0x080e2c1f in for_each_recent_reflog_ent (\n ref=0x818066c \"refs/remotes/git-svn@130115\", \n fn=0x8093030 , ofs=0, cb_data=0xbffff514) at refs.c:1623\n#7 0x080e2d27 in for_each_reflog_ent (\n ref=0x818066c \"refs/remotes/git-svn@130115\", \n fn=0x8093030 , cb_data=0xbffff514) at refs.c:1635\n#8 0x080924d7 in expire_reflog (ref=0x818066c \"refs/remotes/git-svn@130115\", \n sha1=0x819f720 \"\\005\", unused=, cb_data=0xbffff594)\n at builtin/reflog.c:353\n#9 0x08092ab6 in cmd_reflog_expire (argc=2, argv=0xbffffb2c, prefix=0x0)\n at builtin/reflog.c:599\n#10 0x0804b374 in run_builtin (argc=3, argv=0xbffffb28) at git.c:260\n#11 handle_internal_command (argc=3, argv=0xbffffb28) at git.c:416\n#12 0x0804b572 in run_argv (argc=3, argv=0xbffffb28) at git.c:458\n#13 main (argc=3, argv=0xbffffb28) at git.c:529\n"},{"id":"141450","messageId":"20100511050134.GA9413@coredump.intra.peff.net","threadId":"23771","inReplyTo":"0C5B1A89-551A-4EC8-862A-C306A286FAF4@adacore.com","subject":"Re: git gc taking forever to reflog expire --all","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2010-05-11T05:01:34Z","receivedAt":"2010-05-11T05:01:34Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Mon, May 10, 2010 at 01:58:11PM -0400, Geert Bosch wrote:\n\n> On my recent MacBook Pro, I gave up after reflog expire --all\n> had chewed up 30 minutes of perfectly fine CPU time.\n\nThere are some corner cases that can cause the expire-unreachable\nalgorithm to behave badly. The fix is b4ca1db (reflog\n--expire-unreachable: avoid merge-base computation, 2010-04-07), but\nhasn't been included in a release yet.\n\nIn the meantime, you can work around it with:\n\n git config gc.reflogExpireUnreachable never\n\n-Peff\n"},{"id":"141467","messageId":"3924516F-2626-41C0-8648-47A9481FA09F@adacore.com","threadId":"23771","inReplyTo":"20100511050134.GA9413@coredump.intra.peff.net","subject":"Re: git gc taking forever to reflog expire --all","fromName":"Geert Bosch","fromEmail":"bosch@adacore.com","sentAt":"2010-05-11T10:05:14Z","receivedAt":"2010-05-11T10:05:14Z","isPatch":false,"sender":{"key":"bosch@adacore.com","avatar":null},"body":"\nOn May 11, 2010, at 01:01, Jeff King wrote:\n\n> On Mon, May 10, 2010 at 01:58:11PM -0400, Geert Bosch wrote:\n> \n>> On my recent MacBook Pro, I gave up after reflog expire --all\n>> had chewed up 30 minutes of perfectly fine CPU time.\n> \n> There are some corner cases that can cause the expire-unreachable\n> algorithm to behave badly. The fix is b4ca1db (reflog\n> --expire-unreachable: avoid merge-base computation, 2010-04-07), but\n> hasn't been included in a release yet.\n> \nI tried with v1.7.1 + cherry-pick b4ca1db968eb57d9cd869337bffa254e0b2c83bd,\nbut that didn't solve the issue.\n> In the meantime, you can work around it with:\n> \n> git config gc.reflogExpireUnreachable never\n\nI had tried that too, but I guess I'll just have to be very patient:\n22707 ttys001 11:14.27 git reflog expire --all\n\nWhat would be the easiest way for me to repack my repository\nwithout running reflog expire at all?\n\n -Geert\n"},{"id":"141468","messageId":"20100511100800.GA452@coredump.intra.peff.net","threadId":"23771","inReplyTo":"3924516F-2626-41C0-8648-47A9481FA09F@adacore.com","subject":"Re: git gc taking forever to reflog expire --all","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2010-05-11T10:08:00Z","receivedAt":"2010-05-11T10:08:00Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Tue, May 11, 2010 at 06:05:14AM -0400, Geert Bosch wrote:\n\n> > There are some corner cases that can cause the expire-unreachable\n> > algorithm to behave badly. The fix is b4ca1db (reflog\n> > --expire-unreachable: avoid merge-base computation, 2010-04-07), but\n> > hasn't been included in a release yet.\n> > \n> I tried with v1.7.1 + cherry-pick b4ca1db968eb57d9cd869337bffa254e0b2c83bd,\n> but that didn't solve the issue.\n> > In the meantime, you can work around it with:\n> > \n> > git config gc.reflogExpireUnreachable never\n> \n> I had tried that too, but I guess I'll just have to be very patient:\n> 22707 ttys001 11:14.27 git reflog expire --all\n\nHmm. If that didn't solve it, then presumably it is a different problem,\nand I'd like to take a closer look. Is it possible for you to make a\ntarball of the .git directory available?\n\n> What would be the easiest way for me to repack my repository\n> without running reflog expire at all?\n\nTry \"git repack -Ad\".\n\n-Peff\n"},{"id":"141483","messageId":"20100511153048.GA17974@coredump.intra.peff.net","threadId":"23771","inReplyTo":"3924516F-2626-41C0-8648-47A9481FA09F@adacore.com","subject":"Re: git gc taking forever to reflog expire --all","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2010-05-11T15:30:49Z","receivedAt":"2010-05-11T15:30:49Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"[for those reading the list, Geert was kind enough to send\n me a copy off his repo off-list. Here's what I found.]\n\nOn Tue, May 11, 2010 at 06:05:14AM -0400, Geert Bosch wrote:\n\n> > There are some corner cases that can cause the expire-unreachable\n> > algorithm to behave badly. The fix is b4ca1db (reflog\n> > --expire-unreachable: avoid merge-base computation, 2010-04-07), but\n> > hasn't been included in a release yet.\n> > \n> I tried with v1.7.1 + cherry-pick b4ca1db968eb57d9cd869337bffa254e0b2c83bd,\n> but that didn't solve the issue.\n\nAre you absolutely sure you weren't accidentally running an older\nversion of git? I was able to reproduce your problem with git v1.7.0,\nbut current 'master' fixes it. I bisected and ended up at b4ca1db.\n\nThere is the possibility that cherry-picking b4ca1db isn't sufficient\nfor some reason, but I didn't see any related commits that would make a\ndifference. In either case, your problem is just another manifestation\nof the same issue, so it should be taken care of in v1.7.2.\n\n> > In the meantime, you can work around it with:\n> > \n> > git config gc.reflogExpireUnreachable never\n> \n> I had tried that too, but I guess I'll just have to be very patient:\n> 22707 ttys001 11:14.27 git reflog expire --all\n\nSorry, this was my fault. Apparently the \"never\" token had a bug which\nwas fixed in v1.7.0.3. If the git you were testing on was before that,\nthen it would have done nothing.\n\n-Peff\n"}]}