{"thread":{"id":"42783","subject":"reflogs and worktrees?","startedAt":"2016-07-05T15:25:22Z","lastAt":"2016-07-09T14:31:24Z","messageCount":3,"participants":["Johannes Schindelin","Duy Nguyen"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"290844","messageId":"alpine.DEB.2.20.1607051704490.8378@virtualbox","threadId":"42783","inReplyTo":null,"subject":"reflogs and worktrees?","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2016-07-05T15:07:55Z","receivedAt":"2016-07-05T15:25:22Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Duy,\n\never since I started working extensively with worktrees, I end up with\nthese funny gc problems, like broken links and stale reflogs.\n\nIs it maybe possible that the reflogs (which are per-worktree now) are not\ntraversed completely when gc'ing (which is *not* per-worktree)?\n\nCiao,\nDscho\n"},{"id":"290845","messageId":"CACsJy8A=_0RfMsDkqyeyro3Xw3NbS4qU0bkVAkapky52jphH7w@mail.gmail.com","threadId":"42783","inReplyTo":"alpine.DEB.2.20.1607051704490.8378@virtualbox","subject":"Re: reflogs and worktrees?","fromName":"Duy Nguyen","fromEmail":"pclouds@gmail.com","sentAt":"2016-07-05T15:59:11Z","receivedAt":"2016-07-05T15:59:49Z","isPatch":false,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Tue, Jul 5, 2016 at 5:07 PM, Johannes Schindelin\n<Johannes.Schindelin@gmx.de> wrote:\n> Hi Duy,\n>\n> ever since I started working extensively with worktrees, I end up with\n> these funny gc problems, like broken links and stale reflogs.\n\nYeah we have problem with gc not traversing all worktree refs. I had\nsomething that could fix it temporarily [1] but the decision was to\nwait for refs code to support new \"worktree storage\" so that we can\nwalk worktree refs properly, then fix gc [2]. We should get pretty\nclose to that point after [3] lands.\n\n[1] http://thread.gmane.org/gmane.comp.version-control.git/295961/focus=296076\n[2] http://thread.gmane.org/gmane.comp.version-control.git/295961/focus=296219\n[3] http://thread.gmane.org/gmane.comp.version-control.git/296409\n\n> Is it maybe possible that the reflogs (which are per-worktree now) are not\n> traversed completely when gc'ing (which is *not* per-worktree)?\n-- \nDuy\n"},{"id":"291104","messageId":"alpine.DEB.2.20.1607091626270.6426@virtualbox","threadId":"42783","inReplyTo":"CACsJy8A=_0RfMsDkqyeyro3Xw3NbS4qU0bkVAkapky52jphH7w@mail.gmail.com","subject":"Re: reflogs and worktrees?","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2016-07-09T14:31:11Z","receivedAt":"2016-07-09T14:31:24Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Duy,\n\nOn Tue, 5 Jul 2016, Duy Nguyen wrote:\n\n> On Tue, Jul 5, 2016 at 5:07 PM, Johannes Schindelin\n> <Johannes.Schindelin@gmx.de> wrote:\n> > Hi Duy,\n> >\n> > ever since I started working extensively with worktrees, I end up with\n> > these funny gc problems, like broken links and stale reflogs.\n> \n> Yeah we have problem with gc not traversing all worktree refs.\n\nThat is a bit of an understatement.\n\nI get tons of errors like this one, just from the regular auto gc:\n\nerror: Could not read 296ee31af712b02469c4bb606fbf2fac229bcca6\nfatal: Failed to traverse parents of commit 3c22e84e2ccdec5d8243344fc4ec68942b87a393\nerror: failed to run repack\n\nI still have to address all of them, in particular because we do not (yet)\nhave tools to identify how certain objects are expected to be reachable\nwhen they are missing (see my `gc-worktree` branch for my current progress\nof teaching git-fsck to describe the broken links better).\n\nFor the moment, I am working around this problem with the following hook\n(wrapped into a unit test to verify that it does what I expect it to do):\n\n-- snipsnap --\nSubject: [PATCH] Demonstrate a workaround for the worktree/gc problem\n\nWhen gc --auto is called in the presence of worktrees, pretty much all of\nthe reflogs go to hell.\n\nIn the --auto case, we can install a hook that runs before-hand,\naccumulates all the worktree-specific refs and reflogs and installs them\ninto a very special reflog in the common refs namespace. The only purpose\nof this stunt is to let gc pick up on those refs and reflogs, of course,\nand *not* ignore them.\n\nUnfortunately, this still does not address the \"git gc\" case, but\nhopefully a real fix will be here some time soon.\n\nSigned-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>\n---\n t/t6500-gc.sh | 63 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++\n 1 file changed, 63 insertions(+)\n\ndiff --git a/t/t6500-gc.sh b/t/t6500-gc.sh\nindex 5d7d414..518e809 100755\n--- a/t/t6500-gc.sh\n+++ b/t/t6500-gc.sh\n@@ -43,4 +43,67 @@ test_expect_success 'gc is not aborted due to a stale symref' '\n \t)\n '\n \n+test_expect_success 'install pre-auto-gc hook for worktrees' '\n+\tmkdir -p .git/hooks &&\n+\twrite_script .git/hooks/pre-auto-gc <<-\\EOF\n+\techo \"Preserving refs/reflogs of worktrees\" >&2 &&\n+\tdir=\"$(git rev-parse --git-common-dir)\" &&\n+\trefsdir=\"$dir/logs/refs\" &&\n+\trm -f \"$refsdir\"/preserve &&\n+\tident=\"$(GIT_COMMITTER_DATE= git var GIT_COMMITTER_IDENT)\" &&\n+\t(\n+\t\tfind \"$dir\"/logs \"$dir\"/worktrees/*/logs \\\n+\t\t\t-type f -exec cat {} \\; |\n+\t\tcut -d\" \" -f1\n+\t\tfind \"$dir\"/HEAD \"$dir\"/worktrees/*/HEAD \"$dir\"/refs \\\n+\t\t\t\"$dir\"/worktrees/*/refs -type f -exec cat {} \\; |\n+\t\tgrep -v \"/^ref:/\"\n+\t) 2>/dev/null |\n+\tsort |\n+\tuniq |\n+\tsed \"s/.*/& & $ident\tdummy/\" >\"$dir\"/preserve &&\n+\tmkdir -p \"$refsdir\" &&\n+\tmv \"$dir\"/preserve \"$refsdir\"/\n+\tEOF\n+'\n+\n+trigger_auto_gc () {\n+\t# This is unfortunately very, very ugly\n+\tgdir=\"$(git rev-parse --git-common-dir)\" &&\n+\tmkdir -p \"$gdir\"/objects/17 &&\n+\ttouch \"$gdir\"/objects/17/17171717171717171717171717171717171717 &&\n+\ttouch \"$gdir\"/objects/17/17171717171717171717171717171717171718 &&\n+\tgit -c gc.auto=1 -c gc.pruneexpire=now -c gc.autodetach=0 gc --auto\n+}\n+\n+test_expect_success 'gc respects refs/reflogs in all worktrees' '\n+\ttest_commit something &&\n+\tgit worktree add worktree &&\n+\t(\n+\t\tcd worktree &&\n+\t\tgit checkout --detach &&\n+\t\techo 1 >something.t &&\n+\t\ttest_tick &&\n+\t\tgit commit -m worktree-reflog something.t &&\n+\t\tgit rev-parse --verify HEAD >../commit-reflog &&\n+\t\techo 2 >something.t &&\n+\t\ttest_tick &&\n+\t\tgit commit -m worktree-ref something.t &&\n+\t\tgit rev-parse --verify HEAD >../commit-ref\n+\t) &&\n+\ttrigger_auto_gc &&\n+\tgit rev-parse --verify $(cat commit-ref)^{commit} &&\n+\tgit rev-parse --verify $(cat commit-reflog)^{commit} &&\n+\n+\t# Now, add a reflog in the top-level dir and verify that `git gc` in\n+\t# the worktree does not purge that\n+\tgit checkout --detach &&\n+\techo 3 >something.t &&\n+\ttest_tick &&\n+\tgit commit -m commondir-reflog something.t &&\n+\tgit rev-parse --verify HEAD >commondir-reflog &&\n+\t(cd worktree && trigger_auto_gc) &&\n+\tgit rev-parse --verify $(cat commondir-reflog)^{commit}\n+'\n+\n test_done\n-- \n2.9.0.278.g1caae67\n\n"}]}