Re: reflogs and worktrees?

From: Johannes Schindelin <johannes.schindelin@gmx.de>
Date: Jul 9, 2016, 14:31 UTC
Message-ID: <alpine.DEB.2.20.1607091626270.6426@virtualbox>
In-Reply-To: <CACsJy8A=_0RfMsDkqyeyro3Xw3NbS4qU0bkVAkapky52jphH7w@mail.gmail.com>

Hi Duy,

On Tue, 5 Jul 2016, Duy Nguyen wrote:

Show 8 quoted lines

> On Tue, Jul 5, 2016 at 5:07 PM, Johannes Schindelin
> <Johannes.Schindelin@gmx.de> wrote:
> > Hi Duy,
> >
> > ever since I started working extensively with worktrees, I end up with
> > these funny gc problems, like broken links and stale reflogs.
> 
> Yeah we have problem with gc not traversing all worktree refs.

That is a bit of an understatement.

I get tons of errors like this one, just from the regular auto gc:

error: Could not read 296ee31af712b02469c4bb606fbf2fac229bcca6 fatal: Failed to traverse parents of commit 3c22e84e2ccdec5d8243344fc4ec68942b87a393 error: failed to run repack

I still have to address all of them, in particular because we do not (yet) have tools to identify how certain objects are expected to be reachable when they are missing (see my `gc-worktree` branch for my current progress of teaching git-fsck to describe the broken links better).

For the moment, I am working around this problem with the following hook (wrapped into a unit test to verify that it does what I expect it to do):

-- snipsnap --
Subject: [PATCH] Demonstrate a workaround for the worktree/gc problem

When gc --auto is called in the presence of worktrees, pretty much all of the reflogs go to hell.

In the --auto case, we can install a hook that runs before-hand, accumulates all the worktree-specific refs and reflogs and installs them into a very special reflog in the common refs namespace. The only purpose of this stunt is to let gc pick up on those refs and reflogs, of course, and *not* ignore them.

Unfortunately, this still does not address the "git gc" case, but hopefully a real fix will be here some time soon.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 t/t6500-gc.sh | 63 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 63 insertions(+)

diff --git a/t/t6500-gc.sh b/t/t6500-gc.sh
index 5d7d414..518e809 100755
--- a/t/t6500-gc.sh
+++ b/t/t6500-gc.sh
@@ -43,4 +43,67 @@ test_expect_success 'gc is not aborted due to a stale symref' '
 	)
 '
 
+test_expect_success 'install pre-auto-gc hook for worktrees' '
+	mkdir -p .git/hooks &&
+	write_script .git/hooks/pre-auto-gc <<-\EOF
+	echo "Preserving refs/reflogs of worktrees" >&2 &&
+	dir="$(git rev-parse --git-common-dir)" &&
+	refsdir="$dir/logs/refs" &&
+	rm -f "$refsdir"/preserve &&
+	ident="$(GIT_COMMITTER_DATE= git var GIT_COMMITTER_IDENT)" &&
+	(
+		find "$dir"/logs "$dir"/worktrees/*/logs \
+			-type f -exec cat {} \; |
+		cut -d" " -f1
+		find "$dir"/HEAD "$dir"/worktrees/*/HEAD "$dir"/refs \
+			"$dir"/worktrees/*/refs -type f -exec cat {} \; |
+		grep -v "/^ref:/"
+	) 2>/dev/null |
+	sort |
+	uniq |
+	sed "s/.*/& & $ident	dummy/" >"$dir"/preserve &&
+	mkdir -p "$refsdir" &&
+	mv "$dir"/preserve "$refsdir"/
+	EOF
+'
+
+trigger_auto_gc () {
+	# This is unfortunately very, very ugly
+	gdir="$(git rev-parse --git-common-dir)" &&
+	mkdir -p "$gdir"/objects/17 &&
+	touch "$gdir"/objects/17/17171717171717171717171717171717171717 &&
+	touch "$gdir"/objects/17/17171717171717171717171717171717171718 &&
+	git -c gc.auto=1 -c gc.pruneexpire=now -c gc.autodetach=0 gc --auto
+}
+
+test_expect_success 'gc respects refs/reflogs in all worktrees' '
+	test_commit something &&
+	git worktree add worktree &&
+	(
+		cd worktree &&
+		git checkout --detach &&
+		echo 1 >something.t &&
+		test_tick &&
+		git commit -m worktree-reflog something.t &&
+		git rev-parse --verify HEAD >../commit-reflog &&
+		echo 2 >something.t &&
+		test_tick &&
+		git commit -m worktree-ref something.t &&
+		git rev-parse --verify HEAD >../commit-ref
+	) &&
+	trigger_auto_gc &&
+	git rev-parse --verify $(cat commit-ref)^{commit} &&
+	git rev-parse --verify $(cat commit-reflog)^{commit} &&
+
+	# Now, add a reflog in the top-level dir and verify that `git gc` in
+	# the worktree does not purge that
+	git checkout --detach &&
+	echo 3 >something.t &&
+	test_tick &&
+	git commit -m commondir-reflog something.t &&
+	git rev-parse --verify HEAD >commondir-reflog &&
+	(cd worktree && trigger_auto_gc) &&
+	git rev-parse --verify $(cat commondir-reflog)^{commit}
+'
+
 test_done

-- 
2.9.0.278.g1caae67