git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH] fetch-pack: test explicitly that --all can fetch tag references pointing to non-commits

From
Kirill Smelkov <kirr@nexedi.com>
Date
Jun 13, 2018, 18:43 UTC
Message-ID
<20180613184301.GB22854@deco.navytux.spb.ru>
In-Reply-To
<xmqqbmce7fcm.fsf@gitster-ct.c.googlers.com>
On Wed, Jun 13, 2018 at 10:42:33AM -0700, Junio C Hamano wrote:
Show 9 quoted lines
> Kirill Smelkov <kirr@nexedi.com> writes:
> 
> > Fetch-pack --all became broken with respect to unusual tags in
> > 5f0fc64513 (fetch-pack: eliminate spurious error messages, 2012-09-09),
> > and was fixed only recently in e9502c0a7f (fetch-pack: don't try to fetch
> > peel values with --all, 2018-06-11). However the test added in
> > e9502c0a7f does not explicitly cover all funky cases.
> 
> Thanks.  Very much appreciated
Thanks.
Show 37 quoted lines
> > In order to be sure fetching funky tags will never break, let's
> > explicitly test all relevant cases with 4 tag objects pointing to 1) a
> > blob, 2) a tree, 3) a commit, and 4) another tag objects. The referenced
> > tag objects themselves are referenced from under regular refs/tags/*
> > namespace. Before e9502c0a7f `fetch-pack --all` was failing e.g. this way:
> >
> >         .../git/t/trash directory.t5500-fetch-pack/fetchall$ git ls-remote ..
> >         440858748ae905d48259d4fb67a12a7aa1520cf7        HEAD
> >         ...
> >         bc4e9e1fa80662b449805b1ac29fc9b1e4c49187        refs/tags/tag-to-blob                   # <-- NOTE
> >         038f48ad0beaffbea71d186a05084b79e3870cbf        refs/tags/tag-to-blob^{}
> >         520db1f5e1afeaa12b1a8d73ce82db72ca036ee1        refs/tags/tag-to-tree                   # <-- NOTE
> >         7395c100223b7cd760f58ccfa0d3f3d2dd539bb6        refs/tags/tag-to-tree^{}
> >         .../git/t/trash directory.t5500-fetch-pack/fetchall$ git fetch-pack --all ..
> >         fatal: A git upload-pack: not our ref 038f48ad0beaffbea71d186a05084b79e3870cbf
> >         fatal: The remote end hung up unexpectedly
> 
> ... except for this bit.  I am not sure readers understand what
> these two overlong lines want to say.  Would it be easier to
> understand if you wrote the above like this?
> 
>          .../git/t/trash directory.t5500-fetch-pack/fetchall$ git ls-remote ..
>          44085874...        HEAD
>          ...
>          bc4e9e1f...        refs/tags/tag-to-blob
>          038f48ad...        refs/tags/tag-to-blob^{}	# peeled
>          520db1f5...        refs/tags/tag-to-tree
>          7395c100...        refs/tags/tag-to-tree^{}	# peeled
>          .../git/t/trash directory.t5500-fetch-pack/fetchall$ git fetch-pack --all ..
>          fatal: A git upload-pack: not our ref 038f48ad...
>          fatal: The remote end hung up unexpectedly
> 
> Instead of marking the tag-to-blob and tag-to-tree entries (which
> are not where the 'fatal' breakage comes from), I think it makes
> more sense to mark the entries that show peeled version (which also
> matches the object name in the error message), and by shortening the
> line, readers can see more easily which lines are highlighted.
Makes sense. I've amended the patch description with your suggestion.
Show 25 quoted lines
> > +test_expect_success 'test --all wrt tag to non-commits' '
> > +	blob=$(echo "hello blob" | git hash-object -t blob -w --stdin) &&
> > +	git tag -a -m "tag -> blob" tag-to-blob $blob &&
> > + \
> > +	tree=$(printf "100644 blob $blob\tfile" | git mktree) &&
> > +	git tag -a -m "tag -> tree" tag-to-tree $tree &&
> > + \
> > +	tree2=$(printf "100644 blob $blob\tfile2" | git mktree) &&
> > +	commit=$(git commit-tree -m "hello commit" $tree) &&
> > +	git tag -a -m "tag -> commit" tag-to-commit $commit &&
> > + \
> > +	blob2=$(echo "hello blob2" | git hash-object -t blob -w --stdin) &&
> > +	tag=$(printf "object $blob2\ntype blob\ntag tag-to-blob2\n\
> > +tagger author A U Thor <author@example.com> 0 +0000\n\nhello tag" | git mktag) &&
> > +	git tag -a -m "tag -> tag" tag-to-tag $tag &&
> 
> All of the above, while may not be techincallly wrong per-se, look
> unnecessarily complex.
> 
> I guess the reason why you do the above is because you do not want
> to use any object that is reachable via other existing refs and
> instead want to ensure these objects are reachable only via the tags
> you are creating in this test.  Otherwise using HEAD~4:test.txt and
> HEAD^{tree} instead of $blob and $tree constructed via hash-object
> and mktree would be sufficient and more readable.  Oh well.

Yes, it is exactly the reason here. I've added corresponding comment explaining this to the test case.

Kirill
---- 8< ----
From: Kirill Smelkov <kirr@nexedi.com>
Date: Wed, 13 Jun 2018 12:28:21 +0300
Subject: [PATCH v2] fetch-pack: test explicitly that --all can fetch tag
 references pointing to non-commits

Fetch-pack --all became broken with respect to unusual tags in 5f0fc64513 (fetch-pack: eliminate spurious error messages, 2012-09-09), and was fixed only recently in e9502c0a7f (fetch-pack: don't try to fetch peel values with --all, 2018-06-11). However the test added in e9502c0a7f does not explicitly cover all funky cases.

In order to be sure fetching funky tags will never break, let's explicitly test all relevant cases with 4 tag objects pointing to 1) a blob, 2) a tree, 3) a commit, and 4) another tag objects. The referenced tag objects themselves are referenced from under regular refs/tags/* namespace. Before e9502c0a7f `fetch-pack --all` was failing e.g. this way:

        .../git/t/trash directory.t5500-fetch-pack/fetchall$ git ls-remote ..
        44085874...        HEAD
        ...
        bc4e9e1f...        refs/tags/tag-to-blob
        038f48ad...        refs/tags/tag-to-blob^{}	# peeled
        520db1f5...        refs/tags/tag-to-tree
        7395c100...        refs/tags/tag-to-tree^{}	# peeled
        .../git/t/trash directory.t5500-fetch-pack/fetchall$ git fetch-pack --all ..
        fatal: A git upload-pack: not our ref 038f48ad...
        fatal: The remote end hung up unexpectedly
Helped-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Kirill Smelkov <kirr@nexedi.com>
---
 t/t5500-fetch-pack.sh | 31 +++++++++++++++++++++++++++++++
 1 file changed, 31 insertions(+)
diff --git a/t/t5500-fetch-pack.sh b/t/t5500-fetch-pack.sh
index f20bb59d22..5879ee44d7 100755
--- a/t/t5500-fetch-pack.sh
+++ b/t/t5500-fetch-pack.sh
@@ -528,6 +528,37 @@ test_expect_success 'test --all with tag to non-tip' '
 	)
 '
 
+test_expect_success 'test --all wrt tag to non-commits' '
+	# create tag-to-{blob,tree,commit,tag}, making sure all tagged objects
+	# are reachable only via created tag references.
+	blob=$(echo "hello blob" | git hash-object -t blob -w --stdin) &&
+	git tag -a -m "tag -> blob" tag-to-blob $blob &&
+ \
+	tree=$(printf "100644 blob $blob\tfile" | git mktree) &&
+	git tag -a -m "tag -> tree" tag-to-tree $tree &&
+ \
+	tree2=$(printf "100644 blob $blob\tfile2" | git mktree) &&
+	commit=$(git commit-tree -m "hello commit" $tree) &&
+	git tag -a -m "tag -> commit" tag-to-commit $commit &&
+ \
+	blob2=$(echo "hello blob2" | git hash-object -t blob -w --stdin) &&
+	tag=$(printf "object $blob2\ntype blob\ntag tag-to-blob2\n\
+tagger author A U Thor <author@example.com> 0 +0000\n\nhello tag" | git mktag) &&
+	git tag -a -m "tag -> tag" tag-to-tag $tag &&
+ \
+	# `fetch-pack --all` should succeed fetching all those objects.
+	mkdir fetchall &&
+	(
+		cd fetchall &&
+		git init &&
+		git fetch-pack --all .. &&
+		git cat-file blob $blob >/dev/null &&
+		git cat-file tree $tree >/dev/null &&
+		git cat-file commit $commit >/dev/null &&
+		git cat-file tag $tag >/dev/null
+	)
+'
+
 test_expect_success 'shallow fetch with tags does not break the repository' '
 	mkdir repo1 &&
 	(
-- 
2.18.0.rc1.256.g331a1db143
Previous: Junio C HamanoNext: Jeff King
Message 11 of 21 in “fetch-pack: demonstrate --all breakage when remote have tags to non-commit objects”
  1. fetch-pack: demonstrate --all breakage when remote have tags to non-commit objectsKirill Smelkov, Jun 10, 2018
  2. Jeff KingJun 11, 2018
  3. fetch-pack: don't try to fetch peeled values with --allJeff King, Jun 11, 2018
  4. Eric SunshineJun 11, 2018
  5. fetch-pack: don't try to fetch peel values with --allJeff King, Jun 11, 2018
  6. Kirill SmelkovJun 11, 2018
  7. Jeff KingJun 12, 2018
  8. Kirill SmelkovJun 12, 2018
  9. fetch-pack: test explicitly that --all can fetch tag references pointing to non-commitsKirill Smelkov, Jun 13, 2018
  10. Junio C HamanoJun 13, 2018
  11. Kirill SmelkovJun 13, 2018
  12. Jeff KingJun 13, 2018
  13. Jeff KingJun 13, 2018
  14. Kirill SmelkovJun 14, 2018
  15. Junio C HamanoJun 14, 2018
  16. Kirill SmelkovJun 14, 2018
  17. fetch-pack: demonstrate --all failure when remote is emptyKirill Smelkov, Jun 13, 2018
  18. Junio C HamanoJun 13, 2018
  19. Kirill SmelkovJun 13, 2018
  20. Jeff KingJun 13, 2018
  21. Kirill SmelkovJun 14, 2018

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.