git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: Bug report: v2.47.0 cannot fetch version 1 pack indexes

From
Taylor Blau <me@ttaylorr.com>
Date
Oct 22, 2024, 15:18 UTC
Message-ID
<ZxfCTO+COaoablPe@nand.local>
In-Reply-To
<20241022051402.GB1247135@coredump.intra.peff.net>
On Tue, Oct 22, 2024 at 01:14:02AM -0400, Jeff King wrote:
Show 28 quoted lines
> > > +static char *pack_path_from_idx(const char *idx_path)
> > > +{
> > > +	size_t len;
> > > +	if (!strip_suffix(idx_path, ".idx", &len))
> > > +		BUG("idx path does not end in .idx: %s", idx_path);
> > > +	return xstrfmt("%.*s.pack", (int)len, idx_path);
> > > +}
> > > +
> > >  struct packed_git *parse_pack_index(unsigned char *sha1, const char *idx_path)
> > >  {
> > > -	const char *path = sha1_pack_name(sha1);
> > > +	char *path = pack_path_from_idx(idx_path);
> >
> > Huh. I would have thought we have such a helper function already. I
> > guess we probably do, but that it's also defined statically because it's
> > so easy to write.
>
> I thought so, too, but couldn't find one. We have pack_bitmap_filename()
> (and so on for .rev and .midx files) that goes from .pack to those
> extensions. But here we want to go from .idx to .pack. I think most
> stuff goes from ".pack" because that's what we store in the packed_git
> struct.
>
> There's also sha1_pack_index_name(), but that goes from a csum-file hash
> to a filename.
>
> I grepped around and strip_suffix() seems to be par for the course in
> similar situations within pack/repack code, so I think it's OK here.
It would kind of be nice to have a convenient function like:
    const char *pack_ext_name(const struct packed_git *p,
                              const char *ext);
in our codebase, but that is well outside the scope of this patch ;-).
Show 29 quoted lines
> > In any case, this looks like the right thing to do here. It would be
> > nice to have a corresponding test here, since unlike the other
> > finalize_object_file() changes, this one can be provoked
> > deterministically.
> >
> > Would you mind submitting this as a bona-fide patch, which I can then
> > pick up and start merging down?
>
> Yeah, the test is easy:
>
> diff --git a/t/t5550-http-fetch-dumb.sh b/t/t5550-http-fetch-dumb.sh
> index 58189c9f7d..50a7b98813 100755
> --- a/t/t5550-http-fetch-dumb.sh
> +++ b/t/t5550-http-fetch-dumb.sh
> @@ -507,4 +507,14 @@ test_expect_success 'fetching via http alternates works' '
>  	git -c http.followredirects=true clone "$HTTPD_URL/dumb/alt-child.git"
>  '
>
> +test_expect_success 'dumb http can fetch index v1' '
> +	server=$HTTPD_DOCUMENT_ROOT_PATH/idx-v1.git &&
> +	git init --bare "$server" &&
> +	git -C "$server" --work-tree=. commit --allow-empty -m foo &&
> +	git -C "$server" -c pack.indexVersion=1 gc &&
> +
> +	git clone "$HTTPD_URL/dumb/idx-v1.git" &&
> +	git -C idx-v1 fsck
> +'
> +
>  test_done

Beautifully written. I wondered why you didn't use `test_commit -C $server`, but it's because that repository is bare, and so needs `--work-tree=.`, so what you wrote here makes sense.

Show 11 quoted lines
> I raised some other more philosophical issues in the other part of the
> thread, but assuming the answer is "no, let's do the simplest thing",
> then I think this approach is OK.
>
> I'd also like to see if I can clean things up around parse_pack_index(),
> whose semantics I'm changing here (and which violates all manner of
> assumptions that we usually have about packed_git structs). It's used
> only by the dumb-http code, and I think we want to refactor it a bit so
> that nobody else is tempted to use it.
>
> I'll try to send something out tonight or tomorrow.
Sounds all good. Thanks!
> -Peff

Thanks, Taylor

Previous: Jeff KingNext: Jeff King
Message 8 of 31 in “Bug report: v2.47.0 cannot fetch version 1 pack indexes”
  1. foxOct 19, 2024
  2. Eric SunshineOct 20, 2024
  3. Taylor BlauOct 21, 2024
  4. Jeff KingOct 20, 2024
  5. Jeff KingOct 20, 2024
  6. Taylor BlauOct 21, 2024
  7. Jeff KingOct 22, 2024
  8. Taylor BlauOct 22, 2024
  9. 0/11 dumb-http pack index v1 regression + cleanupsJeff King, Oct 25, 2024
  10. 01/11 midx: avoid duplicate packed_git entriesJeff King, Oct 25, 2024
  11. Taylor BlauOct 25, 2024
  12. 02/11 t5550: count fetches in "previously-fetched .idx" testJeff King, Oct 25, 2024
  13. 03/11 dumb-http: store downloaded pack idx as tempfileJeff King, Oct 25, 2024
  14. Taylor BlauOct 25, 2024
  15. Jeff KingOct 26, 2024
  16. Taylor BlauOct 28, 2024
  17. 04/11 packfile: drop has_pack_index()Jeff King, Oct 25, 2024
  18. Taylor BlauOct 25, 2024
  19. 05/11 packfile: drop sha1_pack_name()Jeff King, Oct 25, 2024
  20. 06/11 packfile: drop sha1_pack_index_name()Jeff King, Oct 25, 2024
  21. 07/11 packfile: warn people away from parse_packed_git()Jeff King, Oct 25, 2024
  22. Taylor BlauOct 25, 2024
  23. 08/11 http-walker: use object_id instead of bare hashJeff King, Oct 25, 2024
  24. 09/11 packfile: convert find_sha1_pack() to use object_idJeff King, Oct 25, 2024
  25. 10/11 packfile: use object_id in find_pack_entry_one()Jeff King, Oct 25, 2024
  26. Taylor BlauOct 25, 2024
  27. 11/11 packfile: use oidread() instead of hashcpy() to fill object_idJeff King, Oct 25, 2024
  28. Taylor BlauOct 25, 2024
  29. Taylor BlauOct 21, 2024
  30. Jeff KingOct 22, 2024
  31. Taylor BlauOct 22, 2024

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.