git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: Bug report: v2.47.0 cannot fetch version 1 pack indexes

From
Taylor Blau <me@ttaylorr.com>
Date
Oct 21, 2024, 20:23 UTC
Message-ID
<Zxa4XU+j4+SSmk9c@nand.local>
In-Reply-To
<20241020012455.GA599236@coredump.intra.peff.net>
On Sat, Oct 19, 2024 at 09:24:55PM -0400, Jeff King wrote:
Show 5 quoted lines
> There are a few things I don't like there, though:
>
>   - obviously reversing the tempfile name back to the idx is hacky. We
>     could probably store the original idx that led us to this pack and
>     use that.
TBH, I don't think that this is all that hacky, but...
Show 21 quoted lines
>   - I don't _think_ there is any case where that .idx file is precious.
>     We wouldn't be indexing the .pack file if we didn't just download
>     it, and we wouldn't have downloaded it if we already had a
>     .idx/.pack pair. OTOH the name we got from the other side isn't
>     necessarily the same one we'll use locally; we're feeding the pack
>     via "index-pack --stdin", so it will do its own hash to come up with
>     the name. The other side could have used a different scheme, or even
>     be trying to intentionally trick us.
>
> So I feel like the root of the problem is that we moved the other side's
> "pack-1234abcd.idx" into place at all! We do not plan to use it, but
> only need to access it via our custom packed_git structs to see whether
> we want to download the matching pack. And in fact I'd suspect there are
> other possible bugs/trickery lurking, if the other side gives us a
> broken .idx that does not match its pack (our odb would now have the
> broken file with no matching pack).
>
> So IMHO a cleaner fix is that we should keep the stuff we download from
> the remote marked as temporary files, and then throw them away when we
> complete the fetch (rather than just expecting index-pack to overwrite
> them).

...this seems like a much better idea to me. We shouldn't be keeping the provided *.idx file given that we plan on overwriting it later on in the process.

This feels like more fallout similar to the one described by c177d3dc50 (pack-objects: use finalize_object_file() to rename pack/idx/etc, 2024-09-26), in particularly the paragraph beginning with "This has some test and real-world fallout ..." and the one immediately below it.

So I think that your "cleaner fix" is the way to go.

Thanks, Taylor

Previous: Taylor BlauNext: Jeff King
Message 29 of 31 in “Bug report: v2.47.0 cannot fetch version 1 pack indexes”
  1. foxOct 19, 2024
  2. Eric SunshineOct 20, 2024
  3. Taylor BlauOct 21, 2024
  4. Jeff KingOct 20, 2024
  5. Jeff KingOct 20, 2024
  6. Taylor BlauOct 21, 2024
  7. Jeff KingOct 22, 2024
  8. Taylor BlauOct 22, 2024
  9. 0/11 dumb-http pack index v1 regression + cleanupsJeff King, Oct 25, 2024
  10. 01/11 midx: avoid duplicate packed_git entriesJeff King, Oct 25, 2024
  11. Taylor BlauOct 25, 2024
  12. 02/11 t5550: count fetches in "previously-fetched .idx" testJeff King, Oct 25, 2024
  13. 03/11 dumb-http: store downloaded pack idx as tempfileJeff King, Oct 25, 2024
  14. Taylor BlauOct 25, 2024
  15. Jeff KingOct 26, 2024
  16. Taylor BlauOct 28, 2024
  17. 04/11 packfile: drop has_pack_index()Jeff King, Oct 25, 2024
  18. Taylor BlauOct 25, 2024
  19. 05/11 packfile: drop sha1_pack_name()Jeff King, Oct 25, 2024
  20. 06/11 packfile: drop sha1_pack_index_name()Jeff King, Oct 25, 2024
  21. 07/11 packfile: warn people away from parse_packed_git()Jeff King, Oct 25, 2024
  22. Taylor BlauOct 25, 2024
  23. 08/11 http-walker: use object_id instead of bare hashJeff King, Oct 25, 2024
  24. 09/11 packfile: convert find_sha1_pack() to use object_idJeff King, Oct 25, 2024
  25. 10/11 packfile: use object_id in find_pack_entry_one()Jeff King, Oct 25, 2024
  26. Taylor BlauOct 25, 2024
  27. 11/11 packfile: use oidread() instead of hashcpy() to fill object_idJeff King, Oct 25, 2024
  28. Taylor BlauOct 25, 2024
  29. Taylor BlauOct 21, 2024
  30. Jeff KingOct 22, 2024
  31. Taylor BlauOct 22, 2024

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.