git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v3 0/3] packfile URIs: support concurrent downloads

From
Junio C Hamano <gitster@pobox.com>
Date
Jul 24, 2026, 04:43 UTC
Message-ID
<xmqqldb19evx.fsf@gitster.g>
In-Reply-To
<cover.1784676106.git.tnyman@openai.com>
Ted Nyman <tnyman@openai.com> writes:
Show 21 quoted lines
> Packfile URI and dumb HTTP downloads stage packs at
> objects/pack/pack-<hash>.pack.temp so an interrupted transfer can
> resume. Opening that file in append mode forces every write to its
> current end. Two Git processes fetching the same pack into one object
> database can therefore append duplicate data and corrupt the pack.
> ...
> The tests cover resumption, a completed partial returning 416,
> overlapping 200 and 206 responses, unlinking the staging path while
> index-pack holds its descriptor, and a pre-existing .keep file. The
> unlink test does not require FIFOs, so it can exercise MinGW's sharing
> behavior even though the concurrent-download tests are skipped there.
>
> Changes since v2:
>
>   * Split the --index-pack-arg documentation and error-message cleanup
>     into a preliminary patch, as requested by Junio.
>   * Clarify why per-descriptor offsets keep overlapping writes safe and
>     why MinGW permits the shared staging path to be unlinked.
>   * Add a non-FIFO unlink-while-indexing regression test that can run on
>     MinGW.
>   * Rebase onto the current master.

When merged into 'seen', this topic seems to cause t5550 to hang fairly consistently. It is not surprising, considering that the topic adds roughly 240 lines to the test script in question. It is entirely possible that we are seeing an existing breakage from another topic in 'seen' that is exposed by the additional tests.

The CI run
  https://github.com/git/git/actions/runs/30045343889

is today's seen (excluding this topic) at 728e180b7b; it has breakages in leak checking jobs from other topics, but does not see t5550 hanging.

The CI run
  https://github.com/git/git/actions/runs/30048327878

is seen at 05d0dd408c that merges this topic on top of 728e180b7b above. It breaks the same leak checks, but in addition makes t5550 hang.

Can you help figure out what is going on?
Thanks.
PS. Recent CI runs on 'seen' started to spend so much time on static
    analysis (aka coccinelle) jobs, even though I do not think we
    acquired any new rules recently.  We probably need to figure out
    what is going on there, too.  There is something wrong for these
    CI runs that usually take ~40 minutes to spin for more than 4
    hours.
Previous: Ted NymanNext: Jeff King
Message 26 of 57 in “packfile URIs: support concurrent downloads”
  1. 0/2 packfile URIs: support concurrent downloadsTed Nyman, Jul 13, 2026
  2. 1/2 http: use unique tempfiles for packfile URI downloadsTed Nyman, Jul 13, 2026
  3. Junio C HamanoJul 14, 2026
  4. Ted NymanJul 14, 2026
  5. Taylor BlauJul 14, 2026
  6. Jeff KingJul 14, 2026
  7. Junio C HamanoJul 14, 2026
  8. Ted NymanJul 14, 2026
  9. Taylor BlauJul 14, 2026
  10. Jeff KingJul 14, 2026
  11. Jeff KingJul 14, 2026
  12. 2/2 fetch-pack: accept "pack" output for packfile URIsTed Nyman, Jul 13, 2026
  13. Jeff KingJul 14, 2026
  14. Jeff KingJul 14, 2026
  15. Ted NymanJul 14, 2026
  16. Jeff KingJul 14, 2026
  17. Taylor BlauJul 14, 2026
  18. 0/2 packfile URIs: support concurrent downloadsTed Nyman, Jul 20, 2026
  19. 1/2 http: avoid concurrent appends to partial packsTed Nyman, Jul 20, 2026
  20. Junio C HamanoJul 21, 2026
  21. 2/2 fetch-pack: accept "pack" output for packfile URIsTed Nyman, Jul 20, 2026
  22. 0/3 packfile URIs: support concurrent downloadsTed Nyman, Jul 21, 2026
  23. 1/3 http-fetch: correct --index-pack-arg documentationTed Nyman, Jul 21, 2026
  24. 2/3 http: avoid concurrent appends to partial packsTed Nyman, Jul 21, 2026
  25. 3/3 fetch-pack: accept "pack" output for packfile URIsTed Nyman, Jul 21, 2026
  26. Junio C HamanoJul 24, 2026
  27. Jeff KingJul 25, 2026
  28. Jeff KingJul 25, 2026
  29. Jeff KingJul 25, 2026
  30. Jeff KingJul 25, 2026
  31. Junio C HamanoJul 25, 2026
  32. 0/3 packfile URIs: support concurrent downloadsTed Nyman, Jul 24, 2026
  33. 1/3 http-fetch: correct --index-pack-arg documentationTed Nyman, Jul 24, 2026
  34. Taylor BlauJul 24, 2026
  35. 2/3 http: avoid concurrent appends to partial packsTed Nyman, Jul 24, 2026
  36. 3/3 fetch-pack: accept "pack" output for packfile URIsTed Nyman, Jul 24, 2026
  37. Taylor BlauJul 24, 2026
  38. 0/3 packfile URIs: support concurrent downloadsTed Nyman, Jul 26, 2026
  39. 1/3 http-fetch: correct --index-pack-arg documentationTed Nyman, Jul 26, 2026
  40. 2/3 http: avoid concurrent appends to partial packsTed Nyman, Jul 26, 2026
  41. Jeff KingJul 26, 2026
  42. Ted NymanJul 26, 2026
  43. Jeff KingJul 26, 2026
  44. 3/3 fetch-pack: accept "pack" output for packfile URIsTed Nyman, Jul 26, 2026
  45. Jeff KingJul 26, 2026
  46. 0/6 packfile URIs: support concurrent downloadsTed Nyman, Jul 27, 2026
  47. 1/6 http-fetch: correct --index-pack-arg documentationTed Nyman, Jul 27, 2026
  48. 2/6 http: avoid closing index-pack input twiceTed Nyman, Jul 27, 2026
  49. Jeff KingAug 1, 2026
  50. 3/6 http: accept HTTP 416 for complete partial packsTed Nyman, Jul 27, 2026
  51. Jeff KingAug 1, 2026
  52. 4/6 http: avoid concurrent appends to partial packsTed Nyman, Jul 27, 2026
  53. 5/6 http: permit unlinking partial packs on WindowsTed Nyman, Jul 27, 2026
  54. 6/6 fetch-pack: accept "pack" output for packfile URIsTed Nyman, Jul 27, 2026
  55. Junio C HamanoJul 29, 2026
  56. Jeff KingAug 1, 2026
  57. Junio C HamanoAug 8, 2026

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.