Re: [PATCH 1/2] http: use unique tempfiles for packfile URI downloads
- From
Taylor Blau <ttaylorr@openai.com>
- Date
- Jul 14, 2026, 04:07 UTC
- Message-ID
- <alW2EnNR21VmkESW@com-79390>
- In-Reply-To
- <alWXwAGWgXSXoRJv@com-76773>
On Mon, Jul 13, 2026 at 06:58:24PM -0700, Ted Nyman wrote:
Show 18 quoted lines
> > While that does sound like a safe and correct approach, stepping > > back briefly, would it not be wasteful for the second process to > > download the same packfile that the first has already started > > downloading? > > Yes. If two fetches overlap, the second download is redundant. > > > Are there better ways for these processes to coordinate with each > > other? Instead of appending to the file, what if the second process > > uses a predictable temporary name (which we already use) to open a > > new file with O_CREAT | O_EXCL to avoid this redundant work? > > Using the existing pack-<hash>.pack.temp name with O_CREAT | O_EXCL > would prevent concurrent writes, but EEXIST alone would not > distinguish an in-progress download from one left by an earlier > failed or interrupted invocation. The existing .pack.temp name is not > covered by the tmp_* pruning path, so simply waiting for it to > disappear could leave a fetch stuck after a crash.
Exactly. If two processes are downloading the same pack at the same time to different locations, the effort is of course redundant. But I don't think we can reliably distinguish between that case and one where an earlier process died in the middle of downloading a pack but was unable to clean up after itself.
Thanks, Taylor