Re: [PATCH v2 2/4] csum-file.h: increase hashfile buffer size
- From
Derrick Stolee <stolee@gmail.com>
- Date
- May 18, 2021, 14:44 UTC
- Message-ID
- <6baf7f30-f183-b827-9495-96502ddf1756@gmail.com>
- In-Reply-To
- <YKNtxUtDXoWwaVrc@coredump.intra.peff.net>
On 5/18/2021 3:33 AM, Jeff King wrote:
Show 17 quoted lines
> On Tue, May 18, 2021 at 06:54:57AM +0900, Junio C Hamano wrote: > >> Just after the previous step justified its simplification of its >> progress logic based on how small the buffer is, this step makes it >> 16 times as big, which felt a tiny bit dishonest. We probably >> should say somewhere that 128k is still small enough that the >> rewrite in the previous step is still valid ;-) > > I noticed that, too. I'm not sure if still is small enough. For local > pack writes, etc, it seems fine. But what about "index-pack --stdin" > reading over the network? > > Updating progress every 8k instead of every 128k seems like it would be > more responsive, especially if the network is slow or jittery. I dunno. > Maybe that is too small to care about for the modern world, but I just > want to make sure we are not being blinded by the fast networks all of > us presumably enjoy. :)
This is a good point.
If we combine the earlier suggestion of using the heap to store the buffer, then we can change the buffer size based on the use case: we can use 8k for data that might be streaming from a network, and 128k for local-only data (index, commit-graph, multi-pack-index are all probably safe).
For the case of NFS, I think we should probably assume that the NFS server is not across a slow connection, which is a case we cannot make for streaming a pack-file from a remote server.
Thanks, -Stolee