Re: [PATCH 0/1] Make upload-pack pack write size configurable

From: Jeff King <peff@peff.net>
Date: Dec 14, 2021, 15:12 UTC
Message-ID: <Ybi0dSddal2DKmXQ@coredump.intra.peff.net>
In-Reply-To: <20211213132345.26310-1-jacob@gitlab.com>

On Mon, Dec 13, 2021 at 02:23:44PM +0100, Jacob Vosmaer wrote:

Show 11 quoted lines

> When transfering packfile data, upload-pack.c uses an 8KB buffer.
> This is a reasonable size but when you transfer a lot of packfile
> data, like we do on GitLab.com, we find it is beneficial to use a
> larger buffer size.
> 
> Below you will find a commit where we make the size of this 8KB
> buffer configurable at compile time. It appears pack-objects always
> does 8KB writes so I don't think we should change the default. But
> for GitLab, where we have a cache for the output of pack-objects,
> it is beneficial to use a larger IO size because the cache does
> 64KB writes.

I suspect the big reason that it matters for the cache is that for pack-objects, we're typically CPU bound computing the sha1 of the outgoing pack (for its trailer).

I do suspect we could just always move to something closer to LARGE_PACKET_DATA_MAX (probably minus 1 for the sideband marker). Even if pack-objects only writes in 8k chunks, we may be able to pull several in one read(), especially if the network is a bottleneck (because we'd block writing to the client, and pack-objects would fill up the pipe buffer).

I.e., it doesn't seem like there's any real downside to doing so.

Show 9 quoted lines

> I have also considered converting the packfile copying code to use
> stdio when writing to stdout, but that would be a bigger change
> because we have to be careful not to interleave stdio and stdlib
> writes. And we would have to make the stdout output buffer size
> configurable, because the default stdio buffer size is 4KB which
> is no better than the status quo. A final argument against the stdio
> approach is that it only reduces the number of writes from upload-pack,
> while a larger buffer size reduces both the number of reads and
> writes.

Yeah, I don't think that would help all that much. We really want to size this based on pkt-line limits. That reduces syscalls, but also shrinks the overall size a little (since larger packets minimizes the framing overhead of the pkt-line headers).

-Peff