[PATCH 1/1] upload-pack.c: make output buffer size configurable
- From
Jacob Vosmaer <jacob@gitlab.com>
- Date
- Dec 13, 2021, 13:23 UTC
- Message-ID
- <20211213132345.26310-2-jacob@gitlab.com>
- In-Reply-To
- <20211213132345.26310-1-jacob@gitlab.com>
Add a new compile time constant UPLOAD_PACK_BUFFER_SIZE that makes the size of the static buffer in output_state configurable.
The current size of the buffer is 8192+1. The '+1' is a technical detail orthogonal to this change. On GitLab.com we use GitLab's pack-objects cache which does writes of 65515 bytes. Because of the default 8KB buffer size, propagating these cache writes requires 8 pipe reads and 8 pipe writes from git-upload-pack, and 8 pipe reads from Gitaly (our Git RPC service). If we increase the size of the buffer to the maximum Git packet size, we need only 1 pipe read and 1 pipe write in git-upload-pack, and 1 pipe read in Gitaly to transfer the same amount of data. In benchmarks with a pure fetch and 100% cache hit rate workload we are seeing CPU utilization reductions of over 30%.
Signed-off-by: Jacob Vosmaer <jacob@gitlab.com> --- upload-pack.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)
diff --git a/upload-pack.c b/upload-pack.c index c78d55bc67..b799fbe628 100644 --- a/upload-pack.c +++ b/upload-pack.c @@ -42,6 +42,10 @@ #define ALL_FLAGS (THEY_HAVE | OUR_REF | WANTED | COMMON_KNOWN | SHALLOW | \ NOT_SHALLOW | CLIENT_SHALLOW | HIDDEN_REF) +#ifndef UPLOAD_PACK_BUFFER_SIZE +#define UPLOAD_PACK_BUFFER_SIZE 8192 +#endif + /* Enum for allowed unadvertised object request (UOR) */ enum allow_uor { /* Allow specifying sha1 if it is a ref tip. */ @@ -194,7 +198,7 @@ static int write_one_shallow(const struct commit_graft *graft, void *cb_data) } struct output_state { - char buffer[8193]; + char buffer[UPLOAD_PACK_BUFFER_SIZE+1]; int used; unsigned packfile_uris_started : 1; unsigned packfile_started : 1;
-- 2.33.0