git/list[1] front-page[2] threads[3] people[4] search[5] about
wed 2026-10-07 17:25 UTC

[PATCH v4 03/10] upload-pack: prefer flushing data over sending keepalive

From
Patrick Steinhardt <ps@pks.im>
Date
Mar 13, 2026, 06:45 UTC
Message-ID
<20260313-pks-upload-pack-write-contention-v4-3-7a9668061f7f@pks.im>
In-Reply-To
<20260313-pks-upload-pack-write-contention-v4-0-7a9668061f7f@pks.im>

When using the sideband in git-upload-pack(1) we know to send out keepalive packets in case generating the pack takes too long. These keepalives take the form of a simple empty pktline.

In the preceding commit we have adapted git-upload-pack(1) to buffer data more aggressively before sending it to the client. This creates an obvious optimization opportunity: when we hit the keepalive timeout while we still hold on to some buffered data, then it makes more sense to flush out the data instead of sending the empty keepalive packet.

This is overall not going to be a significant win. Most keepalives will come before the pack data starts, and once pack-objects starts producing data, it tends to do so pretty consistently. And of course we can't send data before we see the PACK header, because the whole point is to buffer the early bit waiting for packfile URIs. But the optimization is easy enough to realize.

Do so and flush out data instead of sending an empty pktline.
Suggested-by: Jeff King <peff@peff.net>
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 upload-pack.c | 21 +++++++++++++++------
 1 file changed, 15 insertions(+), 6 deletions(-)
diff --git a/upload-pack.c b/upload-pack.c
index f6f380a601..7a165d226d 100644
--- a/upload-pack.c
+++ b/upload-pack.c
@@ -466,18 +466,27 @@ static void create_pack_file(struct upload_pack_data *pack_data,
 		}
 
 		/*
-		 * We hit the keepalive timeout without saying anything; send
-		 * an empty message on the data sideband just to let the other
-		 * side know we're still working on it, but don't have any data
-		 * yet.
+		 * We hit the keepalive timeout without saying anything. If we
+		 * have pending data we flush it out to the caller now.
+		 * Otherwise, we send an empty message on the data sideband
+		 * just to let the other side know we're still working on it,
+		 * but don't have any data yet.
 		 *
 		 * If we don't have a sideband channel, there's no room in the
 		 * protocol to say anything, so those clients are just out of
 		 * luck.
 		 */
 		if (!ret && pack_data->use_sideband) {
-			static const char buf[] = "0005\1";
-			write_or_die(1, buf, 5);
+			if (output_state->packfile_started && output_state->used > 1) {
+				send_client_data(1, output_state->buffer, output_state->used - 1,
+						 pack_data->use_sideband);
+				output_state->buffer[0] = output_state->buffer[output_state->used - 1];
+				output_state->used = 1;
+			} else {
+				static const char buf[] = "0005\1";
+				write_or_die(1, buf, 5);
+			}
+
 			last_sent_ms = now_ms;
 		}
 	}
-- 
2.53.0.904.g2727be2e99.dirty
Previous: Patrick SteinhardtNext: Patrick Steinhardt
Message 4 of 11 in “upload-pack: reduce lock contention when writing packfile data”
  1. 00/10 upload-pack: reduce lock contention when writing packfile dataPatrick Steinhardt, Mar 13, 2026
  2. 01/10 upload-pack: fix debug statement when flushing packfile dataPatrick Steinhardt, Mar 13, 2026
  3. 02/10 upload-pack: adapt keepalives based on bufferingPatrick Steinhardt, Mar 13, 2026
  4. 03/10 upload-pack: prefer flushing data over sending keepalivePatrick Steinhardt, Mar 13, 2026
  5. 04/10 upload-pack: reduce lock contention when writing packfile dataPatrick Steinhardt, Mar 13, 2026
  6. 05/10 compat/posix: introduce writev(3p) wrapperPatrick Steinhardt, Mar 13, 2026
  7. 06/10 wrapper: introduce writev(3p) wrappersPatrick Steinhardt, Mar 13, 2026
  8. 07/10 sideband: use writev(3p) to send pktlinesPatrick Steinhardt, Mar 13, 2026
  9. 08/10 csum-file: introduce `hashfd_ext()`Patrick Steinhardt, Mar 13, 2026
  10. 09/10 csum-file: drop `hashfd_throughput()`Patrick Steinhardt, Mar 13, 2026
  11. 10/10 builtin/pack-objects: reduce lock contention when writing packfile dataPatrick Steinhardt, Mar 13, 2026

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.