git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 1/2] upload-pack: send keepalive packets during pack computation

From
Jeff King <peff@peff.net>
Date
Sep 8, 2013, 09:01 UTC
Message-ID
<20130908090131.GA17060@sigill.intra.peff.net>
In-Reply-To
<20130908085915.GA4097@sigill.intra.peff.net>

Whenn upload-pack has started pack-objects, there may be a quiet period while pack-objects prepares the pack (i.e., counting objects and delta compression). Normally we would see (and send to the client) progress information, but if "--quiet" is in effect, pack-objects will produce nothing at all until the pack data is ready. On a large repository, this can take tens of seconds (or even minutes if the system is loaded or the repository is badly packed). Clients or intermediate proxies can sometimes give up in this situation, assuming that the server or connection has hung.

This patch introduces a "keepalive" option; if upload-pack sees no data from pack-objects for a certain number of seconds, it will send an empty sideband data packet to let the other side know that we are still working on it.

Signed-off-by: Jeff King <peff@peff.net>
---
 Documentation/config.txt | 11 +++++++++++
 upload-pack.c            | 25 ++++++++++++++++++++++++-
 2 files changed, 35 insertions(+), 1 deletion(-)
diff --git a/Documentation/config.txt b/Documentation/config.txt
index ec57a15..5d748f7 100644
--- a/Documentation/config.txt
+++ b/Documentation/config.txt
@@ -2216,6 +2216,17 @@ uploadpack.allowtipsha1inwant::
 	of a hidden ref (by default, such a request is rejected).
 	see also `uploadpack.hiderefs`.
 
+uploadpack.keepalive::
+	When `upload-pack` has started `pack-objects`, there may be a
+	quiet period while `pack-objects` prepares the pack. Normally
+	it would output progress information, but if `--quiet` was used
+	for the fetch, `pack-objects` will output nothing at all until
+	the pack data begins. Some clients and networks may consider
+	the server to be hung and give up. Setting this option instructs
+	`upload-pack` to send an empty keepalive packet every
+	`uploadpack.keepalive` seconds. Setting this option to 0
+	disables keepalive packets entirely. The default is 0.
+
 url.<base>.insteadOf::
 	Any URL that starts with this value will be rewritten to
 	start, instead, with <base>. In cases where some site serves a
diff --git a/upload-pack.c b/upload-pack.c
index 127e59a..c3e4a20 100644
--- a/upload-pack.c
+++ b/upload-pack.c
@@ -40,6 +40,7 @@ static unsigned int timeout;
 static struct object_array want_obj;
 static struct object_array extra_edge_obj;
 static unsigned int timeout;
+static int keepalive = -1;
 /* 0 for no sideband,
  * otherwise maximum packet size (up to 65520 bytes).
  */
@@ -200,6 +201,7 @@ static void create_pack_file(void)
 	while (1) {
 		struct pollfd pfd[2];
 		int pe, pu, pollsize;
+		int ret;
 
 		reset_timeout();
 
@@ -222,7 +224,8 @@ static void create_pack_file(void)
 		if (!pollsize)
 			break;
 
-		if (poll(pfd, pollsize, -1) < 0) {
+		ret = poll(pfd, pollsize, 1000 * keepalive);
+		if (ret < 0) {
 			if (errno != EINTR) {
 				error("poll failed, resuming: %s",
 				      strerror(errno));
@@ -284,6 +287,21 @@ static void create_pack_file(void)
 			if (sz < 0)
 				goto fail;
 		}
+
+		/*
+		 * We hit the keepalive timeout without saying anything; send
+		 * an empty message on the data sideband just to let the other
+		 * side know we're still working on it, but don't have any data
+		 * yet.
+		 *
+		 * If we don't have a sideband channel, there's no room in the
+		 * protocol to say anything, so those clients are just out of
+		 * luck.
+		 */
+		if (!ret && use_sideband) {
+			static const char buf[] = "0005\1";
+			write_or_die(1, buf, 5);
+		}
 	}
 
 	if (finish_command(&pack_objects)) {
@@ -785,6 +803,11 @@ static int upload_pack_config(const char *var, const char *value, void *unused)
 {
 	if (!strcmp("uploadpack.allowtipsha1inwant", var))
 		allow_tip_sha1_in_want = git_config_bool(var, value);
+	else if (!strcmp("uploadpack.keepalive", var)) {
+		keepalive = git_config_int(var, value);
+		if (!keepalive)
+			keepalive = -1;
+	}
 	return parse_hide_refs_config(var, value, "uploadpack");
 }
 
-- 
1.8.4.2.g87d4a77
Previous: Jeff KingNext: Eric Sunshine
Message 2 of 4 in “upload-pack keepalive”
  1. 0/2 upload-pack keepaliveJeff King, Sep 8, 2013
  2. 1/2 upload-pack: send keepalive packets during pack computationJeff King, Sep 8, 2013
  3. Eric SunshineSep 9, 2013
  4. 2/2 upload-pack: bump keepalive default to 5 secondsJeff King, Sep 8, 2013

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.