{"thread":{"id":"26782","subject":"[PATCH 3/3] fetch-pack: use smaller handshake window for initial request","startedAt":"2011-03-18T22:27:55Z","lastAt":"2011-03-20T23:12:43Z","messageCount":2,"participants":["Junio C Hamano","Shawn Pearce"],"isPatch":true,"patchVersion":1,"patchTotal":3},"messages":[{"id":"163687","messageId":"7vsjukrrmc.fsf@alter.siamese.dyndns.org","threadId":"26782","inReplyTo":null,"subject":"[PATCH 3/3] fetch-pack: use smaller handshake window for initial request","fromName":"Junio C Hamano","fromEmail":"junio@pobox.com","sentAt":"2011-03-18T22:27:55Z","receivedAt":"2011-03-18T22:27:55Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Start the initial request small by halving the INITIAL_FLUSH (we will try\nto stay one window ahead of the server, so we would end up giving twice as\nmany \"have\" in flight at the very beginning).  We may want to tweak these\nvalues even more, taking MTU into account.\n\nSigned-off-by: Junio C Hamano <junio@pobox.com>\n---\n builtin/fetch-pack.c |    6 ++++--\n 1 files changed, 4 insertions(+), 2 deletions(-)\n\ndiff --git a/builtin/fetch-pack.c b/builtin/fetch-pack.c\nindex b4f34a2..3c2c940 100644\n--- a/builtin/fetch-pack.c\n+++ b/builtin/fetch-pack.c\n@@ -218,12 +218,14 @@ static void send_request(int fd, struct strbuf *buf)\n \t\tsafe_write(fd, buf->buf, buf->len);\n }\n \n-#define INITIAL_FLUSH 32\n+#define INITIAL_FLUSH 16\n #define LARGE_FLUSH 1024\n \n static int next_flush(int count)\n {\n-\tif (count < LARGE_FLUSH)\n+\tif (count < INITIAL_FLUSH * 2)\n+\t\tcount += INITIAL_FLUSH;\n+\telse if (count < LARGE_FLUSH)\n \t\tcount <<= 1;\n \telse\n \t\tcount += LARGE_FLUSH;\n"},{"id":"163846","messageId":"AANLkTinpv-kvzoPiFrge95JKHSFAOWvcE11MAjVywwwq@mail.gmail.com","threadId":"26782","inReplyTo":"7vsjukrrmc.fsf@alter.siamese.dyndns.org","subject":"Re: [PATCH 3/3] fetch-pack: use smaller handshake window for initial request","fromName":"Shawn Pearce","fromEmail":"spearce@spearce.org","sentAt":"2011-03-20T23:12:43Z","receivedAt":"2011-03-20T23:12:43Z","isPatch":true,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"On Fri, Mar 18, 2011 at 15:27, Junio C Hamano <junio@pobox.com> wrote:\n> Start the initial request small by halving the INITIAL_FLUSH (we will try\n> to stay one window ahead of the server, so we would end up giving twice as\n> many \"have\" in flight at the very beginning).  We may want to tweak these\n> values even more, taking MTU into account.\n\nThanks Junio, this patch series looks good to me.\n\nI keep thinking about trying to maximize to the MTU, but this is\ndifficult. If we only consider the anonymous git:// over TCP case,\nclients start the conversation with their \"want\" list, which includes\nthe capability list after the first want line. Because the want list\nisn't regular in size (clients request different branches based on\nwhat the server has offered, and what it is behind on, and what the\nuser may have asked for on the command line) and the capability list\nisn't either (clients request a lot of capabilities these days) sizing\nthe initial \"have\" list to round out to fill an MTU is difficult to do\nwith a static constant. The best way to size the initial transfer to\nan MTU boundary is to keep a running counter of bytes written thus\nfar, write *at least* 16 (or whatever our INITIAL_FLUSH is), and then\nwrite additional \"have\" lines until we cannot fit another one in the\ncurrent MTU.\n\nFor subsequent rounds, yes, we can statically size to an MTU, but\nthere isn't much benefit to doing a static size here if we have the\ncode to dynamically size the first batch based on the capability list\nand the wants.\n\nFor smart HTTP however, sizing to an MTU is much more difficult. The\nHTTP headers sent by libcurl are difficult to predict, and go in front\nof the want list. And subsequent rounds include not just the HTTP\nheaders, but also the prior want list and any prior have lines that\nreceived back ACK %s common from the remote peer. So we probably have\nto use a dynamic size for the subsequent rounds too. To make things\nmore difficult, smart HTTP usually gzips the POST body before\ntransmission, which compresses the want/have list somewhat and makes\nit even harder to predict where an MTU would be full.\n\nLong story short, I'm not sure its worth trying to optimize to fill an\nMTU. But if I'm right, doubling the size of the window on each round\nwill reduce the number of round-trips involved. Over git:// this might\nnot be very noticeable since the server is already sending you TCP ACK\nmessages, and the Git-level ACK/NAKs can be piggy-backed into the TCP\nACKs. But over http:// I think its a big win because the Git-level\nACK/NAKs cannot be used until the entire HTTP request has been\nprocessed. It might not seem like a lot, but if your HTTP client is\nbehind 2 HTTP proxies (e.g. your local LAN proxy, and then the remote\nserver is actually a reverse proxy), the HTTP processing can really\nstart to dominate the round-trip time.\n\n-- \nShawn.\n"}]}