{"thread":{"id":"41274","subject":"[PATCH] attempt connects in parallel for IPv6-capable builds","startedAt":"2016-01-28T11:57:21Z","lastAt":"2016-01-30T13:09:33Z","messageCount":7,"participants":["Eric Wong","Junio C Hamano","Torsten Bögershausen","Johannes Sixt"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"276991","messageId":"20160128115720.GA1827@dcvr.yhbt.net","threadId":"41274","inReplyTo":null,"subject":"[PATCH] attempt connects in parallel for IPv6-capable builds","fromName":"Eric Wong","fromEmail":"normalperson@yhbt.net","sentAt":"2016-01-28T11:57:21Z","receivedAt":"2016-01-28T11:57:21Z","isPatch":true,"sender":{"key":"e@80x24.org","avatar":null},"body":"getaddrinfo() may return multiple addresses, not all of which\nare equally performant.  In some cases, a user behind a non-IPv6\ncapable network may get an IPv6 address which stalls connect().\nInstead of waiting synchronously for a connect() to timeout, use\nnon-blocking connect() in parallel and take the first successful\nconnection.\n\nThis may increase network traffic and server load slightly, but\nmakes the worst-case user experience more bearable when one\nlacks permissions to edit /etc/gai.conf to favor IPv4 addresses.\n\nSigned-off-by: Eric Wong <normalperson@yhbt.net>\n---\n connect.c | 118 ++++++++++++++++++++++++++++++++++++++++++++++++++++++--------\n 1 file changed, 104 insertions(+), 14 deletions(-)\n\ndiff --git a/connect.c b/connect.c\nindex fd7ffe1..74d2bb5 100644\n--- a/connect.c\n+++ b/connect.c\n@@ -14,6 +14,42 @@\n static char *server_capabilities;\n static const char *parse_feature_value(const char *, const char *, int *);\n \n+#ifdef SOCK_NONBLOCK /* Linux-only flag */\n+#  define GIT_SOCK_NONBLOCK SOCK_NONBLOCK\n+#else\n+#  define GIT_SOCK_NONBLOCK 0\n+#endif\n+\n+static int socket_nb(int domain, int type, int protocol)\n+{\n+\tstatic int flags = GIT_SOCK_NONBLOCK;\n+\tint fd = socket(domain, type | flags, protocol);\n+\n+\t/* new headers, old kernel? */\n+\tif (fd < 0 && errno == EINVAL && flags != 0) {\n+\t\tflags = 0;\n+\t\tfd = socket(domain, type, protocol);\n+\t}\n+\n+\t/* couldn't use SOCK_NONBLOCK, set non-blocking the old way */\n+\tif (flags == 0 && fd >= 0) {\n+\t\tint fl = fcntl(fd, F_GETFL);\n+\n+\t\tif (fcntl(fd, F_SETFL, fl | O_NONBLOCK) < 0)\n+\t\t\tdie_errno(\"failed to set nonblocking flag\\n\");\n+\t}\n+\n+\treturn fd;\n+}\n+\n+static void set_blocking(int fd)\n+{\n+\tint fl = fcntl(fd, F_GETFL);\n+\n+\tif (fcntl(fd, F_SETFL, fl & ~O_NONBLOCK) < 0)\n+\t\tdie_errno(\"failed to clear nonblocking flag\\n\");\n+}\n+\n static int check_ref(const char *name, unsigned int flags)\n {\n \tif (!flags)\n@@ -351,6 +387,9 @@ static int git_tcp_connect_sock(char *host, int flags)\n \tstruct addrinfo hints, *ai0, *ai;\n \tint gai;\n \tint cnt = 0;\n+\tnfds_t n = 0, nfds = 0;\n+\tstruct pollfd *fds = NULL;\n+\tstruct addrinfo **inprogress = NULL;\n \n \tget_host_and_port(&host, &port);\n \tif (!*port)\n@@ -371,20 +410,76 @@ static int git_tcp_connect_sock(char *host, int flags)\n \t\tfprintf(stderr, \"done.\\nConnecting to %s (port %s) ... \", host, port);\n \n \tfor (ai0 = ai; ai; ai = ai->ai_next, cnt++) {\n-\t\tsockfd = socket(ai->ai_family,\n-\t\t\t\tai->ai_socktype, ai->ai_protocol);\n-\t\tif ((sockfd < 0) ||\n-\t\t    (connect(sockfd, ai->ai_addr, ai->ai_addrlen) < 0)) {\n+\t\tsize_t cur;\n+\t\tint fd = socket_nb(ai->ai_family, ai->ai_socktype,\n+\t\t\t\t\tai->ai_protocol);\n+\t\tif (fd < 0) {\n \t\t\tstrbuf_addf(&error_message, \"%s[%d: %s]: errno=%s\\n\",\n \t\t\t\t    host, cnt, ai_name(ai), strerror(errno));\n-\t\t\tif (0 <= sockfd)\n-\t\t\t\tclose(sockfd);\n-\t\t\tsockfd = -1;\n \t\t\tcontinue;\n \t\t}\n+\n+\t\tif (connect(fd, ai->ai_addr, ai->ai_addrlen) < 0 &&\n+\t\t\t\t\terrno != EINPROGRESS) {\n+\t\t\tstrbuf_addf(&error_message, \"%s[%d: %s]: errno=%s\\n\",\n+\t\t\t\thost, cnt, ai_name(ai), strerror(errno));\n+\t\t\tclose(fd);\n+\t\t\tcontinue;\n+\t\t}\n+\n \t\tif (flags & CONNECT_VERBOSE)\n-\t\t\tfprintf(stderr, \"%s \", ai_name(ai));\n-\t\tbreak;\n+\t\t\tfprintf(stderr, \"%s (started)\\n\", ai_name(ai));\n+\n+\t\tnfds = n + 1;\n+\t\tcur = n;\n+\t\tALLOC_GROW(fds, nfds, cur);\n+\t\tcur = n;\n+\t\tALLOC_GROW(inprogress, nfds, cur);\n+\t\tinprogress[n] = ai;\n+\t\tfds[n].fd = fd;\n+\t\tfds[n].events = POLLIN|POLLOUT;\n+\t\tfds[n].revents = 0;\n+\t\tn = nfds;\n+\t}\n+\n+\t/*\n+\t * nfds is tiny, no need to limit loop based on poll() retval,\n+\t * just do not let poll sleep forever if nfds is zero\n+\t */\n+\tif (nfds > 0)\n+\t\tpoll(fds, nfds, -1);\n+\n+\tfor (n = 0; n < nfds && sockfd < 0; n++) {\n+\t\tif (fds[n].revents & (POLLERR|POLLHUP))\n+\t\t\tcontinue;\n+\t\tif (fds[n].revents & POLLOUT) {\n+\t\t\tint err;\n+\t\t\tsocklen_t len = (socklen_t)sizeof(err);\n+\t\t\tint rc = getsockopt(fds[n].fd, SOL_SOCKET, SO_ERROR,\n+\t\t\t\t\t\t&err, &len);\n+\t\t\tif (rc != 0)\n+\t\t\t\tdie_errno(\"getsockopt errno=%s\\n\",\n+\t\t\t\t\tstrerror(errno));\n+\t\t\tif (err == 0) { /* success! */\n+\t\t\t\tsockfd = fds[n].fd;\n+\t\t\t\tai = inprogress[n];\n+\t\t\t}\n+\t\t}\n+\t}\n+\n+\t/* cleanup */\n+\tfor (n = 0; n < nfds; n++) {\n+\t\tif (fds[n].fd != sockfd)\n+\t\t\tclose(fds[n].fd);\n+\t}\n+\tfree(inprogress);\n+\tfree(fds);\n+\n+\tif (sockfd >= 0) {\n+\t\tenable_keepalive(sockfd);\n+\t\tset_blocking(sockfd); /* the rest of git expects blocking */\n+\t\tif (flags & CONNECT_VERBOSE)\n+\t\t\tfprintf(stderr, \"%s done.\\n\", ai_name(ai));\n \t}\n \n \tfreeaddrinfo(ai0);\n@@ -392,11 +487,6 @@ static int git_tcp_connect_sock(char *host, int flags)\n \tif (sockfd < 0)\n \t\tdie(\"unable to connect to %s:\\n%s\", host, error_message.buf);\n \n-\tenable_keepalive(sockfd);\n-\n-\tif (flags & CONNECT_VERBOSE)\n-\t\tfprintf(stderr, \"done.\\n\");\n-\n \tstrbuf_release(&error_message);\n \n \treturn sockfd;\n-- \nEW\n"},{"id":"277021","messageId":"xmqqio2dl18i.fsf@gitster.mtv.corp.google.com","threadId":"41274","inReplyTo":"20160128115720.GA1827@dcvr.yhbt.net","subject":"Re: [PATCH] attempt connects in parallel for IPv6-capable builds","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-01-28T23:42:53Z","receivedAt":"2016-01-28T23:42:53Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Eric Wong <normalperson@yhbt.net> writes:\n\n> getaddrinfo() may return multiple addresses, not all of which\n> are equally performant.  In some cases, a user behind a non-IPv6\n> capable network may get an IPv6 address which stalls connect().\n> Instead of waiting synchronously for a connect() to timeout, use\n> non-blocking connect() in parallel and take the first successful\n> connection.\n>\n> This may increase network traffic and server load slightly, but\n> makes the worst-case user experience more bearable when one\n> lacks permissions to edit /etc/gai.conf to favor IPv4 addresses.\n\nUmm.  I am not sure what to think about this change--I generally do\nnot like a selfish \"I'll try to use whatever resource given to me\nto make my process go faster, screw the rest of the world\" approach\nand I cannot decide if this falls into that category.\n\nI'll wait for opinions from others.\n\nThanks.\n\n> Signed-off-by: Eric Wong <normalperson@yhbt.net>\n> ---\n>  connect.c | 118 ++++++++++++++++++++++++++++++++++++++++++++++++++++++--------\n>  1 file changed, 104 insertions(+), 14 deletions(-)\n>\n> diff --git a/connect.c b/connect.c\n> index fd7ffe1..74d2bb5 100644\n> --- a/connect.c\n> +++ b/connect.c\n> @@ -14,6 +14,42 @@\n>  static char *server_capabilities;\n>  static const char *parse_feature_value(const char *, const char *, int *);\n>  \n> +#ifdef SOCK_NONBLOCK /* Linux-only flag */\n> +#  define GIT_SOCK_NONBLOCK SOCK_NONBLOCK\n> +#else\n> +#  define GIT_SOCK_NONBLOCK 0\n> +#endif\n> +\n> +static int socket_nb(int domain, int type, int protocol)\n> +{\n> +\tstatic int flags = GIT_SOCK_NONBLOCK;\n> +\tint fd = socket(domain, type | flags, protocol);\n> +\n> +\t/* new headers, old kernel? */\n> +\tif (fd < 0 && errno == EINVAL && flags != 0) {\n> +\t\tflags = 0;\n> +\t\tfd = socket(domain, type, protocol);\n> +\t}\n> +\n> +\t/* couldn't use SOCK_NONBLOCK, set non-blocking the old way */\n> +\tif (flags == 0 && fd >= 0) {\n> +\t\tint fl = fcntl(fd, F_GETFL);\n> +\n> +\t\tif (fcntl(fd, F_SETFL, fl | O_NONBLOCK) < 0)\n> +\t\t\tdie_errno(\"failed to set nonblocking flag\\n\");\n> +\t}\n> +\n> +\treturn fd;\n> +}\n> +\n> +static void set_blocking(int fd)\n> +{\n> +\tint fl = fcntl(fd, F_GETFL);\n> +\n> +\tif (fcntl(fd, F_SETFL, fl & ~O_NONBLOCK) < 0)\n> +\t\tdie_errno(\"failed to clear nonblocking flag\\n\");\n> +}\n> +\n>  static int check_ref(const char *name, unsigned int flags)\n>  {\n>  \tif (!flags)\n> @@ -351,6 +387,9 @@ static int git_tcp_connect_sock(char *host, int flags)\n>  \tstruct addrinfo hints, *ai0, *ai;\n>  \tint gai;\n>  \tint cnt = 0;\n> +\tnfds_t n = 0, nfds = 0;\n> +\tstruct pollfd *fds = NULL;\n> +\tstruct addrinfo **inprogress = NULL;\n>  \n>  \tget_host_and_port(&host, &port);\n>  \tif (!*port)\n> @@ -371,20 +410,76 @@ static int git_tcp_connect_sock(char *host, int flags)\n>  \t\tfprintf(stderr, \"done.\\nConnecting to %s (port %s) ... \", host, port);\n>  \n>  \tfor (ai0 = ai; ai; ai = ai->ai_next, cnt++) {\n> -\t\tsockfd = socket(ai->ai_family,\n> -\t\t\t\tai->ai_socktype, ai->ai_protocol);\n> -\t\tif ((sockfd < 0) ||\n> -\t\t    (connect(sockfd, ai->ai_addr, ai->ai_addrlen) < 0)) {\n> +\t\tsize_t cur;\n> +\t\tint fd = socket_nb(ai->ai_family, ai->ai_socktype,\n> +\t\t\t\t\tai->ai_protocol);\n> +\t\tif (fd < 0) {\n>  \t\t\tstrbuf_addf(&error_message, \"%s[%d: %s]: errno=%s\\n\",\n>  \t\t\t\t    host, cnt, ai_name(ai), strerror(errno));\n> -\t\t\tif (0 <= sockfd)\n> -\t\t\t\tclose(sockfd);\n> -\t\t\tsockfd = -1;\n>  \t\t\tcontinue;\n>  \t\t}\n> +\n> +\t\tif (connect(fd, ai->ai_addr, ai->ai_addrlen) < 0 &&\n> +\t\t\t\t\terrno != EINPROGRESS) {\n> +\t\t\tstrbuf_addf(&error_message, \"%s[%d: %s]: errno=%s\\n\",\n> +\t\t\t\thost, cnt, ai_name(ai), strerror(errno));\n> +\t\t\tclose(fd);\n> +\t\t\tcontinue;\n> +\t\t}\n> +\n>  \t\tif (flags & CONNECT_VERBOSE)\n> -\t\t\tfprintf(stderr, \"%s \", ai_name(ai));\n> -\t\tbreak;\n> +\t\t\tfprintf(stderr, \"%s (started)\\n\", ai_name(ai));\n> +\n> +\t\tnfds = n + 1;\n> +\t\tcur = n;\n> +\t\tALLOC_GROW(fds, nfds, cur);\n> +\t\tcur = n;\n> +\t\tALLOC_GROW(inprogress, nfds, cur);\n> +\t\tinprogress[n] = ai;\n> +\t\tfds[n].fd = fd;\n> +\t\tfds[n].events = POLLIN|POLLOUT;\n> +\t\tfds[n].revents = 0;\n> +\t\tn = nfds;\n> +\t}\n> +\n> +\t/*\n> +\t * nfds is tiny, no need to limit loop based on poll() retval,\n> +\t * just do not let poll sleep forever if nfds is zero\n> +\t */\n> +\tif (nfds > 0)\n> +\t\tpoll(fds, nfds, -1);\n> +\n> +\tfor (n = 0; n < nfds && sockfd < 0; n++) {\n> +\t\tif (fds[n].revents & (POLLERR|POLLHUP))\n> +\t\t\tcontinue;\n> +\t\tif (fds[n].revents & POLLOUT) {\n> +\t\t\tint err;\n> +\t\t\tsocklen_t len = (socklen_t)sizeof(err);\n> +\t\t\tint rc = getsockopt(fds[n].fd, SOL_SOCKET, SO_ERROR,\n> +\t\t\t\t\t\t&err, &len);\n> +\t\t\tif (rc != 0)\n> +\t\t\t\tdie_errno(\"getsockopt errno=%s\\n\",\n> +\t\t\t\t\tstrerror(errno));\n> +\t\t\tif (err == 0) { /* success! */\n> +\t\t\t\tsockfd = fds[n].fd;\n> +\t\t\t\tai = inprogress[n];\n> +\t\t\t}\n> +\t\t}\n> +\t}\n> +\n> +\t/* cleanup */\n> +\tfor (n = 0; n < nfds; n++) {\n> +\t\tif (fds[n].fd != sockfd)\n> +\t\t\tclose(fds[n].fd);\n> +\t}\n> +\tfree(inprogress);\n> +\tfree(fds);\n> +\n> +\tif (sockfd >= 0) {\n> +\t\tenable_keepalive(sockfd);\n> +\t\tset_blocking(sockfd); /* the rest of git expects blocking */\n> +\t\tif (flags & CONNECT_VERBOSE)\n> +\t\t\tfprintf(stderr, \"%s done.\\n\", ai_name(ai));\n>  \t}\n>  \n>  \tfreeaddrinfo(ai0);\n> @@ -392,11 +487,6 @@ static int git_tcp_connect_sock(char *host, int flags)\n>  \tif (sockfd < 0)\n>  \t\tdie(\"unable to connect to %s:\\n%s\", host, error_message.buf);\n>  \n> -\tenable_keepalive(sockfd);\n> -\n> -\tif (flags & CONNECT_VERBOSE)\n> -\t\tfprintf(stderr, \"done.\\n\");\n> -\n>  \tstrbuf_release(&error_message);\n>  \n>  \treturn sockfd;\n"},{"id":"277027","messageId":"20160129014106.GA8940@dcvr.yhbt.net","threadId":"41274","inReplyTo":"xmqqio2dl18i.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH] attempt connects in parallel for IPv6-capable builds","fromName":"Eric Wong","fromEmail":"normalperson@yhbt.net","sentAt":"2016-01-29T01:41:06Z","receivedAt":"2016-01-29T01:41:06Z","isPatch":true,"sender":{"key":"e@80x24.org","avatar":null},"body":"Junio C Hamano <gitster@pobox.com> wrote:\n> Eric Wong <normalperson@yhbt.net> writes:\n> \n> > getaddrinfo() may return multiple addresses, not all of which\n> > are equally performant.  In some cases, a user behind a non-IPv6\n> > capable network may get an IPv6 address which stalls connect().\n> > Instead of waiting synchronously for a connect() to timeout, use\n> > non-blocking connect() in parallel and take the first successful\n> > connection.\n> >\n> > This may increase network traffic and server load slightly, but\n> > makes the worst-case user experience more bearable when one\n> > lacks permissions to edit /etc/gai.conf to favor IPv4 addresses.\n> \n> Umm.  I am not sure what to think about this change--I generally do\n> not like a selfish \"I'll try to use whatever resource given to me\n> to make my process go faster, screw the rest of the world\" approach\n> and I cannot decide if this falls into that category.\n> \n> I'll wait for opinions from others.\n\nNo problem, I can also make it cheaper for servers to handle\naborted connections in git-daemon:\n\nstandalone:\n\n  1) use recv with MSG_PEEK or FIONREAD to determine if there's\n     readable data in the socket before forking (and avoid\n     forking for zero-bytes-written connections)\n\n  2) use TCP_DEFER_ACCEPT in Linux and dataready filter in FreeBSD\n     for standalone git-daemon to delay accept()\n\ninetd:\n\n  3) suppress die(\"The remote end hung up unexpectedly\")\n     if no bytes are read at all\n\nAt some point in the future, I would love to have git-daemon implement\nsomething like IDLE in IMAP (to avoid having clients poll for updates).\nPerhaps the standalone changes above would make sense there, too.\n"},{"id":"277029","messageId":"xmqqtwlxjdbx.fsf@gitster.mtv.corp.google.com","threadId":"41274","inReplyTo":"20160128115720.GA1827@dcvr.yhbt.net","subject":"Re: [PATCH] attempt connects in parallel for IPv6-capable builds","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-01-29T03:04:34Z","receivedAt":"2016-01-29T03:04:34Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Eric Wong <normalperson@yhbt.net> writes:\n\n> getaddrinfo() may return multiple addresses, not all of which\n> are equally performant.  In some cases, a user behind a non-IPv6\n> capable network may get an IPv6 address which stalls connect().\n\nI'd assume that you are not solving a hypothetical problem, but you\nmay (at least sometimes) have to reach outside world from such a\nnetwork environment.  I further assume that git_tcp_connect() is not\nthe only activity you do from such a network, and other network\nactivities are similarly affected.\n\nHow do you work around the same issue for connections that do not go\nthrough git_tcp_connect()?  The same issue would affect Git traffic\ngoing over git-remote-curl, and also your usual Web browser traffic,\nno?\n\nWhat I am getting at is if it is saner to solve the issue like how\ncurl(1) solves it with its -4/-6 command line options, e.g. by\nadding a pair of configuration variables \"net.ipv[46] = true/false\".\n"},{"id":"277037","messageId":"56AB01C2.3090803@web.de","threadId":"41274","inReplyTo":"xmqqtwlxjdbx.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH] attempt connects in parallel for IPv6-capable builds","fromName":"Torsten Bögershausen","fromEmail":"tboegi@web.de","sentAt":"2016-01-29T06:08:02Z","receivedAt":"2016-01-29T06:08:02Z","isPatch":true,"sender":{"key":"tboegi@web.de","avatar":"https://avatars.githubusercontent.com/u/7138363?v=4"},"body":"On 2016-01-29 04.04, Junio C Hamano wrote:\n> Eric Wong <normalperson@yhbt.net> writes:\n> \n>> getaddrinfo() may return multiple addresses, not all of which\n>> are equally performant.  In some cases, a user behind a non-IPv6\n>> capable network may get an IPv6 address which stalls connect().\n> \n> I'd assume that you are not solving a hypothetical problem, but you\n> may (at least sometimes) have to reach outside world from such a\n> network environment.  I further assume that git_tcp_connect() is not\n> the only activity you do from such a network, and other network\n> activities are similarly affected.\n> \n> How do you work around the same issue for connections that do not go\n> through git_tcp_connect()?  The same issue would affect Git traffic\n> going over git-remote-curl, and also your usual Web browser traffic,\n> no?\n> \n> What I am getting at is if it is saner to solve the issue like how\n> curl(1) solves it with its -4/-6 command line options, e.g. by\n> adding a pair of configuration variables \"net.ipv[46] = true/false\".\n\n(Please don't do parallel connects as a default)\n\nI like the -4 / -6\n\nOut of my head these kind of setups are not desired.\n(getaddrinfo returns an address and the the following connect fails)\nBut it may be hard to fully avoid them in practice,\nand some RFC have been written,  this may be a starting point:\n\nhttps://www.rfc-editor.org/rfc/rfc5014.txt\n"},{"id":"277043","messageId":"56AB1057.6080700@kdbg.org","threadId":"41274","inReplyTo":"20160129014106.GA8940@dcvr.yhbt.net","subject":"Re: [PATCH] attempt connects in parallel for IPv6-capable builds","fromName":"Johannes Sixt","fromEmail":"j6t@kdbg.org","sentAt":"2016-01-29T07:10:15Z","receivedAt":"2016-01-29T07:10:15Z","isPatch":true,"sender":{"key":"j6t@kdbg.org","avatar":"https://avatars.githubusercontent.com/u/14810926?v=4"},"body":"Am 29.01.2016 um 02:41 schrieb Eric Wong:\n> Junio C Hamano <gitster@pobox.com> wrote:\n>> Eric Wong <normalperson@yhbt.net> writes:\n>>\n>>> getaddrinfo() may return multiple addresses, not all of which\n>>> are equally performant.  In some cases, a user behind a non-IPv6\n>>> capable network may get an IPv6 address which stalls connect().\n>>> Instead of waiting synchronously for a connect() to timeout, use\n>>> non-blocking connect() in parallel and take the first successful\n>>> connection.\n>>>\n>>> This may increase network traffic and server load slightly, but\n>>> makes the worst-case user experience more bearable when one\n>>> lacks permissions to edit /etc/gai.conf to favor IPv4 addresses.\n>>\n>> Umm.  I am not sure what to think about this change--I generally do\n>> not like a selfish \"I'll try to use whatever resource given to me\n>> to make my process go faster, screw the rest of the world\" approach\n>> and I cannot decide if this falls into that category.\n>>\n>> I'll wait for opinions from others.\n>\n> No problem, I can also make it cheaper for servers to handle\n> aborted connections in git-daemon:\n>\n> standalone:\n>\n>    1) use recv with MSG_PEEK or FIONREAD to determine if there's\n>       readable data in the socket before forking (and avoid\n>       forking for zero-bytes-written connections)\n>\n>    2) use TCP_DEFER_ACCEPT in Linux and dataready filter in FreeBSD\n>       for standalone git-daemon to delay accept()\n>\n> inetd:\n>\n>    3) suppress die(\"The remote end hung up unexpectedly\")\n>       if no bytes are read at all\n>\n> At some point in the future, I would love to have git-daemon implement\n> something like IDLE in IMAP (to avoid having clients poll for updates).\n> Perhaps the standalone changes above would make sense there, too.\n\nBefore you submit a patch in that direction (or resubmit the patch under \ndiscussion here), could you please find someone to test your patch on \nWindows first? A lot of the infrastructure mentioned may not be \navailable there or may not work as expected. (I admit that I'm just \nhand-waving, I haven't tested your patch.)\n\nThanks,\n-- Hannes\n"},{"id":"277080","messageId":"20160130130933.GA26365@dcvr.yhbt.net","threadId":"41274","inReplyTo":"xmqqtwlxjdbx.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH] attempt connects in parallel for IPv6-capable builds","fromName":"Eric Wong","fromEmail":"normalperson@yhbt.net","sentAt":"2016-01-30T13:09:33Z","receivedAt":"2016-01-30T13:09:33Z","isPatch":true,"sender":{"key":"e@80x24.org","avatar":null},"body":"Junio C Hamano <gitster@pobox.com> wrote:\n> Eric Wong <normalperson@yhbt.net> writes:\n> \n> > getaddrinfo() may return multiple addresses, not all of which\n> > are equally performant.  In some cases, a user behind a non-IPv6\n> > capable network may get an IPv6 address which stalls connect().\n> \n> I'd assume that you are not solving a hypothetical problem, but you\n> may (at least sometimes) have to reach outside world from such a\n> network environment.  I further assume that git_tcp_connect() is not\n> the only activity you do from such a network, and other network\n> activities are similarly affected.\n\nRight.  I only recently started building kernels for this\nnetbook with IPv6 support.   Most of my traffic from this host\nis git/ssh-tunneled and the host I ssh to does not have an AAAA\nrecord.\n\n> How do you work around the same issue for connections that do not go\n> through git_tcp_connect()?  The same issue would affect Git traffic\n> going over git-remote-curl, and also your usual Web browser traffic,\n> no?\n\nSo this is a new issue for me, and I don't use a browser much.\n\nI noticed the curl(1) tool didn't have the problem with http.\n\nLooking at curl, it actually has limited parallelization\n(see HAPPY_EYEBALLS_TIMEOUT timeout handling in lib/connect.c\nof git://github.com/bagder/curl.git).  It starts a parallel\nconnection on a different family if the first address takes\nlonger than 200ms.  This does not seem true of our libcurl\nusage, though...\n\nI tried implementing what curl does in connect.c, but it was\ncomplicated for me (or more likely, I'm easily confused :x).\nIt was much easier for me to start everything up in parallel\nas I did in my original patch.\n\n> What I am getting at is if it is saner to solve the issue like how\n> curl(1) solves it with its -4/-6 command line options, e.g. by\n> adding a pair of configuration variables \"net.ipv[46] = true/false\".\n\nYes, a -4/-6 patch is on the way for command-line switches.  I'm\nnot sure if a config file variable is a good idea since it could\nbe overlooked and cause non-obvious conflicts with /etc/gai.conf\nsettings.\n\nOf course, I would rather have something transparent to the user\nlike curl(1)-style happy eyeballs parallelization if it could be\nimplemented in a straightforward way.\n"}]}