{"thread":{"id":"35277","subject":"[PATCH V4] git clone: is an URL local or ssh","startedAt":"2013-11-04T21:20:49Z","lastAt":"2013-11-06T15:31:14Z","messageCount":7,"participants":["Torsten Bögershausen","Johannes Sixt"],"isPatch":true,"patchVersion":4,"patchTotal":null},"messages":[{"id":"230093","messageId":"201311042220.50178.tboegi@web.de","threadId":"35277","inReplyTo":null,"subject":"[PATCH V4] git clone: is an URL local or ssh","fromName":"Torsten Bögershausen","fromEmail":"tboegi@web.de","sentAt":"2013-11-04T21:20:49Z","receivedAt":"2013-11-04T21:20:49Z","isPatch":true,"sender":{"key":"tboegi@web.de","avatar":"https://avatars.githubusercontent.com/u/7138363?v=4"},"body":"Commit 8d3d28f5 added test cases for URLs which should be ssh.\n\nAdd more tests testing all the combinations:\n -IPv4 or IPv6\n -path starting with \"/\" or with \"/~\"\n -with and without the ssh:// scheme\n\nAdd tests for ssh:// with port number.\n\nWhen a git repository \"foo:bar\" exist, git clone will call\nabsolute_path() and git_connect() will be called with\nsomething like \"/home/user/projects/foo:bar\"\n\nTighten the test and use \"foo:bar\" instead of \"./foo:bar\",\nrefactor clear_ssh() and expect_ssh() into test_clone_url().\n\n\"git clone foo/bar:baz\" should not be ssh:\n  Make the rule\n  \"if there is a slash before the first colon, it is not ssh\"\n  more strict by using the same is_local() in both connect.c\n  and transport.c. Add a test case.\n\nBug fixes for corner cases:\n- Handle clone of [::1]:/~repo correctly (2e7766655):\n  Git should call \"ssh ::1 ~repo\", not ssh ::1 /~repo\n  This was caused by looking at (host != url), which never\n  worked for host names with literal IPv6 adresses, embedded by []\n  Support for the [] URLs was introduced in 356bece0a.\n\n- Do not tamper local URLs starting with '[' which have a ']'\n\nBug fix for msygit in t5601 : use $PWD insted of $(pwd)\n\nHelped-by: Jeff King <peff@peff.net>\nSigned-off-by: Torsten Bögershausen <tboegi@web.de>\n---\n Changes since V3:\n - Integrated Peffs suggestions in t5601\n   (Remove clear_ssh)\n - Decide early if it is ssl or local in connect.c\n - Use \"git fetch\" instead of \"git clone\" in t5601:\n   clone use absolute_path() (adding a / before :), fetch does not\n - Add a test for dos_drive \"C:temp\" for msys\n - Make tests work for msys by using $PWD instead of $(pwd) (file://$PWD)\n\n connect.c        |  50 ++++++++++++++++--------\n connect.h        |   1 +\n t/t5601-clone.sh | 117 ++++++++++++++++++++++++++++++++++++++++++++-----------\n transport.c      |   8 ----\n 4 files changed, 130 insertions(+), 46 deletions(-)\n\ndiff --git a/connect.c b/connect.c\nindex 06e88b0..022d122 100644\n--- a/connect.c\n+++ b/connect.c\n@@ -547,6 +547,25 @@ static char *get_port(char *host)\n \n static struct child_process no_fork;\n \n+int is_local(const char *url)\n+{\n+\tconst char *colon = strchr(url, ':');\n+\tconst char *slash = strchr(url, '/');\n+\tif (has_dos_drive_prefix(url))\n+\t\treturn 1;\n+\tif (!colon)\n+\t\treturn 1;\n+\tif (slash && slash < colon)\n+\t\treturn 1;\n+\tif (url[0] == '[') {\n+\t\tconst char *end = strchr(url + 1, ']');\n+\t\tif (!end)\n+\t\t\treturn 1;\n+\t\treturn is_local(end);\n+\t}\n+\treturn 0;\n+}\n+\n /*\n  * This returns a dummy child_process if the transport protocol does not\n  * need fork(2), or a struct child_process object if it does.  Once done,\n@@ -564,7 +583,7 @@ struct child_process *git_connect(int fd[2], const char *url_orig,\n \tchar *url;\n \tchar *host, *path;\n \tchar *end;\n-\tint c;\n+\tint separator;\n \tstruct child_process *conn = &no_fork;\n \tenum protocol protocol = PROTO_LOCAL;\n \tint free_path = 0;\n@@ -587,17 +606,19 @@ struct child_process *git_connect(int fd[2], const char *url_orig,\n \t\t*host = '\\0';\n \t\tprotocol = get_protocol(url);\n \t\thost += 3;\n-\t\tc = '/';\n+\t\tseparator = '/';\n \t} else {\n \t\thost = url;\n-\t\tc = ':';\n+\t\tprotocol = is_local(url) ? PROTO_LOCAL : PROTO_SSH;\n+\t\tseparator = ':';\n \t}\n \n \t/*\n \t * Don't do destructive transforms with git:// as that\n \t * protocol code does '[]' unwrapping of its own.\n+\t * Don't change local URLs\n \t */\n-\tif (host[0] == '[') {\n+\tif (protocol != PROTO_LOCAL && host[0] == '[') {\n \t\tend = strchr(host + 1, ']');\n \t\tif (end) {\n \t\t\tif (protocol != PROTO_GIT) {\n@@ -610,17 +631,14 @@ struct child_process *git_connect(int fd[2], const char *url_orig,\n \t} else\n \t\tend = host;\n \n-\tpath = strchr(end, c);\n-\tif (path && !has_dos_drive_prefix(end)) {\n-\t\tif (c == ':') {\n-\t\t\tif (host != url || path < strchrnul(host, '/')) {\n-\t\t\t\tprotocol = PROTO_SSH;\n-\t\t\t\t*path++ = '\\0';\n-\t\t\t} else /* '/' in the host part, assume local path */\n-\t\t\t\tpath = end;\n+\tpath = strchr(end, separator);\n+\tif (separator == ':') {\n+\t\tif (path && protocol == PROTO_SSH) {\n+\t\t\t*path++ = '\\0';\n+\t\t} else {/* assume local path */\n+\t\t\tpath = end;\n \t\t}\n-\t} else\n-\t\tpath = end;\n+\t}\n \n \tif (!path || !*path)\n \t\tdie(\"No path specified. See 'man git-pull' for valid url syntax\");\n@@ -629,7 +647,7 @@ struct child_process *git_connect(int fd[2], const char *url_orig,\n \t * null-terminate hostname and point path to ~ for URL's like this:\n \t *    ssh://host.xz/~user/repo\n \t */\n-\tif (protocol != PROTO_LOCAL && host != url) {\n+\tif (protocol != PROTO_LOCAL && separator == '/') {\n \t\tchar *ptr = path;\n \t\tif (path[1] == '~')\n \t\t\tpath++;\n@@ -644,7 +662,7 @@ struct child_process *git_connect(int fd[2], const char *url_orig,\n \t/*\n \t * Add support for ssh port: ssh://host.xy:<port>/...\n \t */\n-\tif (protocol == PROTO_SSH && host != url)\n+\tif (protocol == PROTO_SSH && separator == '/')\n \t\tport = get_port(end);\n \n \tif (protocol == PROTO_GIT) {\ndiff --git a/connect.h b/connect.h\nindex 64fb7db..fb2de9b 100644\n--- a/connect.h\n+++ b/connect.h\n@@ -5,6 +5,7 @@\n extern struct child_process *git_connect(int fd[2], const char *url, const char *prog, int flags);\n extern int finish_connect(struct child_process *conn);\n extern int git_connection_is_socket(struct child_process *conn);\n+extern int is_local(const char *url);\n extern int server_supports(const char *feature);\n extern int parse_feature_request(const char *features, const char *feature);\n extern const char *server_feature_value(const char *feature, int *len_ret);\ndiff --git a/t/t5601-clone.sh b/t/t5601-clone.sh\nindex 1d1c875..5f2b5a0 100755\n--- a/t/t5601-clone.sh\n+++ b/t/t5601-clone.sh\n@@ -30,20 +30,20 @@ test_expect_success 'clone with excess parameters (1)' '\n test_expect_success 'clone with excess parameters (2)' '\n \n \trm -fr dst &&\n-\ttest_must_fail git clone -n \"file://$(pwd)/src\" dst junk\n+\ttest_must_fail git clone -n \"file://$PWD/src\" dst junk\n \n '\n \n test_expect_success C_LOCALE_OUTPUT 'output from clone' '\n \trm -fr dst &&\n-\tgit clone -n \"file://$(pwd)/src\" dst >output 2>&1 &&\n+\tgit clone -n \"file://$PWD/src\" dst >output 2>&1 &&\n \ttest $(grep Clon output | wc -l) = 1\n '\n \n test_expect_success 'clone does not keep pack' '\n \n \trm -fr dst &&\n-\tgit clone -n \"file://$(pwd)/src\" dst &&\n+\tgit clone -n \"file://$PWD/src\" dst &&\n \t! test -f dst/file &&\n \t! (echo dst/.git/objects/pack/pack-* | grep \"\\.keep\")\n \n@@ -172,12 +172,12 @@ test_expect_success 'clone a void' '\n \t(\n \t\tcd src-0 && git init\n \t) &&\n-\tgit clone \"file://$(pwd)/src-0\" target-6 2>err-6 &&\n+\tgit clone \"file://$PWD/src-0\" target-6 2>err-6 &&\n \t! grep \"fatal:\" err-6 &&\n \t(\n \t\tcd src-0 && test_commit A\n \t) &&\n-\tgit clone \"file://$(pwd)/src-0\" target-7 2>err-7 &&\n+\tgit clone \"file://$PWD/src-0\" target-7 2>err-7 &&\n \t! grep \"fatal:\" err-7 &&\n \t# There is no reason to insist they are bit-for-bit\n \t# identical, but this test should suffice for now.\n@@ -291,14 +291,14 @@ test_expect_success 'setup ssh wrapper' '\n \n \tGIT_SSH=\"$TRASH_DIRECTORY/ssh-wrapper\" &&\n \texport GIT_SSH &&\n-\texport TRASH_DIRECTORY\n+\texport TRASH_DIRECTORY &&\n+\t>\"$TRASH_DIRECTORY\"/ssh-output\n '\n \n-clear_ssh () {\n-\t>\"$TRASH_DIRECTORY/ssh-output\"\n-}\n-\n expect_ssh () {\n+\ttest_when_finished '\n+\t  (cd \"$TRASH_DIRECTORY\" && rm -f ssh-expect && >ssh-output)\n+\t' &&\n \t{\n \t\tcase \"$1\" in\n \t\tnone)\n@@ -310,23 +310,96 @@ expect_ssh () {\n \t(cd \"$TRASH_DIRECTORY\" && test_cmp ssh-expect ssh-output)\n }\n \n-test_expect_success 'cloning myhost:src uses ssh' '\n-\tclear_ssh &&\n-\tgit clone myhost:src ssh-clone &&\n-\texpect_ssh myhost src\n-'\n-\n test_expect_success NOT_MINGW,NOT_CYGWIN 'clone local path foo:bar' '\n-\tclear_ssh &&\n \tcp -R src \"foo:bar\" &&\n-\tgit clone \"./foo:bar\" foobar &&\n+\tgit clone \"foo:bar\" foobar &&\n+\texpect_ssh none\n+'\n+\n+test_expect_success 'clone local path foo2' '\n+\tcp -R src \"foo2\" &&\n+\tgit clone \"foo2\" foobar2 &&\n \texpect_ssh none\n '\n \n-test_expect_success 'bracketed hostnames are still ssh' '\n-\tclear_ssh &&\n-\tgit clone \"[myhost:123]:src\" ssh-bracket-clone &&\n-\texpect_ssh myhost:123 src\n+counter=0\n+# $1 url\n+# $2 none|host\n+# $3 path\n+test_fetch_url () {\n+\tcounter=$(($counter + 1))\n+\ttest_might_fail git fetch \"$1\" tmp$counter &&\n+\texpect_ssh \"$2\" \"$3\"\n+}\n+\n+test_expect_success NOT_MINGW 'fetch c:temp is ssl' '\n+\ttest_fetch_url c:temp c temp\n+'\n+\n+test_expect_success MINGW 'fetch c:temp is dos drive' '\n+\ttest_fetch_url c:temp none\n+'\n+\n+#ip v4\n+for repo in rep rep/home/project /~proj 123\n+do\n+\ttest_expect_success \"fetchinghost:$repo\" '\n+\t\ttest_fetch_url host:$repo host $repo\n+\t'\n+done\n+\n+#ipv6\n+for repo in rep rep/home/project /~proj 123\n+do\n+\ttest_expect_success \"fetching[::1]:$repo\" '\n+\t\ttest_fetch_url [::1]:$repo ::1 $repo\n+\t'\n+done\n+\n+# Corner cases\n+for url in [foo]bar/baz:qux [foo/bar]:baz foo/bar:baz\n+do\n+\ttest_expect_success \"fetching$url is not ssh\" '\n+\t\t\ttest_fetch_url $url none\n+\t'\n+done\n+\n+#with ssh:// scheme\n+test_expect_success 'ssh://host.xz/home/user/repo' '\n+\ttest_fetch_url \"ssh://host.xz/home/user/repo\" host.xz \"/home/user/repo\"\n+'\n+\n+# from home directory\n+test_expect_success 'ssh://host.xz/~repo' '\n+\ttest_fetch_url \"ssh://host.xz/~repo\" host.xz \"~repo\"\n+'\n+# with port number\n+test_expect_success 'ssh://host.xz:22/home/user/repo' '\n+\ttest_fetch_url \"ssh://host.xz:22/home/user/repo\" \"-p 22 host.xz\" \"/home/user/repo\"\n+'\n+\n+# from home directory with port number\n+test_expect_success 'ssh://host.xz:22/~repo' '\n+\ttest_fetch_url \"ssh://host.xz:22/~repo\" \"-p 22 host.xz\" \"~repo\"\n+'\n+\n+#IPv6\n+test_expect_success 'ssh://[::1]/home/user/repo' '\n+\ttest_fetch_url \"ssh://[::1]/home/user/repo\" \"::1\" \"/home/user/repo\"\n+'\n+\n+#IPv6 from home directory\n+test_expect_success 'ssh://[::1]/~repo' '\n+\ttest_fetch_url \"ssh://[::1]/~repo\" \"::1\" \"~repo\"\n+'\n+\n+#IPv6 with port number\n+test_expect_success 'ssh://[::1]:22/home/user/repo' '\n+\ttest_fetch_url \"ssh://[::1]:22/home/user/repo\" \"-p 22 ::1\" \"/home/user/repo\"\n+'\n+#IPv6 from home directory with port number\n+test_expect_success 'ssh://[::1]:22/~repo' '\n+\ttest_fetch_url \"ssh://[::1]:22/~repo\" \"-p 22 ::1\" \"~repo\"\n '\n \n test_expect_success 'clone from a repository with two identical branches' '\ndiff --git a/transport.c b/transport.c\nindex 7202b77..a09ba95 100644\n--- a/transport.c\n+++ b/transport.c\n@@ -885,14 +885,6 @@ void transport_take_over(struct transport *transport,\n \ttransport->cannot_reuse = 1;\n }\n \n-static int is_local(const char *url)\n-{\n-\tconst char *colon = strchr(url, ':');\n-\tconst char *slash = strchr(url, '/');\n-\treturn !colon || (slash && slash < colon) ||\n-\t\thas_dos_drive_prefix(url);\n-}\n-\n static int is_file(const char *url)\n {\n \tstruct stat buf;\n-- \n1.8.4.457.g424cb08\n"},{"id":"230111","messageId":"52789AE5.2010702@viscovery.net","threadId":"35277","inReplyTo":"201311042220.50178.tboegi@web.de","subject":"Re: [PATCH V4] git clone: is an URL local or ssh","fromName":"Johannes Sixt","fromEmail":"j.sixt@viscovery.net","sentAt":"2013-11-05T07:14:45Z","receivedAt":"2013-11-05T07:14:45Z","isPatch":true,"sender":{"key":"j6t@kdbg.org","avatar":"https://avatars.githubusercontent.com/u/14810926?v=4"},"body":"Am 11/4/2013 22:20, schrieb Torsten Bögershausen:\n> Bug fix for msygit in t5601 : use $PWD insted of $(pwd)\n\nNot really. $PWD is /c/foo/bar style, but $(pwd) is c:/foo/bar, which is\nequally good.\n\n>  test_expect_success 'clone with excess parameters (2)' '\n>  \n>  \trm -fr dst &&\n> -\ttest_must_fail git clone -n \"file://$(pwd)/src\" dst junk\n> +\ttest_must_fail git clone -n \"file://$PWD/src\" dst junk\n\nThat the code change necessitates this change in the test suite is a sign\nthat there is something wrong with the new code.\n\nAnd indeed, the original works:\n\nC:\\Temp\\gittest>git clone -n file://c:/Temp/gittest/foo good\nCloning into 'good'...\nremote: Counting objects: 2, done.\nremote: Total 2 (delta 0), reused 0 (delta 0)\nReceiving objects: 100% (2/2), done.\nChecking connectivity... done.\n\nbut the changed code does not:\n\nC:\\Temp\\gittest>D:\\src\\mingw-git\\git clone -n file://c:/Temp/gittest/foo bad\nCloning into 'bad'...\nfatal: 'D:/Src/MsysGit/Temp/gittest/foo' does not appear to be a git repository\nfatal: Could not read from remote repository.\n\nPlease make sure you have the correct access rights\nand the repository exists.\n\n\nCan you please make this into a series of small patches so that we can\nmore easily see the good and the bad parts? One of the patches could be a\nclean-up of the current protocol determination and URL dissection, which\nis indigestible spaghetti right now.\n\n-- Hannes\n"},{"id":"230125","messageId":"52794882.2020108@kdbg.org","threadId":"35277","inReplyTo":"52789AE5.2010702@viscovery.net","subject":"[PATCH 1/2] git_connect: remove artificial limit of a remote command","fromName":"Johannes Sixt","fromEmail":"j6t@kdbg.org","sentAt":"2013-11-05T19:35:30Z","receivedAt":"2013-11-05T19:35:30Z","isPatch":true,"sender":{"key":"j6t@kdbg.org","avatar":"https://avatars.githubusercontent.com/u/14810926?v=4"},"body":"Since day one, function git_connect() had a limit on the command line of\nthe command that is invoked to make a connection. 7a33bcbe converted the\ncode that constructs the command to strbuf. This would have been the\nright time to remove the limit, but it did not happen. Remove it now.\n\ngit_connect() uses start_command() to invoke the command; consequently,\nthe limits of the system still apply, but are diagnosed only at execve()\ntime. But these limits are more lenient than the 1K that git_connect()\nimposed.\n\nSigned-off-by: Johannes Sixt <j6t@kdbg.org>\n---\nAm 05.11.2013 08:14, schrieb Johannes Sixt:\n> Can you please make this into a series of small patches so that we can\n> more easily see the good and the bad parts? One of the patches could be a\n> clean-up of the current protocol determination and URL dissection, which\n> is indigestible spaghetti right now.\n\nMaybe start with these two.\n\n connect.c | 7 +------\n 1 file changed, 1 insertion(+), 6 deletions(-)\n\ndiff --git a/connect.c b/connect.c\nindex 06e88b0..6cc1f8d 100644\n--- a/connect.c\n+++ b/connect.c\n@@ -527,8 +527,6 @@ static struct child_process *git_proxy_connect(int fd[2], char *host)\n \treturn proxy;\n }\n \n-#define MAX_CMD_LEN 1024\n-\n static char *get_port(char *host)\n {\n \tchar *end;\n@@ -570,7 +568,7 @@ struct child_process *git_connect(int fd[2], const char *url_orig,\n \tint free_path = 0;\n \tchar *port = NULL;\n \tconst char **arg;\n-\tstruct strbuf cmd;\n+\tstruct strbuf cmd = STRBUF_INIT;\n \n \t/* Without this we cannot rely on waitpid() to tell\n \t * what happened to our children.\n@@ -676,12 +674,9 @@ struct child_process *git_connect(int fd[2], const char *url_orig,\n \n \tconn = xcalloc(1, sizeof(*conn));\n \n-\tstrbuf_init(&cmd, MAX_CMD_LEN);\n \tstrbuf_addstr(&cmd, prog);\n \tstrbuf_addch(&cmd, ' ');\n \tsq_quote_buf(&cmd, path);\n-\tif (cmd.len >= MAX_CMD_LEN)\n-\t\tdie(\"command line too long\");\n \n \tconn->in = conn->out = -1;\n \tconn->argv = arg = xcalloc(7, sizeof(*arg));\n-- \n1.8.4.33.gd68f7e8\n"},{"id":"230126","messageId":"5279496A.2090202@kdbg.org","threadId":"35277","inReplyTo":"52794882.2020108@kdbg.org","subject":"[PATCH 2/2] git_connect: factor out discovery of the protocol and its parts","fromName":"Johannes Sixt","fromEmail":"j6t@kdbg.org","sentAt":"2013-11-05T19:39:22Z","receivedAt":"2013-11-05T19:39:22Z","isPatch":true,"sender":{"key":"j6t@kdbg.org","avatar":"https://avatars.githubusercontent.com/u/14810926?v=4"},"body":"git_connect has grown large due to the many different protocols syntaxes\nthat are supported. Move the part of the function that parses the URL to\nconnect to into a separate function for readability.\n\nSigned-off-by: Johannes Sixt <j6t@kdbg.org>\n---\nApart from this \"simplification\", the protocol parsing code is a complete\nmystery to me. There is more potential for simplification because\ngit_proxy_connect() and git_tcp_connect() do their own host+port parsing.\n\n\n connect.c | 80 ++++++++++++++++++++++++++++++++++++++++++---------------------\n 1 file changed, 53 insertions(+), 27 deletions(-)\n\ndiff --git a/connect.c b/connect.c\nindex 6cc1f8d..a6cf345 100644\n--- a/connect.c\n+++ b/connect.c\n@@ -543,37 +543,20 @@ static char *get_port(char *host)\n \treturn NULL;\n }\n \n-static struct child_process no_fork;\n-\n /*\n- * This returns a dummy child_process if the transport protocol does not\n- * need fork(2), or a struct child_process object if it does.  Once done,\n- * finish the connection with finish_connect() with the value returned from\n- * this function (it is safe to call finish_connect() with NULL to support\n- * the former case).\n- *\n- * If it returns, the connect is successful; it just dies on errors (this\n- * will hopefully be changed in a libification effort, to return NULL when\n- * the connection failed).\n+ * Extract protocol and relevant parts from the specified connection URL.\n+ * The caller must free() the returned strings.\n  */\n-struct child_process *git_connect(int fd[2], const char *url_orig,\n-\t\t\t\t  const char *prog, int flags)\n+static enum protocol parse_connect_url(const char *url_orig, char **ret_host,\n+\t\t\t\t       char **ret_port, char **ret_path)\n {\n \tchar *url;\n \tchar *host, *path;\n \tchar *end;\n \tint c;\n-\tstruct child_process *conn = &no_fork;\n \tenum protocol protocol = PROTO_LOCAL;\n \tint free_path = 0;\n \tchar *port = NULL;\n-\tconst char **arg;\n-\tstruct strbuf cmd = STRBUF_INIT;\n-\n-\t/* Without this we cannot rely on waitpid() to tell\n-\t * what happened to our children.\n-\t */\n-\tsignal(SIGCHLD, SIG_DFL);\n \n \tif (is_url(url_orig))\n \t\turl = url_decode(url_orig);\n@@ -645,6 +628,49 @@ struct child_process *git_connect(int fd[2], const char *url_orig,\n \tif (protocol == PROTO_SSH && host != url)\n \t\tport = get_port(end);\n \n+\t*ret_host = xstrdup(host);\n+\tif (port)\n+\t\t*ret_port = xstrdup(port);\n+\telse\n+\t\t*ret_port = NULL;\n+\tif (free_path)\n+\t\t*ret_path = path;\n+\telse\n+\t\t*ret_path = xstrdup(path);\n+\tfree(url);\n+\treturn protocol;\n+}\n+\n+static struct child_process no_fork;\n+\n+/*\n+ * This returns a dummy child_process if the transport protocol does not\n+ * need fork(2), or a struct child_process object if it does.  Once done,\n+ * finish the connection with finish_connect() with the value returned from\n+ * this function (it is safe to call finish_connect() with NULL to support\n+ * the former case).\n+ *\n+ * If it returns, the connect is successful; it just dies on errors (this\n+ * will hopefully be changed in a libification effort, to return NULL when\n+ * the connection failed).\n+ */\n+struct child_process *git_connect(int fd[2], const char *url,\n+\t\t\t\t  const char *prog, int flags)\n+{\n+\tchar *host, *path;\n+\tstruct child_process *conn = &no_fork;\n+\tenum protocol protocol;\n+\tchar *port;\n+\tconst char **arg;\n+\tstruct strbuf cmd = STRBUF_INIT;\n+\n+\t/* Without this we cannot rely on waitpid() to tell\n+\t * what happened to our children.\n+\t */\n+\tsignal(SIGCHLD, SIG_DFL);\n+\n+\tprotocol = parse_connect_url(url, &host, &port, &path);\n+\n \tif (protocol == PROTO_GIT) {\n \t\t/* These underlying connection commands die() if they\n \t\t * cannot connect.\n@@ -666,9 +692,9 @@ struct child_process *git_connect(int fd[2], const char *url_orig,\n \t\t\t     prog, path, 0,\n \t\t\t     target_host, 0);\n \t\tfree(target_host);\n-\t\tfree(url);\n-\t\tif (free_path)\n-\t\t\tfree(path);\n+\t\tfree(host);\n+\t\tfree(port);\n+\t\tfree(path);\n \t\treturn conn;\n \t}\n \n@@ -709,9 +735,9 @@ struct child_process *git_connect(int fd[2], const char *url_orig,\n \tfd[0] = conn->out; /* read from child's stdout */\n \tfd[1] = conn->in;  /* write to child's stdin */\n \tstrbuf_release(&cmd);\n-\tfree(url);\n-\tif (free_path)\n-\t\tfree(path);\n+\tfree(host);\n+\tfree(port);\n+\tfree(path);\n \treturn conn;\n }\n \n-- \n1.8.4.33.gd68f7e8\n"},{"id":"230128","messageId":"527958E1.2080805@web.de","threadId":"35277","inReplyTo":"5279496A.2090202@kdbg.org","subject":"Re: [PATCH 2/2] git_connect: factor out discovery of the protocol and its parts","fromName":"Torsten Bögershausen","fromEmail":"tboegi@web.de","sentAt":"2013-11-05T20:45:21Z","receivedAt":"2013-11-05T20:45:21Z","isPatch":true,"sender":{"key":"tboegi@web.de","avatar":"https://avatars.githubusercontent.com/u/7138363?v=4"},"body":"On 2013-11-05 20.39, Johannes Sixt wrote:\nThanks for picking this up, please see some minor nits inline,\nand git_connect() is at the end\n\n> -struct child_process *git_connect(int fd[2], const char *url_orig,\n> -\t\t\t\t  const char *prog, int flags)\n> +static enum protocol parse_connect_url(const char *url_orig, char **ret_host,\n> +\t\t\t\t       char **ret_port, char **ret_path)\n>  {\n>  \tchar *url;\n>  \tchar *host, *path;\n>  \tchar *end;\nCan we put all the char * into one single line?\n\n>  \tint c;\n> @@ -645,6 +628,49 @@ struct child_process *git_connect(int fd[2], const char *url_orig,\n>  \tif (protocol == PROTO_SSH && host != url)\n>  \t\tport = get_port(end);\n>  \n> +\t*ret_host = xstrdup(host);\n> +\tif (port)\n> +\t\t*ret_port = xstrdup(port);\n> +\telse\n> +\t\t*ret_port = NULL;\n> +\tif (free_path)\n> +\t\t*ret_path = path;\n> +\telse\n> +\t\t*ret_path = xstrdup(path);\n> +\tfree(url);\n> +\treturn protocol;\n> +}\n> +\n> +static struct child_process no_fork;\n> +\n> +/*\n> + * This returns a dummy child_process if the transport protocol does not\n> + * need fork(2), or a struct child_process object if it does.  Once done,\n> + * finish the connection with finish_connect() with the value returned from\n> + * this function (it is safe to call finish_connect() with NULL to support\n> + * the former case).\n> + *\n> + * If it returns, the connect is successful; it just dies on errors (this\n> + * will hopefully be changed in a libification effort, to return NULL when\n> + * the connection failed).\n> + */\n> +struct child_process *git_connect(int fd[2], const char *url,\n> +\t\t\t\t  const char *prog, int flags)\n> +{\n> +\tchar *host, *path;\n> +\tstruct child_process *conn = &no_fork;\n> +\tenum protocol protocol;\n> +\tchar *port;\n> +\tconst char **arg;\n> +\tstruct strbuf cmd = STRBUF_INIT;\n> +\n> +\t/* Without this we cannot rely on waitpid() to tell\n> +\t * what happened to our children.\n> +\t */\n> +\tsignal(SIGCHLD, SIG_DFL);\n> +\n> +\tprotocol = parse_connect_url(url, &host, &port, &path);\n> +\n>  \tif (protocol == PROTO_GIT) {\n>  \t\t/* These underlying connection commands die() if they\n>  \t\t * cannot connect.\n> @@ -666,9 +692,9 @@ struct child_process *git_connect(int fd[2], const char *url_orig,\n>  \t\t\t     prog, path, 0,\n>  \t\t\t     target_host, 0);\n>  \t\tfree(target_host);\nThis is hard to see in the diff, I think we don't need target_host any more.\n> -\t\tfree(url);\n> -\t\tif (free_path)\n> -\t\t\tfree(path);\n> +\t\tfree(host);\n> +\t\tfree(port);\n> +\t\tfree(path);\n>  \t\treturn conn;\n>  \t}\n>  \n> @@ -709,9 +735,9 @@ struct child_process *git_connect(int fd[2], const char *url_orig,\n>  \tfd[0] = conn->out; /* read from child's stdout */\n>  \tfd[1] = conn->in;  /* write to child's stdin */\n>  \tstrbuf_release(&cmd);\n> -\tfree(url);\n> -\tif (free_path)\n> -\t\tfree(path);\n\nThis \"end of function, free everything and return conn\",\ncould we re-arange so that it is in the code only once ?\n\n> +\tfree(host);\n> +\tfree(port);\n> +\tfree(path);\n>  \treturn conn;\n>  }\n====================\n\nstruct child_process *git_connect(int fd[2], const char *url,\n                  const char *prog, int flags)\n{\n    char *host, *port, *path;\n    struct child_process *conn = &no_fork;\n    enum protocol protocol;\n    const char **arg;\n    struct strbuf cmd = STRBUF_INIT;\n\n    /* Without this we cannot rely on waitpid() to tell\n     * what happened to our children.\n     */\n    signal(SIGCHLD, SIG_DFL);\n\n    protocol = parse_connect_url(url, &host, &port, &path);\n\n    if (protocol == PROTO_GIT) {\n        /* These underlying connection commands die() if they\n         * cannot connect.\n         */\n        if (git_use_proxy(host))\n            conn = git_proxy_connect(fd, host);\n        else\n            git_tcp_connect(fd, host, flags);\n        /*\n         * Separate original protocol components prog and path\n         * from extended host header with a NUL byte.\n         *\n         * Note: Do not add any other headers here!  Doing so\n         * will cause older git-daemon servers to crash.\n         */\n        packet_write(fd[1],\n                 \"%s %s%chost=%s%c\",\n                 prog, path, 0,\n                 host, 0);\n    }\n    else\n    {\n        conn = xcalloc(1, sizeof(*conn));\n\n        strbuf_addstr(&cmd, prog);\n        strbuf_addch(&cmd, ' ');\n        sq_quote_buf(&cmd, path);\n\n        conn->in = conn->out = -1;\n        conn->argv = arg = xcalloc(7, sizeof(*arg));\n        if (protocol == PROTO_SSH) {\n            const char *ssh = getenv(\"GIT_SSH\");\n            int putty = ssh && strcasestr(ssh, \"plink\");\n            if (!ssh) ssh = \"ssh\";\n\n            *arg++ = ssh;\n            if (putty && !strcasestr(ssh, \"tortoiseplink\"))\n                *arg++ = \"-batch\";\n            if (port) {\n                /* P is for PuTTY, p is for OpenSSH */\n                *arg++ = putty ? \"-P\" : \"-p\";\n                *arg++ = port;\n            }\n            *arg++ = host;\n        }\n        else {\n            /* remove repo-local variables from the environment */\n            conn->env = local_repo_env;\n            conn->use_shell = 1;\n        }\n        *arg++ = cmd.buf;\n        *arg = NULL;\n\n        if (start_command(conn))\n            die(\"unable to fork\");\n\n        fd[0] = conn->out; /* read from child's stdout */\n        fd[1] = conn->in;  /* write to child's stdin */\n        strbuf_release(&cmd);\n    }\n    free(host);\n    free(port);\n    free(path);\n    return conn;\n}\n"},{"id":"230131","messageId":"527961AB.2010606@kdbg.org","threadId":"35277","inReplyTo":"527958E1.2080805@web.de","subject":"Re: [PATCH 2/2] git_connect: factor out discovery of the protocol and its parts","fromName":"Johannes Sixt","fromEmail":"j6t@kdbg.org","sentAt":"2013-11-05T21:22:51Z","receivedAt":"2013-11-05T21:22:51Z","isPatch":true,"sender":{"key":"j6t@kdbg.org","avatar":"https://avatars.githubusercontent.com/u/14810926?v=4"},"body":"Am 05.11.2013 21:45, schrieb Torsten Bögershausen:\n> On 2013-11-05 20.39, Johannes Sixt wrote:\n> Thanks for picking this up, please see some minor nits inline,\n> and git_connect() is at the end\n> \n>> -struct child_process *git_connect(int fd[2], const char *url_orig,\n>> -\t\t\t\t  const char *prog, int flags)\n>> +static enum protocol parse_connect_url(const char *url_orig, char **ret_host,\n>> +\t\t\t\t       char **ret_port, char **ret_path)\n>>  {\n>>  \tchar *url;\n>>  \tchar *host, *path;\n>>  \tchar *end;\n> Can we put all the char * into one single line?\n\nThe idea here was to keep the diff minimal, and that further slight\ncleanups should be combined with subsequent rewrites that should happen\nto this function.\n\n>>  \tint c;\n>> @@ -645,6 +628,49 @@ struct child_process *git_connect(int fd[2], const char *url_orig,\n>>  \tif (protocol == PROTO_SSH && host != url)\n>>  \t\tport = get_port(end);\n>>  \n>> +\t*ret_host = xstrdup(host);\n>> +\tif (port)\n>> +\t\t*ret_port = xstrdup(port);\n>> +\telse\n>> +\t\t*ret_port = NULL;\n>> +\tif (free_path)\n>> +\t\t*ret_path = path;\n>> +\telse\n>> +\t\t*ret_path = xstrdup(path);\n>> +\tfree(url);\n>> +\treturn protocol;\n>> +}\n>> +\n>> +static struct child_process no_fork;\n>> +\n>> +/*\n>> + * This returns a dummy child_process if the transport protocol does not\n>> + * need fork(2), or a struct child_process object if it does.  Once done,\n>> + * finish the connection with finish_connect() with the value returned from\n>> + * this function (it is safe to call finish_connect() with NULL to support\n>> + * the former case).\n>> + *\n>> + * If it returns, the connect is successful; it just dies on errors (this\n>> + * will hopefully be changed in a libification effort, to return NULL when\n>> + * the connection failed).\n>> + */\n>> +struct child_process *git_connect(int fd[2], const char *url,\n>> +\t\t\t\t  const char *prog, int flags)\n>> +{\n>> +\tchar *host, *path;\n>> +\tstruct child_process *conn = &no_fork;\n>> +\tenum protocol protocol;\n>> +\tchar *port;\n>> +\tconst char **arg;\n>> +\tstruct strbuf cmd = STRBUF_INIT;\n>> +\n>> +\t/* Without this we cannot rely on waitpid() to tell\n>> +\t * what happened to our children.\n>> +\t */\n>> +\tsignal(SIGCHLD, SIG_DFL);\n>> +\n>> +\tprotocol = parse_connect_url(url, &host, &port, &path);\n>> +\n>>  \tif (protocol == PROTO_GIT) {\n>>  \t\t/* These underlying connection commands die() if they\n>>  \t\t * cannot connect.\n>> @@ -666,9 +692,9 @@ struct child_process *git_connect(int fd[2], const char *url_orig,\n>>  \t\t\t     prog, path, 0,\n>>  \t\t\t     target_host, 0);\n>>  \t\tfree(target_host);\n> This is hard to see in the diff, I think we don't need target_host any more.\n\nI though that as well first, but no, we still need it. Further rewrites\nare needed that move the port discovery from git_proxy_connect() and\ngit_tcp_connect() to the new parse_connect_url() before target_host can\ngo away. And even then it is questionable because target_host is used in\nan error message and is intended to reflect the original combined\nhost+port portion of the URL, if I read the code correctly.\n\n>> -\t\tfree(url);\n>> -\t\tif (free_path)\n>> -\t\t\tfree(path);\n>> +\t\tfree(host);\n>> +\t\tfree(port);\n>> +\t\tfree(path);\n>>  \t\treturn conn;\n>>  \t}\n>>  \n>> @@ -709,9 +735,9 @@ struct child_process *git_connect(int fd[2], const char *url_orig,\n>>  \tfd[0] = conn->out; /* read from child's stdout */\n>>  \tfd[1] = conn->in;  /* write to child's stdin */\n>>  \tstrbuf_release(&cmd);\n>> -\tfree(url);\n>> -\tif (free_path)\n>> -\t\tfree(path);\n> \n> This \"end of function, free everything and return conn\",\n> could we re-arange so that it is in the code only once ?\n\nThat would be quite simple now; just place the part after the first\nreturn into the else branch. That opens opportunities to move variable\ndeclarations from the top of the function into the else branch.\n\nBut all of these changes should go into a separate commit, IMO, so that\nthe function splitting that happens here can be verified more easily.\n\n-- Hannes\n"},{"id":"230142","messageId":"527A60C2.8090607@web.de","threadId":"35277","inReplyTo":"527961AB.2010606@kdbg.org","subject":"Re: [PATCH 2/2] git_connect: factor out discovery of the protocol and its parts","fromName":"Torsten Bögershausen","fromEmail":"tboegi@web.de","sentAt":"2013-11-06T15:31:14Z","receivedAt":"2013-11-06T15:31:14Z","isPatch":true,"sender":{"key":"tboegi@web.de","avatar":"https://avatars.githubusercontent.com/u/7138363?v=4"},"body":"On 2013-11-05 22.22, Johannes Sixt wrote:\n> Am 05.11.2013 21:45, schrieb Torsten Bögershausen:\n>> On 2013-11-05 20.39, Johannes Sixt wrote:\n>> Thanks for picking this up, please see some minor nits inline,\n>> and git_connect() is at the end\n>>\n>>> -struct child_process *git_connect(int fd[2], const char *url_orig,\n>>> -\t\t\t\t  const char *prog, int flags)\n>>> +static enum protocol parse_connect_url(const char *url_orig, char **ret_host,\n>>> +\t\t\t\t       char **ret_port, char **ret_path)\n>>>  {\n>>>  \tchar *url;\n>>>  \tchar *host, *path;\n>>>  \tchar *end;\n>> Can we put all the char * into one single line?\n> \n> The idea here was to keep the diff minimal, and that further slight\n> cleanups should be combined with subsequent rewrites that should happen\n> to this function.\n> \n>>>  \tint c;\n>>> @@ -645,6 +628,49 @@ struct child_process *git_connect(int fd[2], const char *url_orig,\n>>>  \tif (protocol == PROTO_SSH && host != url)\n>>>  \t\tport = get_port(end);\n>>>  \n>>> +\t*ret_host = xstrdup(host);\n>>> +\tif (port)\n>>> +\t\t*ret_port = xstrdup(port);\n>>> +\telse\n>>> +\t\t*ret_port = NULL;\n>>> +\tif (free_path)\n>>> +\t\t*ret_path = path;\n>>> +\telse\n>>> +\t\t*ret_path = xstrdup(path);\n>>> +\tfree(url);\n>>> +\treturn protocol;\n>>> +}\n>>> +\n>>> +static struct child_process no_fork;\n>>> +\n>>> +/*\n>>> + * This returns a dummy child_process if the transport protocol does not\n>>> + * need fork(2), or a struct child_process object if it does.  Once done,\n>>> + * finish the connection with finish_connect() with the value returned from\n>>> + * this function (it is safe to call finish_connect() with NULL to support\n>>> + * the former case).\n>>> + *\n>>> + * If it returns, the connect is successful; it just dies on errors (this\n>>> + * will hopefully be changed in a libification effort, to return NULL when\n>>> + * the connection failed).\n>>> + */\n>>> +struct child_process *git_connect(int fd[2], const char *url,\n>>> +\t\t\t\t  const char *prog, int flags)\n>>> +{\n>>> +\tchar *host, *path;\n>>> +\tstruct child_process *conn = &no_fork;\n>>> +\tenum protocol protocol;\n>>> +\tchar *port;\n>>> +\tconst char **arg;\n>>> +\tstruct strbuf cmd = STRBUF_INIT;\n>>> +\n>>> +\t/* Without this we cannot rely on waitpid() to tell\n>>> +\t * what happened to our children.\n>>> +\t */\n>>> +\tsignal(SIGCHLD, SIG_DFL);\n>>> +\n>>> +\tprotocol = parse_connect_url(url, &host, &port, &path);\n>>> +\n>>>  \tif (protocol == PROTO_GIT) {\n>>>  \t\t/* These underlying connection commands die() if they\n>>>  \t\t * cannot connect.\n>>> @@ -666,9 +692,9 @@ struct child_process *git_connect(int fd[2], const char *url_orig,\n>>>  \t\t\t     prog, path, 0,\n>>>  \t\t\t     target_host, 0);\n>>>  \t\tfree(target_host);\n>> This is hard to see in the diff, I think we don't need target_host any more.\n> \n> I though that as well first, but no, we still need it. Further rewrites\n> are needed that move the port discovery from git_proxy_connect() and\n> git_tcp_connect() to the new parse_connect_url() before target_host can\n> go away. And even then it is questionable because target_host is used in\n> an error message and is intended to reflect the original combined\n> host+port portion of the URL, if I read the code correctly.\n> \n>>> -\t\tfree(url);\n>>> -\t\tif (free_path)\n>>> -\t\t\tfree(path);\n>>> +\t\tfree(host);\n>>> +\t\tfree(port);\n>>> +\t\tfree(path);\n>>>  \t\treturn conn;\n>>>  \t}\n>>>  \n>>> @@ -709,9 +735,9 @@ struct child_process *git_connect(int fd[2], const char *url_orig,\n>>>  \tfd[0] = conn->out; /* read from child's stdout */\n>>>  \tfd[1] = conn->in;  /* write to child's stdin */\n>>>  \tstrbuf_release(&cmd);\n>>> -\tfree(url);\n>>> -\tif (free_path)\n>>> -\t\tfree(path);\n>>\n>> This \"end of function, free everything and return conn\",\n>> could we re-arange so that it is in the code only once ?\n> \n> That would be quite simple now; just place the part after the first\n> return into the else branch. That opens opportunities to move variable\n> declarations from the top of the function into the else branch.\n> \n> But all of these changes should go into a separate commit, IMO, so that\n> the function splitting that happens here can be verified more easily.\n> \n> -- Hannes\nAgreed on all points, (some re-reading was needed)\n\nI will first focus on the test cases,\nsince having god test cases eases us the re-factoring later on.\nThanks\n/Torsten\n"}]}