{"thread":{"id":"8249","subject":"[PATCH 1/3] builtin-fetch--tool: extend \"native-store\" for use in cloning","startedAt":"2007-05-20T17:57:42Z","lastAt":"2007-05-20T17:57:45Z","messageCount":4,"participants":["skimo@liacs.nl"],"isPatch":true,"patchVersion":1,"patchTotal":3},"messages":[{"id":"42733","messageId":"1179683865547-git-send-email-skimo@liacs.nl","threadId":"8249","inReplyTo":null,"subject":"Partial removal of fetching from git-clone","fromName":"","fromEmail":"skimo@liacs.nl","sentAt":"2007-05-20T17:57:42Z","receivedAt":"2007-05-20T17:57:42Z","isPatch":false,"sender":{"key":"skimo@kotnet.org","avatar":null},"body":"From: Sven Verdoolaege <skimo@kotnet.org>\n\nThis was needed for doing submodules in git-fetch,\nbut since I moved the cloning of submodules to git-checkout,\nI no longer need these changes.\nStill, I know Junio wants something like this, so they\ncould be used as a starting point for completely\nremoving all fetching from git-clone.\n\nskimo\n"},{"id":"42732","messageId":"11796838661131-git-send-email-skimo@liacs.nl","threadId":"8249","inReplyTo":"1179683865547-git-send-email-skimo@liacs.nl","subject":"[PATCH 1/3] builtin-fetch--tool: extend \"native-store\" for use in cloning","fromName":"","fromEmail":"skimo@liacs.nl","sentAt":"2007-05-20T17:57:43Z","receivedAt":"2007-05-20T17:57:43Z","isPatch":true,"sender":{"key":"skimo@kotnet.org","avatar":null},"body":"From: Sven Verdoolaege <skimo@kotnet.org>\n\nSigned-off-by: Sven Verdoolaege <skimo@kotnet.org>\n---\n builtin-fetch--tool.c |   52 +++++++++++++++++++++++++++++++++++++++++++-----\n 1 files changed, 46 insertions(+), 6 deletions(-)\n\ndiff --git a/builtin-fetch--tool.c b/builtin-fetch--tool.c\nindex ed4d5de..b3ed3da 100644\n--- a/builtin-fetch--tool.c\n+++ b/builtin-fetch--tool.c\n@@ -207,6 +207,32 @@ static void remove_keep_on_signal(int signo)\n \traise(signo);\n }\n \n+static char *construct_local_name(const char *remote_ref, const char *remote_nick,\n+\t\t\t\t  int use_separate_remote)\n+{\n+\tstatic char local_ref[PATH_MAX];\n+\tint len = strlen(remote_ref);\n+\n+\tif (len >= 3 && !memcmp(remote_ref+len-3, \"^{}\", 3))\n+\t\treturn NULL;\n+\tif (!strcmp(remote_ref, \"HEAD\"))\n+\t\treturn \"REMOTE_HEAD\";\n+\tif (!prefixcmp(remote_ref, \"refs/heads/\")) {\n+\t\tif (snprintf(local_ref, sizeof(local_ref), \"refs/%s%s/%s\",\n+\t\t\t    use_separate_remote ? \"remotes/\" : \"heads\",\n+\t\t\t    use_separate_remote ? remote_nick : \"\",\n+\t\t\t    remote_ref+11) > sizeof(local_ref))\n+\t\t\tdie(\"Local branchname too long\");\n+\t} else if (!prefixcmp(remote_ref, \"refs/tags/\")) {\n+\t\tif (snprintf(local_ref, sizeof(local_ref), \"refs/tags/%s\",\n+\t\t\t    remote_ref+10) > sizeof(local_ref))\n+\t\t\tdie(\"Local branchname too long\");\n+\t} else\n+\t\treturn NULL;\n+\n+\treturn local_ref;\n+}\n+\n static char *find_local_name(const char *remote_name, const char *refs,\n \t\t\t     int *force_p, int *not_for_merge_p)\n {\n@@ -261,7 +287,8 @@ static int fetch_native_store(FILE *fp,\n \t\t\t      const char *remote,\n \t\t\t      const char *remote_nick,\n \t\t\t      const char *refs,\n-\t\t\t      int verbose, int force)\n+\t\t\t      int verbose, int force,\n+\t\t\t      int all, int use_separate_remote)\n {\n \tchar buffer[1024];\n \tint err = 0;\n@@ -273,7 +300,7 @@ static int fetch_native_store(FILE *fp,\n \t\tint len;\n \t\tchar *cp;\n \t\tchar *local_name;\n-\t\tint single_force, not_for_merge;\n+\t\tint single_force = force, not_for_merge = 0;\n \n \t\tfor (cp = buffer; *cp && !isspace(*cp); cp++)\n \t\t\t;\n@@ -294,14 +321,18 @@ static int fetch_native_store(FILE *fp,\n \t\t\tcontinue;\n \t\t}\n \n-\t\tlocal_name = find_local_name(cp, refs,\n-\t\t\t\t\t     &single_force, &not_for_merge);\n+\t\tif (all)\n+\t\t\tlocal_name = construct_local_name(cp, remote_nick,\n+\t\t\t\t\t\t\t  use_separate_remote);\n+\t\telse\n+\t\t\tlocal_name = find_local_name(cp, refs,\n+\t\t\t\t\t\t     &single_force, &not_for_merge);\n \t\tif (!local_name)\n \t\t\tcontinue;\n \t\terr |= append_fetch_head(fp,\n \t\t\t\t\t buffer, remote, cp, remote_nick,\n \t\t\t\t\t local_name, not_for_merge,\n-\t\t\t\t\t verbose, force || single_force);\n+\t\t\t\t\t verbose, single_force);\n \t}\n \treturn err;\n }\n@@ -514,6 +545,8 @@ int cmd_fetch__tool(int argc, const char **argv, const char *prefix)\n \tint verbose = 0;\n \tint force = 0;\n \tint sopt = 0;\n+\tint all = 0;\n+\tint use_separate_remote = 1;\n \n \twhile (1 < argc) {\n \t\tconst char *arg = argv[1];\n@@ -523,6 +556,12 @@ int cmd_fetch__tool(int argc, const char **argv, const char *prefix)\n \t\t\tforce = 1;\n \t\telse if (!strcmp(\"-s\", arg))\n \t\t\tsopt = 1;\n+\t\telse if (!strcmp(\"--all\", arg))\n+\t\t\tall = 1;\n+\t\telse if (!strcmp(\"--use-separate-remote\", arg))\n+\t\t\tuse_separate_remote = 1;\n+\t\telse if (!strcmp(\"--no-separate-remote\", arg))\n+\t\t\tuse_separate_remote = 0;\n \t\telse\n \t\t\tbreak;\n \t\targc--;\n@@ -554,7 +593,8 @@ int cmd_fetch__tool(int argc, const char **argv, const char *prefix)\n \t\t\treturn error(\"fetch-native-store takes 3 args\");\n \t\tfp = fopen(git_path(\"FETCH_HEAD\"), \"a\");\n \t\tresult = fetch_native_store(fp, argv[2], argv[3], argv[4],\n-\t\t\t\t\t    verbose, force);\n+\t\t\t\t\t    verbose, force, all,\n+\t\t\t\t\t    use_separate_remote);\n \t\tfclose(fp);\n \t\treturn result;\n \t}\n-- \n1.5.1.5.g8fc2\n"},{"id":"42735","messageId":"11796838662238-git-send-email-skimo@liacs.nl","threadId":"8249","inReplyTo":"1179683865547-git-send-email-skimo@liacs.nl","subject":"[PATCH 2/3] git-clone: rely on git-fetch for fetching for most protocols","fromName":"","fromEmail":"skimo@liacs.nl","sentAt":"2007-05-20T17:57:44Z","receivedAt":"2007-05-20T17:57:44Z","isPatch":true,"sender":{"key":"skimo@kotnet.org","avatar":null},"body":"From: Sven Verdoolaege <skimo@kotnet.org>\n\nSigned-off-by: Sven Verdoolaege <skimo@kotnet.org>\n---\n git-clone.sh |   20 ++++++++++++--------\n git-fetch.sh |   28 ++++++++++++++++++++++------\n 2 files changed, 34 insertions(+), 14 deletions(-)\n\ndiff --git a/git-clone.sh b/git-clone.sh\nindex fdd354f..823b973 100755\n--- a/git-clone.sh\n+++ b/git-clone.sh\n@@ -159,6 +159,11 @@ then\n \tno_checkout=yes\n \tuse_separate_remote=\n fi\n+if test t = \"$use_separate_remote\"; then\n+\tseparate_remote_flag=\"--use-separate-remote\"\n+else\n+\tseparate_remote_flag=\"--no-separate-remote\"\n+fi\n \n if test -z \"$origin\"\n then\n@@ -219,6 +224,10 @@ then\n \tfi\n fi\n \n+# Write out $origin URL\n+GIT_CONFIG=\"$GIT_DIR/config\"\n+git-config remote.\"$origin\".url \"$repo\" || exit\n+\n rm -f \"$GIT_DIR/CLONE_HEAD\"\n \n # We do local magic only when the user tells us to.\n@@ -299,11 +308,9 @@ yes,yes)\n \t\tfi\n \t\t;;\n \t*)\n-\t\tcase \"$upload_pack\" in\n-\t\t'') git-fetch-pack --all -k $quiet $depth $no_progress \"$repo\";;\n-\t\t*) git-fetch-pack --all -k $quiet \"$upload_pack\" $depth $no_progress \"$repo\" ;;\n-\t\tesac >\"$GIT_DIR/CLONE_HEAD\" ||\n-\t\t\tdie \"fetch-pack from '$repo' failed.\"\n+\t\tgit-fetch --all -k $quiet  ${upload_pack:+\"$upload_pack\"} $depth \\\n+\t\t\t$separate_remote_flag \"$origin\" ||\n+\t\t\tdie \"fetch from '$repo' failed.\"\n \t\t;;\n \tesac\n \t;;\n@@ -387,9 +394,6 @@ then\n \t\torigin_track=\"$remote_top/$head_points_at\" &&\n \t\tgit-update-ref HEAD \"$head_sha1\" &&\n \n-\t\t# Upstream URL\n-\t\tgit-config remote.\"$origin\".url \"$repo\" &&\n-\n \t\t# Set up the mappings to track the remote branches.\n \t\tgit-config remote.\"$origin\".fetch \\\n \t\t\t\"+refs/heads/*:$remote_top/*\" '^$' &&\ndiff --git a/git-fetch.sh b/git-fetch.sh\nindex 0e05cf1..dcb6985 100755\n--- a/git-fetch.sh\n+++ b/git-fetch.sh\n@@ -15,6 +15,7 @@ LF='\n '\n IFS=\"$LF\"\n \n+all=\n no_tags=\n tags=\n append=\n@@ -25,6 +26,7 @@ exec=\n keep=\n shallow_depth=\n no_progress=\n+use_separate_remote=\n test -t 1 || no_progress=--no-progress\n quiet=\n while case \"$#\" in 0) break ;; esac\n@@ -33,6 +35,9 @@ do\n \t-a|--a|--ap|--app|--appe|--appen|--append)\n \t\tappend=t\n \t\t;;\n+\t--al|--all)\n+\t\tall=--all\n+\t\t;;\n \t--upl|--uplo|--uploa|--upload|--upload-|--upload-p|\\\n \t--upload-pa|--upload-pac|--upload-pack)\n \t\tshift\n@@ -63,6 +68,12 @@ do\n \t-v|--verbose)\n \t\tverbose=Yes\n \t\t;;\n+\t--use-separate-remote)\n+\t\tuse_separate_remote=\"--use-separate-remote\"\n+\t\t;;\n+\t--no-separate-remote)\n+\t\tuse_separate_remote=\"--no-separate-remote\"\n+\t\t;;\n \t-k|--k|--ke|--kee|--keep)\n \t\tkeep='-k -k'\n \t\t;;\n@@ -141,7 +152,9 @@ esac\n # branches file, and just fetch those and refspecs explicitly given.\n # Otherwise we do what we always did.\n \n-reflist=$(get_remote_refs_for_fetch \"$@\")\n+if test -z \"$all\"; then\n+\treflist=$(get_remote_refs_for_fetch \"$@\")\n+fi\n if test \"$tags\"\n then\n \ttaglist=`IFS='\t' &&\n@@ -163,8 +176,10 @@ fi\n \n fetch_all_at_once () {\n \n-  eval=$(echo \"$1\" | git-fetch--tool parse-reflist \"-\")\n-  eval \"$eval\"\n+    if test -z \"$all\"; then\n+\teval=$(echo \"$1\" | git-fetch--tool parse-reflist \"-\")\n+\teval \"$eval\"\n+    fi\n \n     ( : subshell because we muck with IFS\n       IFS=\" \t$LF\"\n@@ -177,7 +192,8 @@ fetch_all_at_once () {\n \t    git-bundle unbundle \"$remote\" $rref ||\n \t    echo failed \"$remote\"\n \telse\n-\t\tif\ttest -d \"$remote\" &&\n+\t\tif\ttest -z \"$all\" &&\n+\t\t\ttest -d \"$remote\" &&\n \n \t\t\t# The remote might be our alternate.  With\n \t\t\t# this optimization we will bypass fetch-pack\n@@ -201,7 +217,7 @@ fetch_all_at_once () {\n \t\t\techo \"$ls_remote_result\" | \\\n \t\t\t\tgit-fetch--tool pick-rref \"$rref\" \"-\"\n \t\telse\n-\t\t\tgit-fetch-pack --thin $exec $keep $shallow_depth \\\n+\t\t\tgit-fetch-pack --thin $all $exec $keep $shallow_depth \\\n \t\t\t\t$quiet $no_progress \"$remote\" $rref ||\n \t\t\techo failed \"$remote\"\n \t\tfi\n@@ -212,7 +228,7 @@ fetch_all_at_once () {\n \ttest -n \"$verbose\" && flags=\"$flags -v\"\n \ttest -n \"$force\" && flags=\"$flags -f\"\n \tGIT_REFLOG_ACTION=\"$GIT_REFLOG_ACTION\" \\\n-\t\tgit-fetch--tool $flags native-store \\\n+\t\tgit-fetch--tool $flags $all $use_separate_remote native-store \\\n \t\t\t\"$remote\" \"$remote_nick\" \"$refs\"\n       )\n     ) || exit\n-- \n1.5.1.5.g8fc2\n"},{"id":"42734","messageId":"11796838663961-git-send-email-skimo@liacs.nl","threadId":"8249","inReplyTo":"1179683865547-git-send-email-skimo@liacs.nl","subject":"[PATCH 3/3] git-clone: rely on git-fetch for non-bare fetching over http","fromName":"","fromEmail":"skimo@liacs.nl","sentAt":"2007-05-20T17:57:45Z","receivedAt":"2007-05-20T17:57:45Z","isPatch":true,"sender":{"key":"skimo@kotnet.org","avatar":null},"body":"From: Sven Verdoolaege <skimo@kotnet.org>\n\nSigned-off-by: Sven Verdoolaege <skimo@kotnet.org>\n---\n git-clone.sh |    6 +++---\n git-fetch.sh |   20 ++++++++++++++++++++\n 2 files changed, 23 insertions(+), 3 deletions(-)\n\ndiff --git a/git-clone.sh b/git-clone.sh\nindex 823b973..8d5dc05 100755\n--- a/git-clone.sh\n+++ b/git-clone.sh\n@@ -262,8 +262,8 @@ yes,yes)\n \tgit-ls-remote \"$repo\" >\"$GIT_DIR/CLONE_HEAD\" || exit 1\n \t;;\n *)\n-\tcase \"$repo\" in\n-\trsync://*)\n+\tcase \"$bare,$repo\" in\n+\t*,rsync://*)\n \t\tcase \"$depth\" in\n \t\t\"\") ;;\n \t\t*) die \"shallow over rsync not supported\" ;;\n@@ -295,7 +295,7 @@ yes,yes)\n \t\tfi\n \t\tgit-ls-remote \"$repo\" >\"$GIT_DIR/CLONE_HEAD\" || exit 1\n \t\t;;\n-\thttps://*|http://*|ftp://*)\n+\tyes,https://*|yes,http://*|yes,ftp://*)\n \t\tcase \"$depth\" in\n \t\t\"\") ;;\n \t\t*) die \"shallow over http or ftp not supported\" ;;\ndiff --git a/git-fetch.sh b/git-fetch.sh\nindex dcb6985..68706db 100755\n--- a/git-fetch.sh\n+++ b/git-fetch.sh\n@@ -235,11 +235,31 @@ fetch_all_at_once () {\n \n }\n \n+http_fetch () {\n+\tif [ -n \"$GIT_SSL_NO_VERIFY\" ]; then\n+\t\tcurl_extra_args=\"-k\"\n+\tfi\n+\tif [ -n \"$GIT_CURL_FTP_NO_EPSV\" -o \\\n+\t\t\"`git-config --bool http.noEPSV`\" = true ]; then\n+\t\tcurl_extra_args=\"${curl_extra_args} --disable-epsv\"\n+\tfi\n+\n+\t# $1 = Remote, $2 = Local\n+\tcurl -nsfL $curl_extra_args \"$1\" >\"$2\"\n+}\n+\n fetch_per_ref () {\n   reflist=\"$1\"\n   refs=\n   rref=\n \n+    if test -n \"$all\"; then\n+\treflist=$(canon_refs_list_for_fetch -d \"$remote_nick\" \\\n+\t\t\t\"+refs/heads/*:refs/remotes/$remote_nick/*\")\n+\thttp_fetch \"$remote/HEAD\" \"$GIT_DIR/REMOTE_HEAD\" ||\n+\trm -f \"$GIT_DIR/REMOTE_HEAD\"\n+    fi\n+\n   for ref in $reflist\n   do\n       refs=\"$refs$LF$ref\"\n-- \n1.5.1.5.g8fc2\n"}]}