{"thread":{"id":"5663","subject":"[RFC][RESEND][PATCH] Allow fetching from multiple repositories at once","startedAt":"2006-09-23T16:45:09Z","lastAt":"2006-09-23T19:35:19Z","messageCount":13,"participants":["Petr Baudis","A Large Angry SCM","Johannes Schindelin","Jakub Narebski","Junio C Hamano"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"27482","messageId":"20060923164308.16334.49252.stgit@machine.or.cz","threadId":"5663","inReplyTo":null,"subject":"[RFC][RESEND][PATCH] Allow fetching from multiple repositories at once","fromName":"Petr Baudis","fromEmail":"pasky@suse.cz","sentAt":"2006-09-23T16:45:09Z","receivedAt":"2006-09-23T16:45:09Z","isPatch":true,"sender":{"key":"pasky@ucw.cz","avatar":"https://avatars.githubusercontent.com/u/18439?v=4"},"body":"This patch enables fetching multiple repositories at once over the Git\nprotocol (and SSH, and locally if git-fetch-pack is your cup of coffee\nthere). This is done especially for the xorg people who have tons of\nrepositories and dislike pulls much slower than they were used to with CVS.\nI'm eager to hear how this affects the situation.\n\nIt's kind of \"superproject\" thing, basically, taking reverse approach than\nthe subproject ideas. However, in practice I think it can be used for\nsubprojects quite well and perhaps if I find some spare time during the day\nI will add the lightweight subproject support to Cogito, using this.\n\nSo, you need some kind of porcelain for this. The idea is that instead of\ntelling git-fetch-pack a single repository, you pass multiple --repo=\nparameters and refs always \"belong\" to the latest mentioned repository;\nwhen outputting the new ref values, the appropriate repo is mentioned near\neach ref. In order for this to be useful, on your local side you should\nshare the objects database in some way - either using alternates (then\nyou must fetch to an object database reachable from everywhere) or symlinked\nobject databases.\n\nYou still need to pass git-fetch-pack some URL in addition to the\nrepositories - it is used only for git_connect(), the purpose is that\nrepositories must be local directories so if you want to talk remote, you\nneed to do something like\n\n\tgit-fetch-pack git://kernel.org/pub/scm/git/git.git --repo=/pub/scm/git/git.git master next --repo=/pub/scm/cogito/cogito.git master\n\nThe implementation is, well... rather hackish to say the least. :-)\nWe can do something nicer if it turns out to be a boost huge enough\nin reality too.\n\nThe main target of this are the Xorg people who have plenty of small repos\nand frequently want to update many of them. It would be nice if someone\nfrom Xorg reading this and concerned with this could measure the\ndifference.\n\nSigned-off-by: Petr Baudis <pasky@suse.cz>\n---\n\n This is an even more awful hack than the previous version, because of\ndid_refs. I wonder how ugly would it be to have that in stock Git\nversion, or if there's a good way to eliminate that hack before\nproceeding with full libification.\n\n This is a second resend. The first one was from the end of July but\nflyed by with not much of a response.\n\n cache.h       |    7 ++++-\n connect.c     |   11 +++++---\n daemon.c      |    9 +++++--\n fetch-pack.c  |   58 ++++++++++++++++++++++++++++++---------------\n path.c        |    5 ++++\n peek-remote.c |    4 ++-\n refs.c        |    6 ++---\n send-pack.c   |    4 ++-\n sha1_file.c   |    6 ++---\n upload-pack.c |   74 +++++++++++++++++++++++++++++++++++++++++++++++++--------\n 10 files changed, 136 insertions(+), 48 deletions(-)\n\ndiff --git a/cache.h b/cache.h\nindex a7f673d..59cf0ed 100644\n--- a/cache.h\n+++ b/cache.h\n@@ -240,6 +240,7 @@ int git_config_perm(const char *var, con\n int adjust_shared_perm(const char *path);\n int safe_create_leading_directories(char *path);\n char *enter_repo(char *path, int strict);\n+int security_repo_check(int export_all_trees);\n \n /* Read and unpack a sha1 file into memory, write memory to a sha1 file */\n extern int sha1_object_info(const unsigned char *, char *, unsigned long *);\n@@ -332,6 +333,7 @@ extern struct alternate_object_database \n \tchar base[FLEX_ARRAY]; /* more */\n } *alt_odb_list;\n extern void prepare_alt_odb(void);\n+extern int link_alt_odb_entry(const char * entry, int len, const char * relative_base, int depth, int allow_self);\n \n extern struct packed_git {\n \tstruct packed_git *next;\n@@ -359,6 +361,7 @@ struct ref {\n \tunsigned char new_sha1[20];\n \tunsigned char force;\n \tstruct ref *peer_ref; /* when renaming */\n+\tchar *repo; /* when doing multi-repo operations */\n \tchar name[FLEX_ARRAY]; /* more */\n };\n \n@@ -368,11 +371,11 @@ #define REF_TAGS\t(1u << 2)\n \n extern pid_t git_connect(int fd[2], char *url, const char *prog);\n extern int finish_connect(pid_t pid);\n-extern int path_match(const char *path, int nr, char **match);\n+extern int path_match(const char *path, const char *repo, int nr, char **match, char **match_repo);\n extern int match_refs(struct ref *src, struct ref *dst, struct ref ***dst_tail,\n \t\t      int nr_refspec, char **refspec, int all);\n extern int get_ack(int fd, unsigned char *result_sha1);\n-extern struct ref **get_remote_heads(int in, struct ref **list, int nr_match, char **match, unsigned int flags);\n+extern struct ref **get_remote_heads(int in, struct ref **list, int nr_match, char **match, char *repo, unsigned int flags);\n extern int server_supports(const char *feature);\n \n extern struct packed_git *parse_pack_index(unsigned char *sha1);\ndiff --git a/connect.c b/connect.c\nindex c55a20a..0b62406 100644\n--- a/connect.c\n+++ b/connect.c\n@@ -45,9 +45,8 @@ static int check_ref(const char *name, i\n  */\n struct ref **get_remote_heads(int in, struct ref **list,\n \t\t\t      int nr_match, char **match,\n-\t\t\t      unsigned int flags)\n+\t\t\t      char *repo, unsigned int flags)\n {\n-\t*list = NULL;\n \tfor (;;) {\n \t\tstruct ref *ref;\n \t\tunsigned char old_sha1[20];\n@@ -74,11 +73,12 @@ struct ref **get_remote_heads(int in, st\n \n \t\tif (!check_ref(name, name_len, flags))\n \t\t\tcontinue;\n-\t\tif (nr_match && !path_match(name, nr_match, match))\n+\t\tif (nr_match && !path_match(name, NULL, nr_match, match, NULL))\n \t\t\tcontinue;\n \t\tref = xcalloc(1, sizeof(*ref) + len - 40);\n \t\thashcpy(ref->old_sha1, old_sha1);\n \t\tmemcpy(ref->name, buffer + 41, len - 40);\n+\t\tref->repo = repo;\n \t\t*list = ref;\n \t\tlist = &ref->next;\n \t}\n@@ -112,7 +112,8 @@ int get_ack(int fd, unsigned char *resul\n \tdie(\"git-fetch_pack: expected ACK/NAK, got '%s'\", line);\n }\n \n-int path_match(const char *path, int nr, char **match)\n+int path_match(const char *path, const char *repo, int nr, char **match,\n+               char **match_repo)\n {\n \tint i;\n \tint pathlen = strlen(path);\n@@ -121,6 +122,8 @@ int path_match(const char *path, int nr,\n \t\tchar *s = match[i];\n \t\tint len = strlen(s);\n \n+\t\tif (match_repo && repo != match_repo[i])\n+\t\t\tcontinue;\n \t\tif (!len || len > pathlen)\n \t\t\tcontinue;\n \t\tif (memcmp(path + pathlen - len, s, len))\ndiff --git a/daemon.c b/daemon.c\nindex eb4f3f1..d0b44b2 100644\n--- a/daemon.c\n+++ b/daemon.c\n@@ -321,8 +321,7 @@ static int run_service(struct interp *it\n \t * path_ok() uses enter_repo() and does whitelist checking.\n \t * We only need to make sure the repository is exported.\n \t */\n-\n-\tif (!export_all_trees && access(\"git-daemon-export-ok\", F_OK)) {\n+\tif (!security_repo_check(export_all_trees)) {\n \t\tlogerror(\"'%s': repository not exported.\", path);\n \t\terrno = EACCES;\n \t\treturn -1;\n@@ -355,11 +354,15 @@ static int upload_pack(void)\n {\n \t/* Timeout as string */\n \tchar timeout_buf[64];\n+\tchar checkexport_buf[64];\n \n \tsnprintf(timeout_buf, sizeof timeout_buf, \"--timeout=%u\", timeout);\n \n+\tstrcpy(checkexport_buf, \"--check-export\");\n+\tif (export_all_trees) checkexport_buf[1] = '\\0'; /* XXX */\n+\n \t/* git-upload-pack only ever reads stuff, so this is safe */\n-\texecl_git_cmd(\"upload-pack\", \"--strict\", timeout_buf, \".\", NULL);\n+\texecl_git_cmd(\"upload-pack\", \"--strict\", timeout_buf, checkexport_buf, \".\", NULL);\n \treturn -1;\n }\n \ndiff --git a/fetch-pack.c b/fetch-pack.c\nindex 99ac08b..edbdf19 100644\n--- a/fetch-pack.c\n+++ b/fetch-pack.c\n@@ -282,7 +282,8 @@ static void mark_recent_complete_commits\n \t}\n }\n \n-static void filter_refs(struct ref **refs, int nr_match, char **match)\n+static void filter_refs(struct ref **refs, int nr_match, char **match,\n+                        char **match_repo)\n {\n \tstruct ref **return_refs;\n \tstruct ref *newlist = NULL;\n@@ -313,7 +314,7 @@ static void filter_refs(struct ref **ref\n \t\t\tcontinue;\n \t\t}\n \t\telse {\n-\t\t\tint order = path_match(ref->name, nr_match, match);\n+\t\t\tint order = path_match(ref->name, ref->repo, nr_match, match, match_repo);\n \t\t\tif (order) {\n \t\t\t\treturn_refs[order-1] = ref;\n \t\t\t\tcontinue; /* we will link it later */\n@@ -338,7 +339,8 @@ static void filter_refs(struct ref **ref\n \t*refs = newlist;\n }\n \n-static int everything_local(struct ref **refs, int nr_match, char **match)\n+static int everything_local(struct ref **refs, int nr_match, char **match,\n+                            char **match_repo)\n {\n \tstruct ref *ref;\n \tint retval;\n@@ -387,7 +389,7 @@ static int everything_local(struct ref *\n \t\t}\n \t}\n \n-\tfilter_refs(refs, nr_match, match);\n+\tfilter_refs(refs, nr_match, match, match_repo);\n \n \tfor (retval = 1, ref = *refs; ref ; ref = ref->next) {\n \t\tconst unsigned char *remote = ref->old_sha1;\n@@ -415,13 +417,15 @@ static int everything_local(struct ref *\n \treturn retval;\n }\n \n-static int fetch_pack(int fd[2], int nr_match, char **match)\n+static int fetch_pack(int fd[2], int nr_repo, char **repo,\n+                      int nr_match, char **match, char **match_repo)\n {\n-\tstruct ref *ref;\n+\tstruct ref *ref = NULL, **refn;\n \tunsigned char sha1[20];\n \tint status;\n+\tint i;\n \n-\tget_remote_heads(fd[0], &ref, 0, NULL, 0);\n+\trefn = get_remote_heads(fd[0], &ref, 0, NULL, nr_repo ? repo[0] : NULL, 0);\n \tif (server_supports(\"multi_ack\")) {\n \t\tif (verbose)\n \t\t\tfprintf(stderr, \"Server supports multi_ack\\n\");\n@@ -437,11 +441,19 @@ static int fetch_pack(int fd[2], int nr_\n \t\t\tfprintf(stderr, \"Server supports side-band\\n\");\n \t\tuse_sideband = 1;\n \t}\n+\tif (!server_supports(\"switch-repo\") && nr_repo > 1)\n+\t\tdie(\"server does not support fetching multiple repositories at once\");\n+\n+\tfor (i = 1; i < nr_repo; i++) {\n+\t\tpacket_write(fd[1], \"switch-repo %s\", repo[i]);\n+\t\trefn = get_remote_heads(fd[0], refn, 0, NULL, repo[i], 0);\n+\t}\n+\n \tif (!ref) {\n \t\tpacket_flush(fd[1]);\n \t\tdie(\"no matching remote head\");\n \t}\n-\tif (everything_local(&ref, nr_match, match)) {\n+\tif (everything_local(&ref, nr_match, match, match_repo)) {\n \t\tpacket_flush(fd[1]);\n \t\tgoto all_done;\n \t}\n@@ -462,8 +474,9 @@ static int fetch_pack(int fd[2], int nr_\n \n  all_done:\n \twhile (ref) {\n-\t\tprintf(\"%s %s\\n\",\n-\t\t       sha1_to_hex(ref->old_sha1), ref->name);\n+\t\tprintf(\"%s %s%s%s\\n\",\n+\t\t       sha1_to_hex(ref->old_sha1), ref->name,\n+\t\t       ref->repo ? \" \" : \"\", ref->repo ? : \"\");\n \t\tref = ref->next;\n \t}\n \treturn 0;\n@@ -471,15 +484,16 @@ static int fetch_pack(int fd[2], int nr_\n \n int main(int argc, char **argv)\n {\n-\tint i, ret, nr_heads;\n-\tchar *dest = NULL, **heads;\n+\tint i, ret, nr_repo = 0, nr_heads = 0;\n+\tchar *dest = NULL;\n+\tchar **repo = xmalloc(argc * sizeof(*repo));\n+\tchar **heads = xmalloc(argc * sizeof(*heads));\n+\tchar **heads_repo = xmalloc(argc * sizeof(*heads_repo));\n \tint fd[2];\n \tpid_t pid;\n \n \tsetup_git_directory();\n \n-\tnr_heads = 0;\n-\theads = NULL;\n \tfor (i = 1; i < argc; i++) {\n \t\tchar *arg = argv[i];\n \n@@ -504,16 +518,22 @@ int main(int argc, char **argv)\n \t\t\t\tfetch_all = 1;\n \t\t\t\tcontinue;\n \t\t\t}\n+\t\t\tif (!strncmp(\"--repo=\", arg, 7)) {\n+\t\t\t\trepo[nr_repo++] = arg + 7;\n+\t\t\t\tcontinue;\n+\t\t\t}\n \t\t\tif (!strcmp(\"-v\", arg)) {\n \t\t\t\tverbose = 1;\n \t\t\t\tcontinue;\n \t\t\t}\n \t\t\tusage(fetch_pack_usage);\n \t\t}\n-\t\tdest = arg;\n-\t\theads = argv + i + 1;\n-\t\tnr_heads = argc - i - 1;\n-\t\tbreak;\n+\t\tif (!dest) {\n+\t\t\tdest = arg;\n+\t\t\tcontinue;\n+\t\t}\n+\t\theads_repo[nr_heads] = nr_repo ? repo[nr_repo - 1] : NULL;\n+\t\theads[nr_heads++] = arg;\n \t}\n \tif (!dest)\n \t\tusage(fetch_pack_usage);\n@@ -522,7 +542,7 @@ int main(int argc, char **argv)\n \tpid = git_connect(fd, dest, exec);\n \tif (pid < 0)\n \t\treturn 1;\n-\tret = fetch_pack(fd, nr_heads, heads);\n+\tret = fetch_pack(fd, nr_repo, repo, nr_heads, heads, heads_repo);\n \tclose(fd[0]);\n \tclose(fd[1]);\n \tret |= finish_connect(pid);\ndiff --git a/path.c b/path.c\nindex bb89fb0..d669cac 100644\n--- a/path.c\n+++ b/path.c\n@@ -251,6 +251,11 @@ char *enter_repo(char *path, int strict)\n \treturn NULL;\n }\n \n+int security_repo_check(int export_all_trees)\n+{\n+\treturn (export_all_trees || access(\"git-daemon-export-ok\", F_OK));\n+}\n+\n int adjust_shared_perm(const char *path)\n {\n \tstruct stat st;\ndiff --git a/peek-remote.c b/peek-remote.c\nindex 353da00..2cf9e50 100644\n--- a/peek-remote.c\n+++ b/peek-remote.c\n@@ -8,9 +8,9 @@ static const char *exec = \"git-upload-pa\n \n static int peek_remote(int fd[2], unsigned flags)\n {\n-\tstruct ref *ref;\n+\tstruct ref *ref = NULL;\n \n-\tget_remote_heads(fd[0], &ref, 0, NULL, flags);\n+\tget_remote_heads(fd[0], &ref, 0, NULL, NULL, flags);\n \tpacket_flush(fd[1]);\n \n \twhile (ref) {\ndiff --git a/refs.c b/refs.c\nindex 9b3986a..10e588e 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -144,12 +144,12 @@ nextitem:;\n \treturn list;\n }\n \n+int did_read_loose_refs = 0;\n static struct ref_list *get_loose_refs(void)\n {\n-\tstatic int did_refs = 0;\n \tstatic struct ref_list *refs = NULL;\n \n-\tif (!did_refs) {\n+\tif (!did_read_loose_refs) {\n \t\t/* We need to make sure refs/heads and refs/tags always\n \t\t * go before everything else. */\n \t\tchar *except[] = {\"refs/heads\", \"refs/tags\", NULL};\n@@ -167,7 +167,7 @@ static struct ref_list *get_loose_refs(v\n \t\t\trefs = other_refs;\n \t\t}\n \n-\t\tdid_refs = 1;\n+\t\tdid_read_loose_refs = 1;\n \t}\n \treturn refs;\n }\ndiff --git a/send-pack.c b/send-pack.c\nindex fbd792c..e1cf58b 100644\n--- a/send-pack.c\n+++ b/send-pack.c\n@@ -266,14 +266,14 @@ static int receive_status(int in)\n \n static int send_pack(int in, int out, int nr_refspec, char **refspec)\n {\n-\tstruct ref *ref;\n+\tstruct ref *ref = NULL;\n \tint new_refs;\n \tint ret = 0;\n \tint ask_for_status_report = 0;\n \tint expect_status_report = 0;\n \n \t/* No funny business with the matcher */\n-\tremote_tail = get_remote_heads(in, &remote_refs, 0, NULL, REF_NORMAL);\n+\tremote_tail = get_remote_heads(in, &remote_refs, 0, NULL, NULL, REF_NORMAL);\n \tget_local_heads();\n \n \t/* Does the other end support the reporting? */\ndiff --git a/sha1_file.c b/sha1_file.c\nindex 27b1ebb..14f06c9 100644\n--- a/sha1_file.c\n+++ b/sha1_file.c\n@@ -235,7 +235,7 @@ static void read_info_alternates(const c\n  * SHA1, an extra slash for the first level indirection, and the\n  * terminating NUL.\n  */\n-static int link_alt_odb_entry(const char * entry, int len, const char * relative_base, int depth)\n+int link_alt_odb_entry(const char * entry, int len, const char * relative_base, int depth, int allow_self)\n {\n \tstruct stat st;\n \tconst char *objdir = get_object_directory();\n@@ -285,7 +285,7 @@ static int link_alt_odb_entry(const char\n \t\t\treturn -1;\n \t\t}\n \t}\n-\tif (!memcmp(ent->base, objdir, pfxlen)) {\n+\tif (!allow_self && !memcmp(ent->base, objdir, pfxlen)) {\n \t\tfree(ent);\n \t\treturn -1;\n \t}\n@@ -331,7 +331,7 @@ static void link_alt_odb_entries(const c\n \t\t\t\t\t\trelative_base, last);\n \t\t\t} else {\n \t\t\t\tlink_alt_odb_entry(last, cp - last,\n-\t\t\t\t\t\trelative_base, depth);\n+\t\t\t\t\t\trelative_base, depth, 0);\n \t\t\t}\n \t\t}\n \t\twhile (cp < ep && *cp == sep)\ndiff --git a/upload-pack.c b/upload-pack.c\nindex 4e53edd..57ae127 100644\n--- a/upload-pack.c\n+++ b/upload-pack.c\n@@ -10,7 +10,7 @@ #include \"object.h\"\n #include \"commit.h\"\n #include \"exec_cmd.h\"\n \n-static const char upload_pack_usage[] = \"git-upload-pack [--strict] [--timeout=nn] <dir>\";\n+static const char upload_pack_usage[] = \"git-upload-pack [--strict] [--timeout=nn] [--check-export] <dir>\";\n \n /* bits #0..7 in revision.h, #8..10 in commit.c */\n #define THEY_HAVE\t(1u << 11)\n@@ -30,6 +30,8 @@ static unsigned int timeout;\n  * otherwise maximum packet size (up to 65520 bytes).\n  */\n static int use_sideband;\n+static int check_export;\n+static int strict;\n \n static void reset_timeout(void)\n {\n@@ -455,7 +457,7 @@ static int get_common_commits(void)\n \t}\n }\n \n-static void receive_needs(void)\n+static char *receive_needs(void)\n {\n \tstatic char line[1000];\n \tint len;\n@@ -466,8 +468,10 @@ static void receive_needs(void)\n \t\tlen = packet_read_line(0, line, sizeof(line));\n \t\treset_timeout();\n \t\tif (!len)\n-\t\t\treturn;\n+\t\t\treturn NULL;\n \n+\t\tif (!strncmp(\"switch-repo \", line, 12))\n+\t\t\treturn line+12;\n \t\tif (strncmp(\"want \", line, 5) ||\n \t\t    get_sha1_hex(line+5, sha1_buf))\n \t\t\tdie(\"git-upload-pack: protocol error, \"\n@@ -501,7 +505,7 @@ static void receive_needs(void)\n \n static int send_ref(const char *refname, const unsigned char *sha1, int flag, void *cb_data)\n {\n-\tstatic const char *capabilities = \"multi_ack thin-pack side-band side-band-64k\";\n+\tstatic const char *capabilities = \"multi_ack thin-pack side-band side-band-64k switch-repo\";\n \tstruct object *o = parse_object(sha1);\n \n \tif (!o)\n@@ -524,13 +528,60 @@ static int send_ref(const char *refname,\n \treturn 0;\n }\n \n+extern int did_read_loose_refs;\n static void upload_pack(void)\n {\n-\treset_timeout();\n-\thead_ref(send_ref, NULL);\n-\tfor_each_ref(send_ref, NULL);\n-\tpacket_flush(1);\n-\treceive_needs();\n+\tint multirepo = 0;\n+\n+\twhile (1) {\n+\t\tchar *repo;\n+\t\tchar cwd[PATH_MAX];\n+\n+\t\treset_timeout();\n+\t\thead_ref(send_ref, NULL);\n+\t\tfor_each_ref(send_ref, NULL);\n+\t\tpacket_flush(1);\n+\t\trepo = receive_needs();\n+\t\tif (!repo)\n+\t\t\t/* This is where we go most of the time, unless this is\n+\t\t\t * a multirepo fetch */\n+\t\t\tbreak;\n+\t\tmultirepo++;\n+\n+\t\tfprintf(stderr, \"git-upload-pack: switching to repo %s\\n\", repo);\n+\n+\t\t/* So that we still find objects of the original repository... */\n+\t\tgetcwd(cwd, PATH_MAX);\n+\t\tif (strlen(cwd) < PATH_MAX - 8)\n+\t\t\tstrcat(cwd, \"/objects\");\n+\t\tlink_alt_odb_entry(cwd, strlen(cwd), NULL, 0, 1);\n+\n+\t\tif (!enter_repo(repo, strict) || !security_repo_check(!check_export))\n+\t\t\tdie(\"git-upload-pack: security violation\");\n+\t\t/* This is a hack and a memory leak. If you are seeing this\n+\t\t * in a stock Git version, someone did a horrible mistake. */\n+\t\tdid_read_loose_refs = 0;\n+\t}\n+\n+\tif (multirepo) {\n+#define ALTENV_SIZE 65536\n+\t\t/* Propagate all the repositories to the children */\n+\t\tchar altenv[ALTENV_SIZE], *p = altenv;\n+\t\tstruct alternate_object_database *alt;\n+\t\tstrcpy(p, ALTERNATE_DB_ENVIRONMENT \"=\");\n+\t\tp += sizeof(ALTERNATE_DB_ENVIRONMENT);\n+\t\tfor (alt = alt_odb_list; alt; alt = alt->next) {\n+\t\t\tstrncpy(p, alt->base, alt->name - alt->base);\n+\t\t\tp += alt->name - alt->base;\n+\t\t\tif (p - altenv < ALTENV_SIZE)\n+\t\t\t\t*p++ = ':';\n+\t\t\tif (p - altenv >= ALTENV_SIZE)\n+\t\t\t\tdie(\"fetching too many repositories\");\n+\t\t}\n+\t\tp[-1] = '\\0';\n+\t\tputenv(altenv);\n+\t}\n+\n \tif (want_obj.nr) {\n \t\tget_common_commits();\n \t\tcreate_pack_file();\n@@ -541,7 +592,6 @@ int main(int argc, char **argv)\n {\n \tchar *dir;\n \tint i;\n-\tint strict = 0;\n \n \tfor (i = 1; i < argc; i++) {\n \t\tchar *arg = argv[i];\n@@ -556,6 +606,10 @@ int main(int argc, char **argv)\n \t\t\ttimeout = atoi(arg+10);\n \t\t\tcontinue;\n \t\t}\n+\t\tif (!strcmp(arg, \"--check-export\")) {\n+\t\t\tcheck_export = 1;\n+\t\t\tcontinue;\n+\t\t}\n \t\tif (!strcmp(arg, \"--\")) {\n \t\t\ti++;\n \t\t\tbreak;\n"},{"id":"27484","messageId":"45156780.6070801@gmail.com","threadId":"5663","inReplyTo":"20060923164308.16334.49252.stgit@machine.or.cz","subject":"Re: [RFC][RESEND][PATCH] Allow fetching from multiple repositories at once","fromName":"A Large Angry SCM","fromEmail":"gitzilla@gmail.com","sentAt":"2006-09-23T16:57:36Z","receivedAt":"2006-09-23T16:57:36Z","isPatch":true,"sender":{"key":"gitzilla@gmail.com","avatar":"https://gravatar.com/avatar/354625c442439908ff3dd99757dee330e29e9df7847472384faf7a00add247fb?d=mp&s=160"},"body":"Petr Baudis wrote:\n> This patch enables fetching multiple repositories at once over the Git\n> protocol (and SSH, and locally if git-fetch-pack is your cup of coffee\n> there). This is done especially for the xorg people who have tons of\n> repositories and dislike pulls much slower than they were used to with CVS.\n> I'm eager to hear how this affects the situation.\n> \n> It's kind of \"superproject\" thing, basically, taking reverse approach than\n> the subproject ideas. However, in practice I think it can be used for\n> subprojects quite well and perhaps if I find some spare time during the day\n> I will add the lightweight subproject support to Cogito, using this.\n> \n> So, you need some kind of porcelain for this. The idea is that instead of\n> telling git-fetch-pack a single repository, you pass multiple --repo=\n> parameters and refs always \"belong\" to the latest mentioned repository;\n> when outputting the new ref values, the appropriate repo is mentioned near\n> each ref. In order for this to be useful, on your local side you should\n> share the objects database in some way - either using alternates (then\n> you must fetch to an object database reachable from everywhere) or symlinked\n> object databases.\n> \n> You still need to pass git-fetch-pack some URL in addition to the\n> repositories - it is used only for git_connect(), the purpose is that\n> repositories must be local directories so if you want to talk remote, you\n> need to do something like\n> \n> \tgit-fetch-pack git://kernel.org/pub/scm/git/git.git --repo=/pub/scm/git/git.git master next --repo=/pub/scm/cogito/cogito.git master\n> \n\nCan't this be done with a simple wrapper around git-fetch* ?\n"},{"id":"27485","messageId":"20060923170439.GQ8259@pasky.or.cz","threadId":"5663","inReplyTo":"45156780.6070801@gmail.com","subject":"Re: [RFC][RESEND][PATCH] Allow fetching from multiple repositories at once","fromName":"Petr Baudis","fromEmail":"pasky@suse.cz","sentAt":"2006-09-23T17:04:39Z","receivedAt":"2006-09-23T17:04:39Z","isPatch":true,"sender":{"key":"pasky@ucw.cz","avatar":"https://avatars.githubusercontent.com/u/18439?v=4"},"body":"Dear diary, on Sat, Sep 23, 2006 at 06:57:36PM CEST, I got a letter\nwhere A Large Angry SCM <gitzilla@gmail.com> said that...\n> Petr Baudis wrote:\n> >You still need to pass git-fetch-pack some URL in addition to the\n> >repositories - it is used only for git_connect(), the purpose is that\n> >repositories must be local directories so if you want to talk remote, you\n> >need to do something like\n> >\n> >\tgit-fetch-pack git://kernel.org/pub/scm/git/git.git \n> >\t--repo=/pub/scm/git/git.git master next \n> >\t--repo=/pub/scm/cogito/cogito.git master\n> >\n> \n> Can't this be done with a simple wrapper around git-fetch* ?\n\nThe point is that that is too slow (the difference is noticeable\nespeciall in the case there is no new stuff ready), you need to create a\nconnection and do whole lot of work for each of the repositories.\n\n-- \n\t\t\t\tPetr \"Pasky\" Baudis\nStuff: http://pasky.or.cz/\n#!/bin/perl -sp0777i<X+d*lMLa^*lN%0]dsXx++lMlN/dsM0<j]dsj\n$/=unpack('H*',$_);$_=`echo 16dio\\U$k\"SK$/SM$n\\EsN0p[lN*1\nlK[d2%Sa2/d0$^Ixp\"|dc`;s/\\W//g;$_=pack('H*',/((..)*)$/)\n"},{"id":"27486","messageId":"20060923170621.GR8259@pasky.or.cz","threadId":"5663","inReplyTo":"20060923164308.16334.49252.stgit@machine.or.cz","subject":"Re: [RFC][RESEND][PATCH] Allow fetching from multiple repositories at once","fromName":"Petr Baudis","fromEmail":"pasky@suse.cz","sentAt":"2006-09-23T17:06:21Z","receivedAt":"2006-09-23T17:06:21Z","isPatch":true,"sender":{"key":"pasky@ucw.cz","avatar":"https://avatars.githubusercontent.com/u/18439?v=4"},"body":"Dear diary, on Sat, Sep 23, 2006 at 06:45:09PM CEST, I got a letter\nwhere Petr Baudis <pasky@suse.cz> said that...\n> You still need to pass git-fetch-pack some URL in addition to the\n> repositories - it is used only for git_connect(), the purpose is that\n> repositories must be local directories so if you want to talk remote, you\n> need to do something like\n> \n> \tgit-fetch-pack git://kernel.org/pub/scm/git/git.git --repo=/pub/scm/git/git.git master next --repo=/pub/scm/cogito/cogito.git master\n..snip..\n> The main target of this are the Xorg people who have plenty of small repos\n> and frequently want to update many of them. It would be nice if someone\n> from Xorg reading this and concerned with this could measure the\n> difference.\n\nAttached is a simple porcelain script for this.\n\n-- \n\t\t\t\tPetr \"Pasky\" Baudis\nStuff: http://pasky.or.cz/\n#!/bin/perl -sp0777i<X+d*lMLa^*lN%0]dsXx++lMlN/dsM0<j]dsj\n$/=unpack('H*',$_);$_=`echo 16dio\\U$k\"SK$/SM$n\\EsN0p[lN*1\nlK[d2%Sa2/d0$^Ixp\"|dc`;s/\\W//g;$_=pack('H*',/((..)*)$/)\n\n\n#!/usr/bin/perl\nuse warnings;\nuse strict;\n\n# Remember that you must ensure the obj database is shared - either symlink it\n# or setup alternates!\n#my $remoteurl = 'git://git.kernel.org/pub/scm/git/git.git';\n#my %config = (\n#\t'/pub/scm/git/git.git' => {\n#\t\t'next' => {\n#\t\t\t'/home/xpasky/q/gg' => 'origin'\n#\t\t},\n#\t\t'master' => {\n#\t\t\t'/home/xpasky/q/gg' => 'origin2',\n#\t\t\t'/home/xpasky/q/gg2' => 'origin'\n#\t\t}\n#\t},\n#\t'/pub/scm/cogito/cogito-doc.git' => {\n#\t\tmaster => {\n#\t\t\t'/home/xpasky/q/gg2' => 'origin2'\n#\t\t}\n#\t}\n#);\nmy $remoteurl = 'puturlofremotehosthere';\nmy %config = (\n\t'remoterepo1' => {\n\t\tbranch1 => {\n\t\t\t'localrepo1' => 'origin'\n\t\t},\n\t\tbranch2 => {\n\t\t\t'localrepo1' => 'origin2',\n\t\t\t'localrepo2' => 'origin'\n\t\t}\n\t},\n\t'remoterepo2' => {\n\t\tmaster => {\n\t\t\t'localrepo2' => 'origin2'\n\t\t}\n\t}\n);\n\nmy @args = ($remoteurl);\nforeach my $repo (keys %config) {\n\tpush (@args, '--repo='.$repo);\n\tforeach my $branch (keys %{$config{$repo}}) {\n\t\tpush (@args, $branch);\n\t}\n}\n\nopen (F, '-|', 'git-fetch-pack', @args) or die \"$!\";\nwhile (<F>) {\n\tchomp;\n\tsplit / /, $_;\n\tmy ($sha, $ref, $repo) = @_;\n\t$ref =~ s#^refs/heads/##;\n\tforeach my $lrepo (keys %{$config{$repo}->{$ref}}) {\n\t\tsystem(\"GIT_DIR=$lrepo git-update-ref $config{$repo}->{$ref}->{$lrepo} $sha\");\n\t}\n}\nclose (F);\n"},{"id":"27489","messageId":"Pine.LNX.4.63.0609231921110.25853@wbgn013.biozentrum.uni-wuerzburg.de","threadId":"5663","inReplyTo":"20060923170439.GQ8259@pasky.or.cz","subject":"Re: [RFC][RESEND][PATCH] Allow fetching from multiple repositories at once","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2006-09-23T17:23:01Z","receivedAt":"2006-09-23T17:23:01Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Sat, 23 Sep 2006, Petr Baudis wrote:\n\n> Dear diary, on Sat, Sep 23, 2006 at 06:57:36PM CEST, I got a letter\n> where A Large Angry SCM <gitzilla@gmail.com> said that...\n> > Petr Baudis wrote:\n> > >You still need to pass git-fetch-pack some URL in addition to the\n> > >repositories - it is used only for git_connect(), the purpose is that\n> > >repositories must be local directories so if you want to talk remote, you\n> > >need to do something like\n> > >\n> > >\tgit-fetch-pack git://kernel.org/pub/scm/git/git.git \n> > >\t--repo=/pub/scm/git/git.git master next \n> > >\t--repo=/pub/scm/cogito/cogito.git master\n> > >\n> > \n> > Can't this be done with a simple wrapper around git-fetch* ?\n> \n> The point is that that is too slow (the difference is noticeable\n> especiall in the case there is no new stuff ready), you need to create a\n> connection and do whole lot of work for each of the repositories.\n\nI still firmly believe that it would be way more efficient to fetch all \nthose branches into _one_ proxy repository. Especially since you can reuse \nthe objects with an alternate, which has an additional benefit over your \napproach.\n\nAnd _then_, you can make a wrapper script, which guarantees that the \nalternate is set, and that the ref is actually fetched from the proxy.\n\nCiao,\nDscho\n"},{"id":"27490","messageId":"20060923173943.GA20017@pasky.or.cz","threadId":"5663","inReplyTo":"Pine.LNX.4.63.0609231921110.25853@wbgn013.biozentrum.uni-wuerzburg.de","subject":"Re: [RFC][RESEND][PATCH] Allow fetching from multiple repositories at once","fromName":"Petr Baudis","fromEmail":"pasky@suse.cz","sentAt":"2006-09-23T17:39:43Z","receivedAt":"2006-09-23T17:39:43Z","isPatch":true,"sender":{"key":"pasky@ucw.cz","avatar":"https://avatars.githubusercontent.com/u/18439?v=4"},"body":"  Hi,\n\nDear diary, on Sat, Sep 23, 2006 at 07:23:01PM CEST, I got a letter\nwhere Johannes Schindelin <Johannes.Schindelin@gmx.de> said that...\n> On Sat, 23 Sep 2006, Petr Baudis wrote:\n> \n> > Dear diary, on Sat, Sep 23, 2006 at 06:57:36PM CEST, I got a letter\n> > where A Large Angry SCM <gitzilla@gmail.com> said that...\n> > > Petr Baudis wrote:\n> > > >You still need to pass git-fetch-pack some URL in addition to the\n> > > >repositories - it is used only for git_connect(), the purpose is that\n> > > >repositories must be local directories so if you want to talk remote, you\n> > > >need to do something like\n> > > >\n> > > >\tgit-fetch-pack git://kernel.org/pub/scm/git/git.git \n> > > >\t--repo=/pub/scm/git/git.git master next \n> > > >\t--repo=/pub/scm/cogito/cogito.git master\n> > > >\n> > > \n> > > Can't this be done with a simple wrapper around git-fetch* ?\n> > \n> > The point is that that is too slow (the difference is noticeable\n> > especiall in the case there is no new stuff ready), you need to create a\n> > connection and do whole lot of work for each of the repositories.\n> \n> I still firmly believe that it would be way more efficient to fetch all \n> those branches into _one_ proxy repository. Especially since you can reuse \n> the objects with an alternate, which has an additional benefit over your \n> approach.\n\n  Huh? You can reuse the objects with my approach as well. Actually, it\nis automagically done so.\n\n  With proxy repository, you would still need a server-side setup to\nmaintain that repository, and specialized client-side porcelain to fetch\nfrom it. My approach initially requires some core changes (which aren't\nvery pretty as it is but are not very fundamental or logically intrusive\neither) but in the longer run it pays off since you don't need a\nconvoluted server-side setup for that.\n\n-- \n\t\t\t\tPetr \"Pasky\" Baudis\nStuff: http://pasky.or.cz/\n#!/bin/perl -sp0777i<X+d*lMLa^*lN%0]dsXx++lMlN/dsM0<j]dsj\n$/=unpack('H*',$_);$_=`echo 16dio\\U$k\"SK$/SM$n\\EsN0p[lN*1\nlK[d2%Sa2/d0$^Ixp\"|dc`;s/\\W//g;$_=pack('H*',/((..)*)$/)\n"},{"id":"27492","messageId":"Pine.LNX.4.63.0609231947340.25853@wbgn013.biozentrum.uni-wuerzburg.de","threadId":"5663","inReplyTo":"20060923173943.GA20017@pasky.or.cz","subject":"Re: [RFC][RESEND][PATCH] Allow fetching from multiple repositories at once","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2006-09-23T17:54:10Z","receivedAt":"2006-09-23T17:54:10Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Sat, 23 Sep 2006, Petr Baudis wrote:\n\n> Dear diary, on Sat, Sep 23, 2006 at 07:23:01PM CEST, I got a letter\n> where Johannes Schindelin <Johannes.Schindelin@gmx.de> said that...\n> > On Sat, 23 Sep 2006, Petr Baudis wrote:\n> > \n> > > Dear diary, on Sat, Sep 23, 2006 at 06:57:36PM CEST, I got a letter\n> > > where A Large Angry SCM <gitzilla@gmail.com> said that...\n> > > > Petr Baudis wrote:\n> > > > >You still need to pass git-fetch-pack some URL in addition to the\n> > > > >repositories - it is used only for git_connect(), the purpose is that\n> > > > >repositories must be local directories so if you want to talk remote, you\n> > > > >need to do something like\n> > > > >\n> > > > >\tgit-fetch-pack git://kernel.org/pub/scm/git/git.git \n> > > > >\t--repo=/pub/scm/git/git.git master next \n> > > > >\t--repo=/pub/scm/cogito/cogito.git master\n> > > > >\n> > > > \n> > > > Can't this be done with a simple wrapper around git-fetch* ?\n> > > \n> > > The point is that that is too slow (the difference is noticeable\n> > > especiall in the case there is no new stuff ready), you need to create a\n> > > connection and do whole lot of work for each of the repositories.\n> > \n> > I still firmly believe that it would be way more efficient to fetch all \n> > those branches into _one_ proxy repository. Especially since you can reuse \n> > the objects with an alternate, which has an additional benefit over your \n> > approach.\n> \n>   Huh? You can reuse the objects with my approach as well. Actually, it\n> is automagically done so.\n> \n>   With proxy repository, you would still need a server-side setup to\n> maintain that repository, and specialized client-side porcelain to fetch\n> from it. My approach initially requires some core changes (which aren't\n> very pretty as it is but are not very fundamental or logically intrusive\n> either) but in the longer run it pays off since you don't need a\n> convoluted server-side setup for that.\n\nNo, you do not need _any_ server-side setup. And you do not need any \nspecialized client-side porcelain other than a script, which just does \nthe job.\n\nThe point is, your changes are not only intrusive, they are unnecessary.\n\nYou can even declare the first repository as the proxy. Sketch of script:\n\n\t- find which branches are wanted at all,\n\n\t- do the fetch (of all wanted branches) in the first repository,\n\t  storing the refs under refs/multi/,\n\n\t- for each repository\n\n\t\t- ensure that the first repo is given as alternative\n\n\t\t- store the branches this repo wants (you only need to \n\t\t  update the refs, since you have all the objects)\n\nThis is not particularly complicated, is it?\n\nCiao,\nDscho\n"},{"id":"27493","messageId":"20060923180539.GC20017@pasky.or.cz","threadId":"5663","inReplyTo":"Pine.LNX.4.63.0609231947340.25853@wbgn013.biozentrum.uni-wuerzburg.de","subject":"Re: [RFC][RESEND][PATCH] Allow fetching from multiple repositories at once","fromName":"Petr Baudis","fromEmail":"pasky@suse.cz","sentAt":"2006-09-23T18:05:39Z","receivedAt":"2006-09-23T18:05:39Z","isPatch":true,"sender":{"key":"pasky@ucw.cz","avatar":"https://avatars.githubusercontent.com/u/18439?v=4"},"body":"  Hi!\n\nDear diary, on Sat, Sep 23, 2006 at 07:54:10PM CEST, I got a letter\nwhere Johannes Schindelin <Johannes.Schindelin@gmx.de> said that...\n> On Sat, 23 Sep 2006, Petr Baudis wrote:\n> \n> > Dear diary, on Sat, Sep 23, 2006 at 07:23:01PM CEST, I got a letter\n> > where Johannes Schindelin <Johannes.Schindelin@gmx.de> said that...\n> > > I still firmly believe that it would be way more efficient to fetch all \n> > > those branches into _one_ proxy repository. Especially since you can reuse \n> > > the objects with an alternate, which has an additional benefit over your \n> > > approach.\n> > \n> >   Huh? You can reuse the objects with my approach as well. Actually, it\n> > is automagically done so.\n> > \n> >   With proxy repository, you would still need a server-side setup to\n> > maintain that repository, and specialized client-side porcelain to fetch\n> > from it. My approach initially requires some core changes (which aren't\n> > very pretty as it is but are not very fundamental or logically intrusive\n> > either) but in the longer run it pays off since you don't need a\n> > convoluted server-side setup for that.\n> \n> No, you do not need _any_ server-side setup. And you do not need any \n> specialized client-side porcelain other than a script, which just does \n> the job.\n\n  So how should the proxy repository be set up at the server side? I can\njust imagine a cronjob periodically sweeping the repositories and\npopulate the proxy repository with new stuff (refs and objects) from\nthose.\n\n  Looking at what you wrote again, you talk about fetching all those\nbranches _into_ one proxy repository. Perhaps we misunderstand each\nother. This patch is not about the \"into\" part, it is about being able\nto fetch _FROM_ _multiple_ repositories.\n\n-- \n\t\t\t\tPetr \"Pasky\" Baudis\nStuff: http://pasky.or.cz/\n#!/bin/perl -sp0777i<X+d*lMLa^*lN%0]dsXx++lMlN/dsM0<j]dsj\n$/=unpack('H*',$_);$_=`echo 16dio\\U$k\"SK$/SM$n\\EsN0p[lN*1\nlK[d2%Sa2/d0$^Ixp\"|dc`;s/\\W//g;$_=pack('H*',/((..)*)$/)\n"},{"id":"27495","messageId":"Pine.LNX.4.63.0609232007550.25853@wbgn013.biozentrum.uni-wuerzburg.de","threadId":"5663","inReplyTo":"20060923180539.GC20017@pasky.or.cz","subject":"Re: [RFC][RESEND][PATCH] Allow fetching from multiple repositories at once","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2006-09-23T18:14:28Z","receivedAt":"2006-09-23T18:14:28Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Sat, 23 Sep 2006, Petr Baudis wrote:\n\n>   Hi!\n> \n> Dear diary, on Sat, Sep 23, 2006 at 07:54:10PM CEST, I got a letter\n> where Johannes Schindelin <Johannes.Schindelin@gmx.de> said that...\n> > On Sat, 23 Sep 2006, Petr Baudis wrote:\n> > \n> > > Dear diary, on Sat, Sep 23, 2006 at 07:23:01PM CEST, I got a letter\n> > > where Johannes Schindelin <Johannes.Schindelin@gmx.de> said that...\n> > > > I still firmly believe that it would be way more efficient to fetch all \n> > > > those branches into _one_ proxy repository. Especially since you can reuse \n> > > > the objects with an alternate, which has an additional benefit over your \n> > > > approach.\n> > > \n> > >   Huh? You can reuse the objects with my approach as well. Actually, it\n> > > is automagically done so.\n> > > \n> > >   With proxy repository, you would still need a server-side setup to\n> > > maintain that repository, and specialized client-side porcelain to fetch\n> > > from it. My approach initially requires some core changes (which aren't\n> > > very pretty as it is but are not very fundamental or logically intrusive\n> > > either) but in the longer run it pays off since you don't need a\n> > > convoluted server-side setup for that.\n> > \n> > No, you do not need _any_ server-side setup. And you do not need any \n> > specialized client-side porcelain other than a script, which just does \n> > the job.\n> \n>   So how should the proxy repository be set up at the server side? I can\n> just imagine a cronjob periodically sweeping the repositories and\n> populate the proxy repository with new stuff (refs and objects) from\n> those.\n> \n>   Looking at what you wrote again, you talk about fetching all those\n> branches _into_ one proxy repository. Perhaps we misunderstand each\n> other. This patch is not about the \"into\" part, it is about being able\n> to fetch _FROM_ _multiple_ repositories.\n\nIndeed, I misunderstood.\n\nHmmm. Did you test if this makes things better? If so, could you test with \none proxy repo on the server? I bet that this would be even more \nefficient.\n\nAnd the proxy repo would be updated best by a hook, not a cronjob.\n\nThe thing is, your patch optimizes for a very special case, which special \ncase IMHO should be handled differently to begin with.\n\nCiao,\nDscho\n"},{"id":"27497","messageId":"ef3vb6$69d$1@sea.gmane.org","threadId":"5663","inReplyTo":"20060923164308.16334.49252.stgit@machine.or.cz","subject":"Re: [RFC][RESEND][PATCH] Allow fetching from multiple repositories at once","fromName":"Jakub Narebski","fromEmail":"jnareb@gmail.com","sentAt":"2006-09-23T18:45:01Z","receivedAt":"2006-09-23T18:45:01Z","isPatch":true,"sender":{"key":"jnareb@gmail.com","avatar":"https://avatars.githubusercontent.com/u/2706?v=4"},"body":"Petr Baudis wrote:\n\n> This patch enables fetching multiple repositories at once over the Git\n> protocol (and SSH, and locally if git-fetch-pack is your cup of coffee\n> there). This is done especially for the xorg people who have tons of\n> repositories and dislike pulls much slower than they were used to with CVS.\n> I'm eager to hear how this affects the situation.\n[...]\n\nI see that objects got to the one shared object database. Where the heads\n(refs) go to? Separate projects have separate ref spaces, even if they share\nobject database...\n-- \nJakub Narebski\nWarsaw, Poland\nShadeHawk on #git\n"},{"id":"27499","messageId":"20060923191439.GD20017@pasky.or.cz","threadId":"5663","inReplyTo":"ef3vb6$69d$1@sea.gmane.org","subject":"Re: [RFC][RESEND][PATCH] Allow fetching from multiple repositories at once","fromName":"Petr Baudis","fromEmail":"pasky@suse.cz","sentAt":"2006-09-23T19:14:39Z","receivedAt":"2006-09-23T19:14:39Z","isPatch":true,"sender":{"key":"pasky@ucw.cz","avatar":"https://avatars.githubusercontent.com/u/18439?v=4"},"body":"Dear diary, on Sat, Sep 23, 2006 at 08:45:01PM CEST, I got a letter\nwhere Jakub Narebski <jnareb@gmail.com> said that...\n> Petr Baudis wrote:\n> \n> > This patch enables fetching multiple repositories at once over the Git\n> > protocol (and SSH, and locally if git-fetch-pack is your cup of coffee\n> > there). This is done especially for the xorg people who have tons of\n> > repositories and dislike pulls much slower than they were used to with CVS.\n> > I'm eager to hear how this affects the situation.\n> [...]\n> \n> I see that objects got to the one shared object database. Where the heads\n> (refs) go to? Separate projects have separate ref spaces, even if they share\n> object database...\n\ngit-fetch-pack actually does not save the refs at all, that's the job of\nthe porcelain (either git-fetch or cg-fetch). git-fetch-pack just emits\nthe refs list on stdout. See the script in a followup for an example of\na multirepo-aware porcelain wrapper.\n\n-- \n\t\t\t\tPetr \"Pasky\" Baudis\nStuff: http://pasky.or.cz/\n#!/bin/perl -sp0777i<X+d*lMLa^*lN%0]dsXx++lMlN/dsM0<j]dsj\n$/=unpack('H*',$_);$_=`echo 16dio\\U$k\"SK$/SM$n\\EsN0p[lN*1\nlK[d2%Sa2/d0$^Ixp\"|dc`;s/\\W//g;$_=pack('H*',/((..)*)$/)\n"},{"id":"27500","messageId":"20060923191727.GE20017@pasky.or.cz","threadId":"5663","inReplyTo":"Pine.LNX.4.63.0609232007550.25853@wbgn013.biozentrum.uni-wuerzburg.de","subject":"Re: [RFC][RESEND][PATCH] Allow fetching from multiple repositories at once","fromName":"Petr Baudis","fromEmail":"pasky@suse.cz","sentAt":"2006-09-23T19:17:27Z","receivedAt":"2006-09-23T19:17:27Z","isPatch":true,"sender":{"key":"pasky@ucw.cz","avatar":"https://avatars.githubusercontent.com/u/18439?v=4"},"body":"  Hi,\n\nDear diary, on Sat, Sep 23, 2006 at 08:14:28PM CEST, I got a letter\nwhere Johannes Schindelin <Johannes.Schindelin@gmx.de> said that...\n> Hmmm. Did you test if this makes things better? If so, could you test with \n> one proxy repo on the server? I bet that this would be even more \n> efficient.\n\n  I didn't test it but it _has_ to... ;-))\n\n  I will try to do some real benchmarks (repo.or.cz might come handy\nhere again) but I don't know when will I get to it. I will try the proxy\nsetup as well.\n\n> And the proxy repo would be updated best by a hook, not a cronjob.\n\n  You would have a race window there.\n\n-- \n\t\t\t\tPetr \"Pasky\" Baudis\nStuff: http://pasky.or.cz/\n#!/bin/perl -sp0777i<X+d*lMLa^*lN%0]dsXx++lMlN/dsM0<j]dsj\n$/=unpack('H*',$_);$_=`echo 16dio\\U$k\"SK$/SM$n\\EsN0p[lN*1\nlK[d2%Sa2/d0$^Ixp\"|dc`;s/\\W//g;$_=pack('H*',/((..)*)$/)\n"},{"id":"27502","messageId":"7vpsdms7ew.fsf@assigned-by-dhcp.cox.net","threadId":"5663","inReplyTo":"Pine.LNX.4.63.0609232007550.25853@wbgn013.biozentrum.uni-wuerzburg.de","subject":"Re: [RFC][RESEND][PATCH] Allow fetching from multiple repositories at once","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2006-09-23T19:35:19Z","receivedAt":"2006-09-23T19:35:19Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:\n\n> The thing is, your patch optimizes for a very special case, which special \n> case IMHO should be handled differently to begin with.\n\nI share the feeling, and the usual URL while spelling out the\nrepository name but used only for the site name, because other\nspecial parameters are used to override the repository path\nwithin the site, is just too ugly.\n"}]}