{"thread":{"id":"27448","subject":"[PATCHv2 1/2] Support multiple virtual repositories with a single object store and refs","startedAt":"2011-05-24T21:54:25Z","lastAt":"2011-05-26T04:08:21Z","messageCount":11,"participants":["Jamey Sharp","Junio C Hamano","Johannes Schindelin","Josh Triplett"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"168604","messageId":"1306274066-4092-1-git-send-email-jamey@minilop.net","threadId":"27448","inReplyTo":null,"subject":"[PATCHv2 1/2] Support multiple virtual repositories with a single object store and refs","fromName":"Jamey Sharp","fromEmail":"jamey@minilop.net","sentAt":"2011-05-24T21:54:25Z","receivedAt":"2011-05-24T21:54:25Z","isPatch":false,"sender":{"key":"jamey@minilop.net","avatar":"https://gravatar.com/avatar/979ed4f9190c19c13a84945afe9a6cf141e8884ef6ddaae342db7a74ae28828e?d=mp&s=160"},"body":"Given many repositories with copies of the same objects (such as\nbranches of the same source), sharing a common object store will avoid\nduplication. Alternates provide a single baseline, but don't handle\nongoing activity in the various repositories. Git safely handles\nconcurrent accesses to the same object store across repositories, but\noperations such as gc need to know about all of the refs.\n\nThis change adds support in upload-pack and receive-pack to simulate\nmultiple virtual repositories within the object store and references of\na single underlying repository. The refs and heads of the virtual\nrepositories get stored in the underlying repository using prefixed\nnames specified by the --ref-prefix and --head options; for instance,\n--ref-prefix=repo1/ will use refs/repo1/heads/* and refs/repo1/tags/*.\nupload-pack and receive-pack will not expose any references that do not\nmatch the specified prefix.\n\nThese options implement the underlying mechanism for virtual\nrepositories; the higher-level protocol handler (such as http-backend or\na custom server) can pass these options when invoking upload-pack or\nreceive-pack, providing values based on components of the repository\npath. For a simple local test, git-remote-ext works:\n\ngit clone ext::'git %s --ref-prefix=prefix/ --head=prefix-HEAD /tmp/prefixed.git'\n\nCommit by Josh Triplett and Jamey Sharp.\nSigned-off-by: Josh Triplett \nSigned-off-by: Jamey Sharp \nCc: Shawn O. Pearce \nCc: Johannes Schindelin \nCc: Johannes Sixt \n---\nv2: remove accidentally-included debug message; and add patch 2/2 for\n git-http-backend.\n\n builtin/receive-pack.c | 36 +++++++++++++++++++++++++++---------\n upload-pack.c | 34 +++++++++++++++++++++++++++-------\n 2 files changed, 54 insertions(+), 16 deletions(-)\n\ndiff --git a/builtin/receive-pack.c b/builtin/receive-pack.c\nindex e1ba4dc..76dacd0 100644\n--- a/builtin/receive-pack.c\n+++ b/builtin/receive-pack.c\n@@ -34,6 +34,8 @@ static int prefer_ofs_delta = 1;\n static int auto_update_server_info;\n static int auto_gc = 1;\n static const char *head_name;\n+static const char *head_path = \"HEAD\";\n+static const char *ref_prefix = \"refs/\";\n static int sent_capabilities;\n \n static enum deny_action parse_deny_action(const char *var, const char *value)\n@@ -108,11 +110,12 @@ static int receive_pack_config(const char *var, const char *value, void *cb)\n \n static int show_ref(const char *path, const unsigned char *sha1, int flag, void *cb_data)\n {\n+\tconst char *refnameprefix = cb_data;\n \tif (sent_capabilities)\n-\t\tpacket_write(1, \"%s %s\\n\", sha1_to_hex(sha1), path);\n+\t\tpacket_write(1, \"%s %s%s\\n\", sha1_to_hex(sha1), refnameprefix, path);\n \telse\n-\t\tpacket_write(1, \"%s %s%c%s%s\\n\",\n-\t\t\t sha1_to_hex(sha1), path, 0,\n+\t\tpacket_write(1, \"%s %s%s%c%s%s\\n\",\n+\t\t\t sha1_to_hex(sha1), refnameprefix, path, 0,\n \t\t\t \" report-status delete-refs side-band-64k\",\n \t\t\t prefer_ofs_delta ? \" ofs-delta\" : \"\");\n \tsent_capabilities = 1;\n@@ -121,9 +124,9 @@ static int show_ref(const char *path, const unsigned char *sha1, int flag, void\n \n static void write_head_info(void)\n {\n-\tfor_each_ref(show_ref, NULL);\n+\tfor_each_ref_in(ref_prefix, show_ref, \"refs/\");\n \tif (!sent_capabilities)\n-\t\tshow_ref(\"capabilities^{}\", null_sha1, 0, NULL);\n+\t\tshow_ref(\"capabilities^{}\", null_sha1, 0, \"\");\n \n }\n \n@@ -332,6 +335,8 @@ static void refuse_unconfigured_deny_delete_current(void)\n static const char *update(struct command *cmd)\n {\n \tconst char *name = cmd->ref_name;\n+\tstruct strbuf prefixed_name_buf = STRBUF_INIT;\n+\tconst char *prefixed_name;\n \tunsigned char *old_sha1 = cmd->old_sha1;\n \tunsigned char *new_sha1 = cmd->new_sha1;\n \tstruct ref_lock *lock;\n@@ -342,7 +347,10 @@ static const char *update(struct command *cmd)\n \t\treturn \"funny refname\";\n \t}\n \n-\tif (is_ref_checked_out(name)) {\n+\tstrbuf_addf(&prefixed_name_buf, \"%s%s\", ref_prefix, name + 5);\n+\tprefixed_name = strbuf_detach(&prefixed_name_buf, NULL);\n+\n+\tif (is_ref_checked_out(prefixed_name)) {\n \t\tswitch (deny_current_branch) {\n \t\tcase DENY_IGNORE:\n \t\t\tbreak;\n@@ -370,7 +378,7 @@ static const char *update(struct command *cmd)\n \t\t\treturn \"deletion prohibited\";\n \t\t}\n \n-\t\tif (!strcmp(name, head_name)) {\n+\t\tif (!strcmp(prefixed_name, head_name)) {\n \t\t\tswitch (deny_delete_current) {\n \t\t\tcase DENY_IGNORE:\n \t\t\t\tbreak;\n@@ -426,14 +434,14 @@ static const char *update(struct command *cmd)\n \t\t\trp_warning(\"Allowing deletion of corrupt ref.\");\n \t\t\told_sha1 = NULL;\n \t\t}\n-\t\tif (delete_ref(name, old_sha1, 0)) {\n+\t\tif (delete_ref(prefixed_name, old_sha1, 0)) {\n \t\t\trp_error(\"failed to delete %s\", name);\n \t\t\treturn \"failed to delete\";\n \t\t}\n \t\treturn NULL; /* good */\n \t}\n \telse {\n-\t\tlock = lock_any_ref_for_update(name, old_sha1, 0);\n+\t\tlock = lock_any_ref_for_update(prefixed_name, old_sha1, 0);\n \t\tif (!lock) {\n \t\t\trp_error(\"failed to lock %s\", name);\n \t\t\treturn \"failed to lock\";\n@@ -760,6 +768,16 @@ int cmd_receive_pack(int argc, const char **argv, const char *prefix)\n \t\t\t\tadvertise_refs = 1;\n \t\t\t\tcontinue;\n \t\t\t}\n+\t\t\tif (!prefixcmp(arg, \"--head=\")) {\n+\t\t\t\thead_path = arg+7;\n+\t\t\t\tcontinue;\n+\t\t\t}\n+\t\t\tif (!prefixcmp(arg, \"--ref-prefix=\")) {\n+\t\t\t\tstruct strbuf prefixbuf = STRBUF_INIT;\n+\t\t\t\tstrbuf_addf(&prefixbuf, \"refs/%s\", arg+13);\n+\t\t\t\tref_prefix = strbuf_detach(&prefixbuf, NULL);\n+\t\t\t\tcontinue;\n+\t\t\t}\n \t\t\tif (!strcmp(arg, \"--stateless-rpc\")) {\n \t\t\t\tstateless_rpc = 1;\n \t\t\t\tcontinue;\ndiff --git a/upload-pack.c b/upload-pack.c\nindex ce5cbbe..a1e495f 100644\n--- a/upload-pack.c\n+++ b/upload-pack.c\n@@ -34,6 +34,8 @@ static int shallow_nr;\n static struct object_array have_obj;\n static struct object_array want_obj;\n static struct object_array extra_edge_obj;\n+static const char *head_path = \"HEAD\";\n+static const char *ref_prefix = \"\";\n static unsigned int timeout;\n /* 0 for no sideband,\n * otherwise maximum packet size (up to 65520 bytes).\n@@ -640,17 +642,18 @@ static int send_ref(const char *refname, const unsigned char *sha1, int flag, vo\n \tstatic const char *capabilities = \"multi_ack thin-pack side-band\"\n \t\t\" side-band-64k ofs-delta shallow no-progress\"\n \t\t\" include-tag multi_ack_detailed\";\n+\tconst char *refnameprefix = cb_data;\n \tstruct object *o = parse_object(sha1);\n \n \tif (!o)\n \t\tdie(\"git upload-pack: cannot find object %s:\", sha1_to_hex(sha1));\n \n \tif (capabilities)\n-\t\tpacket_write(1, \"%s %s%c%s%s\\n\", sha1_to_hex(sha1), refname,\n+\t\tpacket_write(1, \"%s %s%s%c%s%s\\n\", sha1_to_hex(sha1), refnameprefix, refname,\n \t\t\t 0, capabilities,\n \t\t\t stateless_rpc ? \" no-done\" : \"\");\n \telse\n-\t\tpacket_write(1, \"%s %s\\n\", sha1_to_hex(sha1), refname);\n+\t\tpacket_write(1, \"%s %s%s\\n\", sha1_to_hex(sha1), refnameprefix, refname);\n \tcapabilities = NULL;\n \tif (!(o->flags & OUR_REF)) {\n \t\to->flags |= OUR_REF;\n@@ -659,7 +662,7 @@ static int send_ref(const char *refname, const unsigned char *sha1, int flag, vo\n \tif (o->type == OBJ_TAG) {\n \t\to = deref_tag(o, refname, 0);\n \t\tif (o)\n-\t\t\tpacket_write(1, \"%s %s^{}\\n\", sha1_to_hex(o->sha1), refname);\n+\t\t\tpacket_write(1, \"%s %s%s^{}\\n\", sha1_to_hex(o->sha1), refnameprefix, refname);\n \t}\n \treturn 0;\n }\n@@ -678,15 +681,24 @@ static int mark_our_ref(const char *refname, const unsigned char *sha1, int flag\n \n static void upload_pack(void)\n {\n+\tstruct strbuf prefix = STRBUF_INIT;\n+\tunsigned char sha1[20];\n+\tint flag;\n+\n+\tstrbuf_addf(&prefix, \"refs/%s\", ref_prefix);\n \tif (advertise_refs || !stateless_rpc) {\n \t\treset_timeout();\n-\t\thead_ref(send_ref, NULL);\n-\t\tfor_each_ref(send_ref, NULL);\n+\t\tif (resolve_ref(head_path, sha1, 1, &flag))\n+\t\t\tsend_ref(\"HEAD\", sha1, flag, \"\");\n+\t\tfor_each_ref_in(prefix.buf, send_ref, \"refs/\");\n \t\tpacket_flush(1);\n \t} else {\n-\t\thead_ref(mark_our_ref, NULL);\n-\t\tfor_each_ref(mark_our_ref, NULL);\n+\t\tif (resolve_ref(head_path, sha1, 1, &flag))\n+\t\t\tmark_our_ref(\"HEAD\", sha1, flag, NULL);\n+\t\tfor_each_ref_in(prefix.buf, mark_our_ref, NULL);\n \t}\n+\tstrbuf_release(&prefix);\n+\n \tif (advertise_refs)\n \t\treturn;\n \n@@ -716,6 +728,14 @@ int main(int argc, char **argv)\n \t\t\tadvertise_refs = 1;\n \t\t\tcontinue;\n \t\t}\n+\t\tif (!prefixcmp(arg, \"--head=\")) {\n+\t\t\thead_path = arg+7;\n+\t\t\tcontinue;\n+\t\t}\n+\t\tif (!prefixcmp(arg, \"--ref-prefix=\")) {\n+\t\t\tref_prefix = arg+13;\n+\t\t\tcontinue;\n+\t\t}\n \t\tif (!strcmp(arg, \"--stateless-rpc\")) {\n \t\t\tstateless_rpc = 1;\n \t\t\tcontinue;\n-- \n1.7.5.1\n"},{"id":"168605","messageId":"1306274066-4092-2-git-send-email-jamey@minilop.net","threadId":"27448","inReplyTo":"1306274066-4092-1-git-send-email-jamey@minilop.net","subject":"[PATCHv2 2/2] Support virtual repositories in smart http-backend, specified by environment","fromName":"Jamey Sharp","fromEmail":"jamey@minilop.net","sentAt":"2011-05-24T21:54:26Z","receivedAt":"2011-05-24T21:54:26Z","isPatch":false,"sender":{"key":"jamey@minilop.net","avatar":"https://gravatar.com/avatar/979ed4f9190c19c13a84945afe9a6cf141e8884ef6ddaae342db7a74ae28828e?d=mp&s=160"},"body":"Translate the GIT_REF_PREFIX and GIT_HEAD environment variables to the\n--ref-prefix= and --head= options to upload-pack and receive-pack.\n\nAdd documentation, including a sample Apache configuration snippet.\n\nCommit by Josh Triplett and Jamey Sharp.\nSigned-off-by: Josh Triplett \nSigned-off-by: Jamey Sharp \nCc: Shawn O. Pearce \nCc: Johannes Schindelin \nCc: Johannes Sixt \n---\nv2: add patch 2/2 for git-http-backend.\n\n Documentation/git-http-backend.txt | 14 ++++++++++++\n http-backend.c | 39 +++++++++++++++++++++++++++++------\n 2 files changed, 46 insertions(+), 7 deletions(-)\n\ndiff --git a/Documentation/git-http-backend.txt b/Documentation/git-http-backend.txt\nindex 277d9e1..4e0b243 100644\n--- a/Documentation/git-http-backend.txt\n+++ b/Documentation/git-http-backend.txt\n@@ -119,6 +119,14 @@ ScriptAliasMatch \\\n \n ScriptAlias /git/ /var/www/cgi-bin/gitweb.cgi/\n ----------------------------------------------------------------\n++\n+To serve multiple virtual repositories from a single storage\n+repository:\n++\n+----------------------------------------------------------------\n+SetEnvIf Request_URI \"^/git/([^/]*)\" GIT_REF_PREFIX=$1/ GIT_HEAD=$1-HEAD\n+ScriptAliasMatch ^/git/[^/]*(.*) /usr/libexec/git-core/git-http-backend/storage.git$1\n+----------------------------------------------------------------\n \n Accelerated static Apache 2.x::\n \tSimilar to the above, but Apache can be used to return static\n@@ -167,6 +175,12 @@ The GIT_HTTP_EXPORT_ALL environmental variable may be passed to\n 'git-http-backend' to bypass the check for the \"git-daemon-export-ok\"\n file in each repository before allowing export of that repository.\n \n+The GIT_REF_PREFIX and GIT_HEAD environment variables allow\n+http-backend to support multiple virtual repositories within a single\n+storage repository. If set, 'git http-backend' will operate on\n+'refs/${GIT_REF_PREFIX}' rather than 'refs/', and '$GIT_HEAD' rather\n+than HEAD.\n+\n The backend process sets GIT_COMMITTER_NAME to '$REMOTE_USER' and\n GIT_COMMITTER_EMAIL to '$\\{REMOTE_USER}@http.$\\{REMOTE_ADDR\\}',\n ensuring that any reflogs created by 'git-receive-pack' contain some\ndiff --git a/http-backend.c b/http-backend.c\nindex 8501504..3d9e3b1 100644\n--- a/http-backend.c\n+++ b/http-backend.c\n@@ -315,16 +315,23 @@ done:\n \tclose(out);\n }\n \n-static void run_service(const char **argv)\n+#define RUN_SERVICE_LAST_EXTRA_ARG 2\n+#define RUN_SERVICE_EXTRA_ARGS NULL, NULL, NULL\n+\n+static void run_service(const char *argv0, const char **argv)\n {\n \tconst char *encoding = getenv(\"HTTP_CONTENT_ENCODING\");\n \tconst char *user = getenv(\"REMOTE_USER\");\n \tconst char *host = getenv(\"REMOTE_ADDR\");\n+\tchar *ref_prefix = getenv(\"GIT_REF_PREFIX\");\n+\tchar *head = getenv(\"GIT_HEAD\");\n \tchar *env[3];\n \tstruct strbuf buf = STRBUF_INIT;\n \tint gzipped_request = 0;\n \tstruct child_process cld;\n \n+\targv += RUN_SERVICE_LAST_EXTRA_ARG;\n+\n \tif (encoding && !strcmp(encoding, \"gzip\"))\n \t\tgzipped_request = 1;\n \telse if (encoding && !strcmp(encoding, \"x-gzip\"))\n@@ -343,6 +350,24 @@ static void run_service(const char **argv)\n \tenv[1] = strbuf_detach(&buf, NULL);\n \tenv[2] = NULL;\n \n+\tif (ref_prefix && !*ref_prefix)\n+\t\tref_prefix = NULL;\n+\tif (ref_prefix) {\n+\t\tstrbuf_addf(&buf, \"--ref-prefix=%s\", ref_prefix);\n+\t\tref_prefix = strbuf_detach(&buf, NULL);\n+\t\t*argv-- = ref_prefix;\n+\t}\n+\n+\tif (head && !*head)\n+\t\thead = NULL;\n+\tif (head) {\n+\t\tstrbuf_addf(&buf, \"--head=%s\", head);\n+\t\thead = strbuf_detach(&buf, NULL);\n+\t\t*argv-- = head;\n+\t}\n+\n+\t*argv = argv0;\n+\n \tmemset(&cld, 0, sizeof(cld));\n \tcld.argv = argv;\n \tcld.env = (const char *const *)env;\n@@ -362,6 +387,8 @@ static void run_service(const char **argv)\n \t\texit(1);\n \tfree(env[0]);\n \tfree(env[1]);\n+\tfree(ref_prefix);\n+\tfree(head);\n \tstrbuf_release(&buf);\n }\n \n@@ -391,7 +418,7 @@ static void get_info_refs(char *arg)\n \thdr_nocache();\n \n \tif (service_name) {\n-\t\tconst char *argv[] = {NULL /* service name */,\n+\t\tconst char *argv[] = {RUN_SERVICE_EXTRA_ARGS,\n \t\t\t\"--stateless-rpc\", \"--advertise-refs\",\n \t\t\t\".\", NULL};\n \t\tstruct rpc_service *svc = select_service(service_name);\n@@ -404,8 +431,7 @@ static void get_info_refs(char *arg)\n \t\tpacket_write(1, \"# service=git-%s\\n\", svc->name);\n \t\tpacket_flush(1);\n \n-\t\targv[0] = svc->name;\n-\t\trun_service(argv);\n+\t\trun_service(svc->name, argv);\n \n \t} else {\n \t\tselect_getanyfile();\n@@ -462,7 +488,7 @@ static void check_content_type(const char *accepted_type)\n \n static void service_rpc(char *service_name)\n {\n-\tconst char *argv[] = {NULL, \"--stateless-rpc\", \".\", NULL};\n+\tconst char *argv[] = {RUN_SERVICE_EXTRA_ARGS, \"--stateless-rpc\", \".\", NULL};\n \tstruct rpc_service *svc = select_service(service_name);\n \tstruct strbuf buf = STRBUF_INIT;\n \n@@ -478,8 +504,7 @@ static void service_rpc(char *service_name)\n \n \tend_headers();\n \n-\targv[0] = svc->name;\n-\trun_service(argv);\n+\trun_service(svc->name, argv);\n \tstrbuf_release(&buf);\n }\n \n-- \n1.7.5.1\n"},{"id":"168614","messageId":"7v7h9f7kzx.fsf@alter.siamese.dyndns.org","threadId":"27448","inReplyTo":"1306274066-4092-1-git-send-email-jamey@minilop.net","subject":"Re: [PATCHv2 1/2] Support multiple virtual repositories with a single object store and refs","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2011-05-24T23:10:42Z","receivedAt":"2011-05-24T23:10:42Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jamey Sharp writes:\n\n> Given many repositories with copies of the same objects (such as\n> branches of the same source), sharing a common object store will avoid\n> duplication. Alternates provide a single baseline, but don't handle\n> ongoing activity in the various repositories. Git safely handles\n> concurrent accesses to the same object store across repositories, but\n> operations such as gc need to know about all of the refs.\n>\n> This change adds support in upload-pack and receive-pack to simulate\n> multiple virtual repositories within the object store and references of\n\nIs it just me to read the above and then have to re-read the first\nsentence of the second paragraph over and over again? There seems to be a\nhuge gap in logic flow, probably largely due to the use of undefined term\n\"virtual repository\".\n\nI think the idea is \"an object store .git/objects/ may have more objects\nthan the refs .git/refs/* in the repository that particular object store\nbelongs to, and it is unsafe to gc there\" (the first paragraph), and then\nwhat is left unsaid is \"to solve it, we propose to add an extra namespace\nin the refs hierarchy of such a repository that lets other repositories to\nborrow its objects from, and store the tips of refs of the borrowing\nrepository there (and call such a repository that lets others borrow a\nvirtual repository)\" or something.\n\nWithout presenting what you are trying to solve...\n\n> a single underlying repository. The refs and heads of the virtual\n> repositories get stored in the underlying repository using prefixed\n> names specified by the --ref-prefix and --head options; for instance,\n> --ref-prefix=repo1/ will use refs/repo1/heads/* and refs/repo1/tags/*.\n> upload-pack and receive-pack will not expose any references that do not\n> match the specified prefix.\n>\n> These options implement the underlying mechanism for virtual\n> repositories; the higher-level protocol handler (such as http-backend or\n> a custom server) can pass these options when invoking upload-pack or\n> receive-pack, providing values based on components of the repository\n> path.\n\n... these are just gibberish describing technical details at too low-level.\n\n> For a simple local test, git-remote-ext works:\n>\n> git clone ext::'git %s --ref-prefix=prefix/ --head=prefix-HEAD /tmp/prefixed.git'\n>\n> Commit by Josh Triplett and Jamey Sharp.\n\nHave a blank line here, as that line is not part of Sign-of chain.\n\n> Signed-off-by: Josh Triplett \n> Signed-off-by: Jamey Sharp \n\n> Cc: Shawn O. Pearce \n> Cc: Johannes Schindelin \n> Cc: Johannes Sixt \n\nAlso personally I do not appreciate seeing Cc: here. You already have them\nat the header of your e-mail; these lines belong there, not here.\n\nI haven't looked deeply at the patch yet.\n"},{"id":"168628","messageId":"alpine.DEB.1.00.1105250847380.2701@bonsai2","threadId":"27448","inReplyTo":"7v7h9f7kzx.fsf@alter.siamese.dyndns.org","subject":"Re: [PATCHv2 1/2] Support multiple virtual repositories with a single object store and refs","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2011-05-25T06:51:07Z","receivedAt":"2011-05-25T06:51:07Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Tue, 24 May 2011, Junio C Hamano wrote:\n\n> Jamey Sharp writes:\n> \n> > Given many repositories with copies of the same objects (such as \n> > branches of the same source), sharing a common object store will avoid \n> > duplication. Alternates provide a single baseline, but don't handle \n> > ongoing activity in the various repositories. Git safely handles \n> > concurrent accesses to the same object store across repositories, but \n> > operations such as gc need to know about all of the refs.\n> >\n> > This change adds support in upload-pack and receive-pack to simulate \n> > multiple virtual repositories within the object store and references \n> > of\n> \n> Is it just me to read the above and then have to re-read the first \n> sentence of the second paragraph over and over again? There seems to be \n> a huge gap in logic flow, probably largely due to the use of undefined \n> term \"virtual repository\".\n\nI had to read the example call to understand that 'virtual repository' \nmeans 'one real catch-em-all repository'.\n\nI wonder about two things, though:\n\n1) Would teaching git clone to understand \"-t this/repo/*\" help?\n\n2) You're extending the protocol by appending the prefix after the SHA-1, \n and I stopped halfway through the patch trying to find information \n which I now think should be in the commit message: a) why? b) why does \n it not break when one of the two sides is a previous version?\n\nCiao,\nDscho\n"},{"id":"168672","messageId":"20110525154405.GA4839@oh.minilop.net","threadId":"27448","inReplyTo":"alpine.DEB.1.00.1105250847380.2701@bonsai2","subject":"Re: [PATCHv2 1/2] Support multiple virtual repositories with a single object store and refs","fromName":"Jamey Sharp","fromEmail":"jamey@minilop.net","sentAt":"2011-05-25T15:44:05Z","receivedAt":"2011-05-25T15:44:05Z","isPatch":false,"sender":{"key":"jamey@minilop.net","avatar":"https://gravatar.com/avatar/979ed4f9190c19c13a84945afe9a6cf141e8884ef6ddaae342db7a74ae28828e?d=mp&s=160"},"body":"On Wed, May 25, 2011 at 08:51:07AM +0200, Johannes Schindelin wrote:\n> On Tue, 24 May 2011, Junio C Hamano wrote:\n> > Jamey Sharp writes:\n> > > Given many repositories with copies of the same objects (such as \n> > > branches of the same source), sharing a common object store will avoid \n> > > duplication. Alternates provide a single baseline, but don't handle \n> > > ongoing activity in the various repositories. Git safely handles \n> > > concurrent accesses to the same object store across repositories, but \n> > > operations such as gc need to know about all of the refs.\n> > >\n> > > This change adds support in upload-pack and receive-pack to simulate \n> > > multiple virtual repositories within the object store and references \n> > > of\n> > \n> > Is it just me to read the above and then have to re-read the first \n> > sentence of the second paragraph over and over again? There seems to be \n> > a huge gap in logic flow, probably largely due to the use of undefined \n> > term \"virtual repository\".\n> \n> I had to read the example call to understand that 'virtual repository' \n> means 'one real catch-em-all repository'.\n> \n> I wonder about two things, though:\n> \n> 1) Would teaching git clone to understand \"-t this/repo/*\" help?\n\nSure, that would be an improvement for our use case, but we expect to\nhave lots of these virtual repositories, so I think we'd rather have the\nserver-side help we proposed in these patches. Seems to me that\nwildcards in git-clone would be nice for other use cases, though.\n\n> 2) You're extending the protocol by appending the prefix after the SHA-1, \n> and I stopped halfway through the patch trying to find information \n> which I now think should be in the commit message: a) why? b) why does \n> it not break when one of the two sides is a previous version?\n\nI don't think we're changing the protocol in any way...? In all of our\ntests, the client was Debian's package of git version 1.7.5.1. And note\nthat none of these patches touch client-side protocol code; it's all in\nthe server-side upload-pack and receive-pack.\n\nIf we have changed the protocol, it was unintentional and we should fix\nit.\n\nJamey\n"},{"id":"168707","messageId":"7vipsy36sb.fsf@alter.siamese.dyndns.org","threadId":"27448","inReplyTo":"20110525154405.GA4839@oh.minilop.net","subject":"Re: [PATCHv2 1/2] Support multiple virtual repositories with a single object store and refs","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2011-05-25T19:43:32Z","receivedAt":"2011-05-25T19:43:32Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jamey Sharp writes:\n\n>> I had to read the example call to understand that 'virtual repository' \n>> means 'one real catch-em-all repository'.\n>> \n>> I wonder about two things, though:\n>> \n>> 1) Would teaching git clone to understand \"-t this/repo/*\" help?\n>\n> Sure, that would be an improvement for our use case,...\n\nHmm, what does the \"-t\" option do? Is there a side-band communication\nbetween you guys that I am not seeing? I didn't see anything to make me\nsay \"Sure\" or otherwise.\n\nI had a feeling that you wanted to keep the illusion to the users that you\nare serving multiple repositories independently, so a solution that does\nnot have to make \"git clone\" on the other side aware of the layout on your\nserver would probably be more preferrable to you anyway.\n"},{"id":"168736","messageId":"alpine.DEB.1.00.1105260152430.2701@bonsai2","threadId":"27448","inReplyTo":"20110525154405.GA4839@oh.minilop.net","subject":"Re: [PATCHv2 1/2] Support multiple virtual repositories with a single object store and refs","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2011-05-25T23:53:42Z","receivedAt":"2011-05-25T23:53:42Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Wed, 25 May 2011, Jamey Sharp wrote:\n\n> On Wed, May 25, 2011 at 08:51:07AM +0200, Johannes Schindelin wrote:\n>\n> > 2) You're extending the protocol by appending the prefix after the \n> > SHA-1, and I stopped halfway through the patch trying to find \n> > information which I now think should be in the commit message: a) \n> > why? b) why does it not break when one of the two sides is a \n> > previous version?\n> \n> I don't think we're changing the protocol in any way...?\n\nDid your patch series not contain a change that sends a capability with \nthe prefix appended?\n\nCiao,\nJohannes\n"},{"id":"168737","messageId":"alpine.DEB.1.00.1105260154440.2701@bonsai2","threadId":"27448","inReplyTo":"7vipsy36sb.fsf@alter.siamese.dyndns.org","subject":"Re: [PATCHv2 1/2] Support multiple virtual repositories with a single object store and refs","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2011-05-25T23:56:08Z","receivedAt":"2011-05-25T23:56:08Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Dear Junio,\n\nOn Wed, 25 May 2011, Junio C Hamano wrote:\n\n> Jamey Sharp writes:\n> \n> >> I had to read the example call to understand that 'virtual \n> >> repository' means 'one real catch-em-all repository'.\n> >> \n> >> I wonder about two things, though:\n> >> \n> >> 1) Would teaching git clone to understand \"-t this/repo/*\" help?\n> >\n> > Sure, that would be an improvement for our use case,...\n> \n> Hmm, what does the \"-t\" option do?\n\nWell, I assumed that \"git clone\" would have adopted \"git remote add\"s -t \noption in the meantime.\n\nHth,\nJohannes\n"},{"id":"168738","messageId":"20110526000104.GA3439@leaf","threadId":"27448","inReplyTo":"alpine.DEB.1.00.1105260152430.2701@bonsai2","subject":"Re: [PATCHv2 1/2] Support multiple virtual repositories with a single object store and refs","fromName":"Josh Triplett","fromEmail":"josh@joshtriplett.org","sentAt":"2011-05-26T00:01:04Z","receivedAt":"2011-05-26T00:01:04Z","isPatch":false,"sender":{"key":"josh@joshtriplett.org","avatar":"https://avatars.githubusercontent.com/u/162737?v=4"},"body":"On Thu, May 26, 2011 at 01:53:42AM +0200, Johannes Schindelin wrote:\n> On Wed, 25 May 2011, Jamey Sharp wrote:\n> > On Wed, May 25, 2011 at 08:51:07AM +0200, Johannes Schindelin wrote:\n> > > 2) You're extending the protocol by appending the prefix after the \n> > > SHA-1, and I stopped halfway through the patch trying to find \n> > > information which I now think should be in the commit message: a) \n> > > why? b) why does it not break when one of the two sides is a \n> > > previous version?\n> > \n> > I don't think we're changing the protocol in any way...?\n> \n> Did your patch series not contain a change that sends a capability with \n> the prefix appended?\n\nNot that we know of. Are we missing something?\n\n- Josh Triplett\n"},{"id":"168739","messageId":"alpine.DEB.1.00.1105260239480.2701@bonsai2","threadId":"27448","inReplyTo":"20110526000104.GA3439@leaf","subject":"Re: [PATCHv2 1/2] Support multiple virtual repositories with a single object store and refs","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2011-05-26T00:40:49Z","receivedAt":"2011-05-26T00:40:49Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Wed, 25 May 2011, Josh Triplett wrote:\n\n> On Thu, May 26, 2011 at 01:53:42AM +0200, Johannes Schindelin wrote:\n> > On Wed, 25 May 2011, Jamey Sharp wrote:\n> > > On Wed, May 25, 2011 at 08:51:07AM +0200, Johannes Schindelin wrote:\n> > > > 2) You're extending the protocol by appending the prefix after the \n> > > > SHA-1, and I stopped halfway through the patch trying to find \n> > > > information which I now think should be in the commit message: a) \n> > > > why? b) why does it not break when one of the two sides is a \n> > > > previous version?\n> > > \n> > > I don't think we're changing the protocol in any way...?\n> > \n> > Did your patch series not contain a change that sends a capability with \n> > the prefix appended?\n> \n> Not that we know of. Are we missing something?\n\nThis is a change of protocol by my understanding:\n\n-- snip --\n \tif (sent_capabilities)\n-\t\tpacket_write(1, \"%s %s\\n\", sha1_to_hex(sha1), path);\n+\t\tpacket_write(1, \"%s %s%s\\n\", sha1_to_hex(sha1), refnameprefix, path);\n-- snap --\n\nOf course, you're free to ignore my comments just like everybody else on \nthe Git mailing list.\n\nCiao,\nJohannes\n"},{"id":"168746","messageId":"7vsjs2yuh6.fsf@alter.siamese.dyndns.org","threadId":"27448","inReplyTo":"alpine.DEB.1.00.1105260239480.2701@bonsai2","subject":"Re: [PATCHv2 1/2] Support multiple virtual repositories with a single object store and refs","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2011-05-26T04:08:21Z","receivedAt":"2011-05-26T04:08:21Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Johannes Schindelin writes:\n\n>> Not that we know of. Are we missing something?\n>\n> This is a change of protocol by my understanding:\n>\n> -- snip --\n> \tif (sent_capabilities)\n> -\t\tpacket_write(1, \"%s %s\\n\", sha1_to_hex(sha1), path);\n> +\t\tpacket_write(1, \"%s %s%s\\n\", sha1_to_hex(sha1), refnameprefix, path);\n> -- snap --\n\nI think this is not a change in the protocol per-se, but rather a botched\nattempt to make refs/virt/one/refsheads/foo this side has appear as if it\nis refs/heads/foo to the other side. In other words, you probably spotted\na bug.\n\nWhen read with the \"chroot\" analogy of Peff and Shawn earlier in the\nthread in mind, shouldn't the sending end that is pretending that only a\npart of its ref namespace is actually the whole thing _stripping_ the\nprefix from the real namespace, rather than appending the extra prefix?\n\nI've been down sick, feeling feverish, this afternoon, so I may be\nremembering what I understood while reading the code incorrectly, but that\nwas the impression I got.\n"}]}