{"thread":{"id":"22796","subject":"[PATCH 3/3] Different views on a repository: HEAD mapping","startedAt":"2010-02-24T15:33:33Z","lastAt":"2010-02-26T21:35:43Z","messageCount":16,"participants":["Andreas Gruenbacher","Shawn O. Pearce","Michael J Gruber","Junio C Hamano","James Pickens","Adam Brewster"],"isPatch":true,"patchVersion":1,"patchTotal":3},"messages":[{"id":"135585","messageId":"f409d0cde7939a833708ed92f86605dbbdd64a49.1267029680.git.agruen@suse.de","threadId":"22796","inReplyTo":"cover.1267029680.git.agruen@suse.de","subject":"[PATCH 1/3] receive-pack: Two small code cleanups","fromName":"Andreas Gruenbacher","fromEmail":"agruen@suse.de","sentAt":"2010-02-24T15:33:33Z","receivedAt":"2010-02-24T15:33:33Z","isPatch":true,"sender":{"key":"agruen@suse.de","avatar":null},"body":"Rename show_ref()'s path parameter to refname.\n\nIn read_head_info(), lines may have trailing capability strings. Throw\naway such strings after evaluation; they are not needed in the command\nstructs.\n\nSigned-off-by: Andreas Gruenbacher \n---\n builtin-receive-pack.c | 12 ++++++------\n 1 files changed, 6 insertions(+), 6 deletions(-)\n\ndiff --git a/builtin-receive-pack.c b/builtin-receive-pack.c\nindex 0559fcc..77cbc2a 100644\n--- a/builtin-receive-pack.c\n+++ b/builtin-receive-pack.c\n@@ -105,13 +105,13 @@ static int receive_pack_config(const char *var, const char *value, void *cb)\n \treturn git_default_config(var, value, cb);\n }\n \n-static int show_ref(const char *path, const unsigned char *sha1, int flag, void *cb_data)\n+static int show_ref(const char *refname, const unsigned char *sha1, int flag, void *cb_data)\n {\n \tif (sent_capabilities)\n-\t\tpacket_write(1, \"%s %s\\n\", sha1_to_hex(sha1), path);\n+\t\tpacket_write(1, \"%s %s\\n\", sha1_to_hex(sha1), refname);\n \telse\n \t\tpacket_write(1, \"%s %s%c%s%s\\n\",\n-\t\t\t sha1_to_hex(sha1), path, 0,\n+\t\t\t sha1_to_hex(sha1), refname, 0,\n \t\t\t \" report-status delete-refs side-band-64k\",\n \t\t\t prefer_ofs_delta ? \" ofs-delta\" : \"\");\n \tsent_capabilities = 1;\n@@ -524,7 +524,7 @@ static void read_head_info(void)\n \t\tstatic char line[1000];\n \t\tunsigned char old_sha1[20], new_sha1[20];\n \t\tstruct command *cmd;\n-\t\tchar *refname;\n+\t\tconst char *refname;\n \t\tint len, reflen;\n \n \t\tlen = packet_read_line(0, line, sizeof(line));\n@@ -548,10 +548,10 @@ static void read_head_info(void)\n \t\t\tif (strstr(refname + reflen + 1, \"side-band-64k\"))\n \t\t\t\tuse_sideband = LARGE_PACKET_MAX;\n \t\t}\n-\t\tcmd = xmalloc(sizeof(struct command) + len - 80);\n+\t\tcmd = xmalloc(sizeof(struct command) + reflen + 1);\n \t\thashcpy(cmd->old_sha1, old_sha1);\n \t\thashcpy(cmd->new_sha1, new_sha1);\n-\t\tmemcpy(cmd->ref_name, line + 82, len - 81);\n+\t\tmemcpy(cmd->ref_name, refname, reflen + 1);\n \t\tcmd->error_string = NULL;\n \t\tcmd->next = NULL;\n \t\t*p = cmd;\n-- \n1.6.6.243.gff6d2\n"},{"id":"135584","messageId":"92fea2335b73265b04d64fcc217055e1170f5e16.1267029680.git.agruen@suse.de","threadId":"22796","inReplyTo":"f409d0cde7939a833708ed92f86605dbbdd64a49.1267029680.git.agruen@suse.de","subject":"[PATCH 2/3] Different views on a repository","fromName":"Andreas Gruenbacher","fromEmail":"agruen@suse.de","sentAt":"2010-02-24T15:57:29Z","receivedAt":"2010-02-24T15:57:29Z","isPatch":true,"sender":{"key":"agruen@suse.de","avatar":null},"body":"Add --view options in upload-pack and receive-pack so that a repository\non the server side can be made to look like several independent\nrepositories on the client side.\n\nThis is implemented by transforming ref names: for example, with\n--view=one/, refs/heads/one/master on the server will look like\nrefs/heads/master to the client, refs/tags/one/v1 will look like\nrefs/tags/v1, etc.\n\nThis allows to transparently share repositories on the server which\nhave a lot of objects in common without complicating things for the\nclient, and without breaking garbage collection.\n\nSigned-off-by: Andreas Gruenbacher \n---\n Documentation/git-receive-pack.txt | 8 +++++-\n Documentation/git-upload-pack.txt | 9 ++++++-\n builtin-receive-pack.c | 20 ++++++++++++++++\n refs.c | 44 ++++++++++++++++++++++++++++++++++++\n refs.h | 3 ++\n upload-pack.c | 11 +++++++++\n 6 files changed, 93 insertions(+), 2 deletions(-)\n\ndiff --git a/Documentation/git-receive-pack.txt b/Documentation/git-receive-pack.txt\nindex 2790eeb..09d7d0c 100644\n--- a/Documentation/git-receive-pack.txt\n+++ b/Documentation/git-receive-pack.txt\n@@ -8,7 +8,7 @@ git-receive-pack - Receive what is pushed into the repository\n \n SYNOPSIS\n --------\n-'git-receive-pack' \n+'git-receive-pack' [--view=] \n \n DESCRIPTION\n -----------\n@@ -34,6 +34,12 @@ are not fast-forwards.\n \n OPTIONS\n -------\n+--view=::\n+\tPrepend to all ref names. For example, --view=one/ will\n+\tturn refs/tags/v1 into refs/tags/one/v1 on the receiving end. Together\n+\twith the --view option of linkgit:git-upload-pack[1], this allows to\n+\tmake one respository look like multiple independent repositories.\n+\n ::\n \tThe repository to sync into.\n \ndiff --git a/Documentation/git-upload-pack.txt b/Documentation/git-upload-pack.txt\nindex 71ca4ef..0eee0ba 100644\n--- a/Documentation/git-upload-pack.txt\n+++ b/Documentation/git-upload-pack.txt\n@@ -8,7 +8,7 @@ git-upload-pack - Send objects packed back to git-fetch-pack\n \n SYNOPSIS\n --------\n-'git-upload-pack' [--strict] [--timeout=] \n+'git-upload-pack' [--strict] [--timeout=] [--view=] \n \n DESCRIPTION\n -----------\n@@ -30,6 +30,13 @@ OPTIONS\n --timeout=::\n \tInterrupt transfer after seconds of inactivity.\n \n+--view=::\n+\tOnly upload refs which start with , and hide from the\n+\tremote side. For example, --view=one/ will skip refs/heads/master\n+\tand turn refs/tags/one/v1 into refs/tags/v1. Together with the --view\n+\toption of linkgit:git-receive-pack, this allows to make one respository\n+\tlook like multiple independent repositories.\n+\n ::\n \tThe repository to sync from.\n \ndiff --git a/builtin-receive-pack.c b/builtin-receive-pack.c\nindex 77cbc2a..44d7055 100644\n--- a/builtin-receive-pack.c\n+++ b/builtin-receive-pack.c\n@@ -32,6 +32,7 @@ static int use_sideband;\n static int prefer_ofs_delta = 1;\n static int auto_update_server_info;\n static int auto_gc = 1;\n+static const char *view;\n static const char *head_name;\n static int sent_capabilities;\n \n@@ -107,6 +108,12 @@ static int receive_pack_config(const char *var, const char *value, void *cb)\n \n static int show_ref(const char *refname, const unsigned char *sha1, int flag, void *cb_data)\n {\n+\tif (view) {\n+\t\trefname = ref_to_view(refname, view);\n+\t\tif (!refname)\n+\t\t\treturn 0;\n+\t}\n+\n \tif (sent_capabilities)\n \t\tpacket_write(1, \"%s %s\\n\", sha1_to_hex(sha1), refname);\n \telse\n@@ -548,6 +555,15 @@ static void read_head_info(void)\n \t\t\tif (strstr(refname + reflen + 1, \"side-band-64k\"))\n \t\t\t\tuse_sideband = LARGE_PACKET_MAX;\n \t\t}\n+\t\tif (view) {\n+\t\t\tconst char *r;\n+\n+\t\t\tr = view_to_ref(refname, view);\n+\t\t\tif (r) {\n+\t\t\t\trefname = r;\n+\t\t\t\treflen = strlen(refname);\n+\t\t\t}\n+\t\t}\n \t\tcmd = xmalloc(sizeof(struct command) + reflen + 1);\n \t\thashcpy(cmd->old_sha1, old_sha1);\n \t\thashcpy(cmd->new_sha1, new_sha1);\n@@ -736,6 +752,10 @@ int cmd_receive_pack(int argc, const char **argv, const char *prefix)\n \t\t\t\tstateless_rpc = 1;\n \t\t\t\tcontinue;\n \t\t\t}\n+\t\t\tif (!prefixcmp(arg, \"--view=\")) {\n+\t\t\t\tview = arg + 7;\n+\t\t\t\tcontinue;\n+\t\t\t}\n \n \t\t\tusage(receive_pack_usage);\n \t\t}\ndiff --git a/refs.c b/refs.c\nindex f3fcbe0..b1f3951 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -1829,3 +1829,47 @@ char *shorten_unambiguous_ref(const char *ref, int strict)\n \tfree(short_name);\n \treturn xstrdup(ref);\n }\n+\n+const char *ref_to_view(const char *refname, const char *view)\n+{\n+\tstatic char *buffer;\n+\tint prefix_len, view_len, suffix_len;\n+\tconst char *r, *suffix;\n+\n+\tif (prefixcmp(refname, \"refs/\"))\n+\t\treturn NULL;\n+\tr = strchr(refname + 5, '/');\n+\tif (!r)\n+\t\treturn NULL;\n+\tr++;\n+\tview_len = strlen(view);\n+\tif (strncmp(r, view, view_len))\n+\t\treturn NULL;\n+\tsuffix = r + view_len;\n+\tprefix_len = r - refname;\n+\tsuffix_len = strlen(suffix);\n+\tbuffer = xrealloc(buffer, prefix_len + suffix_len + 1);\n+\tsprintf(buffer, \"%.*s%s\", prefix_len, refname, suffix);\n+\treturn buffer;\n+}\n+\n+const char *view_to_ref(const char *refname, const char *view)\n+{\n+\tstatic char *buffer;\n+\tint prefix_len, view_len, suffix_len;\n+\tconst char *r, *suffix;\n+\n+\tview_len = strlen(view);\n+\tif (prefixcmp(refname, \"refs/\"))\n+\t\treturn NULL;\n+\tr = strchr(refname + 5, '/');\n+\tif (!r)\n+\t\treturn NULL;\n+\tr++;\n+\tprefix_len = r - refname;\n+\tsuffix = r + view_len;\n+\tsuffix_len = strlen(suffix);\n+\tbuffer = xrealloc(buffer, prefix_len + view_len + suffix_len + 1);\n+\tsprintf(buffer, \"%.*s%s%s\", prefix_len, refname, view, suffix);\n+\treturn buffer;\n+}\ndiff --git a/refs.h b/refs.h\nindex f7648b9..390e812 100644\n--- a/refs.h\n+++ b/refs.h\n@@ -98,4 +98,7 @@ int update_ref(const char *action, const char *refname,\n \t\tconst unsigned char *sha1, const unsigned char *oldval,\n \t\tint flags, enum action_on_err onerr);\n \n+extern const char *ref_to_view(const char *refname, const char *view);\n+extern const char *view_to_ref(const char *refname, const char *view);\n+\n #endif /* REFS_H */\ndiff --git a/upload-pack.c b/upload-pack.c\nindex dc464d7..bc72471 100644\n--- a/upload-pack.c\n+++ b/upload-pack.c\n@@ -34,6 +34,7 @@ static struct object_array have_obj;\n static struct object_array want_obj;\n static struct object_array extra_edge_obj;\n static unsigned int timeout;\n+static const char *view;\n /* 0 for no sideband,\n * otherwise maximum packet size (up to 65520 bytes).\n */\n@@ -629,6 +630,12 @@ static int send_ref(const char *refname, const unsigned char *sha1, int flag, vo\n \tif (!o)\n \t\tdie(\"git upload-pack: cannot find object %s:\", sha1_to_hex(sha1));\n \n+\tif (view) {\n+\t\trefname = ref_to_view(refname, view);\n+\t\tif (!refname)\n+\t\t\treturn 0;\n+\t}\n+\n \tif (capabilities)\n \t\tpacket_write(1, \"%s %s%c%s\\n\", sha1_to_hex(sha1), refname,\n \t\t\t0, capabilities);\n@@ -711,6 +718,10 @@ int main(int argc, char **argv)\n \t\t\tdaemon_mode = 1;\n \t\t\tcontinue;\n \t\t}\n+\t\tif (!prefixcmp(arg, \"--view=\")) {\n+\t\t\tview = arg + 7;\n+\t\t\tcontinue;\n+\t\t}\n \t\tif (!strcmp(arg, \"--\")) {\n \t\t\ti++;\n \t\t\tbreak;\n-- \n1.6.6.243.gff6d2\n"},{"id":"135582","messageId":"d4ad6b45786def21cbe484c97723aa573b069175.1267029680.git.agruen@suse.de","threadId":"22796","inReplyTo":"92fea2335b73265b04d64fcc217055e1170f5e16.1267029680.git.agruen@suse.de","subject":"[PATCH 3/3] Different views on a repository: HEAD mapping","fromName":"Andreas Gruenbacher","fromEmail":"agruen@suse.de","sentAt":"2010-02-24T16:14:03Z","receivedAt":"2010-02-24T16:14:03Z","isPatch":true,"sender":{"key":"agruen@suse.de","avatar":null},"body":"The HEAD ref is not located under .git/refs/heads/, so the trivial\nview mapping doesn't work. Fix this by making refs/heads/HEAD\nappear as HEAD on the client when --view= is used in upload-pack\nand receive-pack.\n\nSigned-off-by: Andreas Gruenbacher \n---\n Documentation/git-receive-pack.txt | 7 ++++---\n Documentation/git-upload-pack.txt | 9 +++++----\n refs.c | 21 ++++++++++++++++++---\n refs.h | 1 +\n upload-pack.c | 9 +++++++--\n 5 files changed, 35 insertions(+), 12 deletions(-)\n\ndiff --git a/Documentation/git-receive-pack.txt b/Documentation/git-receive-pack.txt\nindex 09d7d0c..07e0159 100644\n--- a/Documentation/git-receive-pack.txt\n+++ b/Documentation/git-receive-pack.txt\n@@ -36,9 +36,10 @@ OPTIONS\n -------\n --view=::\n \tPrepend to all ref names. For example, --view=one/ will\n-\tturn refs/tags/v1 into refs/tags/one/v1 on the receiving end. Together\n-\twith the --view option of linkgit:git-upload-pack[1], this allows to\n-\tmake one respository look like multiple independent repositories.\n+\tturn refs/tags/v1 into refs/tags/one/v1 and HEAD into refs/heads/one/HEAD\n+\ton the receiving end. Together with the --view option of\n+\tlinkgit:git-upload-pack[1], this allows to make one respository look like\n+\tmultiple independent repositories.\n \n ::\n \tThe repository to sync into.\ndiff --git a/Documentation/git-upload-pack.txt b/Documentation/git-upload-pack.txt\nindex 0eee0ba..1e8b76b 100644\n--- a/Documentation/git-upload-pack.txt\n+++ b/Documentation/git-upload-pack.txt\n@@ -32,10 +32,11 @@ OPTIONS\n \n --view=::\n \tOnly upload refs which start with , and hide from the\n-\tremote side. For example, --view=one/ will skip refs/heads/master\n-\tand turn refs/tags/one/v1 into refs/tags/v1. Together with the --view\n-\toption of linkgit:git-receive-pack, this allows to make one respository\n-\tlook like multiple independent repositories.\n+\tremote side. For example, --view=one/ will skip refs/heads/master,\n+\tturn refs/tags/one/v1 into refs/tags/v1, and refs/heads/one/HEAD into\n+\tHEAD. Together with the --view option of linkgit:git-receive-pack,\n+\tthis allows to make one respository look like multiple independent\n+\trepositories.\n \n ::\n \tThe repository to sync from.\ndiff --git a/refs.c b/refs.c\nindex b1f3951..77a6267 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -650,16 +650,21 @@ end_each:\n \treturn retval;\n }\n \n-int head_ref(each_ref_fn fn, void *cb_data)\n+int one_ref(each_ref_fn fn, void *cb_data, const char *refname)\n {\n \tunsigned char sha1[20];\n \tint flag;\n \n-\tif (resolve_ref(\"HEAD\", sha1, 1, &flag))\n-\t\treturn fn(\"HEAD\", sha1, flag, cb_data);\n+\tif (resolve_ref(refname, sha1, 1, &flag))\n+\t\treturn fn(refname, sha1, flag, cb_data);\n \treturn 0;\n }\n \n+int head_ref(each_ref_fn fn, void *cb_data)\n+{\n+\treturn one_ref(fn, cb_data, \"HEAD\");\n+}\n+\n int for_each_ref(each_ref_fn fn, void *cb_data)\n {\n \treturn do_for_each_ref(\"refs/\", fn, 0, 0, cb_data);\n@@ -1846,6 +1851,10 @@ const char *ref_to_view(const char *refname, const char *view)\n \tif (strncmp(r, view, view_len))\n \t\treturn NULL;\n \tsuffix = r + view_len;\n+\tif (!strncmp(refname + 5, \"heads/\", 6) &&\n+\t !strcmp(suffix, \"HEAD\"))\n+\t\treturn \"HEAD\";\n+\n \tprefix_len = r - refname;\n \tsuffix_len = strlen(suffix);\n \tbuffer = xrealloc(buffer, prefix_len + suffix_len + 1);\n@@ -1860,6 +1869,12 @@ const char *view_to_ref(const char *refname, const char *view)\n \tconst char *r, *suffix;\n \n \tview_len = strlen(view);\n+\tif (!strcmp(refname, \"HEAD\")) {\n+\t\tbuffer = xrealloc(buffer, view_len + 16);\n+\t\tsprintf(buffer, \"refs/heads/%sHEAD\", view);\n+\t\treturn buffer;\n+\t}\n+\n \tif (prefixcmp(refname, \"refs/\"))\n \t\treturn NULL;\n \tr = strchr(refname + 5, '/');\ndiff --git a/refs.h b/refs.h\nindex 390e812..addcc2d 100644\n--- a/refs.h\n+++ b/refs.h\n@@ -18,6 +18,7 @@ struct ref_lock {\n * and returns the value\n */\n typedef int each_ref_fn(const char *refname, const unsigned char *sha1, int flags, void *cb_data);\n+extern int one_ref(each_ref_fn, void *, const char *);\n extern int head_ref(each_ref_fn, void *);\n extern int for_each_ref(each_ref_fn, void *);\n extern int for_each_ref_in(const char *, each_ref_fn, void *);\ndiff --git a/upload-pack.c b/upload-pack.c\nindex bc72471..b3bf20f 100644\n--- a/upload-pack.c\n+++ b/upload-pack.c\n@@ -668,13 +668,18 @@ static int mark_our_ref(const char *refname, const unsigned char *sha1, int flag\n \n static void upload_pack(void)\n {\n+\tconst char *head = \"HEAD\";\n+\n+\tif (view)\n+\t\thead = view_to_ref(head, view);\n+\n \tif (advertise_refs || !stateless_rpc) {\n \t\treset_timeout();\n-\t\thead_ref(send_ref, NULL);\n+\t\tone_ref(send_ref, NULL, head);\n \t\tfor_each_ref(send_ref, NULL);\n \t\tpacket_flush(1);\n \t} else {\n-\t\thead_ref(mark_our_ref, NULL);\n+\t\tone_ref(mark_our_ref, NULL, head);\n \t\tfor_each_ref(mark_our_ref, NULL);\n \t}\n \tif (advertise_refs)\n-- \n1.6.6.243.gff6d2\n"},{"id":"135583","messageId":"cover.1267029680.git.agruen@suse.de","threadId":"22796","inReplyTo":null,"subject":"[RFC][PATCH 0/3] Different views on a repository","fromName":"Andreas Gruenbacher","fromEmail":"agruen@suse.de","sentAt":"2010-02-24T16:41:20Z","receivedAt":"2010-02-24T16:41:20Z","isPatch":true,"sender":{"key":"agruen@suse.de","avatar":null},"body":"Hello,\n\nwe have a use case with groups of repositories which share lots of\nobjects, but which are logically independent. There is no strict\nhierarchy between the repositories, the development modl is arbitrary.\nThe alternates mechanism for sharig objects between repositories won't\nwork.\n\nThe best idea I came up with so far to solve this was to keep everything\nin the same repository on the server. Then, to keep the logically\nindependent repositories separate, directories are used below refs/heads\nand refs/tags. Receive-pack and upload-pack are modified to hide this\ndirectory structure from clients so that repositories will continue to\nlook \"normal\" to users. For example, the following structure on the\nserver:\n\n\trefs/heads/one/master\n\trefs/tags/one/tag1\n\trefs/heads/two/master\n\trefs/heads/two/branch2\n\nwould appear as two independent repositories to different clients:\n\n\trefs/heads/master\n\trefs/tags/tag1\n\nand:\n\n\trefs/heads/master\n\trefs/heads/branch2\n\nThe following three patches implement this. What do you guys think --\ndoes the basic idea and implementation look sensible, or am I\noverlooking a way to solve this kind of problem with other means?\n\nThanks!\n\n\n receive-pack: Two small code cleanups\n Different views on a repository\n Different views on a repository: HEAD mapping\n\n Documentation/git-receive-pack.txt | 9 ++++-\n Documentation/git-upload-pack.txt | 10 +++++-\n builtin-receive-pack.c | 32 ++++++++++++++---\n refs.c | 65 ++++++++++++++++++++++++++++++++++--\n refs.h | 4 ++\n upload-pack.c | 20 ++++++++++-\n 6 files changed, 127 insertions(+), 13 deletions(-)\n"},{"id":"135586","messageId":"20100224172932.GE18993@spearce.org","threadId":"22796","inReplyTo":"f409d0cde7939a833708ed92f86605dbbdd64a49.1267029680.git.agruen@suse.de","subject":"Re: [PATCH 1/3] receive-pack: Two small code cleanups","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2010-02-24T17:29:32Z","receivedAt":"2010-02-24T17:29:32Z","isPatch":true,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"Andreas Gruenbacher wrote:\n> Rename show_ref()'s path parameter to refname.\n> \n> In read_head_info(), lines may have trailing capability strings. Throw\n> away such strings after evaluation; they are not needed in the command\n> structs.\n> \n> Signed-off-by: Andreas Gruenbacher \n\nAcked-by: Shawn O. Pearce \n\n-- \nShawn.\n"},{"id":"135587","messageId":"20100224174235.GA20567@spearce.org","threadId":"22796","inReplyTo":"92fea2335b73265b04d64fcc217055e1170f5e16.1267029680.git.agruen@suse.de","subject":"Re: [PATCH 2/3] Different views on a repository","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2010-02-24T17:42:35Z","receivedAt":"2010-02-24T17:42:35Z","isPatch":true,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"Andreas Gruenbacher wrote:\n> Add --view options in upload-pack and receive-pack so that a repository\n> on the server side can be made to look like several independent\n> repositories on the client side.\n\nBefore saying this is good... I'd like to know how a repository owner\nis supposed to set these options on the user started invocations\nof other remote side program.\n\nRight now, I don't see how this is too different from just\ndoing the following on a client:\n\n git init\n git remote add origin URL\n git config remote.origin.fetch refs/heads/one/*:refs/remotes/origin/*\n\nand therefore shouldn't just be handled on the *client* side of the\nconnection, as part of the remote setup and push matching refs rules.\n\n(Of course, the push matching ref logic is messy too... adding yet\nmore into that pile might also be ugly.)\n\n> +const char *view_to_ref(const char *refname, const char *view)\n> +{\n> +\tstatic char *buffer;\n...\n> +\tbuffer = xrealloc(buffer, prefix_len + view_len + suffix_len + 1);\n> +\tsprintf(buffer, \"%.*s%s%s\", prefix_len, refname, view, suffix);\n> +\treturn buffer;\n\nI'd rather not use a static buffer like this. Why not alloc and let\nthe caller free? Or have the caller pass in a strbuf you populate\nfor them?\n\n-- \nShawn.\n"},{"id":"135661","messageId":"4B863C77.8040304@drmicha.warpmail.net","threadId":"22796","inReplyTo":"92fea2335b73265b04d64fcc217055e1170f5e16.1267029680.git.agruen@suse.de","subject":"Re: [PATCH 2/3] Different views on a repository","fromName":"Michael J Gruber","fromEmail":"git@drmicha.warpmail.net","sentAt":"2010-02-25T09:01:43Z","receivedAt":"2010-02-25T09:01:43Z","isPatch":true,"sender":{"key":"git@grubix.eu","avatar":"https://avatars.githubusercontent.com/u/233215?v=4"},"body":"Andreas Gruenbacher venit, vidit, dixit 24.02.2010 16:57:\n> Add --view options in upload-pack and receive-pack so that a repository\n> on the server side can be made to look like several independent\n> repositories on the client side.\n> \n> This is implemented by transforming ref names: for example, with\n> --view=one/, refs/heads/one/master on the server will look like\n> refs/heads/master to the client, refs/tags/one/v1 will look like\n> refs/tags/v1, etc.\n> \n> This allows to transparently share repositories on the server which\n> have a lot of objects in common without complicating things for the\n> client, and without breaking garbage collection.\n\nJust from this description, I can't see why the same can't be done with\nappropriate refspecs. (A helper for doing that would be more welcome, of\ncourse.)\n\nMaybe a few tests and documentation (i.e. examples, not just the option\ndescription) would clear this up?\n\nMichael\n"},{"id":"135664","messageId":"201002251025.22881.agruen@suse.de","threadId":"22796","inReplyTo":"4B863C77.8040304@drmicha.warpmail.net","subject":"Re: [PATCH 2/3] Different views on a repository","fromName":"Andreas Gruenbacher","fromEmail":"agruen@suse.de","sentAt":"2010-02-25T09:25:22Z","receivedAt":"2010-02-25T09:25:22Z","isPatch":true,"sender":{"key":"agruen@suse.de","avatar":null},"body":"On Thursday 25 February 2010 10:01:43 Michael J Gruber wrote:\n> Andreas Gruenbacher venit, vidit, dixit 24.02.2010 16:57:\n> > Add --view options in upload-pack and receive-pack so that a repository\n> > on the server side can be made to look like several independent\n> > repositories on the client side.\n> >\n> > This is implemented by transforming ref names: for example, with\n> > --view=one/, refs/heads/one/master on the server will look like\n> > refs/heads/master to the client, refs/tags/one/v1 will look like\n> > refs/tags/v1, etc.\n> >\n> > This allows to transparently share repositories on the server which\n> > have a lot of objects in common without complicating things for the\n> > client, and without breaking garbage collection.\n> \n> Just from this description, I can't see why the same can't be done with\n> appropriate refspecs. (A helper for doing that would be more welcome, of\n> course.)\n\nYou mean on the client side? The problem then is that a simple \"git clone\" \nwould not do the right thing anymore; you would still expose server-side \nimplementation details to clients. Clients shouldn't have to bother with this \nadded complexity. (They might not even have access to some of the views.) When \nyou do the mapping server-side, you can split or merge repositories as needed \nwithout the clients even noticing.\n\n> Maybe a few tests and documentation (i.e. examples, not just the option\n> description) would clear this up?\n\nIndeed, I should add some more background info.\n\nThanks,\nAndreas\n"},{"id":"135673","messageId":"4B866D60.6040306@drmicha.warpmail.net","threadId":"22796","inReplyTo":"201002251025.22881.agruen@suse.de","subject":"Re: [PATCH 2/3] Different views on a repository","fromName":"Michael J Gruber","fromEmail":"git@drmicha.warpmail.net","sentAt":"2010-02-25T12:30:24Z","receivedAt":"2010-02-25T12:30:24Z","isPatch":true,"sender":{"key":"git@grubix.eu","avatar":"https://avatars.githubusercontent.com/u/233215?v=4"},"body":"Andreas Gruenbacher venit, vidit, dixit 25.02.2010 10:25:\n> On Thursday 25 February 2010 10:01:43 Michael J Gruber wrote:\n>> Andreas Gruenbacher venit, vidit, dixit 24.02.2010 16:57:\n>>> Add --view options in upload-pack and receive-pack so that a repository\n>>> on the server side can be made to look like several independent\n>>> repositories on the client side.\n>>>\n>>> This is implemented by transforming ref names: for example, with\n>>> --view=one/, refs/heads/one/master on the server will look like\n>>> refs/heads/master to the client, refs/tags/one/v1 will look like\n>>> refs/tags/v1, etc.\n>>>\n>>> This allows to transparently share repositories on the server which\n>>> have a lot of objects in common without complicating things for the\n>>> client, and without breaking garbage collection.\n>>\n>> Just from this description, I can't see why the same can't be done with\n>> appropriate refspecs. (A helper for doing that would be more welcome, of\n>> course.)\n> \n> You mean on the client side? The problem then is that a simple \"git clone\" \n> would not do the right thing anymore; you would still expose server-side \n> implementation details to clients. Clients shouldn't have to bother with this \n> added complexity. (They might not even have access to some of the views.) When \n> you do the mapping server-side, you can split or merge repositories as needed \n> without the clients even noticing.\n\nBut the client has to request a specific view, doesn't it? You have to\ntell all clients \"don't just clone, use...\", where the \"...\" don't seem\nto be part of the series yet. [I could see 0/3 on gmane only now, by the\nway.]\n\nI just can't help the impression that this is a use case which does not\nneed a new feature, at least not upload/receive-pack wise. It's more a\nmatter of ensuring that all clients use a specific configuration (which\nyou would have to with your patch as well, AFAICT), and this more\ngeneral issue is creeping up again and again, with no agreeable solution\nso far.\n\nCheers,\nMichael\n"},{"id":"135677","messageId":"201002251535.03334.agruen@suse.de","threadId":"22796","inReplyTo":"4B866D60.6040306@drmicha.warpmail.net","subject":"Re: [PATCH 2/3] Different views on a repository","fromName":"Andreas Gruenbacher","fromEmail":"agruen@suse.de","sentAt":"2010-02-25T14:35:03Z","receivedAt":"2010-02-25T14:35:03Z","isPatch":true,"sender":{"key":"agruen@suse.de","avatar":null},"body":"On Thursday 25 February 2010 13:30:24 Michael J Gruber wrote:\n> Andreas Gruenbacher venit, vidit, dixit 25.02.2010 10:25:\n> > On Thursday 25 February 2010 10:01:43 Michael J Gruber wrote:\n> >> Andreas Gruenbacher venit, vidit, dixit 24.02.2010 16:57:\n> >>> Add --view options in upload-pack and receive-pack so that a repository\n> >>> on the server side can be made to look like several independent\n> >>> repositories on the client side.\n> >>>\n> >>> This is implemented by transforming ref names: for example, with\n> >>> --view=one/, refs/heads/one/master on the server will look like\n> >>> refs/heads/master to the client, refs/tags/one/v1 will look like\n> >>> refs/tags/v1, etc.\n> >>>\n> >>> This allows to transparently share repositories on the server which\n> >>> have a lot of objects in common without complicating things for the\n> >>> client, and without breaking garbage collection.\n> >>\n> >> Just from this description, I can't see why the same can't be done with\n> >> appropriate refspecs. (A helper for doing that would be more welcome, of\n> >> course.)\n> >\n> > You mean on the client side? The problem then is that a simple \"git\n> > clone\" would not do the right thing anymore; you would still expose\n> > server-side implementation details to clients. Clients shouldn't have to\n> > bother with this added complexity. (They might not even have access to\n> > some of the views.) When you do the mapping server-side, you can split or\n> > merge repositories as needed without the clients even noticing.\n> \n> But the client has to request a specific view, doesn't it?\n\nNo, it's a server side thing. The git commands affected are upload-pack and \nreceice-pack, and those run on the remote end of a fetch. For example, the \nclient would ask the server for repository /foo/one or /foo/two, and the \nserver would map that to different views of /bar/shared: when the client asks \nthe server to run \"git-upload-pack /foo/one\", the server runs \"git-upload-pack \n--view=one/ /bar/shared\" instead.\n\nThis is relatively easy to set up over ssh using a simple script; for direct \ngit access, a small wrapper daemon would be needed. I'm not sure how this \ncould be hacked into http access, but it doesn't seem all that hard, either.\n\n> I just can't help the impression that this is a use case which does not\n> need a new feature, at least not upload/receive-pack wise.\n\nStill, even with the additional explanation above?\n\n> It's more a matter of ensuring that all clients use a specific configuration\n> (which you would have to with your patch as well, AFAICT), and this more\n> general issue is creeping up again and again, with no agreeable solution\n> so far.\n\nWell that's another problem which we indeed also have for enabling things like \nlocal consistency checks and merge drivers. I don't have a good answer here :)\n\nThanks,\nAndreas\n"},{"id":"135682","messageId":"7vljeh9qcx.fsf@alter.siamese.dyndns.org","threadId":"22796","inReplyTo":"201002251535.03334.agruen@suse.de","subject":"Re: [PATCH 2/3] Different views on a repository","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2010-02-25T17:28:46Z","receivedAt":"2010-02-25T17:28:46Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Andreas Gruenbacher writes:\n\n> No, it's a server side thing.\n\nIf it were a server side thing, then I would expect no change to\nsend/receive pack. Instead your clients will access distinct URL as if\nthey are different repositories.\n\n git clone git://example.com/pub/scm/git/A\n git push example.com:/pub/scm/git/B master\n git pull http://example.com/pub/scm/git/C\n\nThey should not have to care that the server is cheating to save disk\nspace, and they should be able to access your server with Git v1.6.0.\n\nInstead, the server side would:\n\n - have separate repositories, A, B and C, as normal repositories;\n\n - these repositories share their object stores by having their\n .git/objects pointing at a shared location via a symlink;\n\n - on the server side, gc/prune/fsck will have to be updated so that when\n the object store of a repository (say A) is shared with something else,\n they will consider refs in other repositories (B and C) also as the\n root of traversal.\n\nSo if this were a server side solution, I would expect the series would\nadd:\n\n - a way to set up a shared object store;\n\n - a way to maintain a list of backlinks to repositories that share an\n object store;\n\n - a way to create a new repository that shares the object store\n (e.g. create a symlink to the shared store instead of having its own\n .git/objects/, and add itself to the list of backlinks for the shared\n object store);\n\n - a way to retire an existing such repository (rm -rf and remove itself\n from the list of backlinks);\n\n - update gc/prune/fsck to honor such a list of backlinks.\n\nThis would help a \"forks\" setup commonly seen at places like repo.or.cz\nand github.com among others.\n\nOne thing that is missing from the above handwaving outline that your\n\"different views\" offers is a \"consolidated view\", a pseudo-repository\nthat allows you to see refs from individual real (from the client's and\nproject participant's point of view) repositories as if they are in\nindividual subhierarchies of the ref namespace.\n\nI however suspect that you didn't want such a view in the first place if\nthere weren't issues around reachability. In other words, I suspect that\nyou invented it merely as one possible solution to the reachability issue,\nand it was not your goal to have such a consolidated view by itself.\n"},{"id":"135690","messageId":"885649361002251213j4c48f720ree7e70848aafaef5@mail.gmail.com","threadId":"22796","inReplyTo":"cover.1267029680.git.agruen@suse.de","subject":"Re: [RFC][PATCH 0/3] Different views on a repository","fromName":"James Pickens","fromEmail":"jepicken@gmail.com","sentAt":"2010-02-25T20:13:25Z","receivedAt":"2010-02-25T20:13:25Z","isPatch":true,"sender":{"key":"jepicken@gmail.com","avatar":null},"body":"On Wed, Feb 24, 2010, Andreas Gruenbacher wrote:\n> we have a use case with groups of repositories which share lots of\n> objects, but which are logically independent. There is no strict\n> hierarchy between the repositories, the development modl is arbitrary.\n> The alternates mechanism for sharig objects between repositories won't\n> work.\n\nCan you elaborate on why alternates won't work?\n\n> The best idea I came up with so far to solve this\n\nSolve what? You didn't mention any specific problem. Are you just trying\nto save disk space by not storing multiple copies of the same objects?\n\nJames\n"},{"id":"135711","messageId":"201002260145.33960.agruen@suse.de","threadId":"22796","inReplyTo":"7vljeh9qcx.fsf@alter.siamese.dyndns.org","subject":"Re: [PATCH 2/3] Different views on a repository","fromName":"Andreas Gruenbacher","fromEmail":"agruen@suse.de","sentAt":"2010-02-26T00:45:33Z","receivedAt":"2010-02-26T00:45:33Z","isPatch":true,"sender":{"key":"agruen@suse.de","avatar":null},"body":"On Thursday 25 February 2010 18:28:46 Junio C Hamano wrote:\n> Andreas Gruenbacher writes:\n> > No, it's a server side thing.\n> \n> If it were a server side thing, then I would expect no change to\n> send/receive pack.\n>\n> Instead your clients will access distinct URL as if they are different\n> repositories.\n> \n> git clone git://example.com/pub/scm/git/A\n> git push example.com:/pub/scm/git/B master\n> git pull http://example.com/pub/scm/git/C\n>\n> They should not have to care that the server is cheating to save disk\n> space, and they should be able to access your server with Git v1.6.0.\n> \n> Instead, the server side would:\n> \n> - have separate repositories, A, B and C, as normal repositories;\n> \n> - these repositories share their object stores by having their\n> .git/objects pointing at a shared location via a symlink;\n> \n> - on the server side, gc/prune/fsck will have to be updated so that when\n> the object store of a repository (say A) is shared with something else,\n> they will consider refs in other repositories (B and C) also as the\n> root of traversal.\n\nI was proposing to change receive-pack and upload-pack, both of which are \nrunning on the server; there was no mention of send-pack. What I've proposed \nis not a complete solution, but it should suffice to show the idea.\n\nYour alternative proposal would also solve my problem, in a different way. \nWith either approach, what looks like separate repositories A, B, and C to \nclients looks like one repository to gc/prune/fsck.\n\n> So if this were a server side solution, I would expect the series would\n> add:\n> \n> - a way to set up a shared object store;\n> \n> - a way to maintain a list of backlinks to repositories that share an\n> object store;\n> \n> - a way to create a new repository that shares the object store\n> (e.g. create a symlink to the shared store instead of having its own\n> .git/objects/, and add itself to the list of backlinks for the shared\n> object store);\n> \n> - a way to retire an existing such repository (rm -rf and remove itself\n> from the list of backlinks);\n> \n> - update gc/prune/fsck to honor such a list of backlinks.\n> \n> This would help a \"forks\" setup commonly seen at places like repo.or.cz\n> and github.com among others.\n\nYes. I'm don't know how big a problem this is for those kinds of hosters; in \nour case, it is a big problem.\n\n> One thing that is missing from the above handwaving outline that your\n> \"different views\" offers is a \"consolidated view\", a pseudo-repository\n> that allows you to see refs from individual real (from the client's and\n> project participant's point of view) repositories as if they are in\n> individual subhierarchies of the ref namespace.\n\nI have been talking about a repository and different subsets or views of that \nrepository; you call the former a consolidated view and the latter a \nrepository. Those are really just two sides of the same coin.\n\n> I however suspect that you didn't want such a view in the first place if\n> there weren't issues around reachability. In other words, I suspect that\n> you invented it merely as one possible solution to the reachability issue,\n> and it was not your goal to have such a consolidated view by itself.\n\nI'm actually not sure. The \"consolidated view\" as you put it may be useful \nall by itself; it would be a proper, self sufficient git repository -- a \nreally nice property. it may be too painful to maintain this view though.\n\nWhen sharing objects across repositories, the worst-case scenario is that \nsomething goes wrong with the backlinks. You will eventually lose objects, \nbut it may take a while until it happens and until you notice, with a lot of \ndamage. That's nasty.\n\nA combination of the two approaches would be to \"link forward\" instead of \n\"linking back\", so that the consolidated view would maintain itself, with a \nserver repo setup like this:\n\n\t/repos/ABC:\n\t\tobjects\n\t\trefs/tags/A/\n\t\trefs/tags/B/\n\t\trefs/heads/A/\n\t\trefs/heads/B/\n\n\t/repos/A:\n\t\trefs/tags -> /repos/ABC/refs/tags/A/\n\t\trefs/heads -> /repos/ABC/refs/heads/A/\n\t\tobjects -> /repos/ABC/objects/\n\n\t/repos/B:\n\t\trefs/tags -> /repos/ABC/refs/tags/B/\n\t\trefs/heads -> /repos/ABC/refs/heads/B/\n\t\tobjects -> /repos/ABC/objects/\n\nThis could be made safe by not doing garbage collection if objects is a \nsymlink instead of a directory. (The ABC repo could be garbage collected as \nusual.) Am I overlooking anything why this can't work?\n\n\nThanks,\nAndreas\n"},{"id":"135728","messageId":"c376da901002252030p49126bf5tc5ffdca9f2ad13c1@mail.gmail.com","threadId":"22796","inReplyTo":"cover.1267029680.git.agruen@suse.de","subject":"Re: [RFC][PATCH 0/3] Different views on a repository","fromName":"Adam Brewster","fromEmail":"adambrewster@gmail.com","sentAt":"2010-02-26T04:30:33Z","receivedAt":"2010-02-26T04:30:33Z","isPatch":true,"sender":{"key":"adambrewster@gmail.com","avatar":"https://avatars.githubusercontent.com/u/223816?v=4"},"body":"On Wed, Feb 24, 2010 at 11:41 AM, Andreas Gruenbacher wrote:\n\n> Hello,\n>\n> we have a use case with groups of repositories which share lots of\n> objects, but which are logically independent. There is no strict\n> hierarchy between the repositories, the development modl is arbitrary.\n> The alternates mechanism for sharig objects between repositories won't\n> work.\n>\n\nI don't know what you're trying to accomplish, but for what it's\nworth, I do something similar with a couple of shell scripts.\n\nMy goal was to make bundles (basically thin packs) smaller by taking\nadvantage of files I knew were available on the far side of the\nair-gap even if they weren't in a the repository I was bundling that\nparticular day.\n\nThe idea was to push from all of my repositories into a super\nrepository with a fancy (and auto-generated) refspec. The actual code\nis impenetrable, but reconstructing it in everybody's favorite IDE,\ngmail, I came up with\n\n#!/bin/bash\nPROJECTS=$HOME/projects\nSUPER=$HOME/projects/.git-super-repo\n\n[[ -d \"$SUPER\" ]] || \\\n (mkdir \"$SUPER\"; git --git-dir=\"$SUPER\" init)\n\nfor i in $PROJECTS/*/.git; do\n name=$(basename \"$(dirname \"$0\")\")\n echo \"$SUPER/objects\" > $i/.git/objects/info/alternates\n git --git-dir=\"$i\" push -f \"$SUPER\" \"*:refs/$name/*\"\ndone\n\ngit --git-dir=\"$SUPER\" gc --aggressive\n\nfor i in $PROJECTS/*/.git; do\n git --git-dir=\"$i\" repack -Ad #unnecessary?\n git --git-dir=\"$i\" gc --aggressive\ndone\n\nClearly I can lose data if I try to rebase $SUPER or something, but I\nthink it's pretty safe for normal use.\n\nIn your case, the \"projects\" are called \"views\".\n\nAdam\n"},{"id":"135748","messageId":"201002261301.39243.agruen@suse.de","threadId":"22796","inReplyTo":"c376da901002252012s507a6921q922e606bdce4b4fa@mail.gmail.com","subject":"Re: [RFC][PATCH 0/3] Different views on a repository","fromName":"Andreas Gruenbacher","fromEmail":"agruen@suse.de","sentAt":"2010-02-26T12:01:39Z","receivedAt":"2010-02-26T12:01:39Z","isPatch":true,"sender":{"key":"agruen@suse.de","avatar":null},"body":"On Friday 26 February 2010 05:12:45 Adam Brewster wrote:\n> The idea was to push from all of my repositories into a super repository\n> with a fancy (and auto-generated) refspec. The actual code is\n> impenetrable, but reconstructing it in everybody's favorite IDE, gmail, I\n> came up with\n> \n> #!/bin/bash\n> PROJECTS=$HOME/projects\n> SUPER=$HOME/projects/.git-super-repo\n> \n> [[ -d \"$SUPER\" ]] || \\\n> (mkdir \"$SUPER\"; git --git-dir=\"$SUPER\" init)\n> \n> for i in $PROJECTS/*/.git; do\n> name=$(basename \"$(dirname \"$0\")\")\n> echo \"$SUPER/objects\" > $i/.git/objects/info/alternates\n> git --git-dir=\"$i\" push -f \"$SUPER\" \"*:refs/$name/*\"\n> done\n> \n> git --git-dir=\"$SUPER\" gc --aggressive\n> \n> for i in $PROJECTS/*/.git; do\n> git --git-dir=\"$i\" repack -Ad #unnecessary?\n> git --git-dir=\"$i\" gc --aggressive\n> done\n\nI see, when multiple projects share the same objects, you push them into those \nprojects independently first, and the script will later move them to $SUPER.\n\n> Clearly I can lose data if I try to rebase $SUPER or something, but I think\n> it's pretty safe for normal use.\n\nIt looks safe unless somebody messes with $SUPER. A lot of repacking will \nstill occur as part of moving stuff to $SUPER, though.\n\nI was trying to set things up so that this extra work won't be necessary in \nthe first place.\n\nThanks!\n\nAndreas\n"},{"id":"135777","messageId":"201002262235.43929.agruen@suse.de","threadId":"22796","inReplyTo":"201002260145.33960.agruen@suse.de","subject":"Re: [PATCH 2/3] Different views on a repository","fromName":"Andreas Gruenbacher","fromEmail":"agruen@suse.de","sentAt":"2010-02-26T21:35:43Z","receivedAt":"2010-02-26T21:35:43Z","isPatch":true,"sender":{"key":"agruen@suse.de","avatar":null},"body":"On Friday 26 February 2010 01:45:33 Andreas Gruenbacher wrote:\n> A combination of the two approaches would be to \"link forward\" instead of\n> \"linking back\", so that the consolidated view would maintain itself, with a\n> server repo setup like this:\n> \n> \t/repos/ABC:\n> \t\tobjects\n> \t\trefs/tags/A/\n> \t\trefs/tags/B/\n> \t\trefs/heads/A/\n> \t\trefs/heads/B/\n> \n> \t/repos/A:\n> \t\trefs/tags -> /repos/ABC/refs/tags/A/\n> \t\trefs/heads -> /repos/ABC/refs/heads/A/\n> \t\tobjects -> /repos/ABC/objects/\n> \n> \t/repos/B:\n> \t\trefs/tags -> /repos/ABC/refs/tags/B/\n> \t\trefs/heads -> /repos/ABC/refs/heads/B/\n> \t\tobjects -> /repos/ABC/objects/\n> \n> This could be made safe by not doing garbage collection if objects is a\n> symlink instead of a directory. (The ABC repo could be garbage collected\n> as usual.) Am I overlooking anything why this can't work?\n\nSelf reply: reference packing breaks this kind of setup. Crap.\n\nAndreas\n"}]}