{"thread":{"id":"32760","subject":"[PATCH v2 0/3] transfer.hiderefs","startedAt":"2013-01-29T00:13:29Z","lastAt":"2013-01-30T07:19:52Z","messageCount":6,"participants":["Junio C Hamano"],"isPatch":true,"patchVersion":2,"patchTotal":3},"messages":[{"id":"208138","messageId":"1359418412-26602-1-git-send-email-gitster@pobox.com","threadId":"32760","inReplyTo":null,"subject":"[PATCH v2 0/3] transfer.hiderefs","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2013-01-29T00:13:29Z","receivedAt":"2013-01-29T00:13:29Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"So this is the second round, unifying the handling of \"hidden refs\"\na hosting site administrator may want to handle differently from\nnormal refs available to upload-pack and receive-pack.  Big thanks\nto Peff for making me realize that what is shown to \"git push\"\nshould match what is shown to \"git fetch/clone\".\n\nThe configuration variable has been changed to \"transfer.hiderefs\"\nin this round.  I picked refs.[ch] as the shared location to place\nthe machinery that deals with this configuration, but I am not proud\nof the choice---there may be a more suitable place.\n\nThe third patch is new; it prevents a hidden ref to be deleted or\nupdated via \"git push\" to ensure that its contents are controlled\nonly from the server side using some side channel (e.g. pull-request\nUI on the Web).\n\nThere is no \"git fetch $there $exact_sha1_obtained_out_of_band\"\nyet; that must come next, together with a mechanism to control the\navailability of it on the server side.\n\nFor those who missed it, the first round starts at \n\n    http://thread.gmane.org/gmane.comp.version-control.git/213951\n\nJunio C Hamano (3):\n  upload-pack: share more code\n  upload-pack: allow hiding ref hiearchies\n  receive-pack: reject an attempt to update/delete a hidden ref\n\n Documentation/config.txt |  9 +++++++++\n builtin/receive-pack.c   | 24 +++++++++++++++++++++++\n refs.c                   | 41 +++++++++++++++++++++++++++++++++++++++\n refs.h                   |  3 +++\n t/t5512-ls-remote.sh     |  9 +++++++++\n t/t5516-fetch-push.sh    | 24 +++++++++++++++++++++++\n upload-pack.c            | 50 ++++++++++++++++++++++++++++++++----------------\n 7 files changed, 144 insertions(+), 16 deletions(-)\n\n-- \n1.8.1.2.548.g0e4986f\n"},{"id":"208139","messageId":"1359418412-26602-2-git-send-email-gitster@pobox.com","threadId":"32760","inReplyTo":"1359418412-26602-1-git-send-email-gitster@pobox.com","subject":"[PATCH v2 1/3] upload-pack: share more code","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2013-01-29T00:13:30Z","receivedAt":"2013-01-29T00:13:30Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"We mark the objects pointed at our refs with \"OUR_REF\" flag in two\nfunctions (mark_our_ref() and send_ref()), but we can just use the\nformer as a helper for the latter.\n\nUpdate the way mark_our_ref() prepares in-core object to use\nlookup_unknown_object() to delay reading the actual object data,\njust like we did in 435c833 (upload-pack: use peel_ref for ref\nadvertisements, 2012-10-04).\n\nSigned-off-by: Junio C Hamano <gitster@pobox.com>\n---\n upload-pack.c | 31 ++++++++++++++-----------------\n 1 file changed, 14 insertions(+), 17 deletions(-)\n\ndiff --git a/upload-pack.c b/upload-pack.c\nindex 95d8313..3dd220d 100644\n--- a/upload-pack.c\n+++ b/upload-pack.c\n@@ -722,15 +722,28 @@ static void receive_needs(void)\n \tfree(shallows.objects);\n }\n \n+static int mark_our_ref(const char *refname, const unsigned char *sha1, int flag, void *cb_data)\n+{\n+\tstruct object *o = lookup_unknown_object(sha1);\n+\tif (!o)\n+\t\tdie(\"git upload-pack: cannot find object %s:\", sha1_to_hex(sha1));\n+\tif (!(o->flags & OUR_REF)) {\n+\t\to->flags |= OUR_REF;\n+\t\tnr_our_refs++;\n+\t}\n+\treturn 0;\n+}\n+\n static int send_ref(const char *refname, const unsigned char *sha1, int flag, void *cb_data)\n {\n \tstatic const char *capabilities = \"multi_ack thin-pack side-band\"\n \t\t\" side-band-64k ofs-delta shallow no-progress\"\n \t\t\" include-tag multi_ack_detailed\";\n-\tstruct object *o = lookup_unknown_object(sha1);\n \tconst char *refname_nons = strip_namespace(refname);\n \tunsigned char peeled[20];\n \n+\tmark_our_ref(refname, sha1, flag, cb_data);\n+\n \tif (capabilities)\n \t\tpacket_write(1, \"%s %s%c%s%s agent=%s\\n\",\n \t\t\t     sha1_to_hex(sha1), refname_nons,\n@@ -740,27 +753,11 @@ static int send_ref(const char *refname, const unsigned char *sha1, int flag, vo\n \telse\n \t\tpacket_write(1, \"%s %s\\n\", sha1_to_hex(sha1), refname_nons);\n \tcapabilities = NULL;\n-\tif (!(o->flags & OUR_REF)) {\n-\t\to->flags |= OUR_REF;\n-\t\tnr_our_refs++;\n-\t}\n \tif (!peel_ref(refname, peeled))\n \t\tpacket_write(1, \"%s %s^{}\\n\", sha1_to_hex(peeled), refname_nons);\n \treturn 0;\n }\n \n-static int mark_our_ref(const char *refname, const unsigned char *sha1, int flag, void *cb_data)\n-{\n-\tstruct object *o = parse_object(sha1);\n-\tif (!o)\n-\t\tdie(\"git upload-pack: cannot find object %s:\", sha1_to_hex(sha1));\n-\tif (!(o->flags & OUR_REF)) {\n-\t\to->flags |= OUR_REF;\n-\t\tnr_our_refs++;\n-\t}\n-\treturn 0;\n-}\n-\n static void upload_pack(void)\n {\n \tif (advertise_refs || !stateless_rpc) {\n-- \n1.8.1.2.548.g0e4986f\n"},{"id":"208140","messageId":"1359418412-26602-3-git-send-email-gitster@pobox.com","threadId":"32760","inReplyTo":"1359418412-26602-1-git-send-email-gitster@pobox.com","subject":"[PATCH v2 2/3] upload-pack: allow hiding ref hiearchies","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2013-01-29T00:13:31Z","receivedAt":"2013-01-29T00:13:31Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Teach upload-pack to omit some refs from the initial advertisement\nby introducing the uploadPack.hiderefs multivalued configuration\nvariable.  Any ref that is under the hierarchies listed on the value\nof this variable is excluded from responses to \"ls-remote\", \"fetch\"\nor \"clone\" requests.  One typical use case may be\n\n\t[uploadPack]\n\t\thiderefs = refs/changes\n\nNote that the underlying protocol still allows a request to fetch\nobjects at the tip of any ref, including the hidden ones, but on the\nclient side (e.g. fetch-pack), the requests are checked against the\nls-remote response, so it cannot ask for refs/changes/14/432432, or\nfor the object name (which is what eventually goes over the wire on\n\"want\" line) the user may obtain out of band (e.g. Gerrit dashboard).\nA new capability \"allow-tip-sha1-in-want\" is returned by upload-pack\nto signal updated clients that it may be OK to ask for objects that\nwere not advertised.\n\nSigned-off-by: Junio C Hamano <gitster@pobox.com>\n---\n Documentation/config.txt |  8 ++++++++\n builtin/receive-pack.c   |  8 ++++++++\n refs.c                   | 41 +++++++++++++++++++++++++++++++++++++++++\n refs.h                   |  3 +++\n t/t5512-ls-remote.sh     |  9 +++++++++\n upload-pack.c            | 25 +++++++++++++++++++++++--\n 6 files changed, 92 insertions(+), 2 deletions(-)\n\ndiff --git a/Documentation/config.txt b/Documentation/config.txt\nindex ef45c99..8e2853e 100644\n--- a/Documentation/config.txt\n+++ b/Documentation/config.txt\n@@ -2057,6 +2057,14 @@ transfer.fsckObjects::\n \tnot set, the value of this variable is used instead.\n \tDefaults to false.\n \n+transfer.hiderefs::\n+\tString(s) `upload-pack` and `receive-pack` use to decide\n+\twhich refs to omit from their initial advertisement.  Use\n+\tmore than one transfer.hiderefs configuration variables to\n+\tspecify multiple prefix strings. A ref that has one of these\n+\tstrings as its prefix is excluded, and is hidden from\n+\t`git clone --mirror` and `git push :`.\n+\n transfer.unpackLimit::\n \tWhen `fetch.unpackLimit` or `receive.unpackLimit` are\n \tnot set, the value of this variable is used instead.\ndiff --git a/builtin/receive-pack.c b/builtin/receive-pack.c\nindex ff781fe..5ed40be 100644\n--- a/builtin/receive-pack.c\n+++ b/builtin/receive-pack.c\n@@ -59,6 +59,11 @@ static enum deny_action parse_deny_action(const char *var, const char *value)\n \n static int receive_pack_config(const char *var, const char *value, void *cb)\n {\n+\tint status = parse_hide_refs_config(var, value, cb);\n+\n+\tif (status)\n+\t\treturn status;\n+\n \tif (strcmp(var, \"receive.denydeletes\") == 0) {\n \t\tdeny_deletes = git_config_bool(var, value);\n \t\treturn 0;\n@@ -119,6 +124,9 @@ static int receive_pack_config(const char *var, const char *value, void *cb)\n \n static void show_ref(const char *path, const unsigned char *sha1)\n {\n+\tif (ref_is_hidden(path))\n+\t\treturn;\n+\n \tif (sent_capabilities)\n \t\tpacket_write(1, \"%s %s\\n\", sha1_to_hex(sha1), path);\n \telse\ndiff --git a/refs.c b/refs.c\nindex 541fec2..e3574ca 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -3,6 +3,7 @@\n #include \"object.h\"\n #include \"tag.h\"\n #include \"dir.h\"\n+#include \"string-list.h\"\n \n /*\n  * Make sure \"ref\" is something reasonable to have under \".git/refs/\";\n@@ -2556,3 +2557,43 @@ char *shorten_unambiguous_ref(const char *refname, int strict)\n \tfree(short_name);\n \treturn xstrdup(refname);\n }\n+\n+static struct string_list *hide_refs;\n+\n+int parse_hide_refs_config(const char *var, const char *value, void *unused)\n+{\n+\tif (!strcmp(\"transfer.hiderefs\", var)) {\n+\t\tchar *ref;\n+\t\tint len;\n+\n+\t\tif (!value)\n+\t\t\treturn config_error_nonbool(var);\n+\t\tref = xstrdup(value);\n+\t\tlen = strlen(ref);\n+\t\twhile (len && ref[len - 1] == '/')\n+\t\t\tref[--len] = '\\0';\n+\t\tif (!hide_refs) {\n+\t\t\thide_refs = xcalloc(1, sizeof(*hide_refs));\n+\t\t\thide_refs->strdup_strings = 1;\n+\t\t}\n+\t\tstring_list_append(hide_refs, ref);\n+\t}\n+\treturn 0;\n+}\n+\n+int ref_is_hidden(const char *refname)\n+{\n+\tstruct string_list_item *item;\n+\n+\tif (!hide_refs)\n+\t\treturn 0;\n+\tfor_each_string_list_item(item, hide_refs) {\n+\t\tint len;\n+\t\tif (prefixcmp(refname, item->string))\n+\t\t\tcontinue;\n+\t\tlen = strlen(item->string);\n+\t\tif (!refname[len] || refname[len] == '/')\n+\t\t\treturn 1;\n+\t}\n+\treturn 0;\n+}\ndiff --git a/refs.h b/refs.h\nindex d6c2fe2..50b233f 100644\n--- a/refs.h\n+++ b/refs.h\n@@ -147,4 +147,7 @@ int update_ref(const char *action, const char *refname,\n \t\tconst unsigned char *sha1, const unsigned char *oldval,\n \t\tint flags, enum action_on_err onerr);\n \n+extern int parse_hide_refs_config(const char *var, const char *value, void *);\n+extern int ref_is_hidden(const char *);\n+\n #endif /* REFS_H */\ndiff --git a/t/t5512-ls-remote.sh b/t/t5512-ls-remote.sh\nindex d16e5d3..d0702ed 100755\n--- a/t/t5512-ls-remote.sh\n+++ b/t/t5512-ls-remote.sh\n@@ -126,4 +126,13 @@ test_expect_success 'Report match with --exit-code' '\n \ttest_cmp expect actual\n '\n \n+test_expect_success 'Hide some refs' '\n+\ttest_config transfer.hiderefs refs/tags &&\n+\tgit ls-remote . >actual &&\n+\ttest_unconfig transfer.hiderefs &&\n+\tgit ls-remote . |\n+\tsed -e \"/\trefs\\/tags\\//d\" >expect &&\n+\ttest_cmp expect actual\n+'\n+\n test_done\ndiff --git a/upload-pack.c b/upload-pack.c\nindex 3dd220d..31ce954 100644\n--- a/upload-pack.c\n+++ b/upload-pack.c\n@@ -12,6 +12,7 @@\n #include \"run-command.h\"\n #include \"sigchain.h\"\n #include \"version.h\"\n+#include \"string-list.h\"\n \n static const char upload_pack_usage[] = \"git upload-pack [--strict] [--timeout=<n>] <dir>\";\n \n@@ -28,10 +29,12 @@ static const char upload_pack_usage[] = \"git upload-pack [--strict] [--timeout=<\n \n static unsigned long oldest_have;\n \n-static int multi_ack, nr_our_refs;\n+static int multi_ack;\n+static int nr_our_refs; /* This counts both advertised and unadvertised */\n static int no_done;\n static int use_thin_pack, use_ofs_delta, use_include_tag;\n static int no_progress, daemon_mode;\n+static int allow_tip_sha1_in_want;\n static int shallow_nr;\n static struct object_array have_obj;\n static struct object_array want_obj;\n@@ -743,11 +746,14 @@ static int send_ref(const char *refname, const unsigned char *sha1, int flag, vo\n \tunsigned char peeled[20];\n \n \tmark_our_ref(refname, sha1, flag, cb_data);\n+\tif (ref_is_hidden(refname))\n+\t\treturn 0;\n \n \tif (capabilities)\n-\t\tpacket_write(1, \"%s %s%c%s%s agent=%s\\n\",\n+\t\tpacket_write(1, \"%s %s%c%s%s%s agent=%s\\n\",\n \t\t\t     sha1_to_hex(sha1), refname_nons,\n \t\t\t     0, capabilities,\n+\t\t\t     allow_tip_sha1_in_want ? \" allow-tip-sha1-in-want\" : \"\",\n \t\t\t     stateless_rpc ? \" no-done\" : \"\",\n \t\t\t     git_user_agent_sanitized());\n \telse\n@@ -758,11 +764,20 @@ static int send_ref(const char *refname, const unsigned char *sha1, int flag, vo\n \treturn 0;\n }\n \n+static int check_hidden_ref(const char *refname, const unsigned char *sha1, int flag, void *cb_data)\n+{\n+\tif (!ref_is_hidden(refname))\n+\t\treturn 0;\n+\tallow_tip_sha1_in_want = 1;\n+\treturn 1; /* terminate iteration over refs early */\n+}\n+\n static void upload_pack(void)\n {\n \tif (advertise_refs || !stateless_rpc) {\n \t\treset_timeout();\n \t\thead_ref_namespaced(send_ref, NULL);\n+\t\tfor_each_namespaced_ref(check_hidden_ref, NULL);\n \t\tfor_each_namespaced_ref(send_ref, NULL);\n \t\tpacket_flush(1);\n \t} else {\n@@ -779,6 +794,11 @@ static void upload_pack(void)\n \t}\n }\n \n+static int upload_pack_config(const char *var, const char *value, void *unused)\n+{\n+\treturn parse_hide_refs_config(var, value, unused);\n+}\n+\n int main(int argc, char **argv)\n {\n \tchar *dir;\n@@ -830,6 +850,7 @@ int main(int argc, char **argv)\n \t\tdie(\"'%s' does not appear to be a git repository\", dir);\n \tif (is_repository_shallow())\n \t\tdie(\"attempt to fetch/clone from a shallow repository\");\n+\tgit_config(upload_pack_config, NULL);\n \tif (getenv(\"GIT_DEBUG_SEND_PACK\"))\n \t\tdebug_fd = atoi(getenv(\"GIT_DEBUG_SEND_PACK\"));\n \tupload_pack();\n-- \n1.8.1.2.548.g0e4986f\n"},{"id":"208141","messageId":"1359418412-26602-4-git-send-email-gitster@pobox.com","threadId":"32760","inReplyTo":"1359418412-26602-1-git-send-email-gitster@pobox.com","subject":"[PATCH v2 3/3] receive-pack: reject an attempt to update/delete a hidden ref","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2013-01-29T00:13:32Z","receivedAt":"2013-01-29T00:13:32Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"An attempt to update or delete a ref that is hidden from \"git push\"\nis rejected.  With this the server side can implement refs that are\nonly available for its own use, e.g. refs/pull/11/head used to hold\nan incoming pull request at GitHub.\n\nSigned-off-by: Junio C Hamano <gitster@pobox.com>\n---\n Documentation/config.txt |  3 ++-\n builtin/receive-pack.c   | 16 ++++++++++++++++\n t/t5516-fetch-push.sh    | 24 ++++++++++++++++++++++++\n 3 files changed, 42 insertions(+), 1 deletion(-)\n\ndiff --git a/Documentation/config.txt b/Documentation/config.txt\nindex 8e2853e..b7b407b 100644\n--- a/Documentation/config.txt\n+++ b/Documentation/config.txt\n@@ -2063,7 +2063,8 @@ transfer.hiderefs::\n \tmore than one transfer.hiderefs configuration variables to\n \tspecify multiple prefix strings. A ref that has one of these\n \tstrings as its prefix is excluded, and is hidden from\n-\t`git clone --mirror` and `git push :`.\n+\t`git clone --mirror` and `git push :`.  An attempt to update\n+\tor delete a hidden ref by `git push` is\trejected.\n \n transfer.unpackLimit::\n \tWhen `fetch.unpackLimit` or `receive.unpackLimit` are\ndiff --git a/builtin/receive-pack.c b/builtin/receive-pack.c\nindex 5ed40be..a8248d9 100644\n--- a/builtin/receive-pack.c\n+++ b/builtin/receive-pack.c\n@@ -696,6 +696,20 @@ static int iterate_receive_command_list(void *cb_data, unsigned char sha1[20])\n \treturn -1; /* end of list */\n }\n \n+static void reject_updates_to_hidden(struct command *commands)\n+{\n+\tstruct command *cmd;\n+\n+\tfor (cmd = commands; cmd; cmd = cmd->next) {\n+\t\tif (cmd->error_string || !ref_is_hidden(cmd->ref_name))\n+\t\t\tcontinue;\n+\t\tif (is_null_sha1(cmd->new_sha1))\n+\t\t\tcmd->error_string = \"deny deleting a hidden ref\";\n+\t\telse\n+\t\t\tcmd->error_string = \"deny updating a hidden ref\";\n+\t}\n+}\n+\n static void execute_commands(struct command *commands, const char *unpacker_error)\n {\n \tstruct command *cmd;\n@@ -712,6 +726,8 @@ static void execute_commands(struct command *commands, const char *unpacker_erro\n \t\t\t\t       0, &cmd))\n \t\tset_connectivity_errors(commands);\n \n+\treject_updates_to_hidden(commands);\n+\n \tif (run_receive_hook(commands, pre_receive_hook, 0)) {\n \t\tfor (cmd = commands; cmd; cmd = cmd->next) {\n \t\t\tif (!cmd->error_string)\ndiff --git a/t/t5516-fetch-push.sh b/t/t5516-fetch-push.sh\nindex 6009372..8e8287a 100755\n--- a/t/t5516-fetch-push.sh\n+++ b/t/t5516-fetch-push.sh\n@@ -1037,4 +1037,28 @@ test_expect_success 'push --prune refspec' '\n \t! check_push_result $the_first_commit tmp/foo tmp/bar\n '\n \n+test_expect_success 'push to update a hidden ref' '\n+\tmk_test heads/master hidden/one hidden/two hidden/three &&\n+\t(\n+\t\tcd testrepo &&\n+\t\tgit config transfer.hiderefs refs/hidden\n+\t) &&\n+\n+\t# push to unhidden ref succeeds normally\n+\tgit push testrepo master:refs/heads/master &&\n+\tcheck_push_result $the_commit heads/master &&\n+\n+\t# push to update a hidden ref should fail\n+\ttest_must_fail git push testrepo master:refs/hidden/one &&\n+\tcheck_push_result $the_first_commit hidden/one &&\n+\n+\t# push to delete a hidden ref should fail\n+\ttest_must_fail git push testrepo master:refs/hidden/two &&\n+\tcheck_push_result $the_first_commit hidden/two &&\n+\n+\t# idempotent push to update a hidden ref should fail\n+\ttest_must_fail git push testrepo $the_first_commit:refs/hidden/three &&\n+\tcheck_push_result $the_first_commit hidden/three\n+'\n+\n test_done\n-- \n1.8.1.2.548.g0e4986f\n"},{"id":"208151","messageId":"7vham0tvus.fsf@alter.siamese.dyndns.org","threadId":"32760","inReplyTo":"1359418412-26602-1-git-send-email-gitster@pobox.com","subject":"Re: [PATCH v2 0/3] transfer.hiderefs","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2013-01-29T05:13:31Z","receivedAt":"2013-01-29T05:13:31Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Please take this as just a preview of early WIP.  I think I may end\nup doing moderate amount of refactoring as a preparatory step before\nthese patches, so nitpick-reviews are likely to become waste of\nreviewer's time at this point.\n"},{"id":"208250","messageId":"7vfw1jm92f.fsf@alter.siamese.dyndns.org","threadId":"32760","inReplyTo":"7vham0tvus.fsf@alter.siamese.dyndns.org","subject":"Re: [PATCH v2 0/3] transfer.hiderefs","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2013-01-30T07:19:52Z","receivedAt":"2013-01-30T07:19:52Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Junio C Hamano <gitster@pobox.com> writes:\n\n> Please take this as just a preview of early WIP.  I think I may end\n> up doing moderate amount of refactoring as a preparatory step before\n> these patches, so nitpick-reviews are likely to become waste of\n> reviewer's time at this point.\n\nI've pushed out a mostly-done reroll on 'pu'; I'll send them out as\npatches for review tomorrow.  I think I am making a good progress.\n"}]}