{"thread":{"id":"51138","subject":"HEAD and namespaces","startedAt":"2019-05-21T02:59:16Z","lastAt":"2019-05-28T17:01:31Z","messageCount":6,"participants":["Bryan Turner","Jeff King","Ævar Arnfjörð Bjarmason","Junio C Hamano"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"375980","messageId":"CAGyf7-Ew8rS8n67Few9+dT6VXy9rQbaYLswuZkQ3-4j_T_d=qg@mail.gmail.com","threadId":"51138","inReplyTo":null,"subject":"HEAD and namespaces","fromName":"Bryan Turner","fromEmail":"bturner@atlassian.com","sentAt":"2019-05-21T02:59:03Z","receivedAt":"2019-05-21T02:59:16Z","isPatch":false,"sender":{"key":"bturner@atlassian.com","avatar":"https://gravatar.com/avatar/16bcf3167981c1ef7c804e502642366d888a35b0d0b0a4ca01fdc442aa1acb1e?d=mp&s=160"},"body":"When using GIT_NAMESPACE, it appears the \"symref\" added to the\ncapabilities advertisement doesn't get the namespace stripped. The\nnamespace is stripped for the advertised refs, including \"HEAD\", but\nnot on the \"symref\".\n\nLafiel:test bturner$ GIT_NAMESPACE=foo GIT_TRACE_PACKET=1 git ls-remote origin\n19:51:54.012696 pkt-line.c:80           packet:          git<\nb9acca03606d4c674be8b7e79cd788cedbec957c HEAD\\0multi_ack thin-pack\nside-band side-band-64k ofs-delta shallow deepen-since deepen-not\ndeepen-relative no-progress include-tag multi_ack_detailed\nsymref=refs/namespaces/foo/HEAD:refs/namespaces/foo/refs/heads/master\nagent=git/2.20.1\n19:51:54.013017 pkt-line.c:80           packet:          git<\nb9acca03606d4c674be8b7e79cd788cedbec957c refs/heads/master\n19:51:54.013028 pkt-line.c:80           packet:          git< 0000\n19:51:54.013035 pkt-line.c:80           packet:          git> 0000\nb9acca03606d4c674be8b7e79cd788cedbec957c HEAD\nb9acca03606d4c674be8b7e79cd788cedbec957c refs/heads/master\n\nI don't know whether the client somehow does some stripping on the\n\"symref\" to decide that HEAD should be\n\"refs/namespaces/foo/refs/heads/master\", but I'd assume not. Either\nway, since the clone can't find the HEAD value (either because it\ndoesn't see \"refs/namespaces/foo/HEAD\" as HEAD, which I'd assume is\nthe case, or because it can't find\n\"refs/namespaces/foo/refs/heads/master\"), it falls back on the old\nbehavior of picking the first branch with a matching commit. That\nbrings back all the \"wrong branch\" issues with cloning that adding\n\"symref\"  fixed.\n\nIs this a bug? An oversight? An intentional decision? How is HEAD\nsupposed to work when using GIT_NAMESPACE? Perhaps the expectation is\nthat namespaces won't have their own HEADs? I'd say perhaps it's that\neven in the namespace the target ref shouldn't be namespaced, but that\ndoesn't seem like it could possibly be correct since the namespace\ncould contain refs that don't exist outside it, so Git would see the\nsymbolic ref as broken.\n\nAny insight would be greatly appreciated!\n\nBryan\n"},{"id":"376019","messageId":"20190521214630.GD14807@sigill.intra.peff.net","threadId":"51138","inReplyTo":"CAGyf7-Ew8rS8n67Few9+dT6VXy9rQbaYLswuZkQ3-4j_T_d=qg@mail.gmail.com","subject":"Re: HEAD and namespaces","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2019-05-21T21:46:30Z","receivedAt":"2019-05-21T21:46:33Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Mon, May 20, 2019 at 07:59:03PM -0700, Bryan Turner wrote:\n\n> When using GIT_NAMESPACE, it appears the \"symref\" added to the\n> capabilities advertisement doesn't get the namespace stripped. The\n> namespace is stripped for the advertised refs, including \"HEAD\", but\n> not on the \"symref\".\n> \n> Lafiel:test bturner$ GIT_NAMESPACE=foo GIT_TRACE_PACKET=1 git ls-remote origin\n> 19:51:54.012696 pkt-line.c:80           packet:          git<\n> b9acca03606d4c674be8b7e79cd788cedbec957c HEAD\\0multi_ack thin-pack\n> side-band side-band-64k ofs-delta shallow deepen-since deepen-not\n> deepen-relative no-progress include-tag multi_ack_detailed\n> symref=refs/namespaces/foo/HEAD:refs/namespaces/foo/refs/heads/master\n\nYeah, that seems totally broken. It's neat that we find HEAD inside the\nnamespace. But I think there are two bugs here:\n\n  1. We must call it \"HEAD\" and not the nonsense\n     refs/namespaces/foo/HEAD\n\n  2. We need to similarly remove the namespace from its contents. I\n     suppose it's possible that somebody stores the already-truncated\n     name in the symref, but as you noted then such a symref would be\n     totally broken from the perspective of the non-namespaced repo.\n\n> I don't know whether the client somehow does some stripping on the\n> \"symref\" to decide that HEAD should be\n> \"refs/namespaces/foo/refs/heads/master\", but I'd assume not.\n\nI don't think it can. It has no idea that namespaces are in play at all\n(and the point of the feature is for it to be invisible to the client).\n\n> Is this a bug? An oversight? An intentional decision? How is HEAD\n> supposed to work when using GIT_NAMESPACE? Perhaps the expectation is\n> that namespaces won't have their own HEADs? I'd say perhaps it's that\n> even in the namespace the target ref shouldn't be namespaced, but that\n> doesn't seem like it could possibly be correct since the namespace\n> could contain refs that don't exist outside it, so Git would see the\n> symbolic ref as broken.\n\nI think it should be doing:\n\ndiff --git a/upload-pack.c b/upload-pack.c\nindex 24298913c0..4d2129e7fc 100644\n--- a/upload-pack.c\n+++ b/upload-pack.c\n@@ -1037,8 +1037,8 @@ static int find_symref(const char *refname, const struct object_id *oid,\n \tsymref_target = resolve_ref_unsafe(refname, 0, NULL, &flag);\n \tif (!symref_target || (flag & REF_ISSYMREF) == 0)\n \t\tdie(\"'%s' is a symref but it is not?\", refname);\n-\titem = string_list_append(cb_data, refname);\n-\titem->util = xstrdup(symref_target);\n+\titem = string_list_append(cb_data, strip_namespace(refname));\n+\titem->util = xstrdup(strip_namespace(symref_target));\n \treturn 0;\n }\n \n\nand this case has just been broken since 7171d8c15f (upload-pack: send\nsymbolic ref information as capability, 2013-09-17). It mostly works\nbecause of fallback to the old guessing behavior, so nobody really\nnoticed. (I also suspect namespaces are not widely used at all, but I\ncould be wrong).\n\n-Peff\n"},{"id":"376047","messageId":"20190522043146.GA2563@sigill.intra.peff.net","threadId":"51138","inReplyTo":"20190521214630.GD14807@sigill.intra.peff.net","subject":"[PATCH] upload-pack: strip namespace from symref data","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2019-05-22T04:31:46Z","receivedAt":"2019-05-22T04:31:50Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"Since 7171d8c15f (upload-pack: send symbolic ref information as\ncapability, 2013-09-17), we've sent cloning and fetching clients special\ninformation about which branch HEAD is pointing to, so that they don't\nhave to guess based on matching up commit ids.\n\nHowever, this feature has never worked properly with the GIT_NAMESPACE\nfeature.  Because upload-pack uses head_ref_namespaced(find_symref), we\ndo find and report on refs/namespaces/foo/HEAD instead of the actual\nHEAD of the repo. This makes sense, since the branch pointed to by the\ntop-level HEAD may not be advertised at all. But we do two things wrong:\n\n  1. We report the full name refs/namespaces/foo/HEAD, instead of just\n     HEAD. Meaning no client is going to bother doing anything with that\n     symref, since we're not otherwise advertising it.\n\n  2. We report the symref destination using its full name (e.g.,\n     refs/namespaces/foo/refs/heads/master). That's similarly useless to\n     the client, who only saw \"refs/heads/master\" in the advertisement.\n\nWe should be stripping the namespace prefix off of both places (which\nthis patch fixes).\n\nLikely nobody noticed because bug (1) means that from the client's\nperspective, we did not report on HEAD at all. And thus it uses the\npre-7171d8c15f fallback code to guess the correct HEAD, which is usually\nright. It only falls down in ambiguous cases (like the one laid out in\nthe included test).\n\nThis also means that we don't have to worry about breaking anybody who\nwas putting pre-stripped names into their namespace symrefs when we fix\nbug (2). Because of bug (1), nobody would have been using the symref we\nadvertised in the first place (not to mention that those symrefs would\nhave appeared broken for any non-namespaced access).\n\nNote that we have separate fixes here for the v0 and v2 protocols. The\nsymref advertisement moved in v2 to be a part of the ls-refs command.\nThis actually gets part (1) right, since the symref annotation\npiggy-backs on the existing ref advertisement, which is properly\nstripped. But it still needs a fix for part (2). The included tests\ncover both protocols.\n\nReported-by: Bryan Turner <bturner@atlassian.com>\nSigned-off-by: Jeff King <peff@peff.net>\n---\nThis is the same as my earlier fix, but with the v2 bit added, and of\ncourse tests and a commit message. Thanks (as usual) for a helpful bug\nreport.\n\nI don't know if we have a general philosophy for testing v0 versus v2.\nWithout specifying the protocol at all, we'd catch the former on a\nregular run and the latter under a GIT_TEST_PROTOCOL_VERSION=2 run. So\nwe _could_ just rely on that, but since I had to do two separate fixes,\nit made sense to me to include explicit tests.\n\n ls-refs.c                        |  3 ++-\n t/t5509-fetch-push-namespaces.sh | 28 ++++++++++++++++++++++++++++\n upload-pack.c                    |  4 ++--\n 3 files changed, 32 insertions(+), 3 deletions(-)\n\ndiff --git a/ls-refs.c b/ls-refs.c\nindex 0a7dbc6442..818aef70a0 100644\n--- a/ls-refs.c\n+++ b/ls-refs.c\n@@ -57,7 +57,8 @@ static int send_ref(const char *refname, const struct object_id *oid,\n \t\tif (!symref_target)\n \t\t\tdie(\"'%s' is a symref but it is not?\", refname);\n \n-\t\tstrbuf_addf(&refline, \" symref-target:%s\", symref_target);\n+\t\tstrbuf_addf(&refline, \" symref-target:%s\",\n+\t\t\t    strip_namespace(symref_target));\n \t}\n \n \tif (data->peel) {\ndiff --git a/t/t5509-fetch-push-namespaces.sh b/t/t5509-fetch-push-namespaces.sh\nindex c88df78c0b..75cbfcc392 100755\n--- a/t/t5509-fetch-push-namespaces.sh\n+++ b/t/t5509-fetch-push-namespaces.sh\n@@ -124,4 +124,32 @@ test_expect_success 'try to update a hidden full ref' '\n \ttest_must_fail git -C original push pushee-namespaced master\n '\n \n+test_expect_success 'set up ambiguous HEAD' '\n+\tgit init ambiguous &&\n+\t(\n+\t\tcd ambiguous &&\n+\t\tgit commit --allow-empty -m foo &&\n+\t\tgit update-ref refs/namespaces/ns/refs/heads/one HEAD &&\n+\t\tgit update-ref refs/namespaces/ns/refs/heads/two HEAD &&\n+\t\tgit symbolic-ref refs/namespaces/ns/HEAD \\\n+\t\t\trefs/namespaces/ns/refs/heads/two\n+\t)\n+'\n+\n+test_expect_success 'clone chooses correct HEAD (v0)' '\n+\tGIT_NAMESPACE=ns git -c protocol.version=0 \\\n+\t\tclone ambiguous ambiguous-v0 &&\n+\techo refs/heads/two >expect &&\n+\tgit -C ambiguous-v0 symbolic-ref HEAD >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success 'clone chooses correct HEAD (v2)' '\n+\tGIT_NAMESPACE=ns git -c protocol.version=2 \\\n+\t\tclone ambiguous ambiguous-v2 &&\n+\techo refs/heads/two >expect &&\n+\tgit -C ambiguous-v2 symbolic-ref HEAD >actual &&\n+\ttest_cmp expect actual\n+'\n+\n test_done\ndiff --git a/upload-pack.c b/upload-pack.c\nindex 24298913c0..4d2129e7fc 100644\n--- a/upload-pack.c\n+++ b/upload-pack.c\n@@ -1037,8 +1037,8 @@ static int find_symref(const char *refname, const struct object_id *oid,\n \tsymref_target = resolve_ref_unsafe(refname, 0, NULL, &flag);\n \tif (!symref_target || (flag & REF_ISSYMREF) == 0)\n \t\tdie(\"'%s' is a symref but it is not?\", refname);\n-\titem = string_list_append(cb_data, refname);\n-\titem->util = xstrdup(symref_target);\n+\titem = string_list_append(cb_data, strip_namespace(refname));\n+\titem->util = xstrdup(strip_namespace(symref_target));\n \treturn 0;\n }\n \n-- \n2.22.0.rc1.539.g7bfcdfe86d\n\n\n\n"},{"id":"376057","messageId":"875zq2vkvf.fsf@evledraar.gmail.com","threadId":"51138","inReplyTo":"20190522043146.GA2563@sigill.intra.peff.net","subject":"Re: [PATCH] upload-pack: strip namespace from symref data","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2019-05-22T10:33:56Z","receivedAt":"2019-05-22T10:34:02Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Wed, May 22 2019, Jeff King wrote:\n\n> Since 7171d8c15f (upload-pack: send symbolic ref information as\n> capability, 2013-09-17), we've sent cloning and fetching clients special\n> information about which branch HEAD is pointing to, so that they don't\n> have to guess based on matching up commit ids.\n\nI'd add a 2nd paragraph here:\n\n    The client will then use the corresponding code added in a45b5f0552\n    (\"connect: annotate refs with their symref information in\n    get_remote_head()\", 2013-09-17) to find the intended symref. See\n    9907d1359c (\"Merge branch 'jc/upload-pack-send-symref'\", 2013-10-30)\n    for the full set of changes related to this.\n\n...because, see later...\n\n> However, this feature has never worked properly with the GIT_NAMESPACE\n> feature.  Because upload-pack uses head_ref_namespaced(find_symref), we\n> do find and report on refs/namespaces/foo/HEAD instead of the actual\n> HEAD of the repo. This makes sense, since the branch pointed to by the\n> top-level HEAD may not be advertised at all. But we do two things wrong:\n>\n>   1. We report the full name refs/namespaces/foo/HEAD, instead of just\n>      HEAD. Meaning no client is going to bother doing anything with that\n>      symref, since we're not otherwise advertising it.\n>\n>   2. We report the symref destination using its full name (e.g.,\n>      refs/namespaces/foo/refs/heads/master). That's similarly useless to\n>      the client, who only saw \"refs/heads/master\" in the advertisement.\n>\n> We should be stripping the namespace prefix off of both places (which\n> this patch fixes).\n>\n> Likely nobody noticed because bug (1) means that from the client's\n> perspective, we did not report on HEAD at all. And thus it uses the\n> pre-7171d8c15f fallback code to guess the correct HEAD, which is usually\n> right. It only falls down in ambiguous cases (like the one laid out in\n> the included test).\n\n...because here you're talking about \"the client's perspective\" and \"it\nuses the pre-7171d8c15f [...] code\", but this should say \"the\npre-a45b5f0552 code\", i.e. mention the commit that changed the *client*\nlogic.\n\nWell, the client also \"uses\" the server-side code indirectly, but I\nthink it's easier to follow along if we note what both sides are (as I\ndug up when reviewing this...).\n\n> This also means that we don't have to worry about breaking anybody who\n> was putting pre-stripped names into their namespace symrefs when we fix\n> bug (2). Because of bug (1), nobody would have been using the symref we\n> advertised in the first place (not to mention that those symrefs would\n> have appeared broken for any non-namespaced access).\n>\n> Note that we have separate fixes here for the v0 and v2 protocols. The\n> symref advertisement moved in v2 to be a part of the ls-refs command.\n> This actually gets part (1) right, since the symref annotation\n> piggy-backs on the existing ref advertisement, which is properly\n> stripped. But it still needs a fix for part (2). The included tests\n> cover both protocols.\n>\n> Reported-by: Bryan Turner <bturner@atlassian.com>\n> Signed-off-by: Jeff King <peff@peff.net>\n> ---\n> This is the same as my earlier fix, but with the v2 bit added, and of\n> course tests and a commit message. Thanks (as usual) for a helpful bug\n> report.\n>\n> I don't know if we have a general philosophy for testing v0 versus v2.\n> Without specifying the protocol at all, we'd catch the former on a\n> regular run and the latter under a GIT_TEST_PROTOCOL_VERSION=2 run. So\n> we _could_ just rely on that, but since I had to do two separate fixes,\n> it made sense to me to include explicit tests.\n\nYeah, we shouldn't rely on the GIT_TEST_* stuff for coverage, it should\njust be used to find things that fall between the cracks.\n\n>  ls-refs.c                        |  3 ++-\n>  t/t5509-fetch-push-namespaces.sh | 28 ++++++++++++++++++++++++++++\n>  upload-pack.c                    |  4 ++--\n>  3 files changed, 32 insertions(+), 3 deletions(-)\n>\n> diff --git a/ls-refs.c b/ls-refs.c\n> index 0a7dbc6442..818aef70a0 100644\n> --- a/ls-refs.c\n> +++ b/ls-refs.c\n> @@ -57,7 +57,8 @@ static int send_ref(const char *refname, const struct object_id *oid,\n>  \t\tif (!symref_target)\n>  \t\t\tdie(\"'%s' is a symref but it is not?\", refname);\n>\n> -\t\tstrbuf_addf(&refline, \" symref-target:%s\", symref_target);\n> +\t\tstrbuf_addf(&refline, \" symref-target:%s\",\n> +\t\t\t    strip_namespace(symref_target));\n>  \t}\n>\n>  \tif (data->peel) {\n> diff --git a/t/t5509-fetch-push-namespaces.sh b/t/t5509-fetch-push-namespaces.sh\n> index c88df78c0b..75cbfcc392 100755\n> --- a/t/t5509-fetch-push-namespaces.sh\n> +++ b/t/t5509-fetch-push-namespaces.sh\n> @@ -124,4 +124,32 @@ test_expect_success 'try to update a hidden full ref' '\n>  \ttest_must_fail git -C original push pushee-namespaced master\n>  '\n>\n> +test_expect_success 'set up ambiguous HEAD' '\n> +\tgit init ambiguous &&\n> +\t(\n> +\t\tcd ambiguous &&\n> +\t\tgit commit --allow-empty -m foo &&\n> +\t\tgit update-ref refs/namespaces/ns/refs/heads/one HEAD &&\n> +\t\tgit update-ref refs/namespaces/ns/refs/heads/two HEAD &&\n> +\t\tgit symbolic-ref refs/namespaces/ns/HEAD \\\n> +\t\t\trefs/namespaces/ns/refs/heads/two\n> +\t)\n> +'\n> +\n> +test_expect_success 'clone chooses correct HEAD (v0)' '\n> +\tGIT_NAMESPACE=ns git -c protocol.version=0 \\\n> +\t\tclone ambiguous ambiguous-v0 &&\n> +\techo refs/heads/two >expect &&\n> +\tgit -C ambiguous-v0 symbolic-ref HEAD >actual &&\n> +\ttest_cmp expect actual\n> +'\n> +\n> +test_expect_success 'clone chooses correct HEAD (v2)' '\n> +\tGIT_NAMESPACE=ns git -c protocol.version=2 \\\n> +\t\tclone ambiguous ambiguous-v2 &&\n> +\techo refs/heads/two >expect &&\n> +\tgit -C ambiguous-v2 symbolic-ref HEAD >actual &&\n> +\ttest_cmp expect actual\n> +'\n> +\n>  test_done\n> diff --git a/upload-pack.c b/upload-pack.c\n> index 24298913c0..4d2129e7fc 100644\n> --- a/upload-pack.c\n> +++ b/upload-pack.c\n> @@ -1037,8 +1037,8 @@ static int find_symref(const char *refname, const struct object_id *oid,\n>  \tsymref_target = resolve_ref_unsafe(refname, 0, NULL, &flag);\n>  \tif (!symref_target || (flag & REF_ISSYMREF) == 0)\n>  \t\tdie(\"'%s' is a symref but it is not?\", refname);\n> -\titem = string_list_append(cb_data, refname);\n> -\titem->util = xstrdup(symref_target);\n> +\titem = string_list_append(cb_data, strip_namespace(refname));\n> +\titem->util = xstrdup(strip_namespace(symref_target));\n>  \treturn 0;\n>  }\n"},{"id":"376094","messageId":"20190523061121.GB20871@sigill.intra.peff.net","threadId":"51138","inReplyTo":"875zq2vkvf.fsf@evledraar.gmail.com","subject":"[PATCH v2] upload-pack: strip namespace from symref data","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2019-05-23T06:11:21Z","receivedAt":"2019-05-23T06:11:25Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Wed, May 22, 2019 at 12:33:56PM +0200, Ævar Arnfjörð Bjarmason wrote:\n\n> > Likely nobody noticed because bug (1) means that from the client's\n> > perspective, we did not report on HEAD at all. And thus it uses the\n> > pre-7171d8c15f fallback code to guess the correct HEAD, which is usually\n> > right. It only falls down in ambiguous cases (like the one laid out in\n> > the included test).\n> \n> ...because here you're talking about \"the client's perspective\" and \"it\n> uses the pre-7171d8c15f [...] code\", but this should say \"the\n> pre-a45b5f0552 code\", i.e. mention the commit that changed the *client*\n> logic.\n\nHeh. I actually found this split, too, while digging in the history, but\nthought it might be adding distracting details to the commit message.\nBut hey, if you're gonna read them _carefully_, I guess I can be more\nprecise. ;)\n\nThe fallback code isn't even from a45b5f0552; it's much older than that.\nWe started looking at ref annotations at all in fbb074c253 (remote: make\nguess_remote_head() use exact HEAD lookup if it is available,\n2009-02-27). The guessing code that falls back to comes from 8434c2f1af\n(Build in clone, 2008-04-27).  I didn't bother finding the true origin\nin the shell script. ;)\n\nSo instead of mentioning it at the top (because the bug really is just\non the server side), I re-wrote the paragraph about the client a little\nmore clearly and precisely:\n\n    Likely nobody noticed because we tend to do the right thing anyway. Bug\n    (1) means that we said nothing about HEAD (just refs/namespace/foo/HEAD).\n    And so the client half of the code, from a45b5f0552 (connect: annotate\n    refs with their symref information in get_remote_head(), 2013-09-17),\n    does not annotate HEAD, and we use the fallback in guess_remote_head(),\n    matching refs by object id. Which is usually right. It only falls down\n    in ambiguous cases, like the one laid out in the included test.\n\nRevised patch below.\n\n-- >8 --\nSubject: [PATCH] upload-pack: strip namespace from symref data\n\nSince 7171d8c15f (upload-pack: send symbolic ref information as\ncapability, 2013-09-17), we've sent cloning and fetching clients special\ninformation about which branch HEAD is pointing to, so that they don't\nhave to guess based on matching up commit ids.\n\nHowever, this feature has never worked properly with the GIT_NAMESPACE\nfeature.  Because upload-pack uses head_ref_namespaced(find_symref), we\ndo find and report on refs/namespaces/foo/HEAD instead of the actual\nHEAD of the repo. This makes sense, since the branch pointed to by the\ntop-level HEAD may not be advertised at all. But we do two things wrong:\n\n  1. We report the full name refs/namespaces/foo/HEAD, instead of just\n     HEAD. Meaning no client is going to bother doing anything with that\n     symref, since we're not otherwise advertising it.\n\n  2. We report the symref destination using its full name (e.g.,\n     refs/namespaces/foo/refs/heads/master). That's similarly useless to\n     the client, who only saw \"refs/heads/master\" in the advertisement.\n\nWe should be stripping the namespace prefix off of both places (which\nthis patch fixes).\n\nLikely nobody noticed because we tend to do the right thing anyway. Bug\n(1) means that we said nothing about HEAD (just refs/namespace/foo/HEAD).\nAnd so the client half of the code, from a45b5f0552 (connect: annotate\nrefs with their symref information in get_remote_head(), 2013-09-17),\ndoes not annotate HEAD, and we use the fallback in guess_remote_head(),\nmatching refs by object id. Which is usually right. It only falls down\nin ambiguous cases, like the one laid out in the included test.\n\nThis also means that we don't have to worry about breaking anybody who\nwas putting pre-stripped names into their namespace symrefs when we fix\nbug (2). Because of bug (1), nobody would have been using the symref we\nadvertised in the first place (not to mention that those symrefs would\nhave appeared broken for any non-namespaced access).\n\nNote that we have separate fixes here for the v0 and v2 protocols. The\nsymref advertisement moved in v2 to be a part of the ls-refs command.\nThis actually gets part (1) right, since the symref annotation\npiggy-backs on the existing ref advertisement, which is properly\nstripped. But it still needs a fix for part (2). The included tests\ncover both protocols.\n\nReported-by: Bryan Turner <bturner@atlassian.com>\nSigned-off-by: Jeff King <peff@peff.net>\n---\n ls-refs.c                        |  3 ++-\n t/t5509-fetch-push-namespaces.sh | 28 ++++++++++++++++++++++++++++\n upload-pack.c                    |  4 ++--\n 3 files changed, 32 insertions(+), 3 deletions(-)\n\ndiff --git a/ls-refs.c b/ls-refs.c\nindex 0a7dbc6442..818aef70a0 100644\n--- a/ls-refs.c\n+++ b/ls-refs.c\n@@ -57,7 +57,8 @@ static int send_ref(const char *refname, const struct object_id *oid,\n \t\tif (!symref_target)\n \t\t\tdie(\"'%s' is a symref but it is not?\", refname);\n \n-\t\tstrbuf_addf(&refline, \" symref-target:%s\", symref_target);\n+\t\tstrbuf_addf(&refline, \" symref-target:%s\",\n+\t\t\t    strip_namespace(symref_target));\n \t}\n \n \tif (data->peel) {\ndiff --git a/t/t5509-fetch-push-namespaces.sh b/t/t5509-fetch-push-namespaces.sh\nindex c88df78c0b..75cbfcc392 100755\n--- a/t/t5509-fetch-push-namespaces.sh\n+++ b/t/t5509-fetch-push-namespaces.sh\n@@ -124,4 +124,32 @@ test_expect_success 'try to update a hidden full ref' '\n \ttest_must_fail git -C original push pushee-namespaced master\n '\n \n+test_expect_success 'set up ambiguous HEAD' '\n+\tgit init ambiguous &&\n+\t(\n+\t\tcd ambiguous &&\n+\t\tgit commit --allow-empty -m foo &&\n+\t\tgit update-ref refs/namespaces/ns/refs/heads/one HEAD &&\n+\t\tgit update-ref refs/namespaces/ns/refs/heads/two HEAD &&\n+\t\tgit symbolic-ref refs/namespaces/ns/HEAD \\\n+\t\t\trefs/namespaces/ns/refs/heads/two\n+\t)\n+'\n+\n+test_expect_success 'clone chooses correct HEAD (v0)' '\n+\tGIT_NAMESPACE=ns git -c protocol.version=0 \\\n+\t\tclone ambiguous ambiguous-v0 &&\n+\techo refs/heads/two >expect &&\n+\tgit -C ambiguous-v0 symbolic-ref HEAD >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success 'clone chooses correct HEAD (v2)' '\n+\tGIT_NAMESPACE=ns git -c protocol.version=2 \\\n+\t\tclone ambiguous ambiguous-v2 &&\n+\techo refs/heads/two >expect &&\n+\tgit -C ambiguous-v2 symbolic-ref HEAD >actual &&\n+\ttest_cmp expect actual\n+'\n+\n test_done\ndiff --git a/upload-pack.c b/upload-pack.c\nindex 24298913c0..4d2129e7fc 100644\n--- a/upload-pack.c\n+++ b/upload-pack.c\n@@ -1037,8 +1037,8 @@ static int find_symref(const char *refname, const struct object_id *oid,\n \tsymref_target = resolve_ref_unsafe(refname, 0, NULL, &flag);\n \tif (!symref_target || (flag & REF_ISSYMREF) == 0)\n \t\tdie(\"'%s' is a symref but it is not?\", refname);\n-\titem = string_list_append(cb_data, refname);\n-\titem->util = xstrdup(symref_target);\n+\titem = string_list_append(cb_data, strip_namespace(refname));\n+\titem->util = xstrdup(strip_namespace(symref_target));\n \treturn 0;\n }\n \n-- \n2.22.0.rc1.549.gadb183c4cb\n\n"},{"id":"376323","messageId":"xmqq4l5e4iou.fsf@gitster-ct.c.googlers.com","threadId":"51138","inReplyTo":"20190523061121.GB20871@sigill.intra.peff.net","subject":"Re: [PATCH v2] upload-pack: strip namespace from symref data","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2019-05-28T17:01:21Z","receivedAt":"2019-05-28T17:01:31Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jeff King <peff@peff.net> writes:\n\n> Revised patch below.\n\nThanks.  I never took the \"separate namespaces\" thing as a serious\nfeature, so it is understandable that I ignored it completely when I\ndid 7171d8c15f.\n\nThe explanation and the code both look good.  Thanks.\n\n\n> -- >8 --\n> Subject: [PATCH] upload-pack: strip namespace from symref data\n>\n> Since 7171d8c15f (upload-pack: send symbolic ref information as\n> capability, 2013-09-17), we've sent cloning and fetching clients special\n> information about which branch HEAD is pointing to, so that they don't\n> have to guess based on matching up commit ids.\n>\n> However, this feature has never worked properly with the GIT_NAMESPACE\n> feature.  Because upload-pack uses head_ref_namespaced(find_symref), we\n> do find and report on refs/namespaces/foo/HEAD instead of the actual\n> HEAD of the repo. This makes sense, since the branch pointed to by the\n> top-level HEAD may not be advertised at all. But we do two things wrong:\n>\n>   1. We report the full name refs/namespaces/foo/HEAD, instead of just\n>      HEAD. Meaning no client is going to bother doing anything with that\n>      symref, since we're not otherwise advertising it.\n>\n>   2. We report the symref destination using its full name (e.g.,\n>      refs/namespaces/foo/refs/heads/master). That's similarly useless to\n>      the client, who only saw \"refs/heads/master\" in the advertisement.\n>\n> We should be stripping the namespace prefix off of both places (which\n> this patch fixes).\n>\n> Likely nobody noticed because we tend to do the right thing anyway. Bug\n> (1) means that we said nothing about HEAD (just refs/namespace/foo/HEAD).\n> And so the client half of the code, from a45b5f0552 (connect: annotate\n> refs with their symref information in get_remote_head(), 2013-09-17),\n> does not annotate HEAD, and we use the fallback in guess_remote_head(),\n> matching refs by object id. Which is usually right. It only falls down\n> in ambiguous cases, like the one laid out in the included test.\n>\n> This also means that we don't have to worry about breaking anybody who\n> was putting pre-stripped names into their namespace symrefs when we fix\n> bug (2). Because of bug (1), nobody would have been using the symref we\n> advertised in the first place (not to mention that those symrefs would\n> have appeared broken for any non-namespaced access).\n>\n> Note that we have separate fixes here for the v0 and v2 protocols. The\n> symref advertisement moved in v2 to be a part of the ls-refs command.\n> This actually gets part (1) right, since the symref annotation\n> piggy-backs on the existing ref advertisement, which is properly\n> stripped. But it still needs a fix for part (2). The included tests\n> cover both protocols.\n>\n> Reported-by: Bryan Turner <bturner@atlassian.com>\n> Signed-off-by: Jeff King <peff@peff.net>\n> ---\n>  ls-refs.c                        |  3 ++-\n>  t/t5509-fetch-push-namespaces.sh | 28 ++++++++++++++++++++++++++++\n>  upload-pack.c                    |  4 ++--\n>  3 files changed, 32 insertions(+), 3 deletions(-)\n>\n> diff --git a/ls-refs.c b/ls-refs.c\n> index 0a7dbc6442..818aef70a0 100644\n> --- a/ls-refs.c\n> +++ b/ls-refs.c\n> @@ -57,7 +57,8 @@ static int send_ref(const char *refname, const struct object_id *oid,\n>  \t\tif (!symref_target)\n>  \t\t\tdie(\"'%s' is a symref but it is not?\", refname);\n>  \n> -\t\tstrbuf_addf(&refline, \" symref-target:%s\", symref_target);\n> +\t\tstrbuf_addf(&refline, \" symref-target:%s\",\n> +\t\t\t    strip_namespace(symref_target));\n>  \t}\n>  \n>  \tif (data->peel) {\n> diff --git a/t/t5509-fetch-push-namespaces.sh b/t/t5509-fetch-push-namespaces.sh\n> index c88df78c0b..75cbfcc392 100755\n> --- a/t/t5509-fetch-push-namespaces.sh\n> +++ b/t/t5509-fetch-push-namespaces.sh\n> @@ -124,4 +124,32 @@ test_expect_success 'try to update a hidden full ref' '\n>  \ttest_must_fail git -C original push pushee-namespaced master\n>  '\n>  \n> +test_expect_success 'set up ambiguous HEAD' '\n> +\tgit init ambiguous &&\n> +\t(\n> +\t\tcd ambiguous &&\n> +\t\tgit commit --allow-empty -m foo &&\n> +\t\tgit update-ref refs/namespaces/ns/refs/heads/one HEAD &&\n> +\t\tgit update-ref refs/namespaces/ns/refs/heads/two HEAD &&\n> +\t\tgit symbolic-ref refs/namespaces/ns/HEAD \\\n> +\t\t\trefs/namespaces/ns/refs/heads/two\n> +\t)\n> +'\n> +\n> +test_expect_success 'clone chooses correct HEAD (v0)' '\n> +\tGIT_NAMESPACE=ns git -c protocol.version=0 \\\n> +\t\tclone ambiguous ambiguous-v0 &&\n> +\techo refs/heads/two >expect &&\n> +\tgit -C ambiguous-v0 symbolic-ref HEAD >actual &&\n> +\ttest_cmp expect actual\n> +'\n> +\n> +test_expect_success 'clone chooses correct HEAD (v2)' '\n> +\tGIT_NAMESPACE=ns git -c protocol.version=2 \\\n> +\t\tclone ambiguous ambiguous-v2 &&\n> +\techo refs/heads/two >expect &&\n> +\tgit -C ambiguous-v2 symbolic-ref HEAD >actual &&\n> +\ttest_cmp expect actual\n> +'\n> +\n>  test_done\n> diff --git a/upload-pack.c b/upload-pack.c\n> index 24298913c0..4d2129e7fc 100644\n> --- a/upload-pack.c\n> +++ b/upload-pack.c\n> @@ -1037,8 +1037,8 @@ static int find_symref(const char *refname, const struct object_id *oid,\n>  \tsymref_target = resolve_ref_unsafe(refname, 0, NULL, &flag);\n>  \tif (!symref_target || (flag & REF_ISSYMREF) == 0)\n>  \t\tdie(\"'%s' is a symref but it is not?\", refname);\n> -\titem = string_list_append(cb_data, refname);\n> -\titem->util = xstrdup(symref_target);\n> +\titem = string_list_append(cb_data, strip_namespace(refname));\n> +\titem->util = xstrdup(strip_namespace(symref_target));\n>  \treturn 0;\n>  }\n"}]}