{"thread":{"id":"49393","subject":"[PATCH 0/3] Filter alternate references","startedAt":"2018-09-20T18:04:11Z","lastAt":"2018-10-09T14:49:59Z","messageCount":94,"participants":["Taylor Blau","Stefan Beller","Jeff King","Eric Sunshine","Junio C Hamano","brian m. carlson","Ramsay Jones"],"isPatch":true,"patchVersion":1,"patchTotal":3},"messages":[{"id":"358523","messageId":"cover.1537466087.git.me@ttaylorr.com","threadId":"49393","inReplyTo":null,"subject":"[PATCH 0/3] Filter alternate references","fromName":"Taylor Blau","fromEmail":"ttaylorr@github.com","sentAt":"2018-09-20T18:04:05Z","receivedAt":"2018-09-20T18:04:11Z","isPatch":true,"sender":{"key":"ttaylorr@github.com","avatar":"https://gravatar.com/avatar/d5f3476f26b6f99cbb6b467e7ed7482f5762c8157bc73f569196e428bdcbea25?d=mp&s=160"},"body":"Hi,\n\nThis is a series to customize Git's behavior when listing references\nfrom an alternate repository. It is motivated by the following example:\n\nConsider an upstream repository, a fork of it, and a local copy of that\nfork. Ideally, running \"git pull upstream\" from the local copy followed\nby a \"git push fork\" should be a lightweight operation, ideally because\nthe fork already \"knows\" about the new objects introduced upstream.\n\nToday, we do this by means of the special \".have\" references advertised\nby 'git receive-pack'. This special part of the advertisement is\ndesigned to tell the pusher about tips that it might want to know about,\nto avoid sending them again.\n\nThis optimization is a good one and works well, particularly when the\nupstream repository has a relatively normal number of references. When\nthe upstream has a pathologically _large_ number of references, the\nadvertisement alone can be so time consuming, that it's faster to send\nredundant objects to the fork.\n\nTo make the reference advertisement manageable even with a large number\nof references, let's allow the fork to select which ones it thinks might\nbe \"interesting\", and only advertise those. This makes the advertisement\nmuch smaller, and lets us take advantage of the \".have\" references, even\nwhen the upstream contains more references than we're advertising.\n\nThis series implements the above functionality by means of\n\"core.alternateRefsCommand\", and \"core.alternateRefsPrefixes\", either a\ncommand to run in place of \"git for-each-ref\", or arguments to be\nappended to \"git for-each-ref\".\n\nThe order of precedence when listing references from an alternate is as\nfollows:\n\n 1. If the fork configures \"core.alternateRefsCommand\", run that.\n\n 2. If the fork configures \"core.alternateRefsPrefixes\", run 'git\n for-each-ref', limiting results to references that have any of the\n given values as a prefix.\n\n 3. Otherwise, run 'git for-each-ref' in the alternate.\n\nIn a previous version of this series, I taught the configuration\nproperty to the alternate, as in \"these are the references that _I_\nthink _you_ will find interesting,\" rather than the other way around. I\nultimately decided on what is attached here so that the fork does not\nhave to trust the upstream to run arbitrary shell commands.\n\nThanks,\nTaylor\n\nTaylor Blau (3):\n transport.c: extract 'fill_alternate_refs_command'\n transport.c: introduce core.alternateRefsCommand\n transport.c: introduce core.alternateRefsPrefixes\n\n Documentation/config.txt | 12 +++++++++\n t/t5410-receive-pack.sh | 58 ++++++++++++++++++++++++++++++++++++++++\n transport.c | 34 ++++++++++++++++++-----\n 3 files changed, 98 insertions(+), 6 deletions(-)\n create mode 100755 t/t5410-receive-pack.sh\n\n--\n2.19.0\n"},{"id":"358524","messageId":"6e3a58afe7cd18d663f481cdc9eb65cc941765b1.1537466087.git.me@ttaylorr.com","threadId":"49393","inReplyTo":"cover.1537466087.git.me@ttaylorr.com","subject":"[PATCH 1/3] transport.c: extract 'fill_alternate_refs_command'","fromName":"Taylor Blau","fromEmail":"ttaylorr@github.com","sentAt":"2018-09-20T18:04:08Z","receivedAt":"2018-09-20T18:04:13Z","isPatch":true,"sender":{"key":"ttaylorr@github.com","avatar":"https://gravatar.com/avatar/d5f3476f26b6f99cbb6b467e7ed7482f5762c8157bc73f569196e428bdcbea25?d=mp&s=160"},"body":"To list alternate references, 'read_alternate_refs' creates a child\nprocess running 'git for-each-ref' in the alternate's Git directory.\n\nPrepare to run other commands besides 'git for-each-ref' by introducing\nand moving the relevant code from 'read_alternate_refs' to\n'fill_alternate_refs_command'.\n\nSigned-off-by: Taylor Blau \n---\n transport.c | 18 ++++++++++++------\n 1 file changed, 12 insertions(+), 6 deletions(-)\n\ndiff --git a/transport.c b/transport.c\nindex 1c76d64aba..24ae3f375d 100644\n--- a/transport.c\n+++ b/transport.c\n@@ -1325,6 +1325,17 @@ char *transport_anonymize_url(const char *url)\n \treturn xstrdup(url);\n }\n \n+static void fill_alternate_refs_command(struct child_process *cmd,\n+\t\t\t\t\tconst char *repo_path)\n+{\n+\tcmd->git_cmd = 1;\n+\targv_array_pushf(&cmd->args, \"--git-dir=%s\", repo_path);\n+\targv_array_push(&cmd->args, \"for-each-ref\");\n+\targv_array_push(&cmd->args, \"--format=%(objectname) %(refname)\");\n+\tcmd->env = local_repo_env;\n+\tcmd->out = -1;\n+}\n+\n static void read_alternate_refs(const char *path,\n \t\t\t\talternate_ref_fn *cb,\n \t\t\t\tvoid *data)\n@@ -1333,12 +1344,7 @@ static void read_alternate_refs(const char *path,\n \tstruct strbuf line = STRBUF_INIT;\n \tFILE *fh;\n \n-\tcmd.git_cmd = 1;\n-\targv_array_pushf(&cmd.args, \"--git-dir=%s\", path);\n-\targv_array_push(&cmd.args, \"for-each-ref\");\n-\targv_array_push(&cmd.args, \"--format=%(objectname) %(refname)\");\n-\tcmd.env = local_repo_env;\n-\tcmd.out = -1;\n+\tfill_alternate_refs_command(&cmd, path);\n \n \tif (start_command(&cmd))\n \t\treturn;\n-- \n2.19.0\n\n"},{"id":"358525","messageId":"4c4900722cab253b3ce33cb28910c4602ce44536.1537466087.git.me@ttaylorr.com","threadId":"49393","inReplyTo":"cover.1537466087.git.me@ttaylorr.com","subject":"[PATCH 2/3] transport.c: introduce core.alternateRefsCommand","fromName":"Taylor Blau","fromEmail":"ttaylorr@github.com","sentAt":"2018-09-20T18:04:11Z","receivedAt":"2018-09-20T18:04:16Z","isPatch":true,"sender":{"key":"ttaylorr@github.com","avatar":"https://gravatar.com/avatar/d5f3476f26b6f99cbb6b467e7ed7482f5762c8157bc73f569196e428bdcbea25?d=mp&s=160"},"body":"When in a repository containing one or more alternates, Git would\nsometimes like to list references from its alternates. For example, 'git\nreceive-pack' list the objects pointed to by alternate references as\nspecial \".have\" references.\n\nListing \".have\" references is designed to make pushing changes from\nupstream to a fork a lightweight operation, by advertising to the pusher\nthat the fork already has the objects (via its alternate). Thus, the\nclient can avoid sending them.\n\nHowever, when the alternate has a pathologically large number of\nreferences, the initial advertisement is too expensive. In fact, it can\ndominate any such optimization where the pusher avoids sending certain\nobjects.\n\nIntroduce \"core.alternateRefsCommand\" in order to provide a facility to\nlimit or filter alternate references. This can be used, for example, to\nfilter out \"uninteresting\" references from the initial advertisement in\nthe above scenario.\n\nLet the repository that has alternates configure this command to avoid\ntrusting the alternate to provide us a safe command to run in the shell.\nTo behave differently on each alternate (e.g., only list tags from\nalternate A, only heads from B) provide the path of the alternate as the\nfirst argument.\n\nSigned-off-by: Taylor Blau \n---\n Documentation/config.txt | 6 +++++\n t/t5410-receive-pack.sh | 47 ++++++++++++++++++++++++++++++++++++++++\n transport.c | 19 ++++++++++++----\n 3 files changed, 68 insertions(+), 4 deletions(-)\n create mode 100755 t/t5410-receive-pack.sh\n\ndiff --git a/Documentation/config.txt b/Documentation/config.txt\nindex 112041f407..b908bc5825 100644\n--- a/Documentation/config.txt\n+++ b/Documentation/config.txt\n@@ -616,6 +616,12 @@ core.preferSymlinkRefs::\n \tThis is sometimes needed to work with old scripts that\n \texpect HEAD to be a symbolic link.\n \n+core.alternateRefsCommand::\n+\tWhen listing references from an alternate (e.g., in the case of \".have\"), use\n+\tthe shell to execute the specified command instead of\n+\tlinkgit:git-for-each-ref[1]. The first argument is the path of the alternate.\n+\tOutput must be of the form: `%(objectname) SPC %(refname)`.\n+\n core.bare::\n \tIf true this repository is assumed to be 'bare' and has no\n \tworking directory associated with it. If this is the case a\ndiff --git a/t/t5410-receive-pack.sh b/t/t5410-receive-pack.sh\nnew file mode 100755\nindex 0000000000..09fb3f39a1\n--- /dev/null\n+++ b/t/t5410-receive-pack.sh\n@@ -0,0 +1,47 @@\n+#!/bin/sh\n+\n+test_description='git receive-pack test'\n+\n+. ./test-lib.sh\n+\n+test_expect_success 'setup' '\n+\ttest_commit one &&\n+\tgit update-ref refs/heads/a HEAD &&\n+\ttest_commit two &&\n+\tgit update-ref refs/heads/b HEAD &&\n+\ttest_commit three &&\n+\tgit update-ref refs/heads/c HEAD &&\n+\tgit clone --bare . fork &&\n+\tgit clone fork pusher &&\n+\t(\n+\t\tcd fork &&\n+\t\tgit config receive.advertisealternates true &&\n+\t\tgit update-ref -d refs/heads/a &&\n+\t\tgit update-ref -d refs/heads/b &&\n+\t\tgit update-ref -d refs/heads/c &&\n+\t\tgit update-ref -d refs/heads/master &&\n+\t\tgit update-ref -d refs/tags/one &&\n+\t\tgit update-ref -d refs/tags/two &&\n+\t\tgit update-ref -d refs/tags/three &&\n+\t\tprintf \"../../.git/objects\" >objects/info/alternates\n+\t)\n+'\n+\n+extract_haves () {\n+\tdepacketize - | grep -o '^.* \\.have'\n+}\n+\n+test_expect_success 'with core.alternateRefsCommand' '\n+\ttest_config -C fork core.alternateRefsCommand \\\n+\t\t\"git --git-dir=\\\"\\$1\\\" for-each-ref \\\n+\t\t--format=\\\"%(objectname) %(refname)\\\" \\\n+\t\trefs/heads/a refs/heads/c;:\" &&\n+\tcat >expect <<-EOF &&\n+\t$(git rev-parse a) .have\n+\t$(git rev-parse c) .have\n+\tEOF\n+\tprintf \"0000\" | git receive-pack fork | extract_haves >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_done\ndiff --git a/transport.c b/transport.c\nindex 24ae3f375d..e7d2cdf00b 100644\n--- a/transport.c\n+++ b/transport.c\n@@ -1328,10 +1328,21 @@ char *transport_anonymize_url(const char *url)\n static void fill_alternate_refs_command(struct child_process *cmd,\n \t\t\t\t\tconst char *repo_path)\n {\n-\tcmd->git_cmd = 1;\n-\targv_array_pushf(&cmd->args, \"--git-dir=%s\", repo_path);\n-\targv_array_push(&cmd->args, \"for-each-ref\");\n-\targv_array_push(&cmd->args, \"--format=%(objectname) %(refname)\");\n+\tconst char *value;\n+\n+\tif (!git_config_get_value(\"core.alternateRefsCommand\", &value)) {\n+\t\tcmd->use_shell = 1;\n+\n+\t\targv_array_push(&cmd->args, value);\n+\t\targv_array_push(&cmd->args, repo_path);\n+\t} else {\n+\t\tcmd->git_cmd = 1;\n+\n+\t\targv_array_pushf(&cmd->args, \"--git-dir=%s\", repo_path);\n+\t\targv_array_push(&cmd->args, \"for-each-ref\");\n+\t\targv_array_push(&cmd->args, \"--format=%(objectname) %(refname)\");\n+\t}\n+\n \tcmd->env = local_repo_env;\n \tcmd->out = -1;\n }\n-- \n2.19.0\n\n"},{"id":"358526","messageId":"3639e9058859b326f64600fcd0b608171b56ce9f.1537466087.git.me@ttaylorr.com","threadId":"49393","inReplyTo":"cover.1537466087.git.me@ttaylorr.com","subject":"[PATCH 3/3] transport.c: introduce core.alternateRefsPrefixes","fromName":"Taylor Blau","fromEmail":"ttaylorr@github.com","sentAt":"2018-09-20T18:04:13Z","receivedAt":"2018-09-20T18:04:18Z","isPatch":true,"sender":{"key":"ttaylorr@github.com","avatar":"https://gravatar.com/avatar/d5f3476f26b6f99cbb6b467e7ed7482f5762c8157bc73f569196e428bdcbea25?d=mp&s=160"},"body":"The recently-introduced \"core.alternateRefsCommand\" allows callers to\nspecify with high flexibility the tips that they wish to advertise from\nalternates. This flexibility comes at the cost of some inconvenience\nwhen the caller only wishes to limit the advertisement to one or more\nprefixes.\n\nFor example, to advertise only tags, a caller using\n'core.alternateRefsCommand' would have to do:\n\n $ git config core.alternateRefsCommand ' \\\n git -C \"$1\" for-each-ref refs/tags \\\n --format=\"%(objectname) %(refname)\" \\\n '\n\nThe above is cumbersome to write, so let's introduce a\n\"core.alternateRefsPrefixes\" to address this common case. Instead, the\ncaller can run:\n\n $ git config core.alternateRefsPrefixes 'refs/tags'\n\nWhich will behave identically to the longer example using\n\"core.alternateRefsCommand\".\n\nSince the value of \"core.alternateRefsPrefixes\" is appended to 'git\nfor-each-ref' and then executed, include a \"--\" before taking the\nconfigured value to avoid misinterpreting arguments as flags to 'git\nfor-each-ref'.\n\nIn the case that the caller wishes to specify multiple prefixes, they\nmay separate them by whitespace. If \"core.alternateRefsCommand\" is set,\nit will take precedence over \"core.alternateRefsPrefixes\".\n\nSigned-off-by: Taylor Blau \n---\n Documentation/config.txt | 6 ++++++\n t/t5410-receive-pack.sh | 11 +++++++++++\n transport.c | 5 +++++\n 3 files changed, 22 insertions(+)\n\ndiff --git a/Documentation/config.txt b/Documentation/config.txt\nindex b908bc5825..d768c57310 100644\n--- a/Documentation/config.txt\n+++ b/Documentation/config.txt\n@@ -622,6 +622,12 @@ core.alternateRefsCommand::\n \tlinkgit:git-for-each-ref[1]. The first argument is the path of the alternate.\n \tOutput must be of the form: `%(objectname) SPC %(refname)`.\n \n+core.alternateRefsPrefixes::\n+\tWhen listing references from an alternate, list only references that begin\n+\twith the given prefix. To list multiple prefixes, separate them with a\n+\twhitespace character. If `core.alternateRefsCommand` is set, setting\n+\t`core.alternateRefsPrefixes` has no effect.\n+\n core.bare::\n \tIf true this repository is assumed to be 'bare' and has no\n \tworking directory associated with it. If this is the case a\ndiff --git a/t/t5410-receive-pack.sh b/t/t5410-receive-pack.sh\nindex 09fb3f39a1..df2830e9f6 100755\n--- a/t/t5410-receive-pack.sh\n+++ b/t/t5410-receive-pack.sh\n@@ -44,4 +44,15 @@ test_expect_success 'with core.alternateRefsCommand' '\n \ttest_cmp expect actual\n '\n \n+test_expect_success 'with core.alternateRefsPrefixes' '\n+\ttest_config -C fork core.alternateRefsPrefixes \"refs/tags\" &&\n+\tcat >expect <<-EOF &&\n+\t$(git rev-parse one) .have\n+\t$(git rev-parse three) .have\n+\t$(git rev-parse two) .have\n+\tEOF\n+\tprintf \"0000\" | git receive-pack fork | extract_haves >actual &&\n+\ttest_cmp expect actual\n+'\n+\n test_done\ndiff --git a/transport.c b/transport.c\nindex e7d2cdf00b..9323e5c3cd 100644\n--- a/transport.c\n+++ b/transport.c\n@@ -1341,6 +1341,11 @@ static void fill_alternate_refs_command(struct child_process *cmd,\n \t\targv_array_pushf(&cmd->args, \"--git-dir=%s\", repo_path);\n \t\targv_array_push(&cmd->args, \"for-each-ref\");\n \t\targv_array_push(&cmd->args, \"--format=%(objectname) %(refname)\");\n+\n+\t\tif (!git_config_get_value(\"core.alternateRefsPrefixes\", &value)) {\n+\t\t\targv_array_push(&cmd->args, \"--\");\n+\t\t\targv_array_split(&cmd->args, value);\n+\t\t}\n \t}\n \n \tcmd->env = local_repo_env;\n-- \n2.19.0\n"},{"id":"358527","messageId":"CAGZ79kYv-n_x_Z3K=zsiTAQ9RCFMVE=6ZYCWRQMgioGRPnhVOQ@mail.gmail.com","threadId":"49393","inReplyTo":"cover.1537466087.git.me@ttaylorr.com","subject":"Re: [PATCH 0/3] Filter alternate references","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-09-20T18:35:23Z","receivedAt":"2018-09-20T18:35:39Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Thu, Sep 20, 2018 at 11:04 AM Taylor Blau wrote:\n>\n> Hi,\n>\n> This is a series to customize Git's behavior when listing references\n> from an alternate repository. It is motivated by the following example:\n>\n> Consider an upstream repository, a fork of it, and a local copy of that\n> fork. Ideally, running \"git pull upstream\" from the local copy followed\n> by a \"git push fork\" should be a lightweight operation, ideally because\n> the fork already \"knows\" about the new objects introduced upstream.\n>\n> Today, we do this by means of the special \".have\" references advertised\n> by 'git receive-pack'. This special part of the advertisement is\n> designed to tell the pusher about tips that it might want to know about,\n> to avoid sending them again.\n>\n> This optimization is a good one and works well, particularly when the\n> upstream repository has a relatively normal number of references. When\n> the upstream has a pathologically _large_ number of references, the\n> advertisement alone can be so time consuming, that it's faster to send\n> redundant objects to the fork.\n\n(tangent:)\nThe current fetch protocol consists of 2 parts:\nnegotiation + sending the packfile, and the negotiation only tries\nto trim down the size of the packfile to send, without taking its own\ncost (in terms of time and band width) into account, just to produce\na perfect pack to send to the client.\n\nWhen talking about designing protocol v2 for push (which has not\nlanded yet[1]), we had some in-office discussions whether we\nwant to have a proper negotiation on push, as it would help\npushing to remotes that have non-ff pushes, but not necessarily\nregular pushes, as they should be fine with just the refs advertisement.\n\n[1] https://github.com/bmwill/git/commit/57a4e6e5d18a2d4d806fc8dec644b89affd50853\nbmwill@ no longer works on it though.\n\n\n>\n> To make the reference advertisement manageable even with a large number\n> of references, let's allow the fork to select which ones it thinks might\n> be \"interesting\", and only advertise those. This makes the advertisement\n> much smaller, and lets us take advantage of the \".have\" references, even\n> when the upstream contains more references than we're advertising.\n>\n> This series implements the above functionality by means of\n> \"core.alternateRefsCommand\", and \"core.alternateRefsPrefixes\", either a\n> command to run in place of \"git for-each-ref\", or arguments to be\n> appended to \"git for-each-ref\".\n>\n> The order of precedence when listing references from an alternate is as\n> follows:\n>\n> 1. If the fork configures \"core.alternateRefsCommand\", run that.\n>\n> 2. If the fork configures \"core.alternateRefsPrefixes\", run 'git\n> for-each-ref', limiting results to references that have any of the\n> given values as a prefix.\n>\n> 3. Otherwise, run 'git for-each-ref' in the alternate.\n>\n> In a previous version of this series, I taught the configuration\n> property to the alternate, as in \"these are the references that _I_\n> think _you_ will find interesting,\" rather than the other way around. I\n> ultimately decided on what is attached here so that the fork does not\n> have to trust the upstream to run arbitrary shell commands.\n\nWould it make sense to estimate the value of each .have before\nadvertising them and then advertise only the most valuable\n.haves ?\n(e.g. if a .have is only one small commit ahead of origin/master,\nit may not bring a lot of value as the potential savings are small,\nbut if that .have contains history between master..TIP that has lots\nof big blobs or objects in general, this may be valuable to know)\n\nStefan\n"},{"id":"358531","messageId":"20180920185644.GA72694@syl","threadId":"49393","inReplyTo":"CAGZ79kYv-n_x_Z3K=zsiTAQ9RCFMVE=6ZYCWRQMgioGRPnhVOQ@mail.gmail.com","subject":"Re: [PATCH 0/3] Filter alternate references","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2018-09-20T18:56:44Z","receivedAt":"2018-09-20T18:56:50Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"Hi Stefan,\n\nOn Thu, Sep 20, 2018 at 11:35:23AM -0700, Stefan Beller wrote:\n> > To make the reference advertisement manageable even with a large number\n> > of references, let's allow the fork to select which ones it thinks might\n> > be \"interesting\", and only advertise those. This makes the advertisement\n> > much smaller, and lets us take advantage of the \".have\" references, even\n> > when the upstream contains more references than we're advertising.\n> >\n> > This series implements the above functionality by means of\n> > \"core.alternateRefsCommand\", and \"core.alternateRefsPrefixes\", either a\n> > command to run in place of \"git for-each-ref\", or arguments to be\n> > appended to \"git for-each-ref\".\n> >\n> > The order of precedence when listing references from an alternate is as\n> > follows:\n> >\n> > 1. If the fork configures \"core.alternateRefsCommand\", run that.\n> >\n> > 2. If the fork configures \"core.alternateRefsPrefixes\", run 'git\n> > for-each-ref', limiting results to references that have any of the\n> > given values as a prefix.\n> >\n> > 3. Otherwise, run 'git for-each-ref' in the alternate.\n> >\n> > In a previous version of this series, I taught the configuration\n> > property to the alternate, as in \"these are the references that _I_\n> > think _you_ will find interesting,\" rather than the other way around. I\n> > ultimately decided on what is attached here so that the fork does not\n> > have to trust the upstream to run arbitrary shell commands.\n>\n> Would it make sense to estimate the value of each .have before\n> advertising them and then advertise only the most valuable\n> .haves ?\n> (e.g. if a .have is only one small commit ahead of origin/master,\n> it may not bring a lot of value as the potential savings are small,\n> but if that .have contains history between master..TIP that has lots\n> of big blobs or objects in general, this may be valuable to know)\n\nI think that this sort of filtering should be theoretically possible\nby configuring \"core.alternateRefsCommand\", perhaps to execute a script\nlike:\n\n cd \"$1\" &&\n git for-each-ref --format=\"%(objectname) %(refname)\" |\n while read objectname refname; do\n total_size=\"$(git rev-list --objects master...$objectname \\\n | awk '{ print $1 }' \\\n | git cat-file --batch-check='%(objectsize)' \\\n | awk '{ sum+=$1 } END { print $sum }')\"\n\n if [ \"$total_size\" -gt \"$minimum_size\" ]; then\n echo \"$objectname $refname\"\n fi\n done\n\nBut that's quite inefficient to compute, since you're walking the same\nparts of the graph over and over again.\n\nPerhaps we could teach Git to do something better? I suppose that just\n\"core.alternateRefPrefixes\" could do this by default (or with another\nknob) to further optimize the simpler case. But I think that we'd be\nequally OK without it, since push over V2 obviates the need for this\nsort of optimization (as you noted in the unquoted part of this\nresponse).\n\nMy inclination is to avoid teaching this to Git, and let callers\nscript it into their \"core.alternateRefsCommand\" if they really desire\nit.\n\nDoes that seem OK?\n\n\nThanks,\nTaylor\n"},{"id":"358533","messageId":"20180920192117.GA29603@sigill.intra.peff.net","threadId":"49393","inReplyTo":"cover.1537466087.git.me@ttaylorr.com","subject":"Re: [PATCH 0/3] Filter alternate references","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2018-09-20T19:21:18Z","receivedAt":"2018-09-20T19:21:22Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Thu, Sep 20, 2018 at 02:04:05PM -0400, Taylor Blau wrote:\n\n> This is a series to customize Git's behavior when listing references\n> from an alternate repository. It is motivated by the following example:\n> \n> Consider an upstream repository, a fork of it, and a local copy of that\n> fork. Ideally, running \"git pull upstream\" from the local copy followed\n> by a \"git push fork\" should be a lightweight operation, ideally because\n> the fork already \"knows\" about the new objects introduced upstream.\n> \n> Today, we do this by means of the special \".have\" references advertised\n> by 'git receive-pack'. This special part of the advertisement is\n> designed to tell the pusher about tips that it might want to know about,\n> to avoid sending them again.\n\nI think it's important to note that this is just one place where this\noptimization is useful. A few others are:\n\n 1. On fetching, the client similarly advertises the extra tips (not in\n a ref advertisement, but as part of the negotiation).\n\n 2. We don't do it now, but we ought to use those for checking the\n connectivity of incoming objects. Otherwise we end up walking over\n history that we already know we have. Since this is purely local,\n it's not usually as big a deal, but it can matter a lot in large\n repositories, because it makes what should be O(nr_changes)\n fetches into O(size_of_repo). E.g., imagine making a fork of\n linux.git backed by the same shared-object alternate. The initial\n \"fetch\" should be a noop as we realize that we have everything\n already, but we spend 45s of CPU walking the whole graph.\n\n I have patches for this, but haven't sent them, since without the\n optimization you've done here, we'd never be able to turn it on at\n GitHub.\n\n 3. Other scripts may want us to expose this. The patches I have for\n (2) actually implement \"rev-list --alternate-refs\" (since we\n implement the connectivity check there). I don't have other\n particular uses in mind, but it lets you ask questions like \"which\n objects are reachable here versus in the alternate\".\n\nYour patches would affect all of those sites, I and I think that's a\ngood thing. It's giving a consistent view of \"what can I assume is\nreachable from the alternate?\", which is OK to be a subset of the whole\n(and already is, really, since we don't peek into the alternate's\nreflogs).\n\n> In a previous version of this series, I taught the configuration\n> property to the alternate, as in \"these are the references that _I_\n> think _you_ will find interesting,\" rather than the other way around. I\n> ultimately decided on what is attached here so that the fork does not\n> have to trust the upstream to run arbitrary shell commands.\n\nRight, we had a lot of discussion here (which I'm repeating not for you\nbut for the benefit of the list). It might seem conceptually simpler to\nfor the alternate itself to say \"what are my important refs?\". And that\nnicely generalizes if you have multiple alternates. But in our use case,\n\"important\" here is in the eye of the beholder. If a bunch of repos are\nsharing object storage, and repo Y is derived from repo X, then refs\nrelated to X are going to be most important when you're doing an\noperation in Y. But in some repo Q derived from R, that wouldn't be the\ncase.\n\nSo I think you could make an argument either way there. But simplifying\nthe security boundary around core.alternateRefsCommand pushes it in\nfavor of having all of this decided by the repo doing the looking,\nrather than the one it's looking at.\n\n-Peff\n"},{"id":"358534","messageId":"20180920192741.GB29603@sigill.intra.peff.net","threadId":"49393","inReplyTo":"CAGZ79kYv-n_x_Z3K=zsiTAQ9RCFMVE=6ZYCWRQMgioGRPnhVOQ@mail.gmail.com","subject":"Re: [PATCH 0/3] Filter alternate references","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2018-09-20T19:27:41Z","receivedAt":"2018-09-20T19:27:45Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Thu, Sep 20, 2018 at 11:35:23AM -0700, Stefan Beller wrote:\n\n> > This optimization is a good one and works well, particularly when the\n> > upstream repository has a relatively normal number of references. When\n> > the upstream has a pathologically _large_ number of references, the\n> > advertisement alone can be so time consuming, that it's faster to send\n> > redundant objects to the fork.\n> \n> (tangent:)\n> The current fetch protocol consists of 2 parts:\n> negotiation + sending the packfile, and the negotiation only tries\n> to trim down the size of the packfile to send, without taking its own\n> cost (in terms of time and band width) into account, just to produce\n> a perfect pack to send to the client.\n> \n> When talking about designing protocol v2 for push (which has not\n> landed yet[1]), we had some in-office discussions whether we\n> want to have a proper negotiation on push, as it would help\n> pushing to remotes that have non-ff pushes, but not necessarily\n> regular pushes, as they should be fine with just the refs advertisement.\n\nI don't think that materially changes anything. We already do this same\ntrick on fetch (but just with the client advertising the extra haves,\nsince it's the receiver). So if push started doing a real negotiation,\nwe'd still want to feed those haves in the same way.\n\n> Would it make sense to estimate the value of each .have before\n> advertising them and then advertise only the most valuable\n> .haves ?\n> (e.g. if a .have is only one small commit ahead of origin/master,\n> it may not bring a lot of value as the potential savings are small,\n> but if that .have contains history between master..TIP that has lots\n> of big blobs or objects in general, this may be valuable to know)\n\nThat sounds neat, but I think is mostly orthogonal here. We're primarily\ninterested in just narrowing down the initial set of possibilities, so\nyou could cull it further.\n\nAnd I see Taylor just responded with the idea that you could do this in\nyour hook. Which is neat, but definitely not something we are planning\non doing with it immediately. ;)\n\n-Peff\n"},{"id":"358535","messageId":"20180920193751.GC29603@sigill.intra.peff.net","threadId":"49393","inReplyTo":"4c4900722cab253b3ce33cb28910c4602ce44536.1537466087.git.me@ttaylorr.com","subject":"Re: [PATCH 2/3] transport.c: introduce core.alternateRefsCommand","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2018-09-20T19:37:51Z","receivedAt":"2018-09-20T19:37:55Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Thu, Sep 20, 2018 at 02:04:11PM -0400, Taylor Blau wrote:\n\n> diff --git a/Documentation/config.txt b/Documentation/config.txt\n> index 112041f407..b908bc5825 100644\n> --- a/Documentation/config.txt\n> +++ b/Documentation/config.txt\n> @@ -616,6 +616,12 @@ core.preferSymlinkRefs::\n> \tThis is sometimes needed to work with old scripts that\n> \texpect HEAD to be a symbolic link.\n> \n> +core.alternateRefsCommand::\n> +\tWhen listing references from an alternate (e.g., in the case of \".have\"), use\n> +\tthe shell to execute the specified command instead of\n> +\tlinkgit:git-for-each-ref[1]. The first argument is the path of the alternate.\n> +\tOutput must be of the form: `%(objectname) SPC %(refname)`.\n\nWe discussed off-list the notion that this could just be the objectname,\nsince the \".have\" mechanism doesn't care about the actual refnames.\n\nThere's a little prior discussion from the list:\n\n https://public-inbox.org/git/xmqqefzraqbu.fsf@gitster.mtv.corp.google.com/\n\nMy \"rev-list --alternate-refs\" patches _do_ use the refnames, since you\ncould do something like \"--source\" that cares about them. But there's\nsome awkwardness there, because the names are in a different namespace\nthan the rest of the refs. If we were to just say \"nope, you do not get\nto see the names of the alternates\" then that awkwardness goes away. But\nit also loses some information that could _possibly_ be of use to a\ncaller.\n\nBack in that earlier discussion I did not have a strong opinion, but\nhere we are cementing that decision into a user-visible interface. So it\nprobably makes sense to revisit and decide once and for all.\n\n> +test_description='git receive-pack test'\n> +\n> +. ./test-lib.sh\n> +\n> +test_expect_success 'setup' '\n> +\ttest_commit one &&\n> +\tgit update-ref refs/heads/a HEAD &&\n> +\ttest_commit two &&\n> +\tgit update-ref refs/heads/b HEAD &&\n> +\ttest_commit three &&\n> +\tgit update-ref refs/heads/c HEAD &&\n> +\tgit clone --bare . fork &&\n> +\tgit clone fork pusher &&\n> +\t(\n> +\t\tcd fork &&\n> +\t\tgit config receive.advertisealternates true &&\n> +\t\tgit update-ref -d refs/heads/a &&\n> +\t\tgit update-ref -d refs/heads/b &&\n> +\t\tgit update-ref -d refs/heads/c &&\n> +\t\tgit update-ref -d refs/heads/master &&\n> +\t\tgit update-ref -d refs/tags/one &&\n> +\t\tgit update-ref -d refs/tags/two &&\n> +\t\tgit update-ref -d refs/tags/three &&\n\nProbably not worth nit-picking process count, but this could done with a\nsingle \"update-ref --stdin\".\n\n> +\t\tprintf \"../../.git/objects\" >objects/info/alternates\n\nAlso a nitpick, but I think \"echo\" would be more usual here (we handle\nthe lack of a trailing newline just fine, but any use of printf makes me\nwonder if something tricky is going on with line endings).\n\n> +test_expect_success 'with core.alternateRefsCommand' '\n> +\ttest_config -C fork core.alternateRefsCommand \\\n> +\t\t\"git --git-dir=\\\"\\$1\\\" for-each-ref \\\n> +\t\t--format=\\\"%(objectname) %(refname)\\\" \\\n> +\t\trefs/heads/a refs/heads/c;:\" &&\n\nThis is cute and all, but might it be more readable to use\nwrite_script() to stick it into its own script?\n\n> +\tcat >expect <<-EOF &&\n> +\t$(git rev-parse a) .have\n> +\t$(git rev-parse c) .have\n> +\tEOF\n> +\tprintf \"0000\" | git receive-pack fork | extract_haves >actual &&\n\nThere's been a push lately to avoid having git on the left-hand side of\na fork, since we might otherwise miss its exit code (including things\nlike asan/valgrind errors). So maybe:\n\n ... receive-pack fork >actual &&\n extract_haves actual.haves &&\n test_cmp expect actual.haves\n\nor similar?\n\n> diff --git a/transport.c b/transport.c\n> index 24ae3f375d..e7d2cdf00b 100644\n> --- a/transport.c\n> +++ b/transport.c\n> @@ -1328,10 +1328,21 @@ char *transport_anonymize_url(const char *url)\n> static void fill_alternate_refs_command(struct child_process *cmd,\n> \t\t\t\t\tconst char *repo_path)\n> {\n> -\tcmd->git_cmd = 1;\n> -\targv_array_pushf(&cmd->args, \"--git-dir=%s\", repo_path);\n> -\targv_array_push(&cmd->args, \"for-each-ref\");\n> -\targv_array_push(&cmd->args, \"--format=%(objectname) %(refname)\");\n> +\tconst char *value;\n> +\n> +\tif (!git_config_get_value(\"core.alternateRefsCommand\", &value)) {\n> +\t\tcmd->use_shell = 1;\n> +\n> +\t\targv_array_push(&cmd->args, value);\n> +\t\targv_array_push(&cmd->args, repo_path);\n\nSetting use_shell allows the shell trickery in your test, and matches\nthe modern way we run config-based commands. Good.\n\n> +\t} else {\n> +\t\tcmd->git_cmd = 1;\n> +\n> +\t\targv_array_pushf(&cmd->args, \"--git-dir=%s\", repo_path);\n> +\t\targv_array_push(&cmd->args, \"for-each-ref\");\n> +\t\targv_array_push(&cmd->args, \"--format=%(objectname) %(refname)\");\n> +\t}\n> +\n> \tcmd->env = local_repo_env;\n> \tcmd->out = -1;\n\nAnd we still clear local_repo_env for the custom command, which is good\nto avoid confusion like $GIT_DIR being set when the custom command does\n\"cd $1 && git ...\". Good.\n\n-Peff\n"},{"id":"358536","messageId":"20180920194734.GD29603@sigill.intra.peff.net","threadId":"49393","inReplyTo":"3639e9058859b326f64600fcd0b608171b56ce9f.1537466087.git.me@ttaylorr.com","subject":"Re: [PATCH 3/3] transport.c: introduce core.alternateRefsPrefixes","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2018-09-20T19:47:34Z","receivedAt":"2018-09-20T19:47:37Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Thu, Sep 20, 2018 at 02:04:13PM -0400, Taylor Blau wrote:\n\n> The recently-introduced \"core.alternateRefsCommand\" allows callers to\n> specify with high flexibility the tips that they wish to advertise from\n> alternates. This flexibility comes at the cost of some inconvenience\n> when the caller only wishes to limit the advertisement to one or more\n> prefixes.\n\nTo be clear: this isn't something we plan to use at GitHub at all. It\njust seemed like a nice \"in between\" the current inflexible state and\nthe \"incredibly flexible but not trivial to use\" command from patch 2.\n\nNote that unlike core.alternateRefsCommand, there are no security issues\nhere with reading this from the alternate, although:\n\n - it's a little awkward to read the config from the alternate\n\n - since these are clearly related config, it probably makes sense for\n them to be consistent\n\n> For example, to advertise only tags, a caller using\n> 'core.alternateRefsCommand' would have to do:\n> \n> $ git config core.alternateRefsCommand ' \\\n> git -C \"$1\" for-each-ref refs/tags \\\n> --format=\"%(objectname) %(refname)\" \\\n> '\n\nI think it's more likely that advertising only heads would make sense.\nThe pathological repos I see are usually a sane number of branches and\nthen an absurd number of tags.\n\nNot that it's super important, but I wonder if we should give a\nmotivating example like this in the documentation. In which case we'd\nprobably want to give the most plausible one.\n\n> Since the value of \"core.alternateRefsPrefixes\" is appended to 'git\n> for-each-ref' and then executed, include a \"--\" before taking the\n> configured value to avoid misinterpreting arguments as flags to 'git\n> for-each-ref'.\n\nGood idea.\n\n> diff --git a/Documentation/config.txt b/Documentation/config.txt\n> index b908bc5825..d768c57310 100644\n> --- a/Documentation/config.txt\n> +++ b/Documentation/config.txt\n> @@ -622,6 +622,12 @@ core.alternateRefsCommand::\n> \tlinkgit:git-for-each-ref[1]. The first argument is the path of the alternate.\n> \tOutput must be of the form: `%(objectname) SPC %(refname)`.\n> \n> +core.alternateRefsPrefixes::\n> +\tWhen listing references from an alternate, list only references that begin\n> +\twith the given prefix. To list multiple prefixes, separate them with a\n> +\twhitespace character. If `core.alternateRefsCommand` is set, setting\n> +\t`core.alternateRefsPrefixes` has no effect.\n\nI can't remember all of the rules for how for-each-ref matches prefixes,\nbut I remember that it's subtly different than git-branch (and that's\nwhy ref-filter.c has two matching modes). Do we need to spell out the\nrules here (or at least say \"it matches like for-each-ref\")?\n\nAlso, a minor nit, but I think the argv_array_split() helper you're\nusing soaks up arbitrary amounts of whitespace. So maybe \"separate them\nwith whitespace\" instead of \"a whitespace character\". Or maybe we should\nbe strict in what we suggest and liberal in what we parse. ;)\n\n> +test_expect_success 'with core.alternateRefsPrefixes' '\n> +\ttest_config -C fork core.alternateRefsPrefixes \"refs/tags\" &&\n> +\tcat >expect <<-EOF &&\n> +\t$(git rev-parse one) .have\n> +\t$(git rev-parse three) .have\n> +\t$(git rev-parse two) .have\n> +\tEOF\n> +\tprintf \"0000\" | git receive-pack fork | extract_haves >actual &&\n> +\ttest_cmp expect actual\n\nLooks sane, though the same pipe comment applies as before.\n\n> test_done\n> diff --git a/transport.c b/transport.c\n> index e7d2cdf00b..9323e5c3cd 100644\n> --- a/transport.c\n> +++ b/transport.c\n> @@ -1341,6 +1341,11 @@ static void fill_alternate_refs_command(struct child_process *cmd,\n> \t\targv_array_pushf(&cmd->args, \"--git-dir=%s\", repo_path);\n> \t\targv_array_push(&cmd->args, \"for-each-ref\");\n> \t\targv_array_push(&cmd->args, \"--format=%(objectname) %(refname)\");\n> +\n> +\t\tif (!git_config_get_value(\"core.alternateRefsPrefixes\", &value)) {\n> +\t\t\targv_array_push(&cmd->args, \"--\");\n> +\t\t\targv_array_split(&cmd->args, value);\n> +\t\t}\n> \t}\n\nThe implementation ended up delightfully simple.\n\n-Peff\n"},{"id":"358537","messageId":"20180920200034.GA83799@syl","threadId":"49393","inReplyTo":"20180920193751.GC29603@sigill.intra.peff.net","subject":"Re: [PATCH 2/3] transport.c: introduce core.alternateRefsCommand","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2018-09-20T20:00:34Z","receivedAt":"2018-09-20T20:00:39Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"On Thu, Sep 20, 2018 at 03:37:51PM -0400, Jeff King wrote:\n> On Thu, Sep 20, 2018 at 02:04:11PM -0400, Taylor Blau wrote:\n>\n> > diff --git a/Documentation/config.txt b/Documentation/config.txt\n> > index 112041f407..b908bc5825 100644\n> > --- a/Documentation/config.txt\n> > +++ b/Documentation/config.txt\n> > @@ -616,6 +616,12 @@ core.preferSymlinkRefs::\n> > \tThis is sometimes needed to work with old scripts that\n> > \texpect HEAD to be a symbolic link.\n> >\n> > +core.alternateRefsCommand::\n> > +\tWhen listing references from an alternate (e.g., in the case of \".have\"), use\n> > +\tthe shell to execute the specified command instead of\n> > +\tlinkgit:git-for-each-ref[1]. The first argument is the path of the alternate.\n> > +\tOutput must be of the form: `%(objectname) SPC %(refname)`.\n>\n> We discussed off-list the notion that this could just be the objectname,\n> since the \".have\" mechanism doesn't care about the actual refnames.\n>\n> There's a little prior discussion from the list:\n>\n> https://public-inbox.org/git/xmqqefzraqbu.fsf@gitster.mtv.corp.google.com/\n>\n> My \"rev-list --alternate-refs\" patches _do_ use the refnames, since you\n> could do something like \"--source\" that cares about them. But there's\n> some awkwardness there, because the names are in a different namespace\n> than the rest of the refs. If we were to just say \"nope, you do not get\n> to see the names of the alternates\" then that awkwardness goes away. But\n> it also loses some information that could _possibly_ be of use to a\n> caller.\n>\n> Back in that earlier discussion I did not have a strong opinion, but\n> here we are cementing that decision into a user-visible interface. So it\n> probably makes sense to revisit and decide once and for all.\n\nInteresting, and thanks for the link to the prior discussion. I think\nthat I agree mostly with your rationale in [1], which boils down (for\nme) to:\n\n - Other callers (like 'rev-list --alternate-refs') might care about\n them. Even if we don't have those patches in Git today, it's worth\n keeping their use case(s) in mind.\n\n - I didn't measure either, but I can't imagine that we're paying a\n huge price for this. So, it might be easy enough to keep saying,\n \"please write output as '%(objectname) SP %(refname)'\", even if we\n end up throwing out the refname, anyway.\n\n> > +test_description='git receive-pack test'\n> > +\n> > +. ./test-lib.sh\n> > +\n> > +test_expect_success 'setup' '\n> > +\ttest_commit one &&\n> > +\tgit update-ref refs/heads/a HEAD &&\n> > +\ttest_commit two &&\n> > +\tgit update-ref refs/heads/b HEAD &&\n> > +\ttest_commit three &&\n> > +\tgit update-ref refs/heads/c HEAD &&\n> > +\tgit clone --bare . fork &&\n> > +\tgit clone fork pusher &&\n> > +\t(\n> > +\t\tcd fork &&\n> > +\t\tgit config receive.advertisealternates true &&\n> > +\t\tgit update-ref -d refs/heads/a &&\n> > +\t\tgit update-ref -d refs/heads/b &&\n> > +\t\tgit update-ref -d refs/heads/c &&\n> > +\t\tgit update-ref -d refs/heads/master &&\n> > +\t\tgit update-ref -d refs/tags/one &&\n> > +\t\tgit update-ref -d refs/tags/two &&\n> > +\t\tgit update-ref -d refs/tags/three &&\n>\n> Probably not worth nit-picking process count, but this could done with a\n> single \"update-ref --stdin\".\n\nSure, I don't think that 7 `update-ref`'s vs 2 (`cat` + `git update-ref\n--stdin`) will make or break the series, but I can happily shorten it as\nyou suggest ;-).\n\n> > +\t\tprintf \"../../.git/objects\" >objects/info/alternates\n>\n> Also a nitpick, but I think \"echo\" would be more usual here (we handle\n> the lack of a trailing newline just fine, but any use of printf makes me\n> wonder if something tricky is going on with line endings).\n\n'echo' indeed seems to be the way to go. This 'printf' preference is a\nGit LFS-ism ;-).\n\n> > +test_expect_success 'with core.alternateRefsCommand' '\n> > +\ttest_config -C fork core.alternateRefsCommand \\\n> > +\t\t\"git --git-dir=\\\"\\$1\\\" for-each-ref \\\n> > +\t\t--format=\\\"%(objectname) %(refname)\\\" \\\n> > +\t\trefs/heads/a refs/heads/c;:\" &&\n>\n> This is cute and all, but might it be more readable to use\n> write_script() to stick it into its own script?\n\nGood idea, I'll do that.\n\n> > +\tcat >expect <<-EOF &&\n> > +\t$(git rev-parse a) .have\n> > +\t$(git rev-parse c) .have\n> > +\tEOF\n> > +\tprintf \"0000\" | git receive-pack fork | extract_haves >actual &&\n>\n> There's been a push lately to avoid having git on the left-hand side of\n> a fork, since we might otherwise miss its exit code (including things\n> like asan/valgrind errors). So maybe:\n>\n> ... receive-pack fork >actual &&\n> extract_haves actual.haves &&\n> test_cmp expect actual.haves\n>\n> or similar?\n\nSure, I agree that it's a good idea to not miss the exit code (since we\ndon't have pipefail on), etc. I adopted your suggestion into my local\ncopy.\n\n> > diff --git a/transport.c b/transport.c\n> > index 24ae3f375d..e7d2cdf00b 100644\n> > --- a/transport.c\n> > +++ b/transport.c\n> > @@ -1328,10 +1328,21 @@ char *transport_anonymize_url(const char *url)\n> > static void fill_alternate_refs_command(struct child_process *cmd,\n> > \t\t\t\t\tconst char *repo_path)\n> > {\n> > -\tcmd->git_cmd = 1;\n> > -\targv_array_pushf(&cmd->args, \"--git-dir=%s\", repo_path);\n> > -\targv_array_push(&cmd->args, \"for-each-ref\");\n> > -\targv_array_push(&cmd->args, \"--format=%(objectname) %(refname)\");\n> > +\tconst char *value;\n> > +\n> > +\tif (!git_config_get_value(\"core.alternateRefsCommand\", &value)) {\n> > +\t\tcmd->use_shell = 1;\n> > +\n> > +\t\targv_array_push(&cmd->args, value);\n> > +\t\targv_array_push(&cmd->args, repo_path);\n>\n> Setting use_shell allows the shell trickery in your test, and matches\n> the modern way we run config-based commands. Good.\n>\n> > +\t} else {\n> > +\t\tcmd->git_cmd = 1;\n> > +\n> > +\t\targv_array_pushf(&cmd->args, \"--git-dir=%s\", repo_path);\n> > +\t\targv_array_push(&cmd->args, \"for-each-ref\");\n> > +\t\targv_array_push(&cmd->args, \"--format=%(objectname) %(refname)\");\n> > +\t}\n> > +\n> > \tcmd->env = local_repo_env;\n> > \tcmd->out = -1;\n>\n> And we still clear local_repo_env for the custom command, which is good\n> to avoid confusion like $GIT_DIR being set when the custom command does\n> \"cd $1 && git ...\". Good.\n\nThanks,\nTaylor\n\n[1]: https://public-inbox.org/git/20170125195425.q4fpvc4ten5mfjgl@sigill.intra.peff.net/\n"},{"id":"358538","messageId":"20180920200627.GA536@sigill.intra.peff.net","threadId":"49393","inReplyTo":"20180920200034.GA83799@syl","subject":"Re: [PATCH 2/3] transport.c: introduce core.alternateRefsCommand","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2018-09-20T20:06:27Z","receivedAt":"2018-09-20T20:06:30Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Thu, Sep 20, 2018 at 04:00:34PM -0400, Taylor Blau wrote:\n\n> > My \"rev-list --alternate-refs\" patches _do_ use the refnames, since you\n> > could do something like \"--source\" that cares about them. But there's\n> > some awkwardness there, because the names are in a different namespace\n> > than the rest of the refs. If we were to just say \"nope, you do not get\n> > to see the names of the alternates\" then that awkwardness goes away. But\n> > it also loses some information that could _possibly_ be of use to a\n> > caller.\n> >\n> > Back in that earlier discussion I did not have a strong opinion, but\n> > here we are cementing that decision into a user-visible interface. So it\n> > probably makes sense to revisit and decide once and for all.\n> \n> Interesting, and thanks for the link to the prior discussion. I think\n> that I agree mostly with your rationale in [1], which boils down (for\n> me) to:\n> \n> - Other callers (like 'rev-list --alternate-refs') might care about\n> them. Even if we don't have those patches in Git today, it's worth\n> keeping their use case(s) in mind.\n> \n> - I didn't measure either, but I can't imagine that we're paying a\n> huge price for this. So, it might be easy enough to keep saying,\n> \"please write output as '%(objectname) SP %(refname)'\", even if we\n> end up throwing out the refname, anyway.\n\nTBH, the main advantage to me is that it makes the user-visible\ninterface way simpler. We just say \"give us a list of object ids, one\nper line\". I guess the current spec is not too bad, especially given\nthat we can just provide a for-each-ref format that generates it.\n\n> > Probably not worth nit-picking process count, but this could done with a\n> > single \"update-ref --stdin\".\n> \n> Sure, I don't think that 7 `update-ref`'s vs 2 (`cat` + `git update-ref\n> --stdin`) will make or break the series, but I can happily shorten it as\n> you suggest ;-).\n\nYeah, in retrospect I should have not have even mentioned it.\ntest_commit() already adds a bunch of extra processes you may or may not\ncare about (e.g., by making tags, or using \"git add\" when \"commit -a\"\nmight do).\n\n> > > +\tcat >expect <<-EOF &&\n> > > +\t$(git rev-parse a) .have\n> > > +\t$(git rev-parse c) .have\n> > > +\tEOF\n> > > +\tprintf \"0000\" | git receive-pack fork | extract_haves >actual &&\n> >\n> > There's been a push lately to avoid having git on the left-hand side of\n> > a fork, since we might otherwise miss its exit code (including things\n\nHeh, I meant to say \"left-hand side of a pipe\", but you obviously\nfigured out what I meant. :)\n\n-Peff\n"},{"id":"358539","messageId":"20180920201235.GB83799@syl","threadId":"49393","inReplyTo":"20180920194734.GD29603@sigill.intra.peff.net","subject":"Re: [PATCH 3/3] transport.c: introduce core.alternateRefsPrefixes","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2018-09-20T20:12:35Z","receivedAt":"2018-09-20T20:12:41Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"On Thu, Sep 20, 2018 at 03:47:34PM -0400, Jeff King wrote:\n> On Thu, Sep 20, 2018 at 02:04:13PM -0400, Taylor Blau wrote:\n>\n> > The recently-introduced \"core.alternateRefsCommand\" allows callers to\n> > specify with high flexibility the tips that they wish to advertise from\n> > alternates. This flexibility comes at the cost of some inconvenience\n> > when the caller only wishes to limit the advertisement to one or more\n> > prefixes.\n>\n> To be clear: this isn't something we plan to use at GitHub at all. It\n> just seemed like a nice \"in between\" the current inflexible state and\n> the \"incredibly flexible but not trivial to use\" command from patch 2.\n>\n> Note that unlike core.alternateRefsCommand, there are no security issues\n> here with reading this from the alternate, although:\n>\n> - it's a little awkward to read the config from the alternate\n>\n> - since these are clearly related config, it probably makes sense for\n> them to be consistent\n\nAnother note is that the thing we are planning on using\n(\"core.alternateRefsCommand\") could also be implemented as a hook,\ne.g., .git/hooks/gather-alternate-refs.\n\nThat said, I think that this makes more sense when the alternate is\ndoing the configuring, not the ohter way around.\n\n> > For example, to advertise only tags, a caller using\n> > 'core.alternateRefsCommand' would have to do:\n> >\n> > $ git config core.alternateRefsCommand ' \\\n> > git -C \"$1\" for-each-ref refs/tags \\\n> > --format=\"%(objectname) %(refname)\" \\\n> > '\n>\n> I think it's more likely that advertising only heads would make sense.\n> The pathological repos I see are usually a sane number of branches and\n> then an absurd number of tags.\n\nI agree with you. I used \"refs/tags\" as the prefix here since I'd like\ndifferent output than when \"core.alternateRefsPrefixes\" isn't configured\nat all. Since we have a tag for each commit (we use test_commit to do\nso), and refs/heads/{a,b,c,master}, we'd get the same output whether we\nconfigured the prefix to be refs/heads, or didn't configure it at all.\n\nSince using 'git for-each-ref' sorts in order of refname, a prefix of\n\"refs/tags\" sorts in order of tagname, so we'll get different output\nbecause of it.\n\nThat said, I think that this test is a little fragile as-is, since it'll\nbreak if we change the ordering of 'git for-each-ref'. Maybe we should\n`| sort >actual.haves`?\n\n> Not that it's super important, but I wonder if we should give a\n> motivating example like this in the documentation. In which case we'd\n> probably want to give the most plausible one.\n\nMaybe. I don't feel strongly about it, though.\n\n> > Since the value of \"core.alternateRefsPrefixes\" is appended to 'git\n> > for-each-ref' and then executed, include a \"--\" before taking the\n> > configured value to avoid misinterpreting arguments as flags to 'git\n> > for-each-ref'.\n>\n> Good idea.\n>\n> > diff --git a/Documentation/config.txt b/Documentation/config.txt\n> > index b908bc5825..d768c57310 100644\n> > --- a/Documentation/config.txt\n> > +++ b/Documentation/config.txt\n> > @@ -622,6 +622,12 @@ core.alternateRefsCommand::\n> > \tlinkgit:git-for-each-ref[1]. The first argument is the path of the alternate.\n> > \tOutput must be of the form: `%(objectname) SPC %(refname)`.\n> >\n> > +core.alternateRefsPrefixes::\n> > +\tWhen listing references from an alternate, list only references that begin\n> > +\twith the given prefix. To list multiple prefixes, separate them with a\n> > +\twhitespace character. If `core.alternateRefsCommand` is set, setting\n> > +\t`core.alternateRefsPrefixes` has no effect.\n>\n> I can't remember all of the rules for how for-each-ref matches prefixes,\n> but I remember that it's subtly different than git-branch (and that's\n> why ref-filter.c has two matching modes). Do we need to spell out the\n> rules here (or at least say \"it matches like for-each-ref\")?\n\nGood idea. I'll do that.\n\n> Also, a minor nit, but I think the argv_array_split() helper you're\n> using soaks up arbitrary amounts of whitespace. So maybe \"separate them\n> with whitespace\" instead of \"a whitespace character\". Or maybe we should\n> be strict in what we suggest and liberal in what we parse. ;)\n\nYeah, I think that chaning \"a whitespace character\" -> \"with\nwhitespace\" is the easier thing to do ;-).\n\n> > +test_expect_success 'with core.alternateRefsPrefixes' '\n> > +\ttest_config -C fork core.alternateRefsPrefixes \"refs/tags\" &&\n> > +\tcat >expect <<-EOF &&\n> > +\t$(git rev-parse one) .have\n> > +\t$(git rev-parse three) .have\n> > +\t$(git rev-parse two) .have\n> > +\tEOF\n> > +\tprintf \"0000\" | git receive-pack fork | extract_haves >actual &&\n> > +\ttest_cmp expect actual\n>\n> Looks sane, though the same pipe comment applies as before.\n\nThanks. I applied that suggestion in both locations when reading your\nlast mail.\n\n> > test_done\n> > diff --git a/transport.c b/transport.c\n> > index e7d2cdf00b..9323e5c3cd 100644\n> > --- a/transport.c\n> > +++ b/transport.c\n> > @@ -1341,6 +1341,11 @@ static void fill_alternate_refs_command(struct child_process *cmd,\n> > \t\targv_array_pushf(&cmd->args, \"--git-dir=%s\", repo_path);\n> > \t\targv_array_push(&cmd->args, \"for-each-ref\");\n> > \t\targv_array_push(&cmd->args, \"--format=%(objectname) %(refname)\");\n> > +\n> > +\t\tif (!git_config_get_value(\"core.alternateRefsPrefixes\", &value)) {\n> > +\t\t\targv_array_push(&cmd->args, \"--\");\n> > +\t\t\targv_array_split(&cmd->args, value);\n> > +\t\t}\n> > \t}\n>\n> The implementation ended up delightfully simple.\n\nThanks :-). It made me quite happy, too.\n\nThanks,\nTaylor\n"},{"id":"358562","messageId":"CAPig+cT7WTyBCQZ75WSjmBqiui383YrKqoHqbLASQkOaGVTfVA@mail.gmail.com","threadId":"49393","inReplyTo":"3639e9058859b326f64600fcd0b608171b56ce9f.1537466087.git.me@ttaylorr.com","subject":"Re: [PATCH 3/3] transport.c: introduce core.alternateRefsPrefixes","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2018-09-21T07:19:20Z","receivedAt":"2018-09-21T07:19:34Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Thu, Sep 20, 2018 at 2:04 PM Taylor Blau wrote:\n> The recently-introduced \"core.alternateRefsCommand\" allows callers to\n> specify with high flexibility the tips that they wish to advertise from\n> alternates. This flexibility comes at the cost of some inconvenience\n> when the caller only wishes to limit the advertisement to one or more\n> prefixes.\n> [...]\n> Signed-off-by: Taylor Blau \n> ---\n> diff --git a/t/t5410-receive-pack.sh b/t/t5410-receive-pack.sh\n> @@ -44,4 +44,15 @@ test_expect_success 'with core.alternateRefsCommand' '\n> +test_expect_success 'with core.alternateRefsPrefixes' '\n> + test_config -C fork core.alternateRefsPrefixes \"refs/tags\" &&\n> + cat >expect <<-EOF &&\n> + $(git rev-parse one) .have\n> + $(git rev-parse three) .have\n> + $(git rev-parse two) .have\n> + EOF\n\nIt's probably a matter of taste as to which is more readable, but this\nentire \"cat <expect &&\n\nSame comment applies to previous patch, as well.\n\n> + printf \"0000\" | git receive-pack fork | extract_haves >actual &&\n> + test_cmp expect actual\n> +'\n"},{"id":"358566","messageId":"20180921140732.GA43093@syl","threadId":"49393","inReplyTo":"CAPig+cT7WTyBCQZ75WSjmBqiui383YrKqoHqbLASQkOaGVTfVA@mail.gmail.com","subject":"Re: [PATCH 3/3] transport.c: introduce core.alternateRefsPrefixes","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2018-09-21T14:07:32Z","receivedAt":"2018-09-21T14:07:38Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"On Fri, Sep 21, 2018 at 03:19:20AM -0400, Eric Sunshine wrote:\n> On Thu, Sep 20, 2018 at 2:04 PM Taylor Blau wrote:\n> > The recently-introduced \"core.alternateRefsCommand\" allows callers to\n> > specify with high flexibility the tips that they wish to advertise from\n> > alternates. This flexibility comes at the cost of some inconvenience\n> > when the caller only wishes to limit the advertisement to one or more\n> > prefixes.\n> > [...]\n> > Signed-off-by: Taylor Blau \n> > ---\n> > diff --git a/t/t5410-receive-pack.sh b/t/t5410-receive-pack.sh\n> > @@ -44,4 +44,15 @@ test_expect_success 'with core.alternateRefsCommand' '\n> > +test_expect_success 'with core.alternateRefsPrefixes' '\n> > + test_config -C fork core.alternateRefsPrefixes \"refs/tags\" &&\n> > + cat >expect <<-EOF &&\n> > + $(git rev-parse one) .have\n> > + $(git rev-parse three) .have\n> > + $(git rev-parse two) .have\n> > + EOF\n>\n> It's probably a matter of taste as to which is more readable, but this\n> entire \"cat <\n> printf \"%s .have\\n\" $(git rev-parse one three two) >expect &&\n>\n> Same comment applies to previous patch, as well.\n\nThat's a good idea. I amended both patches to replace the 'cat <<-EOF\n...' block with your suggestion above. It's tempting to introduce it as:\n\n expect_haves() {\n printf \"%s .have\\n\" $(git rev-parse -- $@)\n }\n\nAnd call it as:\n\n expect_haves one three two >expect\n\nBut I'm not sure whether I think that this is better or worse than\nwriting it twice inline. I think that the test is small enough that it\ndoesn't really matter either way, but I think that I've convinced myself\nwhile composing this email that expect_haves() is an OK idea.\n\nIf you feel strongly that it isn't, please let me know, and I'll write\nthem inline before sending v2.\n\nThanks,\nTaylor\n"},{"id":"358610","messageId":"xmqqlg7ux0st.fsf@gitster-ct.c.googlers.com","threadId":"49393","inReplyTo":"4c4900722cab253b3ce33cb28910c4602ce44536.1537466087.git.me@ttaylorr.com","subject":"Re: [PATCH 2/3] transport.c: introduce core.alternateRefsCommand","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2018-09-21T16:39:14Z","receivedAt":"2018-09-21T16:39:19Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Taylor Blau writes:\n\n> +extract_haves () {\n> +\tdepacketize - | grep -o '^.* \\.have'\n\nNot portable, isn't it?\n\ncf. http://pubs.opengroup.org/onlinepubs/9699919799/utilities/grep.html\n\n"},{"id":"358611","messageId":"xmqqk1nex0rg.fsf@gitster-ct.c.googlers.com","threadId":"49393","inReplyTo":"CAPig+cT7WTyBCQZ75WSjmBqiui383YrKqoHqbLASQkOaGVTfVA@mail.gmail.com","subject":"Re: [PATCH 3/3] transport.c: introduce core.alternateRefsPrefixes","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2018-09-21T16:40:03Z","receivedAt":"2018-09-21T16:40:07Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Eric Sunshine writes:\n\n> On Thu, Sep 20, 2018 at 2:04 PM Taylor Blau wrote:\n>> The recently-introduced \"core.alternateRefsCommand\" allows callers to\n>> specify with high flexibility the tips that they wish to advertise from\n>> alternates. This flexibility comes at the cost of some inconvenience\n>> when the caller only wishes to limit the advertisement to one or more\n>> prefixes.\n>> [...]\n>> Signed-off-by: Taylor Blau \n>> ---\n>> diff --git a/t/t5410-receive-pack.sh b/t/t5410-receive-pack.sh\n>> @@ -44,4 +44,15 @@ test_expect_success 'with core.alternateRefsCommand' '\n>> +test_expect_success 'with core.alternateRefsPrefixes' '\n>> + test_config -C fork core.alternateRefsPrefixes \"refs/tags\" &&\n>> + cat >expect <<-EOF &&\n>> + $(git rev-parse one) .have\n>> + $(git rev-parse three) .have\n>> + $(git rev-parse two) .have\n>> + EOF\n>\n> It's probably a matter of taste as to which is more readable, but this\n> entire \"cat <\n> printf \"%s .have\\n\" $(git rev-parse one three two) >expect &&\n>\n> Same comment applies to previous patch, as well.\n\nIf the expected pattern is expected to stay to be just a sequence of\n\" .have\" and nothing else for the foreseeable future, I think\nit is a good idea.\n\n>\n>> + printf \"0000\" | git receive-pack fork | extract_haves >actual &&\n>> + test_cmp expect actual\n>> +'\n"},{"id":"358612","messageId":"xmqqfty2x0iw.fsf@gitster-ct.c.googlers.com","threadId":"49393","inReplyTo":"20180921140732.GA43093@syl","subject":"Re: [PATCH 3/3] transport.c: introduce core.alternateRefsPrefixes","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2018-09-21T16:45:11Z","receivedAt":"2018-09-21T16:45:16Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Taylor Blau writes:\n\n> ...' block with your suggestion above. It's tempting to introduce it as:\n>\n> expect_haves() {\n> printf \"%s .have\\n\" $(git rev-parse -- $@)\n> }\n>\n> And call it as:\n>\n> expect_haves one three two >expect\n>\n> But I'm not sure whether I think that this is better or worse than\n> writing it twice inline.\n\nIf the expected pattern is expected to stay to be just a sequence of\n\" .have\" and nothing else for the foreseeable future, I think\nit is a good idea to introduce such a helper function. Spelling it\nout at the use site, e.g.\n\n\tprintf \"%s .have\\n\" $(git rev-parse a b c) >expect\n\nwill become cumbersome once the set of objects you need to show\nstarts growing.\n\n\texpect_haves a b c >expect\n\nwould be shorter, of course. And as long as we expect to have ONLY\n\" .have\" lines and nothing else, there is no downside that the\ndetails of the format is hidden away inside the helper.\n\n"},{"id":"358630","messageId":"20180921174825.GA17718@syl","threadId":"49393","inReplyTo":"xmqqlg7ux0st.fsf@gitster-ct.c.googlers.com","subject":"Re: [PATCH 2/3] transport.c: introduce core.alternateRefsCommand","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2018-09-21T17:48:25Z","receivedAt":"2018-09-21T17:48:31Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"On Fri, Sep 21, 2018 at 09:39:14AM -0700, Junio C Hamano wrote:\n> Taylor Blau writes:\n>\n> > +extract_haves () {\n> > +\tdepacketize - | grep -o '^.* \\.have'\n>\n> Not portable, isn't it?\n>\n> cf. http://pubs.opengroup.org/onlinepubs/9699919799/utilities/grep.html\n\nGood catch. Definitely not portable, per the link that you shared above.\n\nSince 'depacketize()' will give us a \"\\0\", we can pull it and anything\nafter it out with 'sed', instead. Any lines that don't contain a \"\\0\"\nonly contain an OID and the literal, \".have\", and are fine as-is.\n\nSomething like this:\n\n extract_haves () {\n depacketize - | grep '^.* \\.have' | sed -e 's/\\\\0.*$//g'\n }\n\nHarder to read--at least for me--but infinitely more portable.\n\nI'll wait until a little later today, and then send you v2. Thanks for\nreviewing :-).\n\nThanks,\nTaylor\n"},{"id":"358631","messageId":"20180921174923.GB17718@syl","threadId":"49393","inReplyTo":"xmqqfty2x0iw.fsf@gitster-ct.c.googlers.com","subject":"Re: [PATCH 3/3] transport.c: introduce core.alternateRefsPrefixes","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2018-09-21T17:49:23Z","receivedAt":"2018-09-21T17:49:29Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"On Fri, Sep 21, 2018 at 09:45:11AM -0700, Junio C Hamano wrote:\n> Taylor Blau writes:\n>\n> > ...' block with your suggestion above. It's tempting to introduce it as:\n> >\n> > expect_haves() {\n> > printf \"%s .have\\n\" $(git rev-parse -- $@)\n> > }\n> >\n> > And call it as:\n> >\n> > expect_haves one three two >expect\n> >\n> > But I'm not sure whether I think that this is better or worse than\n> > writing it twice inline.\n>\n> If the expected pattern is expected to stay to be just a sequence of\n> \" .have\" and nothing else for the foreseeable future, I think\n> it is a good idea to introduce such a helper function. Spelling it\n> out at the use site, e.g.\n>\n> \tprintf \"%s .have\\n\" $(git rev-parse a b c) >expect\n>\n> will become cumbersome once the set of objects you need to show\n> starts growing.\n\nThat's a good reason, and I hadn't thought of it.\n\n> \texpect_haves a b c >expect\n>\n> would be shorter, of course. And as long as we expect to have ONLY\n> \" .have\" lines and nothing else, there is no downside that the\n> details of the format is hidden away inside the helper.\n\nYeah, I don't expect this to to change much at all, so I think that\n'expect_haves()' is good.\n\nThanks,\nTaylor\n"},{"id":"358632","messageId":"20180921175739.GA21514@syl","threadId":"49393","inReplyTo":"20180921174825.GA17718@syl","subject":"Re: [PATCH 2/3] transport.c: introduce core.alternateRefsCommand","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2018-09-21T17:57:39Z","receivedAt":"2018-09-21T17:57:44Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"On Fri, Sep 21, 2018 at 01:48:25PM -0400, Taylor Blau wrote:\n> On Fri, Sep 21, 2018 at 09:39:14AM -0700, Junio C Hamano wrote:\n> > Taylor Blau writes:\n> >\n> > > +extract_haves () {\n> > > +\tdepacketize - | grep -o '^.* \\.have'\n> >\n> > Not portable, isn't it?\n> >\n> > cf. http://pubs.opengroup.org/onlinepubs/9699919799/utilities/grep.html\n>\n> Good catch. Definitely not portable, per the link that you shared above.\n>\n> Since 'depacketize()' will give us a \"\\0\", we can pull it and anything\n> after it out with 'sed', instead. Any lines that don't contain a \"\\0\"\n> only contain an OID and the literal, \".have\", and are fine as-is.\n>\n> Something like this:\n>\n> extract_haves () {\n> depacketize - | grep '^.* \\.have' | sed -e 's/\\\\0.*$//g'\n> }\n>\n> Harder to read--at least for me--but infinitely more portable.\n\nIn fact, I think that we can go even further: since we don't need to\ncatch the beginning '^.*' (without -o), we can instead:\n\n extract_haves () {\n depacketize - | grep '\\.have' | sed -e 's/\\\\0.*$//g'\n }\n\nThanks,\nTaylor\n"},{"id":"358636","messageId":"cover.1537555544.git.me@ttaylorr.com","threadId":"49393","inReplyTo":"cover.1537466087.git.me@ttaylorr.com","subject":"[PATCH v2 0/3] Filter alternate references","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2018-09-21T18:47:36Z","receivedAt":"2018-09-21T18:47:42Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"Hi,\n\nAttached is the second re-roll of my series to teach\n\"core.alternateRefsCommand\" and \"core.alternateRefsPrefixes\".\n\nI have included a range-diff below (which I have taught my scripts to do\nby default now), but will summarize the changes as usual:\n\n * Clean up t5410 according to Peff's suggestions in [1]:\n\n * Simplify many `git update-ref -d`'s into one `git update-ref\n --stdin`.\n\n * Use `echo >`, instead of `printf >` to write an alternate\n repository.\n\n * Avoid placing Git on the left-hand side of a pipe.\n\n * Use 'write_script', instead of embedding the same code in a\n lengthy 'test_config'.\n\n * Add a motivating example in Documentation/config.txt, per Peff's\n suggestion in [1].\n\n * Use `printf \"%s .have\\n\"` with many arguments instead of another\n `cat <<-EOF` block and extract it into `expect_haves`, per [2].\n\n * Do not use `grep -o` in `extract_haves`, thus making it portable.\n Per [3].\n\n[1]: https://public-inbox.org/git/20180920193751.GC29603@sigill.intra.peff.net/\n[2]: https://public-inbox.org/git/CAPig+cT7WTyBCQZ75WSjmBqiui383YrKqoHqbLASQkOaGVTfVA@mail.gmail.com/\n[3]: https://public-inbox.org/git/xmqqlg7ux0st.fsf@gitster-ct.c.googlers.com/\n\nTaylor Blau (3):\n transport.c: extract 'fill_alternate_refs_command'\n transport.c: introduce core.alternateRefsCommand\n transport.c: introduce core.alternateRefsPrefixes\n\n Documentation/config.txt | 18 ++++++++++++\n t/t5410-receive-pack.sh | 62 ++++++++++++++++++++++++++++++++++++++++\n transport.c | 34 ++++++++++++++++++----\n 3 files changed, 108 insertions(+), 6 deletions(-)\n create mode 100755 t/t5410-receive-pack.sh\n\nRange-diff against v1:\n1: 6e3a58afe7 = 1: 6e3a58afe7 transport.c: extract 'fill_alternate_refs_command'\n2: 4c4900722c ! 2: 9797f52551 transport.c: introduce core.alternateRefsCommand\n @@ -42,6 +42,11 @@\n +\tthe shell to execute the specified command instead of\n +\tlinkgit:git-for-each-ref[1]. The first argument is the path of the alternate.\n +\tOutput must be of the form: `%(objectname) SPC %(refname)`.\n +++\n ++This is useful when a repository only wishes to advertise some of its\n ++alternate's references as \".have\"'s. For example, to only advertise branch\n ++heads, configure `core.alternateRefsCommand` to the path of a script which runs\n ++`git --git-dir=\"$1\" for-each-ref refs/heads`.\n +\n core.bare::\n \tIf true this repository is assumed to be 'bare' and has no\n @@ -70,32 +75,39 @@\n +\t(\n +\t\tcd fork &&\n +\t\tgit config receive.advertisealternates true &&\n -+\t\tgit update-ref -d refs/heads/a &&\n -+\t\tgit update-ref -d refs/heads/b &&\n -+\t\tgit update-ref -d refs/heads/c &&\n -+\t\tgit update-ref -d refs/heads/master &&\n -+\t\tgit update-ref -d refs/tags/one &&\n -+\t\tgit update-ref -d refs/tags/two &&\n -+\t\tgit update-ref -d refs/tags/three &&\n -+\t\tprintf \"../../.git/objects\" >objects/info/alternates\n ++\t\tcat <<-EOF | git update-ref --stdin &&\n ++\t\tdelete refs/heads/a\n ++\t\tdelete refs/heads/b\n ++\t\tdelete refs/heads/c\n ++\t\tdelete refs/heads/master\n ++\t\tdelete refs/tags/one\n ++\t\tdelete refs/tags/two\n ++\t\tdelete refs/tags/three\n ++\t\tEOF\n ++\t\techo \"../../.git/objects\" >objects/info/alternates\n +\t)\n +'\n +\n ++expect_haves () {\n ++\tprintf \"%s .have\\n\" $(git rev-parse $@) >expect\n ++}\n ++\n +extract_haves () {\n -+\tdepacketize - | grep -o '^.* \\.have'\n ++\tdepacketize - | grep '\\.have' | sed -e 's/\\\\0.*$//g'\n +}\n +\n +test_expect_success 'with core.alternateRefsCommand' '\n -+\ttest_config -C fork core.alternateRefsCommand \\\n -+\t\t\"git --git-dir=\\\"\\$1\\\" for-each-ref \\\n -+\t\t--format=\\\"%(objectname) %(refname)\\\" \\\n -+\t\trefs/heads/a refs/heads/c;:\" &&\n -+\tcat >expect <<-EOF &&\n -+\t$(git rev-parse a) .have\n -+\t$(git rev-parse c) .have\n ++\twrite_script fork/alternate-refs <<-\\EOF &&\n ++\t\tgit --git-dir=\"$1\" for-each-ref \\\n ++\t\t\t--format=\"%(objectname) %(refname)\" \\\n ++\t\t\trefs/heads/a \\\n ++\t\t\trefs/heads/c\n +\tEOF\n -+\tprintf \"0000\" | git receive-pack fork | extract_haves >actual &&\n -+\ttest_cmp expect actual\n ++\ttest_config -C fork core.alternateRefsCommand alternate-refs &&\n ++\texpect_haves a c >expect &&\n ++\tprintf \"0000\" | git receive-pack fork >actual &&\n ++\textract_haves actual.haves &&\n ++\ttest_cmp expect actual.haves\n +'\n +\n +test_done\n3: 3639e90588 ! 3: 6e8f65a16d transport.c: introduce core.alternateRefsPrefixes\n @@ -40,13 +40,14 @@\n --- a/Documentation/config.txt\n +++ b/Documentation/config.txt\n @@\n - \tlinkgit:git-for-each-ref[1]. The first argument is the path of the alternate.\n - \tOutput must be of the form: `%(objectname) SPC %(refname)`.\n + heads, configure `core.alternateRefsCommand` to the path of a script which runs\n + `git --git-dir=\"$1\" for-each-ref refs/heads`.\n\n +core.alternateRefsPrefixes::\n +\tWhen listing references from an alternate, list only references that begin\n -+\twith the given prefix. To list multiple prefixes, separate them with a\n -+\twhitespace character. If `core.alternateRefsCommand` is set, setting\n ++\twith the given prefix. Prefixes match as if they were given as arguments to\n ++\tlinkgit:git-for-each-ref[1]. To list multiple prefixes, separate them with\n ++\twhitespace. If `core.alternateRefsCommand` is set, setting\n +\t`core.alternateRefsPrefixes` has no effect.\n +\n core.bare::\n @@ -57,18 +58,15 @@\n --- a/t/t5410-receive-pack.sh\n +++ b/t/t5410-receive-pack.sh\n @@\n - \ttest_cmp expect actual\n + \ttest_cmp expect actual.haves\n '\n\n +test_expect_success 'with core.alternateRefsPrefixes' '\n +\ttest_config -C fork core.alternateRefsPrefixes \"refs/tags\" &&\n -+\tcat >expect <<-EOF &&\n -+\t$(git rev-parse one) .have\n -+\t$(git rev-parse three) .have\n -+\t$(git rev-parse two) .have\n -+\tEOF\n -+\tprintf \"0000\" | git receive-pack fork | extract_haves >actual &&\n -+\ttest_cmp expect actual\n ++\texpect_haves one three two >expect &&\n ++\tprintf \"0000\" | git receive-pack fork >actual &&\n ++\textract_haves actual.haves &&\n ++\ttest_cmp expect actual.haves\n +'\n +\n test_done\n--\n2.19.0.221.g150f307af\n"},{"id":"358637","messageId":"6e3a58afe7cd18d663f481cdc9eb65cc941765b1.1537555544.git.me@ttaylorr.com","threadId":"49393","inReplyTo":"cover.1537555544.git.me@ttaylorr.com","subject":"[PATCH v2 1/3] transport.c: extract 'fill_alternate_refs_command'","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2018-09-21T18:47:39Z","receivedAt":"2018-09-21T18:47:45Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"To list alternate references, 'read_alternate_refs' creates a child\nprocess running 'git for-each-ref' in the alternate's Git directory.\n\nPrepare to run other commands besides 'git for-each-ref' by introducing\nand moving the relevant code from 'read_alternate_refs' to\n'fill_alternate_refs_command'.\n\nSigned-off-by: Taylor Blau \n---\n transport.c | 18 ++++++++++++------\n 1 file changed, 12 insertions(+), 6 deletions(-)\n\ndiff --git a/transport.c b/transport.c\nindex 1c76d64aba..24ae3f375d 100644\n--- a/transport.c\n+++ b/transport.c\n@@ -1325,6 +1325,17 @@ char *transport_anonymize_url(const char *url)\n \treturn xstrdup(url);\n }\n \n+static void fill_alternate_refs_command(struct child_process *cmd,\n+\t\t\t\t\tconst char *repo_path)\n+{\n+\tcmd->git_cmd = 1;\n+\targv_array_pushf(&cmd->args, \"--git-dir=%s\", repo_path);\n+\targv_array_push(&cmd->args, \"for-each-ref\");\n+\targv_array_push(&cmd->args, \"--format=%(objectname) %(refname)\");\n+\tcmd->env = local_repo_env;\n+\tcmd->out = -1;\n+}\n+\n static void read_alternate_refs(const char *path,\n \t\t\t\talternate_ref_fn *cb,\n \t\t\t\tvoid *data)\n@@ -1333,12 +1344,7 @@ static void read_alternate_refs(const char *path,\n \tstruct strbuf line = STRBUF_INIT;\n \tFILE *fh;\n \n-\tcmd.git_cmd = 1;\n-\targv_array_pushf(&cmd.args, \"--git-dir=%s\", path);\n-\targv_array_push(&cmd.args, \"for-each-ref\");\n-\targv_array_push(&cmd.args, \"--format=%(objectname) %(refname)\");\n-\tcmd.env = local_repo_env;\n-\tcmd.out = -1;\n+\tfill_alternate_refs_command(&cmd, path);\n \n \tif (start_command(&cmd))\n \t\treturn;\n-- \n2.19.0.221.g150f307af\n\n"},{"id":"358638","messageId":"9797f525517142b3494cfbd17a10dfeb3bf586e2.1537555544.git.me@ttaylorr.com","threadId":"49393","inReplyTo":"cover.1537555544.git.me@ttaylorr.com","subject":"[PATCH v2 2/3] transport.c: introduce core.alternateRefsCommand","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2018-09-21T18:47:43Z","receivedAt":"2018-09-21T18:47:49Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"When in a repository containing one or more alternates, Git would\nsometimes like to list references from its alternates. For example, 'git\nreceive-pack' list the objects pointed to by alternate references as\nspecial \".have\" references.\n\nListing \".have\" references is designed to make pushing changes from\nupstream to a fork a lightweight operation, by advertising to the pusher\nthat the fork already has the objects (via its alternate). Thus, the\nclient can avoid sending them.\n\nHowever, when the alternate has a pathologically large number of\nreferences, the initial advertisement is too expensive. In fact, it can\ndominate any such optimization where the pusher avoids sending certain\nobjects.\n\nIntroduce \"core.alternateRefsCommand\" in order to provide a facility to\nlimit or filter alternate references. This can be used, for example, to\nfilter out \"uninteresting\" references from the initial advertisement in\nthe above scenario.\n\nLet the repository that has alternates configure this command to avoid\ntrusting the alternate to provide us a safe command to run in the shell.\nTo behave differently on each alternate (e.g., only list tags from\nalternate A, only heads from B) provide the path of the alternate as the\nfirst argument.\n\nSigned-off-by: Taylor Blau \n---\n Documentation/config.txt | 11 ++++++++\n t/t5410-receive-pack.sh | 54 ++++++++++++++++++++++++++++++++++++++++\n transport.c | 19 +++++++++++---\n 3 files changed, 80 insertions(+), 4 deletions(-)\n create mode 100755 t/t5410-receive-pack.sh\n\ndiff --git a/Documentation/config.txt b/Documentation/config.txt\nindex 112041f407..526557e494 100644\n--- a/Documentation/config.txt\n+++ b/Documentation/config.txt\n@@ -616,6 +616,17 @@ core.preferSymlinkRefs::\n \tThis is sometimes needed to work with old scripts that\n \texpect HEAD to be a symbolic link.\n \n+core.alternateRefsCommand::\n+\tWhen listing references from an alternate (e.g., in the case of \".have\"), use\n+\tthe shell to execute the specified command instead of\n+\tlinkgit:git-for-each-ref[1]. The first argument is the path of the alternate.\n+\tOutput must be of the form: `%(objectname) SPC %(refname)`.\n++\n+This is useful when a repository only wishes to advertise some of its\n+alternate's references as \".have\"'s. For example, to only advertise branch\n+heads, configure `core.alternateRefsCommand` to the path of a script which runs\n+`git --git-dir=\"$1\" for-each-ref refs/heads`.\n+\n core.bare::\n \tIf true this repository is assumed to be 'bare' and has no\n \tworking directory associated with it. If this is the case a\ndiff --git a/t/t5410-receive-pack.sh b/t/t5410-receive-pack.sh\nnew file mode 100755\nindex 0000000000..2f21f1cb8f\n--- /dev/null\n+++ b/t/t5410-receive-pack.sh\n@@ -0,0 +1,54 @@\n+#!/bin/sh\n+\n+test_description='git receive-pack test'\n+\n+. ./test-lib.sh\n+\n+test_expect_success 'setup' '\n+\ttest_commit one &&\n+\tgit update-ref refs/heads/a HEAD &&\n+\ttest_commit two &&\n+\tgit update-ref refs/heads/b HEAD &&\n+\ttest_commit three &&\n+\tgit update-ref refs/heads/c HEAD &&\n+\tgit clone --bare . fork &&\n+\tgit clone fork pusher &&\n+\t(\n+\t\tcd fork &&\n+\t\tgit config receive.advertisealternates true &&\n+\t\tcat <<-EOF | git update-ref --stdin &&\n+\t\tdelete refs/heads/a\n+\t\tdelete refs/heads/b\n+\t\tdelete refs/heads/c\n+\t\tdelete refs/heads/master\n+\t\tdelete refs/tags/one\n+\t\tdelete refs/tags/two\n+\t\tdelete refs/tags/three\n+\t\tEOF\n+\t\techo \"../../.git/objects\" >objects/info/alternates\n+\t)\n+'\n+\n+expect_haves () {\n+\tprintf \"%s .have\\n\" $(git rev-parse $@) >expect\n+}\n+\n+extract_haves () {\n+\tdepacketize - | grep '\\.have' | sed -e 's/\\\\0.*$//g'\n+}\n+\n+test_expect_success 'with core.alternateRefsCommand' '\n+\twrite_script fork/alternate-refs <<-\\EOF &&\n+\t\tgit --git-dir=\"$1\" for-each-ref \\\n+\t\t\t--format=\"%(objectname) %(refname)\" \\\n+\t\t\trefs/heads/a \\\n+\t\t\trefs/heads/c\n+\tEOF\n+\ttest_config -C fork core.alternateRefsCommand alternate-refs &&\n+\texpect_haves a c >expect &&\n+\tprintf \"0000\" | git receive-pack fork >actual &&\n+\textract_haves actual.haves &&\n+\ttest_cmp expect actual.haves\n+'\n+\n+test_done\ndiff --git a/transport.c b/transport.c\nindex 24ae3f375d..e7d2cdf00b 100644\n--- a/transport.c\n+++ b/transport.c\n@@ -1328,10 +1328,21 @@ char *transport_anonymize_url(const char *url)\n static void fill_alternate_refs_command(struct child_process *cmd,\n \t\t\t\t\tconst char *repo_path)\n {\n-\tcmd->git_cmd = 1;\n-\targv_array_pushf(&cmd->args, \"--git-dir=%s\", repo_path);\n-\targv_array_push(&cmd->args, \"for-each-ref\");\n-\targv_array_push(&cmd->args, \"--format=%(objectname) %(refname)\");\n+\tconst char *value;\n+\n+\tif (!git_config_get_value(\"core.alternateRefsCommand\", &value)) {\n+\t\tcmd->use_shell = 1;\n+\n+\t\targv_array_push(&cmd->args, value);\n+\t\targv_array_push(&cmd->args, repo_path);\n+\t} else {\n+\t\tcmd->git_cmd = 1;\n+\n+\t\targv_array_pushf(&cmd->args, \"--git-dir=%s\", repo_path);\n+\t\targv_array_push(&cmd->args, \"for-each-ref\");\n+\t\targv_array_push(&cmd->args, \"--format=%(objectname) %(refname)\");\n+\t}\n+\n \tcmd->env = local_repo_env;\n \tcmd->out = -1;\n }\n-- \n2.19.0.221.g150f307af\n\n"},{"id":"358639","messageId":"6e8f65a16dc0be84234d2be93bb4a5c9a585dd57.1537555544.git.me@ttaylorr.com","threadId":"49393","inReplyTo":"cover.1537555544.git.me@ttaylorr.com","subject":"[PATCH v2 3/3] transport.c: introduce core.alternateRefsPrefixes","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2018-09-21T18:47:45Z","receivedAt":"2018-09-21T18:47:51Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"The recently-introduced \"core.alternateRefsCommand\" allows callers to\nspecify with high flexibility the tips that they wish to advertise from\nalternates. This flexibility comes at the cost of some inconvenience\nwhen the caller only wishes to limit the advertisement to one or more\nprefixes.\n\nFor example, to advertise only tags, a caller using\n'core.alternateRefsCommand' would have to do:\n\n $ git config core.alternateRefsCommand ' \\\n git -C \"$1\" for-each-ref refs/tags \\\n --format=\"%(objectname) %(refname)\" \\\n '\n\nThe above is cumbersome to write, so let's introduce a\n\"core.alternateRefsPrefixes\" to address this common case. Instead, the\ncaller can run:\n\n $ git config core.alternateRefsPrefixes 'refs/tags'\n\nWhich will behave identically to the longer example using\n\"core.alternateRefsCommand\".\n\nSince the value of \"core.alternateRefsPrefixes\" is appended to 'git\nfor-each-ref' and then executed, include a \"--\" before taking the\nconfigured value to avoid misinterpreting arguments as flags to 'git\nfor-each-ref'.\n\nIn the case that the caller wishes to specify multiple prefixes, they\nmay separate them by whitespace. If \"core.alternateRefsCommand\" is set,\nit will take precedence over \"core.alternateRefsPrefixes\".\n\nSigned-off-by: Taylor Blau \n---\n Documentation/config.txt | 7 +++++++\n t/t5410-receive-pack.sh | 8 ++++++++\n transport.c | 5 +++++\n 3 files changed, 20 insertions(+)\n\ndiff --git a/Documentation/config.txt b/Documentation/config.txt\nindex 526557e494..7df6c22925 100644\n--- a/Documentation/config.txt\n+++ b/Documentation/config.txt\n@@ -627,6 +627,13 @@ alternate's references as \".have\"'s. For example, to only advertise branch\n heads, configure `core.alternateRefsCommand` to the path of a script which runs\n `git --git-dir=\"$1\" for-each-ref refs/heads`.\n \n+core.alternateRefsPrefixes::\n+\tWhen listing references from an alternate, list only references that begin\n+\twith the given prefix. Prefixes match as if they were given as arguments to\n+\tlinkgit:git-for-each-ref[1]. To list multiple prefixes, separate them with\n+\twhitespace. If `core.alternateRefsCommand` is set, setting\n+\t`core.alternateRefsPrefixes` has no effect.\n+\n core.bare::\n \tIf true this repository is assumed to be 'bare' and has no\n \tworking directory associated with it. If this is the case a\ndiff --git a/t/t5410-receive-pack.sh b/t/t5410-receive-pack.sh\nindex 2f21f1cb8f..b656c9b30c 100755\n--- a/t/t5410-receive-pack.sh\n+++ b/t/t5410-receive-pack.sh\n@@ -51,4 +51,12 @@ test_expect_success 'with core.alternateRefsCommand' '\n \ttest_cmp expect actual.haves\n '\n \n+test_expect_success 'with core.alternateRefsPrefixes' '\n+\ttest_config -C fork core.alternateRefsPrefixes \"refs/tags\" &&\n+\texpect_haves one three two >expect &&\n+\tprintf \"0000\" | git receive-pack fork >actual &&\n+\textract_haves actual.haves &&\n+\ttest_cmp expect actual.haves\n+'\n+\n test_done\ndiff --git a/transport.c b/transport.c\nindex e7d2cdf00b..9323e5c3cd 100644\n--- a/transport.c\n+++ b/transport.c\n@@ -1341,6 +1341,11 @@ static void fill_alternate_refs_command(struct child_process *cmd,\n \t\targv_array_pushf(&cmd->args, \"--git-dir=%s\", repo_path);\n \t\targv_array_push(&cmd->args, \"for-each-ref\");\n \t\targv_array_push(&cmd->args, \"--format=%(objectname) %(refname)\");\n+\n+\t\tif (!git_config_get_value(\"core.alternateRefsPrefixes\", &value)) {\n+\t\t\targv_array_push(&cmd->args, \"--\");\n+\t\t\targv_array_split(&cmd->args, value);\n+\t\t}\n \t}\n \n \tcmd->env = local_repo_env;\n-- \n2.19.0.221.g150f307af\n"},{"id":"358642","messageId":"xmqqr2hmvcyz.fsf@gitster-ct.c.googlers.com","threadId":"49393","inReplyTo":"20180921175739.GA21514@syl","subject":"Re: [PATCH 2/3] transport.c: introduce core.alternateRefsCommand","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2018-09-21T19:59:16Z","receivedAt":"2018-09-21T19:59:22Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Taylor Blau writes:\n\n> In fact, I think that we can go even further: since we don't need to\n> catch the beginning '^.*' (without -o), we can instead:\n>\n> extract_haves () {\n> depacketize - | grep '\\.have' | sed -e 's/\\\\0.*$//g'\n> }\n\nDo not pipe grep into sed, unless you have an overly elaborate set\nof patterns to filter with, e.g. something along the lines of...\n\n\tsed -ne '/\\.have/s/...//p'\n\n"},{"id":"358644","messageId":"CAPig+cSx0vxegy+ENtiYgV=x-A=0upFZGuFbW-=cwwknko_f4A@mail.gmail.com","threadId":"49393","inReplyTo":"9797f525517142b3494cfbd17a10dfeb3bf586e2.1537555544.git.me@ttaylorr.com","subject":"Re: [PATCH v2 2/3] transport.c: introduce core.alternateRefsCommand","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2018-09-21T20:18:03Z","receivedAt":"2018-09-21T20:18:18Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Fri, Sep 21, 2018 at 2:47 PM Taylor Blau wrote:\n> When in a repository containing one or more alternates, Git would\n> sometimes like to list references from its alternates. For example, 'git\n> receive-pack' list the objects pointed to by alternate references as\n> special \".have\" references.\n> [...]\n> Signed-off-by: Taylor Blau \n> ---\n> diff --git a/t/t5410-receive-pack.sh b/t/t5410-receive-pack.sh\n> @@ -0,0 +1,54 @@\n> +expect_haves () {\n> + printf \"%s .have\\n\" $(git rev-parse $@) >expect\n> +}\n\nMagic quoting behavior only kicks in when $@ is itself quoted, so this\nshould be:\n\n printf \"%s .have\\n\" $(git rev-parse \"$@\") >expect\n\nHowever, as it's unlikely that you need magic quoting in this case,\nyou might get by with plain $* (unquoted).\n"},{"id":"358656","messageId":"xmqqfty2v9qj.fsf@gitster-ct.c.googlers.com","threadId":"49393","inReplyTo":"9797f525517142b3494cfbd17a10dfeb3bf586e2.1537555544.git.me@ttaylorr.com","subject":"Re: [PATCH v2 2/3] transport.c: introduce core.alternateRefsCommand","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2018-09-21T21:09:08Z","receivedAt":"2018-09-21T21:09:13Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Taylor Blau writes:\n\n> +core.alternateRefsCommand::\n> +\tWhen listing references from an alternate (e.g., in the case of \".have\"), use\n\nIt is not clear how (e.g.,...) connects to what is said in the\nsentence. \"When advertising tips of available history from an\nalternate, use ...\" without saying \".have\" may be less cryptic. \n\nI dunno.\n\n> +\tthe shell to execute the specified command instead of\n> +\tlinkgit:git-for-each-ref[1]. The first argument is the path of the alternate.\n\n\"The path\" meaning the absolute path? Relative to the original\nobject store? Something else?\n\n> +\tOutput must be of the form: `%(objectname) SPC %(refname)`.\n> ++\n> +This is useful when a repository only wishes to advertise some of its\n> +alternate's references as \".have\"'s. For example, to only advertise branch\n> +heads, configure `core.alternateRefsCommand` to the path of a script which runs\n> +`git --git-dir=\"$1\" for-each-ref refs/heads`.\n> +\n> core.bare::\n> \tIf true this repository is assumed to be 'bare' and has no\n> \tworking directory associated with it. If this is the case a\n> diff --git a/t/t5410-receive-pack.sh b/t/t5410-receive-pack.sh\n> new file mode 100755\n> index 0000000000..2f21f1cb8f\n> --- /dev/null\n> +++ b/t/t5410-receive-pack.sh\n> @@ -0,0 +1,54 @@\n> +#!/bin/sh\n> +\n> +test_description='git receive-pack test'\n> +\n> +. ./test-lib.sh\n> +\n> +test_expect_success 'setup' '\n> +\ttest_commit one &&\n> +\tgit update-ref refs/heads/a HEAD &&\n> +\ttest_commit two &&\n> +\tgit update-ref refs/heads/b HEAD &&\n> +\ttest_commit three &&\n> +\tgit update-ref refs/heads/c HEAD &&\n> +\tgit clone --bare . fork &&\n> +\tgit clone fork pusher &&\n> +\t(\n> +\t\tcd fork &&\n> +\t\tgit config receive.advertisealternates true &&\n\nHmph. Do we have code to support this configuration variable?\n\n> +\t\tcat <<-EOF | git update-ref --stdin &&\n\nStyle: writing \"<<-\\EOF\" instead would allow readers' eyes to\ncoast over without having to look for $variable_references in\nthe here-doc.\n\n> +\t\tdelete refs/heads/a\n> +\t\tdelete refs/heads/b\n> +\t\tdelete refs/heads/c\n> +\t\tdelete refs/heads/master\n> +\t\tdelete refs/tags/one\n> +\t\tdelete refs/tags/two\n> +\t\tdelete refs/tags/three\n\nSo, the original created one/two/three/a/b/c/master, fork is a bare\nclone of it and has all these things, and then you deleted all of\nthese? What does fork have after this is done? HEAD that is\ndangling?\n\n> +\t\tEOF\n> +\t\techo \"../../.git/objects\" >objects/info/alternates\n\nWhen viewed from fork/objects, ../../.git is the GIT_DIR of the\nprimary test repository, so that is where we borrow objects from.\n\nIf we pruned the objects from fork's object store before this echo,\nwe would have an almost empty repository that borrows from its\nalternates everything, which may make a more realistic sample case,\nbut because you are only focusing on the ref advertisement, it does\nnot matter that your fork is full of duplicate objects that are\navailable from the alternates.\n\n> +expect_haves () {\n> +\tprintf \"%s .have\\n\" $(git rev-parse $@) >expect\n\nQuote $@ inside dq pair, like $(git rev-parse \"$@\").\n\n> +extract_haves () {\n> +\tdepacketize - | grep '\\.have' | sed -e 's/\\\\0.*$//g'\n> +}\n\n\nDon't pipe grep into sed, especially when both the pattern to filter\nand the operation to perform are simple.\n\nI am not sure what you are trying to achive with 'g' in\ns/pattern$//g; The anchor at the rightmost end of the pattern makes\nsure that the pattern matches only once per line at the end anyway,\nso \"do this howmanyever times as we have match on each line\" would\nnot make any difference, no?\n\n> diff --git a/transport.c b/transport.c\n> index 24ae3f375d..e7d2cdf00b 100644\n> --- a/transport.c\n> +++ b/transport.c\n> @@ -1328,10 +1328,21 @@ char *transport_anonymize_url(const char *url)\n> static void fill_alternate_refs_command(struct child_process *cmd,\n> \t\t\t\t\tconst char *repo_path)\n> {\n> -\tcmd->git_cmd = 1;\n> -\targv_array_pushf(&cmd->args, \"--git-dir=%s\", repo_path);\n> -\targv_array_push(&cmd->args, \"for-each-ref\");\n> -\targv_array_push(&cmd->args, \"--format=%(objectname) %(refname)\");\n> +\tconst char *value;\n> +\n> +\tif (!git_config_get_value(\"core.alternateRefsCommand\", &value)) {\n> +\t\tcmd->use_shell = 1;\n> +\n> +\t\targv_array_push(&cmd->args, value);\n> +\t\targv_array_push(&cmd->args, repo_path);\n> +\t} else {\n> +\t\tcmd->git_cmd = 1;\n> +\n> +\t\targv_array_pushf(&cmd->args, \"--git-dir=%s\", repo_path);\n> +\t\targv_array_push(&cmd->args, \"for-each-ref\");\n> +\t\targv_array_push(&cmd->args, \"--format=%(objectname) %(refname)\");\n> +\t}\n> +\n> \tcmd->env = local_repo_env;\n> \tcmd->out = -1;\n> }\n"},{"id":"358657","messageId":"CAPig+cRzY1LXuz3rYnPa5TqGv=Yur7o=h_6H4KNb6=fCV_ofFw@mail.gmail.com","threadId":"49393","inReplyTo":"9797f525517142b3494cfbd17a10dfeb3bf586e2.1537555544.git.me@ttaylorr.com","subject":"Re: [PATCH v2 2/3] transport.c: introduce core.alternateRefsCommand","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2018-09-21T21:10:06Z","receivedAt":"2018-09-21T21:10:20Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Fri, Sep 21, 2018 at 2:47 PM Taylor Blau wrote:\n> When in a repository containing one or more alternates, Git would\n> sometimes like to list references from its alternates. For example, 'git\n> receive-pack' list the objects pointed to by alternate references as\n> special \".have\" references.\n> [...]\n> Signed-off-by: Taylor Blau \n> ---\n> diff --git a/t/t5410-receive-pack.sh b/t/t5410-receive-pack.sh\n> @@ -0,0 +1,54 @@\n> +expect_haves () {\n> + printf \"%s .have\\n\" $(git rev-parse $@) >expect\n> +}\n> +\n> +test_expect_success 'with core.alternateRefsCommand' '\n> + [...]\n> + expect_haves a c >expect &&\n\nThis is not great. Both the caller of expect_haves() and\nexpect_haves() itself redirect to a file named \"expect\". This works,\nbut only by accident.\n\nBetter would be to make expect_haves() simply a generator to stdout\nand let the caller redirect to the file rather than hardcoding the\nfilename in the function itself (much as extract_haves() takes it its\ninput on stdin rather than hardcoding a filename). If you take this\napproach, then you'd probably want to rename the function, as well;\nperhaps call it emit_haves() or something.\n\n> + printf \"0000\" | git receive-pack fork >actual &&\n> + extract_haves actual.haves &&\n> + test_cmp expect actual.haves\n> +'\n"},{"id":"358658","messageId":"xmqqbm8qv9hy.fsf@gitster-ct.c.googlers.com","threadId":"49393","inReplyTo":"6e8f65a16dc0be84234d2be93bb4a5c9a585dd57.1537555544.git.me@ttaylorr.com","subject":"Re: [PATCH v2 3/3] transport.c: introduce core.alternateRefsPrefixes","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2018-09-21T21:14:17Z","receivedAt":"2018-09-21T21:14:21Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Taylor Blau writes:\n\n> +core.alternateRefsPrefixes::\n> +\tWhen listing references from an alternate, list only references that begin\n> +\twith the given prefix. Prefixes match as if they were given as arguments to\n> +\tlinkgit:git-for-each-ref[1]. To list multiple prefixes, separate them with\n> +\twhitespace. If `core.alternateRefsCommand` is set, setting\n> +\t`core.alternateRefsPrefixes` has no effect.\n\nWe do not allow anything elaborate like \"refs/tags/release-*\" but we\nstill allow \"refs/tags/\" and \"refs/heads/\" by listing them together,\nand because these are only prefixes, whitespace is a reasonable list\nseparator as they cannot appear anywhere in a refname. OK.\n\nWhy is this \"core\"? I thought this was more about receive-pack;\neven if this is going to be extended to upload-pack's negotiation,\n\"core\" is way too wide a hierarchy. We have \"transport.*\" for\nthings like this, no?\n\nThe exact same comment applies to 2/3, of course.\n\n\n> diff --git a/t/t5410-receive-pack.sh b/t/t5410-receive-pack.sh\n> index 2f21f1cb8f..b656c9b30c 100755\n> --- a/t/t5410-receive-pack.sh\n> +++ b/t/t5410-receive-pack.sh\n> @@ -51,4 +51,12 @@ test_expect_success 'with core.alternateRefsCommand' '\n> \ttest_cmp expect actual.haves\n> '\n> \n> +test_expect_success 'with core.alternateRefsPrefixes' '\n> +\ttest_config -C fork core.alternateRefsPrefixes \"refs/tags\" &&\n> +\texpect_haves one three two >expect &&\n> +\tprintf \"0000\" | git receive-pack fork >actual &&\n> +\textract_haves actual.haves &&\n> +\ttest_cmp expect actual.haves\n> +'\n> +\n> test_done\n> diff --git a/transport.c b/transport.c\n> index e7d2cdf00b..9323e5c3cd 100644\n> --- a/transport.c\n> +++ b/transport.c\n> @@ -1341,6 +1341,11 @@ static void fill_alternate_refs_command(struct child_process *cmd,\n> \t\targv_array_pushf(&cmd->args, \"--git-dir=%s\", repo_path);\n> \t\targv_array_push(&cmd->args, \"for-each-ref\");\n> \t\targv_array_push(&cmd->args, \"--format=%(objectname) %(refname)\");\n> +\n> +\t\tif (!git_config_get_value(\"core.alternateRefsPrefixes\", &value)) {\n> +\t\t\targv_array_push(&cmd->args, \"--\");\n> +\t\t\targv_array_split(&cmd->args, value);\n> +\t\t}\n> \t}\n> \n> \tcmd->env = local_repo_env;\n"},{"id":"358660","messageId":"20180921213753.GA11177@sigill.intra.peff.net","threadId":"49393","inReplyTo":"xmqqbm8qv9hy.fsf@gitster-ct.c.googlers.com","subject":"Re: [PATCH v2 3/3] transport.c: introduce core.alternateRefsPrefixes","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2018-09-21T21:37:53Z","receivedAt":"2018-09-21T21:37:57Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Fri, Sep 21, 2018 at 02:14:17PM -0700, Junio C Hamano wrote:\n\n> Taylor Blau writes:\n> \n> > +core.alternateRefsPrefixes::\n> > +\tWhen listing references from an alternate, list only references that begin\n> > +\twith the given prefix. Prefixes match as if they were given as arguments to\n> > +\tlinkgit:git-for-each-ref[1]. To list multiple prefixes, separate them with\n> > +\twhitespace. If `core.alternateRefsCommand` is set, setting\n> > +\t`core.alternateRefsPrefixes` has no effect.\n> \n> We do not allow anything elaborate like \"refs/tags/release-*\" but we\n> still allow \"refs/tags/\" and \"refs/heads/\" by listing them together,\n> and because these are only prefixes, whitespace is a reasonable list\n> separator as they cannot appear anywhere in a refname. OK.\n> \n> Why is this \"core\"? I thought this was more about receive-pack;\n> even if this is going to be extended to upload-pack's negotiation,\n> \"core\" is way too wide a hierarchy. We have \"transport.*\" for\n> things like this, no?\n\nThere's no extension necessary; these should already affect upload-pack\nas well. I agree transport.* would cover both upload-pack and\nreceive-pack. If we extend it to check_everything_connected(), would it\nmake sense as part of transport.*, too?\n\nI dunno. I guess I could see an argument either way.\n\nIf we do add \"rev-list --alternate-refs\", that pushes it even further\naway from transport.*, though.\n\n-Peff\n"},{"id":"358663","messageId":"xmqqtvmitsi4.fsf@gitster-ct.c.googlers.com","threadId":"49393","inReplyTo":"20180921213753.GA11177@sigill.intra.peff.net","subject":"Re: [PATCH v2 3/3] transport.c: introduce core.alternateRefsPrefixes","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2018-09-21T22:06:43Z","receivedAt":"2018-09-21T22:06:48Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jeff King writes:\n\n> There's no extension necessary; these should already affect upload-pack\n> as well. I agree transport.* would cover both upload-pack and\n> receive-pack. If we extend it to check_everything_connected(), would it\n> make sense as part of transport.*, too?\n>\n> I dunno. I guess I could see an argument either way.\n\nSorry but I do not quite follow. Are you saying that something that\ncovers check-everything-connected would the result be too wide to\nfit inside transport.*? or something that does not cover\ncheck-everything-connected falls short of transport.*? Or something\nelse? Either way, core.* is way too wide for what this hook does, I\nwould think.\n"},{"id":"358665","messageId":"20180921221317.GB11177@sigill.intra.peff.net","threadId":"49393","inReplyTo":"xmqqfty2v9qj.fsf@gitster-ct.c.googlers.com","subject":"Re: [PATCH v2 2/3] transport.c: introduce core.alternateRefsCommand","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2018-09-21T22:13:18Z","receivedAt":"2018-09-21T22:13:21Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Fri, Sep 21, 2018 at 02:09:08PM -0700, Junio C Hamano wrote:\n\n> > +test_expect_success 'setup' '\n> > +\ttest_commit one &&\n> > +\tgit update-ref refs/heads/a HEAD &&\n> > +\ttest_commit two &&\n> > +\tgit update-ref refs/heads/b HEAD &&\n> > +\ttest_commit three &&\n> > +\tgit update-ref refs/heads/c HEAD &&\n> > +\tgit clone --bare . fork &&\n> > +\tgit clone fork pusher &&\n> > +\t(\n> > +\t\tcd fork &&\n> > +\t\tgit config receive.advertisealternates true &&\n> \n> Hmph. Do we have code to support this configuration variable?\n\nSorry, I should have caught that. Our existing solution is to disable\nalternates in the advertisement entirely (since the optimization\nbackfires for us). So this line is a leftover from testing it against\nour fork, and should be dropped.\n\nIf anybody is interested, we can share those patches, though they're\nunsurprisingly trivial. I suspect we may end up discarding them if this\ncustom-command thing works, but it's possible we'll still need to be\nable to shut them off completely for some truly pathological cases.\n\n> > +\t\tcat <<-EOF | git update-ref --stdin &&\n> \n> Style: writing \"<<-\\EOF\" instead would allow readers' eyes to\n> coast over without having to look for $variable_references in\n> the here-doc.\n\nAlso, useless-use-of-cat in the original, which could be:\n\n git update-ref --stdin <<-\\EOF\n\n> [...]\n\nYeah, I second all the other bits you mentioned.\n\n-Peff\n"},{"id":"358666","messageId":"20180921221832.GC11177@sigill.intra.peff.net","threadId":"49393","inReplyTo":"xmqqtvmitsi4.fsf@gitster-ct.c.googlers.com","subject":"Re: [PATCH v2 3/3] transport.c: introduce core.alternateRefsPrefixes","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2018-09-21T22:18:32Z","receivedAt":"2018-09-21T22:18:36Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Fri, Sep 21, 2018 at 03:06:43PM -0700, Junio C Hamano wrote:\n\n> Jeff King writes:\n> \n> > There's no extension necessary; these should already affect upload-pack\n> > as well. I agree transport.* would cover both upload-pack and\n> > receive-pack. If we extend it to check_everything_connected(), would it\n> > make sense as part of transport.*, too?\n> >\n> > I dunno. I guess I could see an argument either way.\n> \n> Sorry but I do not quite follow. Are you saying that something that\n> covers check-everything-connected would the result be too wide to\n> fit inside transport.*? or something that does not cover\n> check-everything-connected falls short of transport.*? Or something\n> else? Either way, core.* is way too wide for what this hook does, I\n> would think.\n\nI was suggesting that check_everything_connected() is not strictly\ntransport-related, so would be inappropriate for transport.*, and we'd\nneed a more generic name. And my \"either way\" was that I could see\nan argument that it _is_ transport related, since we only call it now\nwhen receiving a pack. But that doesn't have to be the case, and\ncertainly implementing it with \"rev-list --alternate-refs\" muddies that\nconsiderably.\n\nI agree that core.* is kind of a kitchen sink, but I'm not sure that's\nall that bad. Is \"here is how Git finds refs in an alternate\" any more\nor less core than \"here is how Git invokes ssh\"?\n\n-Peff\n"},{"id":"358668","messageId":"CAGZ79kaT=zvwLxJys76DOHc5HABpvu2fwqaq7dZ=j7PiCgaTYQ@mail.gmail.com","threadId":"49393","inReplyTo":"20180921221832.GC11177@sigill.intra.peff.net","subject":"Re: [PATCH v2 3/3] transport.c: introduce core.alternateRefsPrefixes","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2018-09-21T22:23:06Z","receivedAt":"2018-09-21T22:23:20Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Fri, Sep 21, 2018 at 3:18 PM Jeff King wrote:\n\n> I agree that core.* is kind of a kitchen sink, but I'm not sure that's\n> all that bad. Is \"here is how Git finds refs in an alternate\" any more\n\nThis touches both \"refs\" and \"alternates\", which are Git concepts\nwhereas ssh is not.\n\n> or less core than \"here is how Git invokes ssh\"?\n\nArguably core.sshCommand should be deprecated and re-introduced\nas transport.\"ssh\".command. :-P\n"},{"id":"358669","messageId":"xmqqpnx6trpv.fsf@gitster-ct.c.googlers.com","threadId":"49393","inReplyTo":"20180921221317.GB11177@sigill.intra.peff.net","subject":"Re: [PATCH v2 2/3] transport.c: introduce core.alternateRefsCommand","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2018-09-21T22:23:40Z","receivedAt":"2018-09-21T22:23:44Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jeff King writes:\n\n> On Fri, Sep 21, 2018 at 02:09:08PM -0700, Junio C Hamano wrote:\n>\n>> > +test_expect_success 'setup' '\n>> > +\ttest_commit one &&\n>> > +\tgit update-ref refs/heads/a HEAD &&\n>> > +\ttest_commit two &&\n>> > +\tgit update-ref refs/heads/b HEAD &&\n>> > +\ttest_commit three &&\n>> > +\tgit update-ref refs/heads/c HEAD &&\n>> > +\tgit clone --bare . fork &&\n>> > +\tgit clone fork pusher &&\n>> > +\t(\n>> > +\t\tcd fork &&\n>> > +\t\tgit config receive.advertisealternates true &&\n>> \n>> Hmph. Do we have code to support this configuration variable?\n>\n> Sorry, I should have caught that. Our existing solution is to disable\n> alternates in the advertisement entirely (since the optimization\n> backfires for us). So this line is a leftover from testing it against\n> our fork, and should be dropped.\n>\n> If anybody is interested, we can share those patches, though they're\n> unsurprisingly trivial.\n\nHeh, I guessed correctly what is going on ;-)\n\nEven though there may not be much interest in the \"all-or-none\"\nboolean configuration, in order to upstream this custom thing, it\nmay be the cleanest to upstream that all-or-none thing as well.\nOtherwise, you'd need to keep a patch to this test script that is\nprivate for your \"all-or-none\" feature. That's your maintenance\nburden so it ultimately is your call ;-)\n> Also, useless-use-of-cat in the original, which could be:\n>\n> git update-ref --stdin <<-\\EOF\n\nYup.\n\nThanks.\n"},{"id":"358671","messageId":"20180921222728.GE11177@sigill.intra.peff.net","threadId":"49393","inReplyTo":"xmqqpnx6trpv.fsf@gitster-ct.c.googlers.com","subject":"Re: [PATCH v2 2/3] transport.c: introduce core.alternateRefsCommand","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2018-09-21T22:27:28Z","receivedAt":"2018-09-21T22:27:32Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Fri, Sep 21, 2018 at 03:23:40PM -0700, Junio C Hamano wrote:\n\n> >> > +\t\tgit config receive.advertisealternates true &&\n> >> \n> >> Hmph. Do we have code to support this configuration variable?\n> >\n> > Sorry, I should have caught that. Our existing solution is to disable\n> > alternates in the advertisement entirely (since the optimization\n> > backfires for us). So this line is a leftover from testing it against\n> > our fork, and should be dropped.\n> >\n> > If anybody is interested, we can share those patches, though they're\n> > unsurprisingly trivial.\n> \n> Heh, I guessed correctly what is going on ;-)\n> \n> Even though there may not be much interest in the \"all-or-none\"\n> boolean configuration, in order to upstream this custom thing, it\n> may be the cleanest to upstream that all-or-none thing as well.\n> Otherwise, you'd need to keep a patch to this test script that is\n> private for your \"all-or-none\" feature. That's your maintenance\n> burden so it ultimately is your call ;-)\n\nEasy one-liners in test scripts are the least of my ongoing maintenance\nburden. ;)\n\nI think in this case, though, the line is not even necessary, as our\npatches leave the default as \"true\" (which is certainly what we would\nwant upstream, as well, for compatibility).\n\n-Peff\n"},{"id":"358701","messageId":"20180922180231.GD432229@genre.crustytoothpaste.net","threadId":"49393","inReplyTo":"9797f525517142b3494cfbd17a10dfeb3bf586e2.1537555544.git.me@ttaylorr.com","subject":"Re: [PATCH v2 2/3] transport.c: introduce core.alternateRefsCommand","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2018-09-22T18:02:31Z","receivedAt":"2018-09-22T18:02:40Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On Fri, Sep 21, 2018 at 02:47:43PM -0400, Taylor Blau wrote:\n> +expect_haves () {\n> +\tprintf \"%s .have\\n\" $(git rev-parse $@) >expect\n> +}\n> +\n> +extract_haves () {\n> +\tdepacketize - | grep '\\.have' | sed -e 's/\\\\0.*$//g'\n\nIt looks like you're trying to match a NUL here in the sed expression,\nbut from my reading of it, POSIX doesn't permit BREs to match NUL.\n\nPerhaps someone can come up with a better solution, but I'd write this\nas the following:\n\n depacketize - | perl -ne 'next unless /\\.have/; s/\\0.*$//g; print'\n-- \nbrian m. carlson: Houston, Texas, US\nOpenPGP: https://keybase.io/bk2204\n"},{"id":"358713","messageId":"20180922195258.GA20983@sigill.intra.peff.net","threadId":"49393","inReplyTo":"20180922180231.GD432229@genre.crustytoothpaste.net","subject":"Re: [PATCH v2 2/3] transport.c: introduce core.alternateRefsCommand","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2018-09-22T19:52:58Z","receivedAt":"2018-09-22T19:53:03Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Sat, Sep 22, 2018 at 06:02:31PM +0000, brian m. carlson wrote:\n\n> On Fri, Sep 21, 2018 at 02:47:43PM -0400, Taylor Blau wrote:\n> > +expect_haves () {\n> > +\tprintf \"%s .have\\n\" $(git rev-parse $@) >expect\n> > +}\n> > +\n> > +extract_haves () {\n> > +\tdepacketize - | grep '\\.have' | sed -e 's/\\\\0.*$//g'\n> \n> It looks like you're trying to match a NUL here in the sed expression,\n> but from my reading of it, POSIX doesn't permit BREs to match NUL.\n\nNo, it's trying to literally match backslash followed by 0. The\ndepacketize() script will have undone the NUL already. In perl, no less,\nmaking it more or less equivalent to your suggestion. ;)\n\nSo I think this is fine (modulo that the grep and sed can be combined).\nYet another option would be to simply strip away everything except the\nobject id (which is all we care about), like:\n\n depacketize | perl -lne '/^(\\S+) \\.have/ and print $1'\n\nOr the equivalent in sed. I am happy with any solution that does the\ncorrect thing.\n\n-Peff\n"},{"id":"358723","messageId":"20180923145350.GE432229@genre.crustytoothpaste.net","threadId":"49393","inReplyTo":"20180922195258.GA20983@sigill.intra.peff.net","subject":"Re: [PATCH v2 2/3] transport.c: introduce core.alternateRefsCommand","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2018-09-23T14:53:51Z","receivedAt":"2018-09-23T14:53:59Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On Sat, Sep 22, 2018 at 03:52:58PM -0400, Jeff King wrote:\n> On Sat, Sep 22, 2018 at 06:02:31PM +0000, brian m. carlson wrote:\n> \n> > On Fri, Sep 21, 2018 at 02:47:43PM -0400, Taylor Blau wrote:\n> > > +expect_haves () {\n> > > +\tprintf \"%s .have\\n\" $(git rev-parse $@) >expect\n> > > +}\n> > > +\n> > > +extract_haves () {\n> > > +\tdepacketize - | grep '\\.have' | sed -e 's/\\\\0.*$//g'\n> > \n> > It looks like you're trying to match a NUL here in the sed expression,\n> > but from my reading of it, POSIX doesn't permit BREs to match NUL.\n> \n> No, it's trying to literally match backslash followed by 0. The\n> depacketize() script will have undone the NUL already. In perl, no less,\n> making it more or less equivalent to your suggestion. ;)\n\nAh, okay. That makes more sense.\n\n> So I think this is fine (modulo that the grep and sed can be combined).\n> Yet another option would be to simply strip away everything except the\n> object id (which is all we care about), like:\n> \n> depacketize | perl -lne '/^(\\S+) \\.have/ and print $1'\n> \n> Or the equivalent in sed. I am happy with any solution that does the\n> correct thing.\n\nYeah, I agree that with that context, no change is needed.\n-- \nbrian m. carlson: Houston, Texas, US\nOpenPGP: https://keybase.io/bk2204\n"},{"id":"358757","messageId":"xmqqftxzsz5x.fsf@gitster-ct.c.googlers.com","threadId":"49393","inReplyTo":"20180921221832.GC11177@sigill.intra.peff.net","subject":"Re: [PATCH v2 3/3] transport.c: introduce core.alternateRefsPrefixes","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2018-09-24T15:17:14Z","receivedAt":"2018-09-24T15:17:22Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jeff King writes:\n\n> I was suggesting that check_everything_connected() is not strictly\n> transport-related, so would be inappropriate for transport.*, and we'd\n> need a more generic name. And my \"either way\" was that I could see\n> an argument that it _is_ transport related, since we only call it now\n> when receiving a pack. But that doesn't have to be the case, and\n> certainly implementing it with \"rev-list --alternate-refs\" muddies that\n> considerably.\n\nEven after 7043c707 (\"check_everything_connected: use a struct with\nnamed options\", 2016-07-15) unified many into check_connected(),\nthere still are different reasons why we call to find out about the\nconnectivity, and I doubt we can afford to have a single knob that\nis shared both for transport and other kind of connectivity checks\n(like fsck or repack). Do we want to be affected by \"we pretend\nthat these are the only refs exported from that alternate object\nstore\" when repacking and pruning only local objects and keep us\nrely on the alternate, for example?\n\nIn any case it is good that these configuration variables are\ndefined on _our_ side, not in the alternate---it means that we do\nnot have to worry about the case where the alternateRefsCommand lies\nand tells us that an object that the alternate does not actually\nhave exists at a tip of a ref in an attempt to confuse us, etc.\n"},{"id":"358769","messageId":"20180924181011.GA24781@sigill.intra.peff.net","threadId":"49393","inReplyTo":"xmqqftxzsz5x.fsf@gitster-ct.c.googlers.com","subject":"Re: [PATCH v2 3/3] transport.c: introduce core.alternateRefsPrefixes","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2018-09-24T18:10:12Z","receivedAt":"2018-09-24T18:10:15Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Mon, Sep 24, 2018 at 08:17:14AM -0700, Junio C Hamano wrote:\n\n> Jeff King writes:\n> \n> > I was suggesting that check_everything_connected() is not strictly\n> > transport-related, so would be inappropriate for transport.*, and we'd\n> > need a more generic name. And my \"either way\" was that I could see\n> > an argument that it _is_ transport related, since we only call it now\n> > when receiving a pack. But that doesn't have to be the case, and\n> > certainly implementing it with \"rev-list --alternate-refs\" muddies that\n> > considerably.\n> \n> Even after 7043c707 (\"check_everything_connected: use a struct with\n> named options\", 2016-07-15) unified many into check_connected(),\n> there still are different reasons why we call to find out about the\n> connectivity, and I doubt we can afford to have a single knob that\n> is shared both for transport and other kind of connectivity checks\n> (like fsck or repack). Do we want to be affected by \"we pretend\n> that these are the only refs exported from that alternate object\n> store\" when repacking and pruning only local objects and keep us\n> rely on the alternate, for example?\n\nActually, yes, I think there is value in a single knob. At least that's\nwhat I'd want for our (GitHub's) use case.\n\nRemember that these alternate refs might not exist at all (the\nalternates mechanism can work with just a bare \"objects\" directory,\nunconnected from a real git repo). So I think anything using them has to\nview it as a \"best effort\" optimization: we might or might not know\nabout some ref tips that might or might not cover the whole set of\nobjects in the alternate. They're the things we _guarantee_ that the\nalternate has full connectivity for, and it might have more.\n\nSo I think it's conceptually consistent to always show a subset. I did\nqualify with \"for our use case\" because some people might be primarily\nconcerned with the bandwidth of sending .haves across the network.\nWhereas at our scale, even enumerating them at all is prohibitively\nexpensive.\n\nOne thing we could do is add a \"core\" config now (whether it's in core.*\nor wherever). And then if later somebody wants receive-pack to behave\ndifferently, we have an out: we can add transfer.alternateRefsCommand or\neven receive.alternateRefsCommand that take precedence in those\nsituations.\n\nOf course we could add the more restricted ones now, and add the \"core\"\none later as new uses grow. But that's more work now, since we'd have to\nplumb through that context to the for_each_alternate_ref() interface.\nI'd rather punt on that work until later (because I suspect that \"later\"\nwill never actually come).\n\n> In any case it is good that these configuration variables are\n> defined on _our_ side, not in the alternate---it means that we do\n> not have to worry about the case where the alternateRefsCommand lies\n> and tells us that an object that the alternate does not actually\n> have exists at a tip of a ref in an attempt to confuse us, etc.\n\nYes. It also makes it easy to use \"git -c\" to override the scheme if you\nwant to (as opposed to mucking with on-disk files in the alternate).\n\n-Peff\n"},{"id":"358779","messageId":"xmqqa7o6skkl.fsf@gitster-ct.c.googlers.com","threadId":"49393","inReplyTo":"20180924181011.GA24781@sigill.intra.peff.net","subject":"Re: [PATCH v2 3/3] transport.c: introduce core.alternateRefsPrefixes","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2018-09-24T20:32:26Z","receivedAt":"2018-09-24T20:32:30Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jeff King writes:\n\n> So I think it's conceptually consistent to always show a subset.\n\nOK. Then I agree with you that it is a good approach to first adopt\ncore.* knobs that universally apply, and add specialized ones as\nthey are needed later.\n\nThanks.\n"},{"id":"358780","messageId":"20180924205022.GA26936@sigill.intra.peff.net","threadId":"49393","inReplyTo":"xmqqa7o6skkl.fsf@gitster-ct.c.googlers.com","subject":"Re: [PATCH v2 3/3] transport.c: introduce core.alternateRefsPrefixes","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2018-09-24T20:50:22Z","receivedAt":"2018-09-24T20:50:26Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Mon, Sep 24, 2018 at 01:32:26PM -0700, Junio C Hamano wrote:\n\n> Jeff King writes:\n> \n> > So I think it's conceptually consistent to always show a subset.\n> \n> OK. Then I agree with you that it is a good approach to first adopt\n> core.* knobs that universally apply, and add specialized ones as\n> they are needed later.\n\nThanks. There's one other major decision for this series, I think.\n\nDo you have an opinion on whether for_each_alternate_refs() interface\nshould stop passing back refnames? By the \"they may not even exist\"\nrationale in this sub-thread, I think it's probably foolish for any\ncaller to actually depend on the names being meaningful.\n\nWe need to decide now because the idea of which data is relevant is\ngetting baked into the documented alternateRefsCmd output format.\n\n-Peff\n"},{"id":"358786","messageId":"20180924210127.GA6255@sigill.intra.peff.net","threadId":"49393","inReplyTo":"20180924205022.GA26936@sigill.intra.peff.net","subject":"Re: [PATCH v2 3/3] transport.c: introduce core.alternateRefsPrefixes","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2018-09-24T21:01:27Z","receivedAt":"2018-09-24T21:01:31Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Mon, Sep 24, 2018 at 04:50:22PM -0400, Jeff King wrote:\n\n> On Mon, Sep 24, 2018 at 01:32:26PM -0700, Junio C Hamano wrote:\n> \n> > Jeff King writes:\n> > \n> > > So I think it's conceptually consistent to always show a subset.\n> > \n> > OK. Then I agree with you that it is a good approach to first adopt\n> > core.* knobs that universally apply, and add specialized ones as\n> > they are needed later.\n> \n> Thanks. There's one other major decision for this series, I think.\n> \n> Do you have an opinion on whether for_each_alternate_refs() interface\n> should stop passing back refnames? By the \"they may not even exist\"\n> rationale in this sub-thread, I think it's probably foolish for any\n> caller to actually depend on the names being meaningful.\n> \n> We need to decide now because the idea of which data is relevant is\n> getting baked into the documented alternateRefsCmd output format.\n\nJust to sketch it out further, I was thinking that we'd do something\nlike this at the front of Taylor's series (with the rest rebased as\nappropriate on top).\n\n-- >8 --\nSubject: [PATCH] transport: drop refnames from for_each_alternate_ref\n\nNone of the current callers use the refname parameter we pass to their\ncallbacks. In theory somebody _could_ do so, but it's actually quite\nweird if you think about it: it's a ref in somebody else's repository.\nSo the name has no meaning locally, and in fact there may be duplicates\nif there are multiple alternates.\n\nThe users of this interface really only care about seeing some ref tips,\nsince that promises that the alternate has the full commit graph\nreachable from there. So let's keep the information we pass back to the\nbare minimum.\n\nSigned-off-by: Jeff King \n---\n builtin/receive-pack.c | 3 +--\n fetch-pack.c | 3 +--\n transport.c | 6 +++---\n transport.h | 2 +-\n 4 files changed, 6 insertions(+), 8 deletions(-)\n\ndiff --git a/builtin/receive-pack.c b/builtin/receive-pack.c\nindex a3bb13af10..39993f2bcf 100644\n--- a/builtin/receive-pack.c\n+++ b/builtin/receive-pack.c\n@@ -281,8 +281,7 @@ static int show_ref_cb(const char *path_full, const struct object_id *oid,\n \treturn 0;\n }\n \n-static void show_one_alternate_ref(const char *refname,\n-\t\t\t\t const struct object_id *oid,\n+static void show_one_alternate_ref(const struct object_id *oid,\n \t\t\t\t void *data)\n {\n \tstruct oidset *seen = data;\ndiff --git a/fetch-pack.c b/fetch-pack.c\nindex 75047a4b2a..b643de143b 100644\n--- a/fetch-pack.c\n+++ b/fetch-pack.c\n@@ -76,8 +76,7 @@ struct alternate_object_cache {\n \tsize_t nr, alloc;\n };\n \n-static void cache_one_alternate(const char *refname,\n-\t\t\t\tconst struct object_id *oid,\n+static void cache_one_alternate(const struct object_id *oid,\n \t\t\t\tvoid *vcache)\n {\n \tstruct alternate_object_cache *cache = vcache;\ndiff --git a/transport.c b/transport.c\nindex 1c76d64aba..2e0bc414d0 100644\n--- a/transport.c\n+++ b/transport.c\n@@ -1336,7 +1336,7 @@ static void read_alternate_refs(const char *path,\n \tcmd.git_cmd = 1;\n \targv_array_pushf(&cmd.args, \"--git-dir=%s\", path);\n \targv_array_push(&cmd.args, \"for-each-ref\");\n-\targv_array_push(&cmd.args, \"--format=%(objectname) %(refname)\");\n+\targv_array_push(&cmd.args, \"--format=%(objectname)\");\n \tcmd.env = local_repo_env;\n \tcmd.out = -1;\n \n@@ -1348,13 +1348,13 @@ static void read_alternate_refs(const char *path,\n \t\tstruct object_id oid;\n \n \t\tif (get_oid_hex(line.buf, &oid) ||\n-\t\t line.buf[GIT_SHA1_HEXSZ] != ' ') {\n+\t\t line.buf[GIT_SHA1_HEXSZ]) {\n \t\t\twarning(_(\"invalid line while parsing alternate refs: %s\"),\n \t\t\t\tline.buf);\n \t\t\tbreak;\n \t\t}\n \n-\t\tcb(line.buf + GIT_SHA1_HEXSZ + 1, &oid, data);\n+\t\tcb(&oid, data);\n \t}\n \n \tfclose(fh);\ndiff --git a/transport.h b/transport.h\nindex 01e717c29e..9baeca2d7a 100644\n--- a/transport.h\n+++ b/transport.h\n@@ -261,6 +261,6 @@ int transport_refs_pushed(struct ref *ref);\n void transport_print_push_status(const char *dest, struct ref *refs,\n \t\t int verbose, int porcelain, unsigned int *reject_reasons);\n \n-typedef void alternate_ref_fn(const char *refname, const struct object_id *oid, void *);\n+typedef void alternate_ref_fn(const struct object_id *oid, void *);\n extern void for_each_alternate_ref(alternate_ref_fn, void *);\n #endif\n-- \n2.19.0.764.g0a058409ab\n\n"},{"id":"358797","messageId":"xmqqva6ur24y.fsf@gitster-ct.c.googlers.com","threadId":"49393","inReplyTo":"20180924205022.GA26936@sigill.intra.peff.net","subject":"Re: [PATCH v2 3/3] transport.c: introduce core.alternateRefsPrefixes","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2018-09-24T21:55:57Z","receivedAt":"2018-09-24T21:56:02Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jeff King writes:\n\n> Do you have an opinion on whether for_each_alternate_refs() interface\n> should stop passing back refnames? By the \"they may not even exist\"\n> rationale in this sub-thread, I think it's probably foolish for any\n> caller to actually depend on the names being meaningful.\n\nI personally do not mind they were all \".have\" or unnamed.\n\nThe primary motivatgion behind for-each-alternate-refs was that we\nwanted to find more anchoring points to help the common ancestry\nnegotiation and for-each-*-ref was the obvious way to do so; the\nuser did not care anything about names.\n"},{"id":"358806","messageId":"20180924231455.GA7702@sigill.intra.peff.net","threadId":"49393","inReplyTo":"xmqqva6ur24y.fsf@gitster-ct.c.googlers.com","subject":"Re: [PATCH v2 3/3] transport.c: introduce core.alternateRefsPrefixes","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2018-09-24T23:14:56Z","receivedAt":"2018-09-24T23:14:59Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Mon, Sep 24, 2018 at 02:55:57PM -0700, Junio C Hamano wrote:\n\n> Jeff King writes:\n> \n> > Do you have an opinion on whether for_each_alternate_refs() interface\n> > should stop passing back refnames? By the \"they may not even exist\"\n> > rationale in this sub-thread, I think it's probably foolish for any\n> > caller to actually depend on the names being meaningful.\n> \n> I personally do not mind they were all \".have\" or unnamed.\n> \n> The primary motivatgion behind for-each-alternate-refs was that we\n> wanted to find more anchoring points to help the common ancestry\n> negotiation and for-each-*-ref was the obvious way to do so; the\n> user did not care anything about names.\n\nRight, I think that is totally fine for the current uses. I guess my\nquestion was: do you envision cutting the interface down to only the\noids to bite us in the future?\n\nI was on the fence during past discussions, but I think I've come over\nto the idea that the refnames actively confuse things.\n\n-Peff\n"},{"id":"358846","messageId":"xmqq1s9hqxtt.fsf@gitster-ct.c.googlers.com","threadId":"49393","inReplyTo":"20180924231455.GA7702@sigill.intra.peff.net","subject":"Re: [PATCH v2 3/3] transport.c: introduce core.alternateRefsPrefixes","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2018-09-25T17:41:18Z","receivedAt":"2018-09-25T17:41:24Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jeff King writes:\n\n> Right, I think that is totally fine for the current uses. I guess my\n> question was: do you envision cutting the interface down to only the\n> oids to bite us in the future?\n>\n> I was on the fence during past discussions, but I think I've come over\n> to the idea that the refnames actively confuse things.\n\nAlternates are sort-of repositories that you interact with via more\nnormal transports like fetch or push, and at the object store level\n(i.e. the one that helps you build your local history) you do not\nreally care what refnames other people use in their repository.\nE.g. it does not matter if a pull request to you asks you to pull\ntheir 'frotz' branch or 'nitfol' branch, as long as the work they\ndid on that branch is what you expected them to do. And I think\n\"I am aware that I can get to the objects that are reachable from\nthese objects I can borrow from that alternate when I need them\" is\nquite similar in spirit; the borrower has even less need to be aware\nof the refnames as there isn't even a need to \"git pull\" from it (at\nthat only one single point, you would care what name they used in\ntheir pull request).\n\nSo, I think we probably are better off without names.\n"},{"id":"358911","messageId":"20180925224645.GG4364@syl","threadId":"49393","inReplyTo":"xmqq1s9hqxtt.fsf@gitster-ct.c.googlers.com","subject":"Re: [PATCH v2 3/3] transport.c: introduce core.alternateRefsPrefixes","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2018-09-25T22:46:45Z","receivedAt":"2018-09-25T22:46:51Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"On Tue, Sep 25, 2018 at 10:41:18AM -0700, Junio C Hamano wrote:\n> Jeff King writes:\n>\n> > Right, I think that is totally fine for the current uses. I guess my\n> > question was: do you envision cutting the interface down to only the\n> > oids to bite us in the future?\n> >\n> > I was on the fence during past discussions, but I think I've come over\n> > to the idea that the refnames actively confuse things.\n>\n> [ ... ]\n>\n> So, I think we probably are better off without names.\n\nSorry for re-entering the thread a little later. I was travelling\nyesterday, and was surprised when I discovered that our \"grep | sed\" vs.\n\"sed\" discussion had grown so much ;-).\n\nMy reading of this is threefold:\n\n 1. There are some cosmetic changes that need to occur in t5410 and\n documentation, which are mentioned above. Those seem self\n explanatory, and I've applied the necessary bits already on my\n local version of this topic.\n\n 2. The core.alternateRefsCommand vs transport.* discussion was\n resolved in [1] as \"let's use core.alternateRefsCommand and\n core.alternateRefsPrefixes\" for now, and others contributors can\n change this as is needed.\n\n 3. We can apply Peff's patch to remove the refname requirement before\n mine, as well as any relevant changes in my series as have been\n affected by Peff's patch (e.g., documentation mentioning\n '%(refname)', etc).\n\nDoes this all sound sane to you (and match your recollection/reading of\nthe thread)? If so, I'll send v3 hopefully tomorrow.\n\nSorry for repeating what's already been said in this thread, but I felt\nit was important to ensure that we had matching understandings of one\nanother.\n\nThanks,\nTaylor\n\n[1]: https://public-inbox.org/git/xmqqa7o6skkl.fsf@gitster-ct.c.googlers.com/\n"},{"id":"358914","messageId":"xmqq4lednnc4.fsf@gitster-ct.c.googlers.com","threadId":"49393","inReplyTo":"20180925224645.GG4364@syl","subject":"Re: [PATCH v2 3/3] transport.c: introduce core.alternateRefsPrefixes","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2018-09-25T23:56:11Z","receivedAt":"2018-09-25T23:56:18Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Taylor Blau writes:\n\n> My reading of this is threefold:\n>\n> 1. There are some cosmetic changes that need to occur in t5410 and\n> documentation, which are mentioned above. Those seem self\n> explanatory, and I've applied the necessary bits already on my\n> local version of this topic.\n>\n> 2. The core.alternateRefsCommand vs transport.* discussion was\n> resolved in [1] as \"let's use core.alternateRefsCommand and\n> core.alternateRefsPrefixes\" for now, and others contributors can\n> change this as is needed.\n>\n> 3. We can apply Peff's patch to remove the refname requirement before\n> mine, as well as any relevant changes in my series as have been\n> affected by Peff's patch (e.g., documentation mentioning\n> '%(refname)', etc).\n\nI do think it makes sense to allow alternateRefsCommand to output\njust the object names without adding any refnames, and to keep the\nparser simple, we should not even make the refname optional\n(i.e. \"allow\" above becomes \"require\"), and make the default one\ndone via an invocation of for-each-ref also do the same.\n\nI do not think there was a strong concensus that we need to change\nthe internal C API signature, though. If the function signature for\nthe callback between each_ref_fn and alternate_ref_fn were the same,\nI would have opposed to the change, but because they are already\ndifferent, I do not think it is necessary to keep the dummy refname\nparameter that is always passed a meaningless value.\n\nThe final series would be\n\n 1/4: peff's \"refnames in alternates do nto matter\"\n\n 2/4: your \"hardcoded for-each-ref becomes just a default\"\n\n 3/4: your \"config can affect what command enumerates alternate's tips\"\n\n 4/4: your \"with prefix config, you don't need a fully custom command\"\n\nI guess?\n"},{"id":"358915","messageId":"20180926005620.GH4364@syl","threadId":"49393","inReplyTo":"xmqqr2hmvcyz.fsf@gitster-ct.c.googlers.com","subject":"Re: [PATCH 2/3] transport.c: introduce core.alternateRefsCommand","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2018-09-26T00:56:20Z","receivedAt":"2018-09-26T00:56:26Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"On Fri, Sep 21, 2018 at 12:59:16PM -0700, Junio C Hamano wrote:\n> Taylor Blau writes:\n>\n> > In fact, I think that we can go even further: since we don't need to\n> > catch the beginning '^.*' (without -o), we can instead:\n> >\n> > extract_haves () {\n> > depacketize - | grep '\\.have' | sed -e 's/\\\\0.*$//g'\n> > }\n>\n> Do not pipe grep into sed, unless you have an overly elaborate set\n> of patterns to filter with, e.g. something along the lines of...\n>\n> \tsed -ne '/\\.have/s/...//p'\n\nThanks, I'm not sure why I thought that this was a good idea to send\n(even after discussing it to myself twice publicly on the list\nbeforehand).\n\nAnyway, in my local copy, I adopted Peff's suggestion below in the\nthread, which is:\n\n extract_haves () {\n depacketize - | perl -lne '/^(\\S+) \\.have/ and print $1'\n }\n\nI think that that should be OK, but I sent it here to double check\nbefore sending you real patches.\n\nThanks,\nTaylor\n"},{"id":"358916","messageId":"20180926005914.GI4364@syl","threadId":"49393","inReplyTo":"CAPig+cSx0vxegy+ENtiYgV=x-A=0upFZGuFbW-=cwwknko_f4A@mail.gmail.com","subject":"Re: [PATCH v2 2/3] transport.c: introduce core.alternateRefsCommand","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2018-09-26T00:59:14Z","receivedAt":"2018-09-26T00:59:19Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"On Fri, Sep 21, 2018 at 04:18:03PM -0400, Eric Sunshine wrote:\n> On Fri, Sep 21, 2018 at 2:47 PM Taylor Blau wrote:\n> > When in a repository containing one or more alternates, Git would\n> > sometimes like to list references from its alternates. For example, 'git\n> > receive-pack' list the objects pointed to by alternate references as\n> > special \".have\" references.\n> > [...]\n> > Signed-off-by: Taylor Blau \n> > ---\n> > diff --git a/t/t5410-receive-pack.sh b/t/t5410-receive-pack.sh\n> > @@ -0,0 +1,54 @@\n> > +expect_haves () {\n> > + printf \"%s .have\\n\" $(git rev-parse $@) >expect\n> > +}\n>\n> Magic quoting behavior only kicks in when $@ is itself quoted, so this\n> should be:\n>\n> printf \"%s .have\\n\" $(git rev-parse \"$@\") >expect\n>\n> However, as it's unlikely that you need magic quoting in this case,\n> you might get by with plain $* (unquoted).\n\nYep, thanks for catching my mistake. I rewrote my local copy with \"$@\"\n(instead of $@), and also applied your suggestion of not redirecting to\n`>expect`, and renaming the function.\n\nThese both ended up becoming moot points, though, because of the\nPerl-ism that Peff suggested and I adopted throughout this thread.\n\nThe Perl Peff wrote does not capture the \" .have\" suffix at all, and\ninstead only the object identifiers. Hence, all we really need is a call\nto 'git-rev-parse(1)'. I doubt that this will ever change, so I removed\nthe function entirely.\n\nThanks,\nTaylor\n"},{"id":"358917","messageId":"20180926010606.GJ4364@syl","threadId":"49393","inReplyTo":"xmqqfty2v9qj.fsf@gitster-ct.c.googlers.com","subject":"Re: [PATCH v2 2/3] transport.c: introduce core.alternateRefsCommand","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2018-09-26T01:06:06Z","receivedAt":"2018-09-26T01:06:11Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"On Fri, Sep 21, 2018 at 02:09:08PM -0700, Junio C Hamano wrote:\n> Taylor Blau writes:\n>\n> > +core.alternateRefsCommand::\n> > +\tWhen listing references from an alternate (e.g., in the case of \".have\"), use\n>\n> It is not clear how (e.g.,...) connects to what is said in the\n> sentence. \"When advertising tips of available history from an\n> alternate, use ...\" without saying \".have\" may be less cryptic.\n>\n> I dunno.\n\nThanks, I think that I tend to overuse both \"e.g.,\" and \"i.e.,\". I took\nyour suggestion as above, which I think looks better than my original\nprose.\n\n> > +\tthe shell to execute the specified command instead of\n> > +\tlinkgit:git-for-each-ref[1]. The first argument is the path of the alternate.\n>\n> \"The path\" meaning the absolute path? Relative to the original\n> object store? Something else?\n\nIt's the absolute path, and I've updated the documentation to clarify it\nas such.\n\n> > +\tOutput must be of the form: `%(objectname) SPC %(refname)`.\n> > ++\n> > +This is useful when a repository only wishes to advertise some of its\n> > +alternate's references as \".have\"'s. For example, to only advertise branch\n> > +heads, configure `core.alternateRefsCommand` to the path of a script which runs\n> > +`git --git-dir=\"$1\" for-each-ref refs/heads`.\n> > +\n> > core.bare::\n> > \tIf true this repository is assumed to be 'bare' and has no\n> > \tworking directory associated with it. If this is the case a\n> > diff --git a/t/t5410-receive-pack.sh b/t/t5410-receive-pack.sh\n> > new file mode 100755\n> > index 0000000000..2f21f1cb8f\n> > --- /dev/null\n> > +++ b/t/t5410-receive-pack.sh\n> > @@ -0,0 +1,54 @@\n> > +#!/bin/sh\n> > +\n> > +test_description='git receive-pack test'\n> > +\n> > +. ./test-lib.sh\n> > +\n> > +test_expect_success 'setup' '\n> > +\ttest_commit one &&\n> > +\tgit update-ref refs/heads/a HEAD &&\n> > +\ttest_commit two &&\n> > +\tgit update-ref refs/heads/b HEAD &&\n> > +\ttest_commit three &&\n> > +\tgit update-ref refs/heads/c HEAD &&\n> > +\tgit clone --bare . fork &&\n> > +\tgit clone fork pusher &&\n> > +\t(\n> > +\t\tcd fork &&\n> > +\t\tgit config receive.advertisealternates true &&\n>\n> Hmph. Do we have code to support this configuration variable?\n\nWe don't ;-). Peff's explanation of why is accurate, and the mistake is\nmine.\n\n> > +\t\tcat <<-EOF | git update-ref --stdin &&\n>\n> Style: writing \"<<-\\EOF\" instead would allow readers' eyes to\n> coast over without having to look for $variable_references in\n> the here-doc.\n>\n> > +\t\tdelete refs/heads/a\n> > +\t\tdelete refs/heads/b\n> > +\t\tdelete refs/heads/c\n> > +\t\tdelete refs/heads/master\n> > +\t\tdelete refs/tags/one\n> > +\t\tdelete refs/tags/two\n> > +\t\tdelete refs/tags/three\n\nThanks, it ended up being much cleaner to write <<-\\EOF, and avoid the\nunnecessary cat(1) entirely.\n\n> So, the original created one/two/three/a/b/c/master, fork is a bare\n> clone of it and has all these things, and then you deleted all of\n> these? What does fork have after this is done? HEAD that is\n> dangling?\n>\n> > +\t\tEOF\n> > +\t\techo \"../../.git/objects\" >objects/info/alternates\n>\n> When viewed from fork/objects, ../../.git is the GIT_DIR of the\n> primary test repository, so that is where we borrow objects from.\n>\n> If we pruned the objects from fork's object store before this echo,\n> we would have an almost empty repository that borrows from its\n> alternates everything, which may make a more realistic sample case,\n> but because you are only focusing on the ref advertisement, it does\n> not matter that your fork is full of duplicate objects that are\n> available from the alternates.\n\nI could go either way. You're right in that we have only a dangling HEAD\nreference in the fork, and that all of the objects are still there. I\nsuppose that we could gc the objects that are there, but I think (as you\nnote above) that it doesn't make a huge difference either way.\n\n> > +expect_haves () {\n> > +\tprintf \"%s .have\\n\" $(git rev-parse $@) >expect\n>\n> Quote $@ inside dq pair, like $(git rev-parse \"$@\").\n\nThanks, I fixed this (per your and Eric's suggestion), but ended up\nremoving the function entirely anyway.\n\n> > +extract_haves () {\n> > +\tdepacketize - | grep '\\.have' | sed -e 's/\\\\0.*$//g'\n> > +}\n>\n> Don't pipe grep into sed, especially when both the pattern to filter\n> and the operation to perform are simple.\n>\n> I am not sure what you are trying to achive with 'g' in\n> s/pattern$//g; The anchor at the rightmost end of the pattern makes\n> sure that the pattern matches only once per line at the end anyway,\n> so \"do this howmanyever times as we have match on each line\" would\n> not make any difference, no?\n\nI admit to not fully understanding when the trailing `/g` is and is not\nuseful. Anyway, I took Peff's suggestion below to convert this 'grep |\nsed' pipeline into a Perl invocation, which I think ended up much\ncleaner.\n\nThanks,\nTaylor\n"},{"id":"358918","messageId":"20180926010935.GK4364@syl","threadId":"49393","inReplyTo":"20180922195258.GA20983@sigill.intra.peff.net","subject":"Re: [PATCH v2 2/3] transport.c: introduce core.alternateRefsCommand","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2018-09-26T01:09:35Z","receivedAt":"2018-09-26T01:09:39Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"On Sat, Sep 22, 2018 at 03:52:58PM -0400, Jeff King wrote:\n> On Sat, Sep 22, 2018 at 06:02:31PM +0000, brian m. carlson wrote:\n>\n> > On Fri, Sep 21, 2018 at 02:47:43PM -0400, Taylor Blau wrote:\n> > > +expect_haves () {\n> > > +\tprintf \"%s .have\\n\" $(git rev-parse $@) >expect\n> > > +}\n> > > +\n> > > +extract_haves () {\n> > > +\tdepacketize - | grep '\\.have' | sed -e 's/\\\\0.*$//g'\n> >\n> > It looks like you're trying to match a NUL here in the sed expression,\n> > but from my reading of it, POSIX doesn't permit BREs to match NUL.\n>\n> No, it's trying to literally match backslash followed by 0. The\n> depacketize() script will have undone the NUL already. In perl, no less,\n> making it more or less equivalent to your suggestion. ;)\n>\n> So I think this is fine (modulo that the grep and sed can be combined).\n> Yet another option would be to simply strip away everything except the\n> object id (which is all we care about), like:\n>\n> depacketize | perl -lne '/^(\\S+) \\.have/ and print $1'\n\nThanks for this. This is the suggestion I ended up taking (modulo taking\n'-' as the first argument to 'depacketize').\n\nThe 'print $1' part of this makes things a lot nicer, actually, having\nremoved the \" .have\" suffix. We can get rid of the expect_haves()\nfunction above, and instead call 'git rev-parse' inline and get the\nright results.\n\n> Or the equivalent in sed. I am happy with any solution that does the\n> correct thing.\n\nMe too :-). Thanks again.\n\nThanks,\nTaylor\n"},{"id":"358919","messageId":"20180926011838.GL4364@syl","threadId":"49393","inReplyTo":"xmqq4lednnc4.fsf@gitster-ct.c.googlers.com","subject":"Re: [PATCH v2 3/3] transport.c: introduce core.alternateRefsPrefixes","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2018-09-26T01:18:38Z","receivedAt":"2018-09-26T01:18:44Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"On Tue, Sep 25, 2018 at 04:56:11PM -0700, Junio C Hamano wrote:\n> Taylor Blau writes:\n>\n> > My reading of this is threefold:\n> >\n> > 1. There are some cosmetic changes that need to occur in t5410 and\n> > documentation, which are mentioned above. Those seem self\n> > explanatory, and I've applied the necessary bits already on my\n> > local version of this topic.\n> >\n> > 2. The core.alternateRefsCommand vs transport.* discussion was\n> > resolved in [1] as \"let's use core.alternateRefsCommand and\n> > core.alternateRefsPrefixes\" for now, and others contributors can\n> > change this as is needed.\n> >\n> > 3. We can apply Peff's patch to remove the refname requirement before\n> > mine, as well as any relevant changes in my series as have been\n> > affected by Peff's patch (e.g., documentation mentioning\n> > '%(refname)', etc).\n>\n> I do think it makes sense to allow alternateRefsCommand to output\n> just the object names without adding any refnames, and to keep the\n> parser simple, we should not even make the refname optional\n> (i.e. \"allow\" above becomes \"require\"), and make the default one\n> done via an invocation of for-each-ref also do the same.\n>\n> I do not think there was a strong concensus that we need to change\n> the internal C API signature, though. If the function signature for\n> the callback between each_ref_fn and alternate_ref_fn were the same,\n> I would have opposed to the change, but because they are already\n> different, I do not think it is necessary to keep the dummy refname\n> parameter that is always passed a meaningless value.\n>\n> The final series would be\n>\n> 1/4: peff's \"refnames in alternates do nto matter\"\n>\n> 2/4: your \"hardcoded for-each-ref becomes just a default\"\n>\n> 3/4: your \"config can affect what command enumerates alternate's tips\"\n>\n> 4/4: your \"with prefix config, you don't need a fully custom command\"\n>\n> I guess?\n\nPerfect -- we are in agreement on how the rerolled series should be\norganized. I don't anticipate much further comment on v2 in this thread,\nbut I'll let it sit overnight to make sure that the dust has all settled\nafter my new mail.\n\nI have a version of what will likely become 'v3', pushed here: [1].\n\nThanks,\nTaylor\n\n[1]: https://github.com/ttaylorr/git/tree/tb/alternate-refs-cmd\n"},{"id":"358921","messageId":"20180926031651.GA32064@sigill.intra.peff.net","threadId":"49393","inReplyTo":"xmqq4lednnc4.fsf@gitster-ct.c.googlers.com","subject":"Re: [PATCH v2 3/3] transport.c: introduce core.alternateRefsPrefixes","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2018-09-26T03:16:52Z","receivedAt":"2018-09-26T03:16:55Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Tue, Sep 25, 2018 at 04:56:11PM -0700, Junio C Hamano wrote:\n\n> Taylor Blau writes:\n> \n> > My reading of this is threefold:\n> >\n> > 1. There are some cosmetic changes that need to occur in t5410 and\n> > documentation, which are mentioned above. Those seem self\n> > explanatory, and I've applied the necessary bits already on my\n> > local version of this topic.\n> >\n> > 2. The core.alternateRefsCommand vs transport.* discussion was\n> > resolved in [1] as \"let's use core.alternateRefsCommand and\n> > core.alternateRefsPrefixes\" for now, and others contributors can\n> > change this as is needed.\n> >\n> > 3. We can apply Peff's patch to remove the refname requirement before\n> > mine, as well as any relevant changes in my series as have been\n> > affected by Peff's patch (e.g., documentation mentioning\n> > '%(refname)', etc).\n\nYeah, these three sound right to me.\n\n> I do think it makes sense to allow alternateRefsCommand to output\n> just the object names without adding any refnames, and to keep the\n> parser simple, we should not even make the refname optional\n> (i.e. \"allow\" above becomes \"require\"), and make the default one\n> done via an invocation of for-each-ref also do the same.\n\nYeah, making it optional is just the worst of both worlds, IMHO. Then\ncallers sometimes get a real value and sometimes just whatever garbage\nwe fill in, and can't rely on it.\n\n> I do not think there was a strong concensus that we need to change\n> the internal C API signature, though. If the function signature for\n> the callback between each_ref_fn and alternate_ref_fn were the same,\n> I would have opposed to the change, but because they are already\n> different, I do not think it is necessary to keep the dummy refname\n> parameter that is always passed a meaningless value.\n\nAgreed. I adjusted my \"rev-list --alternate-refs\" patch for the proposed\nnew world order (just because it's the likely user of the refname\nfield). Since the function signatures aren't the same, I already had a\ncustom callback. It did chain to the existing each_ref_fn one, so I had\nto adjust it like so:\n\ndiff --git a/revision.c b/revision.c\nindex 3988275fde..8dfe2fd4c0 100644\n--- a/revision.c\n+++ b/revision.c\n@@ -1396,11 +1396,10 @@ void add_index_objects_to_pending(struct rev_info *revs, unsigned int flags)\n \tfree_worktrees(worktrees);\n }\n \n-static void handle_one_alternate_ref(const char *refname,\n-\t\t\t\t const struct object_id *oid,\n+static void handle_one_alternate_ref(const struct object_id *oid,\n \t\t\t\t void *data)\n {\n-\thandle_one_ref(refname, oid, 0, data);\n+\thandle_one_ref(\".have\", oid, 0, data);\n }\n \n static int add_parents_only(struct rev_info *revs, const char *arg_, int flags,\n\nBut I think that's fine. We have to handle the lack of name _somewhere_\nin the call stack, so I'd just as soon it be here in the callback, where\nwe know what it will be used for (or not used at all).\n\n> The final series would be\n> \n> 1/4: peff's \"refnames in alternates do nto matter\"\n> \n> 2/4: your \"hardcoded for-each-ref becomes just a default\"\n> \n> 3/4: your \"config can affect what command enumerates alternate's tips\"\n> \n> 4/4: your \"with prefix config, you don't need a fully custom command\"\n\nYep, that's what I'd expect from the new series.\n\n-Peff\n"},{"id":"358922","messageId":"20180926032107.GB32064@sigill.intra.peff.net","threadId":"49393","inReplyTo":"20180926010606.GJ4364@syl","subject":"Re: [PATCH v2 2/3] transport.c: introduce core.alternateRefsCommand","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2018-09-26T03:21:08Z","receivedAt":"2018-09-26T03:21:12Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Tue, Sep 25, 2018 at 06:06:06PM -0700, Taylor Blau wrote:\n\n> > > +extract_haves () {\n> > > +\tdepacketize - | grep '\\.have' | sed -e 's/\\\\0.*$//g'\n> > > +}\n> >\n> > Don't pipe grep into sed, especially when both the pattern to filter\n> > and the operation to perform are simple.\n> >\n> > I am not sure what you are trying to achive with 'g' in\n> > s/pattern$//g; The anchor at the rightmost end of the pattern makes\n> > sure that the pattern matches only once per line at the end anyway,\n> > so \"do this howmanyever times as we have match on each line\" would\n> > not make any difference, no?\n> \n> I admit to not fully understanding when the trailing `/g` is and is not\n> useful. Anyway, I took Peff's suggestion below to convert this 'grep |\n> sed' pipeline into a Perl invocation, which I think ended up much\n> cleaner.\n\nIt makes the replacement global in the line. Without we substitute only\nthe first match. So try:\n\n echo foo | sed s/o/X/\n\nversus:\n\n echo foo | sed s/o/X/g\n\n-Peff\n"},{"id":"358923","messageId":"20180926033337.GC32064@sigill.intra.peff.net","threadId":"49393","inReplyTo":"20180926010935.GK4364@syl","subject":"Re: [PATCH v2 2/3] transport.c: introduce core.alternateRefsCommand","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2018-09-26T03:33:37Z","receivedAt":"2018-09-26T03:33:42Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Tue, Sep 25, 2018 at 06:09:35PM -0700, Taylor Blau wrote:\n\n> > So I think this is fine (modulo that the grep and sed can be combined).\n> > Yet another option would be to simply strip away everything except the\n> > object id (which is all we care about), like:\n> >\n> > depacketize | perl -lne '/^(\\S+) \\.have/ and print $1'\n> \n> Thanks for this. This is the suggestion I ended up taking (modulo taking\n> '-' as the first argument to 'depacketize').\n\nI don't think depacketize takes any arguments. It always reads from\nstdin directly, doesn't it? Your \"-\" is not hurting anything, but it is\ntotally ignored.\n\nA perl tangent if you're interested:\n\n Normally for shell functions like this that are just wrappers around\n perl snippets, I would suggest to pass \"$@\" from the function's\n arguments to perl. So for example if we had:\n\n haves_from_packets () {\n\tperl -lne '/^(\\S+) \\.have/ and print $1' \"$@\"\n }\n\n then you could call it with a filename:\n\n haves_from_packets packets\n\n or input on stdin:\n\n haves_from_packets )\" explicitly in your program).\n\n But because depacketize() has to use byte-wise read() calls, it\n doesn't get that magic for free. And it did not seem worth the effort\n to implement, when shell redirections are so easy. ;)\n\n Just skimming through test-lib-functions.sh, though, it does seem that\n we often deviate from that pattern (e.g., all of the q_to_nul family).\n And has seemed to mind.\n\n> The 'print $1' part of this makes things a lot nicer, actually, having\n> removed the \" .have\" suffix. We can get rid of the expect_haves()\n> function above, and instead call 'git rev-parse' inline and get the\n> right results.\n\nYes. You can even do it all in a single rev-parse call.\n\n-Peff\n"},{"id":"358942","messageId":"20180926133956.GA25697@syl","threadId":"49393","inReplyTo":"20180926033337.GC32064@sigill.intra.peff.net","subject":"Re: [PATCH v2 2/3] transport.c: introduce core.alternateRefsCommand","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2018-09-26T13:39:56Z","receivedAt":"2018-09-26T13:40:02Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"On Tue, Sep 25, 2018 at 11:33:37PM -0400, Jeff King wrote:\n> On Tue, Sep 25, 2018 at 06:09:35PM -0700, Taylor Blau wrote:\n>\n> > > So I think this is fine (modulo that the grep and sed can be combined).\n> > > Yet another option would be to simply strip away everything except the\n> > > object id (which is all we care about), like:\n> > >\n> > > depacketize | perl -lne '/^(\\S+) \\.have/ and print $1'\n> >\n> > Thanks for this. This is the suggestion I ended up taking (modulo taking\n> > '-' as the first argument to 'depacketize').\n>\n> I don't think depacketize takes any arguments. It always reads from\n> stdin directly, doesn't it? Your \"-\" is not hurting anything, but it is\n> totally ignored.\n\nYep, certainly. I think that I was drawn to this claim because I watched\nt5410 fail after applying the above recommendation, so thusly assumed\nthat it was my fault for not passing `-` to 'depacketize()`.\n\nIn the end, I'm not sure why the test failed originally (it's likely\nthat I hadn't removed the \".have\" part of 'expect_haves()', yet). But, I\nremoved the `-` in my local copy of v3, and the tests passes on all\nrevisions of this series that have it.\n\n> A perl tangent if you're interested:\n>\n> Normally for shell functions like this that are just wrappers around\n> perl snippets, I would suggest to pass \"$@\" from the function's\n> arguments to perl. So for example if we had:\n>\n> haves_from_packets () {\n> \tperl -lne '/^(\\S+) \\.have/ and print $1' \"$@\"\n> }\n>\n> then you could call it with a filename:\n>\n> haves_from_packets packets\n>\n> or input on stdin:\n>\n> haves_from_packets \n> and either works (this is magic from perl's \"-p\" loop, but you get the\n> same if you write \"while (<>)\" explicitly in your program).\n>\n> But because depacketize() has to use byte-wise read() calls, it\n> doesn't get that magic for free. And it did not seem worth the effort\n> to implement, when shell redirections are so easy. ;)\n\nTo be clear, we ought to leave this function as:\n\n extract_haves () {\n depacketize | perl -lne '/^(\\S+) \\.have/ and print $1'\n }\n\nOr are you suggesting that we change it to:\n\n extract_haves () {\n perl -lne '/^(\\S+) \\.have/ and print $1'\n }\n\nAnd call it as:\n\n printf \"0000\" | git receive-pack fork >actual &&\n depacketize actual.packets\n extract_haves actual.haves &&\n\nFrankly, (and I think that this is what you're getting at in your reply\nabove), I think that the former (e.g., calling 'depacketize()' in\n'extract_haves()') is cleaner. This approach leaves us with \"actual\" and\n\"actual.haves\", and obviates the need for another intermediary,\n\"actual.packets\".\n\n> > The 'print $1' part of this makes things a lot nicer, actually, having\n> > removed the \" .have\" suffix. We can get rid of the expect_haves()\n> > function above, and instead call 'git rev-parse' inline and get the\n> > right results.\n>\n> Yes. You can even do it all in a single rev-parse call.\n\nIndeed.\n\nThanks,\nTaylor\n"},{"id":"358974","messageId":"20180926183853.GB30680@sigill.intra.peff.net","threadId":"49393","inReplyTo":"20180926133956.GA25697@syl","subject":"Re: [PATCH v2 2/3] transport.c: introduce core.alternateRefsCommand","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2018-09-26T18:38:53Z","receivedAt":"2018-09-26T18:38:56Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Wed, Sep 26, 2018 at 06:39:56AM -0700, Taylor Blau wrote:\n\n> > A perl tangent if you're interested:\n> [...]\n> \n> To be clear, we ought to leave this function as:\n> \n> extract_haves () {\n> depacketize | perl -lne '/^(\\S+) \\.have/ and print $1'\n> }\n\nYes, I agree. You cannot do the \"$@\" there because it relies on\ndepacketize, which only handles stdin.\n\n> Or are you suggesting that we change it to:\n> \n> extract_haves () {\n> perl -lne '/^(\\S+) \\.have/ and print $1'\n> }\n\nNo, sorry. I just used the \".have\" snippet as filler text, but I see\nthat muddied my meaning considerably. This really was just a tangent for\nthe future. What you've written above is the best thing for this case.\n\n> And call it as:\n> \n> printf \"0000\" | git receive-pack fork >actual &&\n> depacketize actual.packets\n> extract_haves actual.haves &&\n> \n> Frankly, (and I think that this is what you're getting at in your reply\n> above), I think that the former (e.g., calling 'depacketize()' in\n> 'extract_haves()') is cleaner. This approach leaves us with \"actual\" and\n> \"actual.haves\", and obviates the need for another intermediary,\n> \"actual.packets\".\n\nYeah. I have no problem with the three-liner you wrote above, but I do\nnot see any particular reason for it.\n\n-Peff\n"},{"id":"359176","messageId":"20180928023901.GB89283@syl","threadId":"49393","inReplyTo":"20180926183853.GB30680@sigill.intra.peff.net","subject":"Re: [PATCH v2 2/3] transport.c: introduce core.alternateRefsCommand","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2018-09-28T02:39:01Z","receivedAt":"2018-09-28T02:39:06Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"On Wed, Sep 26, 2018 at 02:38:53PM -0400, Jeff King wrote:\n> On Wed, Sep 26, 2018 at 06:39:56AM -0700, Taylor Blau wrote:\n>\n> > > A perl tangent if you're interested:\n> > [...]\n> >\n> > To be clear, we ought to leave this function as:\n> >\n> > extract_haves () {\n> > depacketize | perl -lne '/^(\\S+) \\.have/ and print $1'\n> > }\n>\n> Yes, I agree. You cannot do the \"$@\" there because it relies on\n> depacketize, which only handles stdin.\n>\n> > Or are you suggesting that we change it to:\n> >\n> > extract_haves () {\n> > perl -lne '/^(\\S+) \\.have/ and print $1'\n> > }\n>\n> No, sorry. I just used the \".have\" snippet as filler text, but I see\n> that muddied my meaning considerably. This really was just a tangent for\n> the future. What you've written above is the best thing for this case.\n\nI see, and I had assumed that you meant the later, not that including\n\" .have\" was a good way to go forward. So I think that we're in\nagreement here.\n\n> > And call it as:\n> >\n> > printf \"0000\" | git receive-pack fork >actual &&\n> > depacketize actual.packets\n> > extract_haves actual.haves &&\n> >\n> > Frankly, (and I think that this is what you're getting at in your reply\n> > above), I think that the former (e.g., calling 'depacketize()' in\n> > 'extract_haves()') is cleaner. This approach leaves us with \"actual\" and\n> > \"actual.haves\", and obviates the need for another intermediary,\n> > \"actual.packets\".\n>\n> Yeah. I have no problem with the three-liner you wrote above, but I do\n> not see any particular reason for it.\n\nGood. That's the version that I'll send shortly, then.\n\nThanks,\nTaylor\n"},{"id":"359179","messageId":"cover.1538108385.git.me@ttaylorr.com","threadId":"49393","inReplyTo":"cover.1537466087.git.me@ttaylorr.com","subject":"[PATCH v3 0/4] Filter alternate references","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2018-09-28T04:25:34Z","receivedAt":"2018-09-28T04:25:39Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"Hi,\n\nAttached is the third re-roll of mine and Peff's series to introduce\n'core.alternateRefsCommand', and 'core.alternateRefsPrefixes' to filter\nthe initial \".have\" advertisement when an alternate has a pathologically\nlarge number of references.\n\nA range-diff against v2 is included below, but the major changes between\nthe two revisions are as follows:\n\n 1. Documentation and testing clean-up, per helpful input from Junio,\n Peff, and brian carlson.\n\n 2. Included also is a preparatory patch from Peff, to change the\n requirement that we provide refnames for alternate references. We\n no longer allow this, and the first commit sent makes that such\n change.\n\nI imagine that we may hit one more re-roll, depending on the outcome of\nthis review. The series has not fundamentally changed since v2, so I\nthink that we are at a point of stasis there. Anything that is left\noutstanding from v3 should hopefully be similarly-not-earth-shattering\n;-).\n\nThanks in advance for your review.\n\nThanks,\nTaylor\n\nJeff King (1):\n transport: drop refnames from for_each_alternate_ref\n\nTaylor Blau (3):\n transport.c: extract 'fill_alternate_refs_command'\n transport.c: introduce core.alternateRefsCommand\n transport.c: introduce core.alternateRefsPrefixes\n\n Documentation/config.txt | 18 +++++++++++++\n builtin/receive-pack.c | 3 +--\n fetch-pack.c | 3 +--\n t/t5410-receive-pack.sh | 57 ++++++++++++++++++++++++++++++++++++++++\n transport.c | 38 +++++++++++++++++++++------\n transport.h | 2 +-\n 6 files changed, 108 insertions(+), 13 deletions(-)\n create mode 100755 t/t5410-receive-pack.sh\n\nRange-diff against v2:\n-: ---------- > 1: 037273dab0 transport: drop refnames from for_each_alternate_ref\n1: 6e3a58afe7 ! 2: 9479470cb1 transport.c: extract 'fill_alternate_refs_command'\n @@ -24,7 +24,7 @@\n +\tcmd->git_cmd = 1;\n +\targv_array_pushf(&cmd->args, \"--git-dir=%s\", repo_path);\n +\targv_array_push(&cmd->args, \"for-each-ref\");\n -+\targv_array_push(&cmd->args, \"--format=%(objectname) %(refname)\");\n ++\targv_array_push(&cmd->args, \"--format=%(objectname)\");\n +\tcmd->env = local_repo_env;\n +\tcmd->out = -1;\n +}\n @@ -39,7 +39,7 @@\n -\tcmd.git_cmd = 1;\n -\targv_array_pushf(&cmd.args, \"--git-dir=%s\", path);\n -\targv_array_push(&cmd.args, \"for-each-ref\");\n --\targv_array_push(&cmd.args, \"--format=%(objectname) %(refname)\");\n +-\targv_array_push(&cmd.args, \"--format=%(objectname)\");\n -\tcmd.env = local_repo_env;\n -\tcmd.out = -1;\n +\tfill_alternate_refs_command(&cmd, path);\n2: 9797f52551 ! 3: 2dbcd54190 transport.c: introduce core.alternateRefsCommand\n @@ -3,24 +3,24 @@\n transport.c: introduce core.alternateRefsCommand\n\n When in a repository containing one or more alternates, Git would\n - sometimes like to list references from its alternates. For example, 'git\n - receive-pack' list the objects pointed to by alternate references as\n - special \".have\" references.\n + sometimes like to list references from those alternates. For example,\n + 'git receive-pack' lists the \"tips\" pointed to by references in those\n + alternates as special \".have\" references.\n\n Listing \".have\" references is designed to make pushing changes from\n upstream to a fork a lightweight operation, by advertising to the pusher\n that the fork already has the objects (via its alternate). Thus, the\n client can avoid sending them.\n\n - However, when the alternate has a pathologically large number of\n - references, the initial advertisement is too expensive. In fact, it can\n - dominate any such optimization where the pusher avoids sending certain\n - objects.\n + However, when the alternate (upstream, in the previous example) has a\n + pathologically large number of references, the initial advertisement is\n + too expensive. In fact, it can dominate any such optimization where the\n + pusher avoids sending certain objects.\n\n Introduce \"core.alternateRefsCommand\" in order to provide a facility to\n limit or filter alternate references. This can be used, for example, to\n - filter out \"uninteresting\" references from the initial advertisement in\n - the above scenario.\n + filter out references the alternate does not wish to send (for space\n + concerns, or otherwise) during the initial advertisement.\n\n Let the repository that has alternates configure this command to avoid\n trusting the alternate to provide us a safe command to run in the shell.\n @@ -38,15 +38,15 @@\n \texpect HEAD to be a symbolic link.\n\n +core.alternateRefsCommand::\n -+\tWhen listing references from an alternate (e.g., in the case of \".have\"), use\n -+\tthe shell to execute the specified command instead of\n -+\tlinkgit:git-for-each-ref[1]. The first argument is the path of the alternate.\n -+\tOutput must be of the form: `%(objectname) SPC %(refname)`.\n ++\tWhen advertising tips of available history from an alternate, use the shell to\n ++\texecute the specified command instead of linkgit:git-for-each-ref[1]. The\n ++\tfirst argument is the absolute path of the alternate. Output must be of the\n ++\tform: `%(objectname)`, where multiple tips are separated by newlines.\n ++\n +This is useful when a repository only wishes to advertise some of its\n +alternate's references as \".have\"'s. For example, to only advertise branch\n +heads, configure `core.alternateRefsCommand` to the path of a script which runs\n -+`git --git-dir=\"$1\" for-each-ref refs/heads`.\n ++`git --git-dir=\"$1\" for-each-ref --format='%(objectname)' refs/heads`.\n +\n core.bare::\n \tIf true this repository is assumed to be 'bare' and has no\n @@ -74,8 +74,7 @@\n +\tgit clone fork pusher &&\n +\t(\n +\t\tcd fork &&\n -+\t\tgit config receive.advertisealternates true &&\n -+\t\tcat <<-EOF | git update-ref --stdin &&\n ++\t\tgit update-ref --stdin <<-\\EOF &&\n +\t\tdelete refs/heads/a\n +\t\tdelete refs/heads/b\n +\t\tdelete refs/heads/c\n @@ -88,23 +87,19 @@\n +\t)\n +'\n +\n -+expect_haves () {\n -+\tprintf \"%s .have\\n\" $(git rev-parse $@) >expect\n -+}\n -+\n +extract_haves () {\n -+\tdepacketize - | grep '\\.have' | sed -e 's/\\\\0.*$//g'\n ++\tdepacketize | perl -lne '/^(\\S+) \\.have/ and print $1'\n +}\n +\n +test_expect_success 'with core.alternateRefsCommand' '\n +\twrite_script fork/alternate-refs <<-\\EOF &&\n +\t\tgit --git-dir=\"$1\" for-each-ref \\\n -+\t\t\t--format=\"%(objectname) %(refname)\" \\\n ++\t\t\t--format=\"%(objectname)\" \\\n +\t\t\trefs/heads/a \\\n +\t\t\trefs/heads/c\n +\tEOF\n +\ttest_config -C fork core.alternateRefsCommand alternate-refs &&\n -+\texpect_haves a c >expect &&\n ++\tgit rev-parse a c >expect &&\n +\tprintf \"0000\" | git receive-pack fork >actual &&\n +\textract_haves actual.haves &&\n +\ttest_cmp expect actual.haves\n @@ -122,7 +117,7 @@\n -\tcmd->git_cmd = 1;\n -\targv_array_pushf(&cmd->args, \"--git-dir=%s\", repo_path);\n -\targv_array_push(&cmd->args, \"for-each-ref\");\n --\targv_array_push(&cmd->args, \"--format=%(objectname) %(refname)\");\n +-\targv_array_push(&cmd->args, \"--format=%(objectname)\");\n +\tconst char *value;\n +\n +\tif (!git_config_get_value(\"core.alternateRefsCommand\", &value)) {\n @@ -135,7 +130,7 @@\n +\n +\t\targv_array_pushf(&cmd->args, \"--git-dir=%s\", repo_path);\n +\t\targv_array_push(&cmd->args, \"for-each-ref\");\n -+\t\targv_array_push(&cmd->args, \"--format=%(objectname) %(refname)\");\n ++\t\targv_array_push(&cmd->args, \"--format=%(objectname)\");\n +\t}\n +\n \tcmd->env = local_repo_env;\n3: 6e8f65a16d ! 4: 48eb774c9e transport.c: introduce core.alternateRefsPrefixes\n @@ -12,9 +12,7 @@\n 'core.alternateRefsCommand' would have to do:\n\n $ git config core.alternateRefsCommand ' \\\n - git -C \"$1\" for-each-ref refs/tags \\\n - --format=\"%(objectname) %(refname)\" \\\n - '\n + git -C \"$1\" for-each-ref refs/tags --format=\"%(objectname)\"'\n\n The above is cumbersome to write, so let's introduce a\n \"core.alternateRefsPrefixes\" to address this common case. Instead, the\n @@ -41,7 +39,7 @@\n +++ b/Documentation/config.txt\n @@\n heads, configure `core.alternateRefsCommand` to the path of a script which runs\n - `git --git-dir=\"$1\" for-each-ref refs/heads`.\n + `git --git-dir=\"$1\" for-each-ref --format='%(objectname)' refs/heads`.\n\n +core.alternateRefsPrefixes::\n +\tWhen listing references from an alternate, list only references that begin\n @@ -63,7 +61,7 @@\n\n +test_expect_success 'with core.alternateRefsPrefixes' '\n +\ttest_config -C fork core.alternateRefsPrefixes \"refs/tags\" &&\n -+\texpect_haves one three two >expect &&\n ++\tgit rev-parse one three two >expect &&\n +\tprintf \"0000\" | git receive-pack fork >actual &&\n +\textract_haves actual.haves &&\n +\ttest_cmp expect actual.haves\n @@ -77,7 +75,7 @@\n @@\n \t\targv_array_pushf(&cmd->args, \"--git-dir=%s\", repo_path);\n \t\targv_array_push(&cmd->args, \"for-each-ref\");\n - \t\targv_array_push(&cmd->args, \"--format=%(objectname) %(refname)\");\n + \t\targv_array_push(&cmd->args, \"--format=%(objectname)\");\n +\n +\t\tif (!git_config_get_value(\"core.alternateRefsPrefixes\", &value)) {\n +\t\t\targv_array_push(&cmd->args, \"--\");\n--\n2.19.0.221.g150f307af\n"},{"id":"359180","messageId":"037273dab0f861e5f10241de1e32cddaabeab0cc.1538108385.git.me@ttaylorr.com","threadId":"49393","inReplyTo":"cover.1538108385.git.me@ttaylorr.com","subject":"[PATCH v3 1/4] transport: drop refnames from for_each_alternate_ref","fromName":"Jeff King","fromEmail":"me@ttaylorr.com","sentAt":"2018-09-28T04:25:37Z","receivedAt":"2018-09-28T04:25:42Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"None of the current callers use the refname parameter we pass to their\ncallbacks. In theory somebody _could_ do so, but it's actually quite\nweird if you think about it: it's a ref in somebody else's repository.\nSo the name has no meaning locally, and in fact there may be duplicates\nif there are multiple alternates.\n\nThe users of this interface really only care about seeing some ref tips,\nsince that promises that the alternate has the full commit graph\nreachable from there. So let's keep the information we pass back to the\nbare minimum.\n\nSigned-off-by: Jeff King \n---\n builtin/receive-pack.c | 3 +--\n fetch-pack.c | 3 +--\n transport.c | 6 +++---\n transport.h | 2 +-\n 4 files changed, 6 insertions(+), 8 deletions(-)\n\ndiff --git a/builtin/receive-pack.c b/builtin/receive-pack.c\nindex 4d30001950..6792291f5e 100644\n--- a/builtin/receive-pack.c\n+++ b/builtin/receive-pack.c\n@@ -281,8 +281,7 @@ static int show_ref_cb(const char *path_full, const struct object_id *oid,\n \treturn 0;\n }\n \n-static void show_one_alternate_ref(const char *refname,\n-\t\t\t\t const struct object_id *oid,\n+static void show_one_alternate_ref(const struct object_id *oid,\n \t\t\t\t void *data)\n {\n \tstruct oidset *seen = data;\ndiff --git a/fetch-pack.c b/fetch-pack.c\nindex 75047a4b2a..b643de143b 100644\n--- a/fetch-pack.c\n+++ b/fetch-pack.c\n@@ -76,8 +76,7 @@ struct alternate_object_cache {\n \tsize_t nr, alloc;\n };\n \n-static void cache_one_alternate(const char *refname,\n-\t\t\t\tconst struct object_id *oid,\n+static void cache_one_alternate(const struct object_id *oid,\n \t\t\t\tvoid *vcache)\n {\n \tstruct alternate_object_cache *cache = vcache;\ndiff --git a/transport.c b/transport.c\nindex 1c76d64aba..2e0bc414d0 100644\n--- a/transport.c\n+++ b/transport.c\n@@ -1336,7 +1336,7 @@ static void read_alternate_refs(const char *path,\n \tcmd.git_cmd = 1;\n \targv_array_pushf(&cmd.args, \"--git-dir=%s\", path);\n \targv_array_push(&cmd.args, \"for-each-ref\");\n-\targv_array_push(&cmd.args, \"--format=%(objectname) %(refname)\");\n+\targv_array_push(&cmd.args, \"--format=%(objectname)\");\n \tcmd.env = local_repo_env;\n \tcmd.out = -1;\n \n@@ -1348,13 +1348,13 @@ static void read_alternate_refs(const char *path,\n \t\tstruct object_id oid;\n \n \t\tif (get_oid_hex(line.buf, &oid) ||\n-\t\t line.buf[GIT_SHA1_HEXSZ] != ' ') {\n+\t\t line.buf[GIT_SHA1_HEXSZ]) {\n \t\t\twarning(_(\"invalid line while parsing alternate refs: %s\"),\n \t\t\t\tline.buf);\n \t\t\tbreak;\n \t\t}\n \n-\t\tcb(line.buf + GIT_SHA1_HEXSZ + 1, &oid, data);\n+\t\tcb(&oid, data);\n \t}\n \n \tfclose(fh);\ndiff --git a/transport.h b/transport.h\nindex 01e717c29e..9baeca2d7a 100644\n--- a/transport.h\n+++ b/transport.h\n@@ -261,6 +261,6 @@ int transport_refs_pushed(struct ref *ref);\n void transport_print_push_status(const char *dest, struct ref *refs,\n \t\t int verbose, int porcelain, unsigned int *reject_reasons);\n \n-typedef void alternate_ref_fn(const char *refname, const struct object_id *oid, void *);\n+typedef void alternate_ref_fn(const struct object_id *oid, void *);\n extern void for_each_alternate_ref(alternate_ref_fn, void *);\n #endif\n-- \n2.19.0.221.g150f307af\n\n"},{"id":"359181","messageId":"9479470cb1d5bdf8ee140d723fb913b9a95d3a32.1538108385.git.me@ttaylorr.com","threadId":"49393","inReplyTo":"cover.1538108385.git.me@ttaylorr.com","subject":"[PATCH v3 2/4] transport.c: extract 'fill_alternate_refs_command'","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2018-09-28T04:25:39Z","receivedAt":"2018-09-28T04:25:44Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"To list alternate references, 'read_alternate_refs' creates a child\nprocess running 'git for-each-ref' in the alternate's Git directory.\n\nPrepare to run other commands besides 'git for-each-ref' by introducing\nand moving the relevant code from 'read_alternate_refs' to\n'fill_alternate_refs_command'.\n\nSigned-off-by: Taylor Blau \n---\n transport.c | 18 ++++++++++++------\n 1 file changed, 12 insertions(+), 6 deletions(-)\n\ndiff --git a/transport.c b/transport.c\nindex 2e0bc414d0..2825debac5 100644\n--- a/transport.c\n+++ b/transport.c\n@@ -1325,6 +1325,17 @@ char *transport_anonymize_url(const char *url)\n \treturn xstrdup(url);\n }\n \n+static void fill_alternate_refs_command(struct child_process *cmd,\n+\t\t\t\t\tconst char *repo_path)\n+{\n+\tcmd->git_cmd = 1;\n+\targv_array_pushf(&cmd->args, \"--git-dir=%s\", repo_path);\n+\targv_array_push(&cmd->args, \"for-each-ref\");\n+\targv_array_push(&cmd->args, \"--format=%(objectname)\");\n+\tcmd->env = local_repo_env;\n+\tcmd->out = -1;\n+}\n+\n static void read_alternate_refs(const char *path,\n \t\t\t\talternate_ref_fn *cb,\n \t\t\t\tvoid *data)\n@@ -1333,12 +1344,7 @@ static void read_alternate_refs(const char *path,\n \tstruct strbuf line = STRBUF_INIT;\n \tFILE *fh;\n \n-\tcmd.git_cmd = 1;\n-\targv_array_pushf(&cmd.args, \"--git-dir=%s\", path);\n-\targv_array_push(&cmd.args, \"for-each-ref\");\n-\targv_array_push(&cmd.args, \"--format=%(objectname)\");\n-\tcmd.env = local_repo_env;\n-\tcmd.out = -1;\n+\tfill_alternate_refs_command(&cmd, path);\n \n \tif (start_command(&cmd))\n \t\treturn;\n-- \n2.19.0.221.g150f307af\n\n"},{"id":"359182","messageId":"2dbcd5419073f06def007be3746ce90fffaf6a6d.1538108385.git.me@ttaylorr.com","threadId":"49393","inReplyTo":"cover.1538108385.git.me@ttaylorr.com","subject":"[PATCH v3 3/4] transport.c: introduce core.alternateRefsCommand","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2018-09-28T04:25:42Z","receivedAt":"2018-09-28T04:25:47Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"When in a repository containing one or more alternates, Git would\nsometimes like to list references from those alternates. For example,\n'git receive-pack' lists the \"tips\" pointed to by references in those\nalternates as special \".have\" references.\n\nListing \".have\" references is designed to make pushing changes from\nupstream to a fork a lightweight operation, by advertising to the pusher\nthat the fork already has the objects (via its alternate). Thus, the\nclient can avoid sending them.\n\nHowever, when the alternate (upstream, in the previous example) has a\npathologically large number of references, the initial advertisement is\ntoo expensive. In fact, it can dominate any such optimization where the\npusher avoids sending certain objects.\n\nIntroduce \"core.alternateRefsCommand\" in order to provide a facility to\nlimit or filter alternate references. This can be used, for example, to\nfilter out references the alternate does not wish to send (for space\nconcerns, or otherwise) during the initial advertisement.\n\nLet the repository that has alternates configure this command to avoid\ntrusting the alternate to provide us a safe command to run in the shell.\nTo behave differently on each alternate (e.g., only list tags from\nalternate A, only heads from B) provide the path of the alternate as the\nfirst argument.\n\nSigned-off-by: Taylor Blau \n---\n Documentation/config.txt | 11 +++++++++\n t/t5410-receive-pack.sh | 49 ++++++++++++++++++++++++++++++++++++++++\n transport.c | 19 ++++++++++++----\n 3 files changed, 75 insertions(+), 4 deletions(-)\n create mode 100755 t/t5410-receive-pack.sh\n\ndiff --git a/Documentation/config.txt b/Documentation/config.txt\nindex ad0f4510c3..afcb18331a 100644\n--- a/Documentation/config.txt\n+++ b/Documentation/config.txt\n@@ -616,6 +616,17 @@ core.preferSymlinkRefs::\n \tThis is sometimes needed to work with old scripts that\n \texpect HEAD to be a symbolic link.\n \n+core.alternateRefsCommand::\n+\tWhen advertising tips of available history from an alternate, use the shell to\n+\texecute the specified command instead of linkgit:git-for-each-ref[1]. The\n+\tfirst argument is the absolute path of the alternate. Output must be of the\n+\tform: `%(objectname)`, where multiple tips are separated by newlines.\n++\n+This is useful when a repository only wishes to advertise some of its\n+alternate's references as \".have\"'s. For example, to only advertise branch\n+heads, configure `core.alternateRefsCommand` to the path of a script which runs\n+`git --git-dir=\"$1\" for-each-ref --format='%(objectname)' refs/heads`.\n+\n core.bare::\n \tIf true this repository is assumed to be 'bare' and has no\n \tworking directory associated with it. If this is the case a\ndiff --git a/t/t5410-receive-pack.sh b/t/t5410-receive-pack.sh\nnew file mode 100755\nindex 0000000000..503dde35a4\n--- /dev/null\n+++ b/t/t5410-receive-pack.sh\n@@ -0,0 +1,49 @@\n+#!/bin/sh\n+\n+test_description='git receive-pack test'\n+\n+. ./test-lib.sh\n+\n+test_expect_success 'setup' '\n+\ttest_commit one &&\n+\tgit update-ref refs/heads/a HEAD &&\n+\ttest_commit two &&\n+\tgit update-ref refs/heads/b HEAD &&\n+\ttest_commit three &&\n+\tgit update-ref refs/heads/c HEAD &&\n+\tgit clone --bare . fork &&\n+\tgit clone fork pusher &&\n+\t(\n+\t\tcd fork &&\n+\t\tgit update-ref --stdin <<-\\EOF &&\n+\t\tdelete refs/heads/a\n+\t\tdelete refs/heads/b\n+\t\tdelete refs/heads/c\n+\t\tdelete refs/heads/master\n+\t\tdelete refs/tags/one\n+\t\tdelete refs/tags/two\n+\t\tdelete refs/tags/three\n+\t\tEOF\n+\t\techo \"../../.git/objects\" >objects/info/alternates\n+\t)\n+'\n+\n+extract_haves () {\n+\tdepacketize | perl -lne '/^(\\S+) \\.have/ and print $1'\n+}\n+\n+test_expect_success 'with core.alternateRefsCommand' '\n+\twrite_script fork/alternate-refs <<-\\EOF &&\n+\t\tgit --git-dir=\"$1\" for-each-ref \\\n+\t\t\t--format=\"%(objectname)\" \\\n+\t\t\trefs/heads/a \\\n+\t\t\trefs/heads/c\n+\tEOF\n+\ttest_config -C fork core.alternateRefsCommand alternate-refs &&\n+\tgit rev-parse a c >expect &&\n+\tprintf \"0000\" | git receive-pack fork >actual &&\n+\textract_haves actual.haves &&\n+\ttest_cmp expect actual.haves\n+'\n+\n+test_done\ndiff --git a/transport.c b/transport.c\nindex 2825debac5..e271b66603 100644\n--- a/transport.c\n+++ b/transport.c\n@@ -1328,10 +1328,21 @@ char *transport_anonymize_url(const char *url)\n static void fill_alternate_refs_command(struct child_process *cmd,\n \t\t\t\t\tconst char *repo_path)\n {\n-\tcmd->git_cmd = 1;\n-\targv_array_pushf(&cmd->args, \"--git-dir=%s\", repo_path);\n-\targv_array_push(&cmd->args, \"for-each-ref\");\n-\targv_array_push(&cmd->args, \"--format=%(objectname)\");\n+\tconst char *value;\n+\n+\tif (!git_config_get_value(\"core.alternateRefsCommand\", &value)) {\n+\t\tcmd->use_shell = 1;\n+\n+\t\targv_array_push(&cmd->args, value);\n+\t\targv_array_push(&cmd->args, repo_path);\n+\t} else {\n+\t\tcmd->git_cmd = 1;\n+\n+\t\targv_array_pushf(&cmd->args, \"--git-dir=%s\", repo_path);\n+\t\targv_array_push(&cmd->args, \"for-each-ref\");\n+\t\targv_array_push(&cmd->args, \"--format=%(objectname)\");\n+\t}\n+\n \tcmd->env = local_repo_env;\n \tcmd->out = -1;\n }\n-- \n2.19.0.221.g150f307af\n\n"},{"id":"359183","messageId":"48eb774c9e36f468549a278fd8cf703d8a34af28.1538108385.git.me@ttaylorr.com","threadId":"49393","inReplyTo":"cover.1538108385.git.me@ttaylorr.com","subject":"[PATCH v3 4/4] transport.c: introduce core.alternateRefsPrefixes","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2018-09-28T04:25:45Z","receivedAt":"2018-09-28T04:25:50Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"The recently-introduced \"core.alternateRefsCommand\" allows callers to\nspecify with high flexibility the tips that they wish to advertise from\nalternates. This flexibility comes at the cost of some inconvenience\nwhen the caller only wishes to limit the advertisement to one or more\nprefixes.\n\nFor example, to advertise only tags, a caller using\n'core.alternateRefsCommand' would have to do:\n\n $ git config core.alternateRefsCommand ' \\\n git -C \"$1\" for-each-ref refs/tags --format=\"%(objectname)\"'\n\nThe above is cumbersome to write, so let's introduce a\n\"core.alternateRefsPrefixes\" to address this common case. Instead, the\ncaller can run:\n\n $ git config core.alternateRefsPrefixes 'refs/tags'\n\nWhich will behave identically to the longer example using\n\"core.alternateRefsCommand\".\n\nSince the value of \"core.alternateRefsPrefixes\" is appended to 'git\nfor-each-ref' and then executed, include a \"--\" before taking the\nconfigured value to avoid misinterpreting arguments as flags to 'git\nfor-each-ref'.\n\nIn the case that the caller wishes to specify multiple prefixes, they\nmay separate them by whitespace. If \"core.alternateRefsCommand\" is set,\nit will take precedence over \"core.alternateRefsPrefixes\".\n\nSigned-off-by: Taylor Blau \n---\n Documentation/config.txt | 7 +++++++\n t/t5410-receive-pack.sh | 8 ++++++++\n transport.c | 5 +++++\n 3 files changed, 20 insertions(+)\n\ndiff --git a/Documentation/config.txt b/Documentation/config.txt\nindex afcb18331a..9ef792ef0d 100644\n--- a/Documentation/config.txt\n+++ b/Documentation/config.txt\n@@ -627,6 +627,13 @@ alternate's references as \".have\"'s. For example, to only advertise branch\n heads, configure `core.alternateRefsCommand` to the path of a script which runs\n `git --git-dir=\"$1\" for-each-ref --format='%(objectname)' refs/heads`.\n \n+core.alternateRefsPrefixes::\n+\tWhen listing references from an alternate, list only references that begin\n+\twith the given prefix. Prefixes match as if they were given as arguments to\n+\tlinkgit:git-for-each-ref[1]. To list multiple prefixes, separate them with\n+\twhitespace. If `core.alternateRefsCommand` is set, setting\n+\t`core.alternateRefsPrefixes` has no effect.\n+\n core.bare::\n \tIf true this repository is assumed to be 'bare' and has no\n \tworking directory associated with it. If this is the case a\ndiff --git a/t/t5410-receive-pack.sh b/t/t5410-receive-pack.sh\nindex 503dde35a4..3449967cc7 100755\n--- a/t/t5410-receive-pack.sh\n+++ b/t/t5410-receive-pack.sh\n@@ -46,4 +46,12 @@ test_expect_success 'with core.alternateRefsCommand' '\n \ttest_cmp expect actual.haves\n '\n \n+test_expect_success 'with core.alternateRefsPrefixes' '\n+\ttest_config -C fork core.alternateRefsPrefixes \"refs/tags\" &&\n+\tgit rev-parse one three two >expect &&\n+\tprintf \"0000\" | git receive-pack fork >actual &&\n+\textract_haves actual.haves &&\n+\ttest_cmp expect actual.haves\n+'\n+\n test_done\ndiff --git a/transport.c b/transport.c\nindex e271b66603..83474add28 100644\n--- a/transport.c\n+++ b/transport.c\n@@ -1341,6 +1341,11 @@ static void fill_alternate_refs_command(struct child_process *cmd,\n \t\targv_array_pushf(&cmd->args, \"--git-dir=%s\", repo_path);\n \t\targv_array_push(&cmd->args, \"for-each-ref\");\n \t\targv_array_push(&cmd->args, \"--format=%(objectname)\");\n+\n+\t\tif (!git_config_get_value(\"core.alternateRefsPrefixes\", &value)) {\n+\t\t\targv_array_push(&cmd->args, \"--\");\n+\t\t\targv_array_split(&cmd->args, value);\n+\t\t}\n \t}\n \n \tcmd->env = local_repo_env;\n-- \n2.19.0.221.g150f307af\n"},{"id":"359186","messageId":"20180928045858.GA25850@sigill.intra.peff.net","threadId":"49393","inReplyTo":"037273dab0f861e5f10241de1e32cddaabeab0cc.1538108385.git.me@ttaylorr.com","subject":"Re: [PATCH v3 1/4] transport: drop refnames from for_each_alternate_ref","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2018-09-28T04:58:58Z","receivedAt":"2018-09-28T04:59:02Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"> From: Jeff King \n\nPretty sure that isn't right. :)\n\nThe preferred way to send a patch with a different author is to have\nactual email be \"From:\" you, but then include a:\n\n From: Jeff King \n\nas the first line of the body (which git-am will then pick up).\ngit-send-email will do this for you automatically. Other scripts (like\nsay, if you're sending the output of format-patch into mutt) used to\nhave to implement it themselves, but these days we have \"format-patch\n--from\", which should directly output what you want.\n\n> ---\n> builtin/receive-pack.c | 3 +--\n> fetch-pack.c | 3 +--\n> transport.c | 6 +++---\n> transport.h | 2 +-\n> 4 files changed, 6 insertions(+), 8 deletions(-)\n\nThe patch itself is flawless, of course. ;)\n\n-Peff\n"},{"id":"359187","messageId":"20180928045945.GB25850@sigill.intra.peff.net","threadId":"49393","inReplyTo":"9479470cb1d5bdf8ee140d723fb913b9a95d3a32.1538108385.git.me@ttaylorr.com","subject":"Re: [PATCH v3 2/4] transport.c: extract 'fill_alternate_refs_command'","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2018-09-28T04:59:45Z","receivedAt":"2018-09-28T04:59:49Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Thu, Sep 27, 2018 at 09:25:39PM -0700, Taylor Blau wrote:\n\n> To list alternate references, 'read_alternate_refs' creates a child\n> process running 'git for-each-ref' in the alternate's Git directory.\n> \n> Prepare to run other commands besides 'git for-each-ref' by introducing\n> and moving the relevant code from 'read_alternate_refs' to\n> 'fill_alternate_refs_command'.\n> \n> Signed-off-by: Taylor Blau \n> ---\n> transport.c | 18 ++++++++++++------\n> 1 file changed, 12 insertions(+), 6 deletions(-)\n\nSame as before, but moving the slightly modified code. Makes sense.\n\n-Peff\n"},{"id":"359188","messageId":"20180928052613.GC25850@sigill.intra.peff.net","threadId":"49393","inReplyTo":"2dbcd5419073f06def007be3746ce90fffaf6a6d.1538108385.git.me@ttaylorr.com","subject":"Re: [PATCH v3 3/4] transport.c: introduce core.alternateRefsCommand","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2018-09-28T05:26:13Z","receivedAt":"2018-09-28T05:26:23Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Thu, Sep 27, 2018 at 09:25:42PM -0700, Taylor Blau wrote:\n\n> Let the repository that has alternates configure this command to avoid\n> trusting the alternate to provide us a safe command to run in the shell.\n> To behave differently on each alternate (e.g., only list tags from\n> alternate A, only heads from B) provide the path of the alternate as the\n> first argument.\n\nWell, you also need to pass the path so it knows which repo to look at.\nWhich I think is the primary reason we do it, but behaving differently\nfor each alternate is another option.\n\n> +core.alternateRefsCommand::\n> +\tWhen advertising tips of available history from an alternate, use the shell to\n> +\texecute the specified command instead of linkgit:git-for-each-ref[1]. The\n> +\tfirst argument is the absolute path of the alternate. Output must be of the\n> +\tform: `%(objectname)`, where multiple tips are separated by newlines.\n\nI wonder if people may be confused about the %(objectname) syntax, since\nit's specific to for-each-ref. Now that we've simplified the output\nformat to a single value, perhaps we should define it more directly.\nE.g., like:\n\n The output should contain one hex object id per line (i.e., the same\n as produced by `git for-each-ref --format='%(objectname)'`).\n\nNow that we've dropped the refname requirement from the output, it is\nmore clear that this really does not have to be about refs at all. In\nthe most technical sense, what we really allow in the output is any\nobject id X for which the alternate promises it has all objects\nreachable from X. Ref tips are a convenient and efficient way of\nproviding that, but they are not the only possibility (and likewise, it\nis fine to omit duplicates or even tips that are ancestors of other\ntips).\n\nI think that's probably getting _too_ technical, though. It probably\nmakes sense to just keep thinking of these as \"what are the ref tips\".\n\n> +This is useful when a repository only wishes to advertise some of its\n> +alternate's references as \".have\"'s. For example, to only advertise branch\n\nMaybe put \".have\" into backticks for formatting?\n\n> +heads, configure `core.alternateRefsCommand` to the path of a script which runs\n> +`git --git-dir=\"$1\" for-each-ref --format='%(objectname)' refs/heads`.\n\nDoes that script actually work? Because of the way we invoke shell\ncommands with arguments, I think we'd end up with:\n\n git --git-dir=\"$1\" for-each-ref --format='%(objectname)' refs/heads \"$@\"\n\nPossibly for-each-ref would ignore the extra path argument (thinking\nit's a ref pattern that just doesn't match), but it's definitely not\nwhat you intended. You'd have to write:\n\n f() { git --git-dir=$1 ...etc; } f\n\nin the usual way. That's a minor pain, but it's what makes the more\ndirect:\n\n /my/script\n\nwork.\n\nThe other alternative is to pass $GIT_DIR in the environment on behalf\nof the program. Then writing:\n\n git for-each-ref --format='%(objectname)' refs/heads\n\nwould Just Work. But it's a bit subtle, since it is not immediately\nobvious that the command is meant to run in a different repository.\n\n> diff --git a/t/t5410-receive-pack.sh b/t/t5410-receive-pack.sh\n> new file mode 100755\n> index 0000000000..503dde35a4\n> --- /dev/null\n> +++ b/t/t5410-receive-pack.sh\n> @@ -0,0 +1,49 @@\n> +#!/bin/sh\n> +\n> +test_description='git receive-pack test'\n\nThe name of this test file and the description are pretty vague. Can we\nsay something like \"test handling of receive-pack with alternate-refs\nconfig\"?\n\n> +test_expect_success 'setup' '\n> +\ttest_commit one &&\n> +\tgit update-ref refs/heads/a HEAD &&\n> +\ttest_commit two &&\n> +\tgit update-ref refs/heads/b HEAD &&\n> +\ttest_commit three &&\n> +\tgit update-ref refs/heads/c HEAD &&\n> +\tgit clone --bare . fork &&\n> +\tgit clone fork pusher &&\n> +\t(\n> +\t\tcd fork &&\n> +\t\tgit update-ref --stdin <<-\\EOF &&\n> +\t\tdelete refs/heads/a\n> +\t\tdelete refs/heads/b\n> +\t\tdelete refs/heads/c\n> +\t\tdelete refs/heads/master\n> +\t\tdelete refs/tags/one\n> +\t\tdelete refs/tags/two\n> +\t\tdelete refs/tags/three\n> +\t\tEOF\n> +\t\techo \"../../.git/objects\" >objects/info/alternates\n> +\t)\n> +'\n\nThis setup is kind of convoluted. You're deleting those refs in the\nfork, I think, because we don't want them to suppress the duplicate\n.have lines from the alternate. Might it be easier to just create the\n.have lines we're interested in after the fact?\n\nI think we can also use \"clone -s\" to make the setup of the alternate a\nlittle simpler.\n\nI don't see the \"pusher\" repo being used for anything here. Leftover\ncruft from when you were using \"git push\" to test?\n\nSo all together, perhaps something like:\n\n # we have a fork which points back to us as an alternate\n test_commit base &&\n git clone -s . fork &&\n\n # the alternate has two refs with new tips, in two separate hierarchies\n git checkout -b public/branch master &&\n test_commit public &&\n git checkout -b private/branch master &&\n test_commit private\n\nAnd then...\n\n> +test_expect_success 'with core.alternateRefsCommand' '\n> +\twrite_script fork/alternate-refs <<-\\EOF &&\n> +\t\tgit --git-dir=\"$1\" for-each-ref \\\n> +\t\t\t--format=\"%(objectname)\" \\\n> +\t\t\trefs/heads/a \\\n> +\t\t\trefs/heads/c\n> +\tEOF\n\n...this can just look for refs/heads/public/, and...\n\n> +\ttest_config -C fork core.alternateRefsCommand alternate-refs &&\n> +\tgit rev-parse a c >expect &&\n\n...we verify that we saw public/branch but not private/branch.\n\nIt's not that much shorter, but I had trouble understanding from the\nsetup why we needed to delete all those refs (and why we cared about\nthose tags in the first place).\n\n> diff --git a/transport.c b/transport.c\n> index 2825debac5..e271b66603 100644\n> --- a/transport.c\n> +++ b/transport.c\n> @@ -1328,10 +1328,21 @@ char *transport_anonymize_url(const char *url)\n> static void fill_alternate_refs_command(struct child_process *cmd,\n> \t\t\t\t\tconst char *repo_path)\n\nThe code change itself looks good to me.\n\n-Peff\n"},{"id":"359189","messageId":"20180928053057.GD25850@sigill.intra.peff.net","threadId":"49393","inReplyTo":"48eb774c9e36f468549a278fd8cf703d8a34af28.1538108385.git.me@ttaylorr.com","subject":"Re: [PATCH v3 4/4] transport.c: introduce core.alternateRefsPrefixes","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2018-09-28T05:30:57Z","receivedAt":"2018-09-28T05:31:01Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Thu, Sep 27, 2018 at 09:25:45PM -0700, Taylor Blau wrote:\n\n> The recently-introduced \"core.alternateRefsCommand\" allows callers to\n> specify with high flexibility the tips that they wish to advertise from\n> alternates. This flexibility comes at the cost of some inconvenience\n> when the caller only wishes to limit the advertisement to one or more\n> prefixes.\n> \n> For example, to advertise only tags, a caller using\n> 'core.alternateRefsCommand' would have to do:\n> \n> $ git config core.alternateRefsCommand ' \\\n> git -C \"$1\" for-each-ref refs/tags --format=\"%(objectname)\"'\n\nThis has the same \"$@\" issue as the previous one, I think (which only\nmakes your point about it being cumbersome more true!).\n\n> In the case that the caller wishes to specify multiple prefixes, they\n> may separate them by whitespace. If \"core.alternateRefsCommand\" is set,\n> it will take precedence over \"core.alternateRefsPrefixes\".\n\nJust a meta-comment: I don't particularly mind this discussion in the\ncommit message, but since these points ought to be in the documentation\nanyway, it may make sense to omit them here in the name of brevity.\n\n> +core.alternateRefsPrefixes::\n> +\tWhen listing references from an alternate, list only references that begin\n> +\twith the given prefix. Prefixes match as if they were given as arguments to\n> +\tlinkgit:git-for-each-ref[1]. To list multiple prefixes, separate them with\n> +\twhitespace. If `core.alternateRefsCommand` is set, setting\n> +\t`core.alternateRefsPrefixes` has no effect.\n\nLooks good.\n\n> diff --git a/t/t5410-receive-pack.sh b/t/t5410-receive-pack.sh\n> index 503dde35a4..3449967cc7 100755\n> --- a/t/t5410-receive-pack.sh\n> +++ b/t/t5410-receive-pack.sh\n> @@ -46,4 +46,12 @@ test_expect_success 'with core.alternateRefsCommand' '\n> \ttest_cmp expect actual.haves\n> '\n> \n> +test_expect_success 'with core.alternateRefsPrefixes' '\n> +\ttest_config -C fork core.alternateRefsPrefixes \"refs/tags\" &&\n> +\tgit rev-parse one three two >expect &&\n> +\tprintf \"0000\" | git receive-pack fork >actual &&\n> +\textract_haves actual.haves &&\n> +\ttest_cmp expect actual.haves\n> +'\n\nIf you follow my suggestion on the test setup from the last patch, it\nwould make sense to just put \"refs/heads/public/\" here. Although neither\nthat nor what you have here tests the whitespace separation. Possibly\nthere should be a third hierarchy.\n\n> diff --git a/transport.c b/transport.c\n> index e271b66603..83474add28 100644\n> --- a/transport.c\n> +++ b/transport.c\n> @@ -1341,6 +1341,11 @@ static void fill_alternate_refs_command(struct child_process *cmd,\n> \t\targv_array_pushf(&cmd->args, \"--git-dir=%s\", repo_path);\n> \t\targv_array_push(&cmd->args, \"for-each-ref\");\n> \t\targv_array_push(&cmd->args, \"--format=%(objectname)\");\n> +\n> +\t\tif (!git_config_get_value(\"core.alternateRefsPrefixes\", &value)) {\n> +\t\t\targv_array_push(&cmd->args, \"--\");\n> +\t\t\targv_array_split(&cmd->args, value);\n> +\t\t}\n\nAnd this part looks good.\n\n-Peff\n"},{"id":"359203","messageId":"20180928142151.GD23652@syl","threadId":"49393","inReplyTo":"20180928045858.GA25850@sigill.intra.peff.net","subject":"Re: [PATCH v3 1/4] transport: drop refnames from for_each_alternate_ref","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2018-09-28T14:21:51Z","receivedAt":"2018-09-28T14:21:55Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"On Fri, Sep 28, 2018 at 12:58:58AM -0400, Jeff King wrote:\n> > From: Jeff King \n>\n> Pretty sure that isn't right. :)\n\nIndeed that isn't right :-). I try my best to review my patches\ndiligently before submitting them, but here's an interesting side-story\nif you're interested:\n\nI use a script 'git mail' which is essentially doing:\n\n git format-patch --stdout >mbox && mutt -f mbox\n\nSo, by the time that I've reviewed the diff via:\n\n $ git format-patch --stdout | less\n\nI assume that the patches are ready to send (since, after all, running\n'git format-patch' more than once shouldn't change anything.) So, I open\nmutt with 'git mail', write my cover letter, and send each of the\npatches to the list.\n\nIt was during that last phase that I ignored the From: Jeff King\n, which I agree with you is certainly incorrect :-).\n\nI was going to ask Junio to fix this up when queuing, but it seems (from\na quick skim of the rest of your review), that we will reach v4, so I'll\nsee if I can't teach 'git mail' to do the right thing for me.\n\n> The patch itself is flawless, of course. ;)\n\nObviously ;-).\n\nThanks,\nTaylor\n"},{"id":"359256","messageId":"20180928220410.GA45367@syl","threadId":"49393","inReplyTo":"20180928052613.GC25850@sigill.intra.peff.net","subject":"Re: [PATCH v3 3/4] transport.c: introduce core.alternateRefsCommand","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2018-09-28T22:04:10Z","receivedAt":"2018-09-28T22:04:18Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"On Fri, Sep 28, 2018 at 01:26:13AM -0400, Jeff King wrote:\n> On Thu, Sep 27, 2018 at 09:25:42PM -0700, Taylor Blau wrote:\n>\n> > Let the repository that has alternates configure this command to avoid\n> > trusting the alternate to provide us a safe command to run in the shell.\n> > To behave differently on each alternate (e.g., only list tags from\n> > alternate A, only heads from B) provide the path of the alternate as the\n> > first argument.\n>\n> Well, you also need to pass the path so it knows which repo to look at.\n> Which I think is the primary reason we do it, but behaving differently\n> for each alternate is another option.\n\nYeah. I think that the clearer argument is yours, so I'll amend my copy.\nI am thinking of:\n\n To find the alternate, pass its absolute path as the first argument.\n\nHow does that sound?\n\n> > +core.alternateRefsCommand::\n> > + When advertising tips of available history from an alternate, use the shell to\n> > + execute the specified command instead of linkgit:git-for-each-ref[1]. The\n> > + first argument is the absolute path of the alternate. Output must be of the\n> > + form: `%(objectname)`, where multiple tips are separated by newlines.\n>\n> I wonder if people may be confused about the %(objectname) syntax, since\n> it's specific to for-each-ref. Now that we've simplified the output\n> format to a single value, perhaps we should define it more directly.\n> E.g., like:\n>\n> The output should contain one hex object id per line (i.e., the same\n> as produced by `git for-each-ref --format='%(objectname)'`).\n\nI think that that's clearer, thanks. I applied it pretty much as you\nsuggested, but changed 'should' to 'must' and dropped the leading 'the'.\n\n> Now that we've dropped the refname requirement from the output, it is\n> more clear that this really does not have to be about refs at all. In\n> the most technical sense, what we really allow in the output is any\n> object id X for which the alternate promises it has all objects\n> reachable from X. Ref tips are a convenient and efficient way of\n> providing that, but they are not the only possibility (and likewise, it\n> is fine to omit duplicates or even tips that are ancestors of other\n> tips).\n>\n> I think that's probably getting _too_ technical, though. It probably\n> makes sense to just keep thinking of these as \"what are the ref tips\".\n\nYep, I agree completely.\n\n> > +This is useful when a repository only wishes to advertise some of its\n> > +alternate's references as \".have\"'s. For example, to only advertise branch\n>\n> Maybe put \".have\" into backticks for formatting?\n\nGood idea, thanks. I took this locally as suggested.\n\n> > +heads, configure `core.alternateRefsCommand` to the path of a script which runs\n> > +`git --git-dir=\"$1\" for-each-ref --format='%(objectname)' refs/heads`.\n>\n> Does that script actually work? Because of the way we invoke shell\n> commands with arguments, I think we'd end up with:\n>\n> git --git-dir=\"$1\" for-each-ref --format='%(objectname)' refs/heads \"$@\"\n\nI think that you're right...\n\n> Possibly for-each-ref would ignore the extra path argument (thinking\n> it's a ref pattern that just doesn't match), but it's definitely not\n> what you intended. You'd have to write:\n>\n> f() { git --git-dir=$1 ...etc; } f\n>\n> in the usual way. That's a minor pain, but it's what makes the more\n> direct:\n>\n> /my/script\n>\n> work.\n\n...but this was what I was trying to get across with saying \"...to the\npath of a script which runs...\", such that we would get the implicit\nscoping that you make explicit in your example with \"f() { ... }; f\".\n\nDoes that seem OK as-is after the additional context? I think that after\nreading your response, it seems to be confusing, so perhaps it should be\nchanged...\n\n> The other alternative is to pass $GIT_DIR in the environment on behalf\n> of the program. Then writing:\n>\n> git for-each-ref --format='%(objectname)' refs/heads\n>\n> would Just Work. But it's a bit subtle, since it is not immediately\n> obvious that the command is meant to run in a different repository.\n\nI think that we discussed this approach a bit off-list, and I had the\nidea that it was too fragile to work in practice, and that it would be\ntoo surprising for callers to suddenly be in a different world.\n\nI say this not because it wouldn't make this particular scenario more\nconvenient, which it uncountably would, but because it would make other\nscenarios _more_ complicated.\n\nFor example, if a caller uses an alternate reference backed, perhaps,\nMySQL (or anything that _isn't_ Git), they're not going to want to have\nthese GIT_ environment variable set.\n\nSo, I think that the greatest common denominator between the two is to\npass the alternate's absolute path as the first argument.\n\n> > diff --git a/t/t5410-receive-pack.sh b/t/t5410-receive-pack.sh\n> > new file mode 100755\n> > index 0000000000..503dde35a4\n> > --- /dev/null\n> > +++ b/t/t5410-receive-pack.sh\n> > @@ -0,0 +1,49 @@\n> > +#!/bin/sh\n> > +\n> > +test_description='git receive-pack test'\n>\n> The name of this test file and the description are pretty vague. Can we\n> say something like \"test handling of receive-pack with alternate-refs\n> config\"?\n\nI left it intentionally vague, since I'd like for it to contain more\ntests about 'git receive-pack'-specific things in the future.\n\nI'm happy to change the name, though I wonder if we should change the\nfilename accordingly, and if so, to what.\n\n> > +test_expect_success 'setup' '\n> > + test_commit one &&\n> > + git update-ref refs/heads/a HEAD &&\n> > + test_commit two &&\n> > + git update-ref refs/heads/b HEAD &&\n> > + test_commit three &&\n> > + git update-ref refs/heads/c HEAD &&\n> > + git clone --bare . fork &&\n> > + git clone fork pusher &&\n> > + (\n> > + cd fork &&\n> > + git update-ref --stdin <<-\\EOF &&\n> > + delete refs/heads/a\n> > + delete refs/heads/b\n> > + delete refs/heads/c\n> > + delete refs/heads/master\n> > + delete refs/tags/one\n> > + delete refs/tags/two\n> > + delete refs/tags/three\n> > + EOF\n> > + echo \"../../.git/objects\" >objects/info/alternates\n> > + )\n> > +'\n>\n> This setup is kind of convoluted. You're deleting those refs in the\n> fork, I think, because we don't want them to suppress the duplicate\n> .have lines from the alternate. Might it be easier to just create the\n> .have lines we're interested in after the fact?\n> I think we can also use \"clone -s\" to make the setup of the alternate a\n> little simpler.\n>\n> I don't see the \"pusher\" repo being used for anything here. Leftover\n> cruft from when you were using \"git push\" to test?\n>\n> So all together, perhaps something like:\n>\n> # we have a fork which points back to us as an alternate\n> test_commit base &&\n> git clone -s . fork &&\n>\n> # the alternate has two refs with new tips, in two separate hierarchies\n> git checkout -b public/branch master &&\n> test_commit public &&\n> git checkout -b private/branch master &&\n> test_commit private\n>\n> And then...\n>\n> > +test_expect_success 'with core.alternateRefsCommand' '\n> > + write_script fork/alternate-refs <<-\\EOF &&\n> > + git --git-dir=\"$1\" for-each-ref \\\n> > + --format=\"%(objectname)\" \\\n> > + refs/heads/a \\\n> > + refs/heads/c\n> > + EOF\n>\n> ...this can just look for refs/heads/public/, and...\n>\n> > + test_config -C fork core.alternateRefsCommand alternate-refs &&\n> > + git rev-parse a c >expect &&\n>\n> ...we verify that we saw public/branch but not private/branch.\n>\n> It's not that much shorter, but I had trouble understanding from the\n> setup why we needed to delete all those refs (and why we cared about\n> those tags in the first place).\n\nI agree with all of this. It's certainly roughly the same length, but I\nthink that it makes it much easier to grok, and it addresses a comment\nthat Junio made in an earlier response to this thread. So, two wins for\nthe price of one :-).\n\nI had to make a couple of other changes that you didn't recommend:\n\n - Since we used to create fork with 'git clone --bare', the path of\n `core.alternateRefsCommand` grew an extra `../`, since we have to\n also traverse _out_ of the .git directory in a non-bare repository.\n\n Instead of this, I opted for both, with 'git clone -s --bare .\n fork', which means we don't have to check out a working copy, and we\n can avoid changing the line mentioned above.\n\n - Another thing that I had to decide on was what to give as a prefix\n for the test exercising 'core.alternateRefsPrefixes', which I\n decided to use 'refs/heads/private' for, which makes sure that we're\n seeing something different than 'core.alternateRefsCommand'.\n\nThe diff is kind of long (so I'm avoiding sending it here), but I think\nthat it's mostly self-explanatory from what you recommended to me and\nwhat I said above.\n\n> > diff --git a/transport.c b/transport.c\n> > index 2825debac5..e271b66603 100644\n> > --- a/transport.c\n> > +++ b/transport.c\n> > @@ -1328,10 +1328,21 @@ char *transport_anonymize_url(const char *url)\n> > static void fill_alternate_refs_command(struct child_process *cmd,\n> > const char *repo_path)\n>\n> The code change itself looks good to me.\n\nThanks for your review, as always.\n\nI'll wait until Monday to re-roll, just to make sure that there isn't\nany new feedback between now and then.\n\nThanks,\nTaylor\n"},{"id":"359258","messageId":"20180928220557.GB45367@syl","threadId":"49393","inReplyTo":"20180928053057.GD25850@sigill.intra.peff.net","subject":"Re: [PATCH v3 4/4] transport.c: introduce core.alternateRefsPrefixes","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2018-09-28T22:05:57Z","receivedAt":"2018-09-28T22:06:02Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"On Fri, Sep 28, 2018 at 01:30:57AM -0400, Jeff King wrote:\n> On Thu, Sep 27, 2018 at 09:25:45PM -0700, Taylor Blau wrote:\n>\n> > The recently-introduced \"core.alternateRefsCommand\" allows callers to\n> > specify with high flexibility the tips that they wish to advertise from\n> > alternates. This flexibility comes at the cost of some inconvenience\n> > when the caller only wishes to limit the advertisement to one or more\n> > prefixes.\n> >\n> > For example, to advertise only tags, a caller using\n> > 'core.alternateRefsCommand' would have to do:\n> >\n> > $ git config core.alternateRefsCommand ' \\\n> > git -C \"$1\" for-each-ref refs/tags --format=\"%(objectname)\"'\n>\n> This has the same \"$@\" issue as the previous one, I think (which only\n> makes your point about it being cumbersome more true!).\n\nHmm. I'll be curious to how you respond to my other message about the\nsame topic. I feel that whatever the outcome there is will affect both\nlocations in the same way.\n\n> > In the case that the caller wishes to specify multiple prefixes, they\n> > may separate them by whitespace. If \"core.alternateRefsCommand\" is set,\n> > it will take precedence over \"core.alternateRefsPrefixes\".\n>\n> Just a meta-comment: I don't particularly mind this discussion in the\n> commit message, but since these points ought to be in the documentation\n> anyway, it may make sense to omit them here in the name of brevity.\n\nSure, that makes sense.\n\n> > +core.alternateRefsPrefixes::\n> > +\tWhen listing references from an alternate, list only references that begin\n> > +\twith the given prefix. Prefixes match as if they were given as arguments to\n> > +\tlinkgit:git-for-each-ref[1]. To list multiple prefixes, separate them with\n> > +\twhitespace. If `core.alternateRefsCommand` is set, setting\n> > +\t`core.alternateRefsPrefixes` has no effect.\n>\n> Looks good.\n>\n> > diff --git a/t/t5410-receive-pack.sh b/t/t5410-receive-pack.sh\n> > index 503dde35a4..3449967cc7 100755\n> > --- a/t/t5410-receive-pack.sh\n> > +++ b/t/t5410-receive-pack.sh\n> > @@ -46,4 +46,12 @@ test_expect_success 'with core.alternateRefsCommand' '\n> > \ttest_cmp expect actual.haves\n> > '\n> >\n> > +test_expect_success 'with core.alternateRefsPrefixes' '\n> > +\ttest_config -C fork core.alternateRefsPrefixes \"refs/tags\" &&\n> > +\tgit rev-parse one three two >expect &&\n> > +\tprintf \"0000\" | git receive-pack fork >actual &&\n> > +\textract_haves actual.haves &&\n> > +\ttest_cmp expect actual.haves\n> > +'\n>\n> If you follow my suggestion on the test setup from the last patch, it\n> would make sense to just put \"refs/heads/public/\" here. Although neither\n> that nor what you have here tests the whitespace separation. Possibly\n> there should be a third hierarchy.\n\nSounds good; that's what I did.\n\n> > diff --git a/transport.c b/transport.c\n> > index e271b66603..83474add28 100644\n> > --- a/transport.c\n> > +++ b/transport.c\n> > @@ -1341,6 +1341,11 @@ static void fill_alternate_refs_command(struct child_process *cmd,\n> > \t\targv_array_pushf(&cmd->args, \"--git-dir=%s\", repo_path);\n> > \t\targv_array_push(&cmd->args, \"for-each-ref\");\n> > \t\targv_array_push(&cmd->args, \"--format=%(objectname)\");\n> > +\n> > +\t\tif (!git_config_get_value(\"core.alternateRefsPrefixes\", &value)) {\n> > +\t\t\targv_array_push(&cmd->args, \"--\");\n> > +\t\t\targv_array_split(&cmd->args, value);\n> > +\t\t}\n>\n> And this part looks good.\n\nThanks for the review of this patch, too.\n\nThanks,\nTaylor\n"},{"id":"359273","messageId":"20180929073138.GB2174@sigill.intra.peff.net","threadId":"49393","inReplyTo":"20180928220410.GA45367@syl","subject":"Re: [PATCH v3 3/4] transport.c: introduce core.alternateRefsCommand","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2018-09-29T07:31:38Z","receivedAt":"2018-09-29T07:31:42Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Fri, Sep 28, 2018 at 03:04:10PM -0700, Taylor Blau wrote:\n\n> > Well, you also need to pass the path so it knows which repo to look at.\n> > Which I think is the primary reason we do it, but behaving differently\n> > for each alternate is another option.\n> \n> Yeah. I think that the clearer argument is yours, so I'll amend my copy.\n> I am thinking of:\n> \n> To find the alternate, pass its absolute path as the first argument.\n> \n> How does that sound?\n\nSounds good.\n\n> > > +heads, configure `core.alternateRefsCommand` to the path of a script which runs\n> > > +`git --git-dir=\"$1\" for-each-ref --format='%(objectname)' refs/heads`.\n> >\n> > Does that script actually work? Because of the way we invoke shell\n> > commands with arguments, I think we'd end up with:\n> >\n> > git --git-dir=\"$1\" for-each-ref --format='%(objectname)' refs/heads \"$@\"\n> [...]\n> ...but this was what I was trying to get across with saying \"...to the\n> path of a script which runs...\", such that we would get the implicit\n> scoping that you make explicit in your example with \"f() { ... }; f\".\n>\n> Does that seem OK as-is after the additional context? I think that after\n> reading your response, it seems to be confusing, so perhaps it should be\n> changed...\n\nAh, OK. I totally missed that \"path of a script\" part. What you have is\ncorrect, then, but I do wonder if we could make it less subtle.\n\nMaybe something like:\n\n For example, if `/path/to/script` runs `git --git-dir=\"$1\"\n for-each-ref --format='%(objectname)' refs/heads/`, then putting\n `/path/to/script` in `core.alternateRefsCommand` will show only the\n branch heads from the alternate.\n\nI dunno. It's certainly clunkier. I wonder if we would be less awkward\nto show the sample script in a fenced block, with the `#!/bin/sh` and\neverything.\n\nOr maybe just keep the text you have and add a note at the end like:\n\n Note that writing that `for-each-ref` command directly in the config\n option doesn't quite work, as it has to handle the path argument\n specially.\n\nI don't think we need to hand-hold a user through the f() shell-snippet\ntrickery. I just don't want somebody thinking they can blindly paste\nthat into their config.\n\n> > The other alternative is to pass $GIT_DIR in the environment on behalf\n> > of the program. Then writing:\n> >\n> > git for-each-ref --format='%(objectname)' refs/heads\n> >\n> > would Just Work. But it's a bit subtle, since it is not immediately\n> > obvious that the command is meant to run in a different repository.\n> \n> I think that we discussed this approach a bit off-list, and I had the\n> idea that it was too fragile to work in practice, and that it would be\n> too surprising for callers to suddenly be in a different world.\n> \n> I say this not because it wouldn't make this particular scenario more\n> convenient, which it uncountably would, but because it would make other\n> scenarios _more_ complicated.\n> \n> For example, if a caller uses an alternate reference backed, perhaps,\n> MySQL (or anything that _isn't_ Git), they're not going to want to have\n> these GIT_ environment variable set.\n\nIf they're not using Git under the hood, then GIT_* probably isn't\nhurting anything. But it is still pretty subtle. Let's forget I\nmentioned it. Just chaining for-each-ref with a prefix is pretty\nawkward, but that's why we have the next patch with\nalternateRefsPrefixes.\n\nYour response did make me think of one other thing, though. The\nalternate file points to a directory with objects, and the\nfor_each_alternate_ref() code checks to see if that looks vaguely like\nthe objects/ directory of a git repo. But would anybody want to run\nsomething like alternateRefsCommand on _just_ the object directory?\nI.e., you don't have a real git repo there, but your script can\n\"somehow\" come up with a list of valid tips.\n\nThat isn't inconceivable to me for the kind of multi-fork storage we do\nat GitHub. E.g., imagine a shared object directory with no refs, and\nthen a script that goes out to the other related forks to look at their\nref tips. I don't think we have any immediate plans for it, though (and\nthere are a lot of subtle bits that I won't go into here that make it\nnon-trivial). So I'm OK to punt on it for now. I also think in a pinch\nthat you could easily fool the alternates code by just having a dummy\n\"refs/\" directory.\n\n> > > diff --git a/t/t5410-receive-pack.sh b/t/t5410-receive-pack.sh\n> [...]\n> > > +test_description='git receive-pack test'\n> >\n> > The name of this test file and the description are pretty vague. Can we\n> > say something like \"test handling of receive-pack with alternate-refs\n> > config\"?\n> \n> I left it intentionally vague, since I'd like for it to contain more\n> tests about 'git receive-pack'-specific things in the future.\n> \n> I'm happy to change the name, though I wonder if we should change the\n> filename accordingly, and if so, to what.\n\nI think we'd want to have a separate script for other receive-pack tests\nthat aren't related to alternates. There's some startup overhead to each\nscript so we don't want to make them _too_ small, but there are benefits\nto having small test scripts:\n\n - they're our unit of parallelism, so we want to be able to keep a\n reasonable number of processors full\n\n - each test script starts with a clean slate, so there's less chance\n for unexpected interactions between individual tests (e.g., when\n modifying or adding a test in the middle of the script)\n\n - it's less annoying when you're debugging a failing test near the end\n of a script ;)\n\nI actually think we'd benefit from splitting up a few of the longer\nscripts. On my quad-core laptop, running the tests in slow-to-fast order\nkeeps the processors pretty busy, and the slowest test takes less time\nthan the whole suite. But I've also tried running on a 40-core box. It\nburns through the short tests quickly, but you can never get faster than\nthe slowest single test, which takes something like 35 seconds. So\ninstead of being 10 times faster, it's more like two times faster, as\nmost of the processors idle waiting for that one script to finish.\n\nBut that's all pretty tangential here. My point is just that this\nprobably ought to be remain its own script. :)\n\nI'd probably name it \"t5410-receive-pack-alternates\" or similar.\n\n> I'll wait until Monday to re-roll, just to make sure that there isn't\n> any new feedback between now and then.\n\nSounds good. Thanks for working on this.\n\n-Peff\n"},{"id":"359274","messageId":"20180929073426.GC2174@sigill.intra.peff.net","threadId":"49393","inReplyTo":"20180928220557.GB45367@syl","subject":"Re: [PATCH v3 4/4] transport.c: introduce core.alternateRefsPrefixes","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2018-09-29T07:34:26Z","receivedAt":"2018-09-29T07:34:29Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Fri, Sep 28, 2018 at 03:05:57PM -0700, Taylor Blau wrote:\n\n> > > For example, to advertise only tags, a caller using\n> > > 'core.alternateRefsCommand' would have to do:\n> > >\n> > > $ git config core.alternateRefsCommand ' \\\n> > > git -C \"$1\" for-each-ref refs/tags --format=\"%(objectname)\"'\n> >\n> > This has the same \"$@\" issue as the previous one, I think (which only\n> > makes your point about it being cumbersome more true!).\n> \n> Hmm. I'll be curious to how you respond to my other message about the\n> same topic. I feel that whatever the outcome there is will affect both\n> locations in the same way.\n\nI think they're separate issues, right? I was just confused on the\nearlier patch, but the \"git config\" command you show above is the actual\nbroken case isn't it?\n\nI'm not overly concerned since this isn't recommending the technique to\nend users (and in fact the whole point is to give an alternative), but\nit may be worth showing a working command in case anybody runs across\nit.\n\n-Peff\n"},{"id":"359396","messageId":"20181002015619.GE96979@syl","threadId":"49393","inReplyTo":"20180929073138.GB2174@sigill.intra.peff.net","subject":"Re: [PATCH v3 3/4] transport.c: introduce core.alternateRefsCommand","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2018-10-02T01:56:19Z","receivedAt":"2018-10-02T01:56:24Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"On Sat, Sep 29, 2018 at 03:31:38AM -0400, Jeff King wrote:\n> On Fri, Sep 28, 2018 at 03:04:10PM -0700, Taylor Blau wrote:\n>\n> > > Well, you also need to pass the path so it knows which repo to look at.\n> > > Which I think is the primary reason we do it, but behaving differently\n> > > for each alternate is another option.\n> >\n> > Yeah. I think that the clearer argument is yours, so I'll amend my copy.\n> > I am thinking of:\n> >\n> > To find the alternate, pass its absolute path as the first argument.\n> >\n> > How does that sound?\n>\n> Sounds good.\n>\n> > > > +heads, configure `core.alternateRefsCommand` to the path of a script which runs\n> > > > +`git --git-dir=\"$1\" for-each-ref --format='%(objectname)' refs/heads`.\n> > >\n> > > Does that script actually work? Because of the way we invoke shell\n> > > commands with arguments, I think we'd end up with:\n> > >\n> > > git --git-dir=\"$1\" for-each-ref --format='%(objectname)' refs/heads \"$@\"\n> > [...]\n> > ...but this was what I was trying to get across with saying \"...to the\n> > path of a script which runs...\", such that we would get the implicit\n> > scoping that you make explicit in your example with \"f() { ... }; f\".\n> >\n> > Does that seem OK as-is after the additional context? I think that after\n> > reading your response, it seems to be confusing, so perhaps it should be\n> > changed...\n>\n> Ah, OK. I totally missed that \"path of a script\" part. What you have is\n> correct, then, but I do wonder if we could make it less subtle.\n>\n> Maybe something like:\n>\n> For example, if `/path/to/script` runs `git --git-dir=\"$1\"\n> for-each-ref --format='%(objectname)' refs/heads/`, then putting\n> `/path/to/script` in `core.alternateRefsCommand` will show only the\n> branch heads from the alternate.\n>\n> I dunno. It's certainly clunkier. I wonder if we would be less awkward\n> to show the sample script in a fenced block, with the `#!/bin/sh` and\n> everything.\n>\n> Or maybe just keep the text you have and add a note at the end like:\n>\n> Note that writing that `for-each-ref` command directly in the config\n> option doesn't quite work, as it has to handle the path argument\n> specially.\n>\n> I don't think we need to hand-hold a user through the f() shell-snippet\n> trickery. I just don't want somebody thinking they can blindly paste\n> that into their config.\n\nYeah, I agree with your later suggestion, and I'm glad that we're on the\nsame page. I certianly don't think that we need to do an extra amount of\nhand holding through the 'f() { ... }; f' pattern, but I added an extra\nbit to say that 'git for-each-ref' by itself doesn't work, since you\nhave to handle the path argument.\n\n> > > The other alternative is to pass $GIT_DIR in the environment on behalf\n> > > of the program. Then writing:\n> > >\n> > > git for-each-ref --format='%(objectname)' refs/heads\n> > >\n> > > would Just Work. But it's a bit subtle, since it is not immediately\n> > > obvious that the command is meant to run in a different repository.\n> >\n> > I think that we discussed this approach a bit off-list, and I had the\n> > idea that it was too fragile to work in practice, and that it would be\n> > too surprising for callers to suddenly be in a different world.\n> >\n> > I say this not because it wouldn't make this particular scenario more\n> > convenient, which it uncountably would, but because it would make other\n> > scenarios _more_ complicated.\n> >\n> > For example, if a caller uses an alternate reference backed, perhaps,\n> > MySQL (or anything that _isn't_ Git), they're not going to want to have\n> > these GIT_ environment variable set.\n>\n> If they're not using Git under the hood, then GIT_* probably isn't\n> hurting anything. But it is still pretty subtle. Let's forget I\n> mentioned it. Just chaining for-each-ref with a prefix is pretty\n> awkward, but that's why we have the next patch with\n> alternateRefsPrefixes.\n>\n> Your response did make me think of one other thing, though. The\n> alternate file points to a directory with objects, and the\n> for_each_alternate_ref() code checks to see if that looks vaguely like\n> the objects/ directory of a git repo. But would anybody want to run\n> something like alternateRefsCommand on _just_ the object directory?\n> I.e., you don't have a real git repo there, but your script can\n> \"somehow\" come up with a list of valid tips.\n>\n> That isn't inconceivable to me for the kind of multi-fork storage we do\n> at GitHub. E.g., imagine a shared object directory with no refs, and\n> then a script that goes out to the other related forks to look at their\n> ref tips. I don't think we have any immediate plans for it, though (and\n> there are a lot of subtle bits that I won't go into here that make it\n> non-trivial). So I'm OK to punt on it for now. I also think in a pinch\n> that you could easily fool the alternates code by just having a dummy\n> \"refs/\" directory.\n\nI'm not opposed to the idea in general, and I think that it's a good\none, but I am opposed to it in this series. I think that the series\nas-is is concise, and unlocks a path towards implementing this feature\nat GitHub, and for other users, too.\n\nCertainly we can invent more complicated examples, and I think that many\nof them (yours included) are worth building the extra support for. But\nin this initial version, I think that we'd be fine to leave it off.\n\n> > > > diff --git a/t/t5410-receive-pack.sh b/t/t5410-receive-pack.sh\n> > [...]\n> > > > +test_description='git receive-pack test'\n> > >\n> > > The name of this test file and the description are pretty vague. Can we\n> > > say something like \"test handling of receive-pack with alternate-refs\n> > > config\"?\n> >\n> > I left it intentionally vague, since I'd like for it to contain more\n> > tests about 'git receive-pack'-specific things in the future.\n> >\n> > I'm happy to change the name, though I wonder if we should change the\n> > filename accordingly, and if so, to what.\n>\n> I think we'd want to have a separate script for other receive-pack tests\n> that aren't related to alternates. There's some startup overhead to each\n> script so we don't want to make them _too_ small, but there are benefits\n> to having small test scripts:\n>\n> - they're our unit of parallelism, so we want to be able to keep a\n> reasonable number of processors full\n>\n> - each test script starts with a clean slate, so there's less chance\n> for unexpected interactions between individual tests (e.g., when\n> modifying or adding a test in the middle of the script)\n>\n> - it's less annoying when you're debugging a failing test near the end\n> of a script ;)\n\nAll good points, so I'm convinced ;-).\n\n> I actually think we'd benefit from splitting up a few of the longer\n> scripts. On my quad-core laptop, running the tests in slow-to-fast order\n> keeps the processors pretty busy, and the slowest test takes less time\n> than the whole suite. But I've also tried running on a 40-core box. It\n> burns through the short tests quickly, but you can never get faster than\n> the slowest single test, which takes something like 35 seconds. So\n> instead of being 10 times faster, it's more like two times faster, as\n> most of the processors idle waiting for that one script to finish.\n>\n> But that's all pretty tangential here. My point is just that this\n> probably ought to be remain its own script. :)\n>\n> I'd probably name it \"t5410-receive-pack-alternates\" or similar.\n\nSounds good, I'll do that and update the name of the test to be\n'receive-pack with alternate ref filtering'.\n\n> > I'll wait until Monday to re-roll, just to make sure that there isn't\n> > any new feedback between now and then.\n>\n> Sounds good. Thanks for working on this.\n\nIt's been my pleasure. Thanks for all of your help.\n\nThanks,\nTaylor\n"},{"id":"359397","messageId":"20181002015737.GF96979@syl","threadId":"49393","inReplyTo":"20180929073426.GC2174@sigill.intra.peff.net","subject":"Re: [PATCH v3 4/4] transport.c: introduce core.alternateRefsPrefixes","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2018-10-02T01:57:37Z","receivedAt":"2018-10-02T01:57:42Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"On Sat, Sep 29, 2018 at 03:34:26AM -0400, Jeff King wrote:\n> On Fri, Sep 28, 2018 at 03:05:57PM -0700, Taylor Blau wrote:\n>\n> > > > For example, to advertise only tags, a caller using\n> > > > 'core.alternateRefsCommand' would have to do:\n> > > >\n> > > > $ git config core.alternateRefsCommand ' \\\n> > > > git -C \"$1\" for-each-ref refs/tags --format=\"%(objectname)\"'\n> > >\n> > > This has the same \"$@\" issue as the previous one, I think (which only\n> > > makes your point about it being cumbersome more true!).\n> >\n> > Hmm. I'll be curious to how you respond to my other message about the\n> > same topic. I feel that whatever the outcome there is will affect both\n> > locations in the same way.\n>\n> I think they're separate issues, right? I was just confused on the\n> earlier patch, but the \"git config\" command you show above is the actual\n> broken case isn't it?\n\nAh, I certainly had these mixed up on Saturday when I wrote what is\nquoted here. As I understand it now, you were talking about the\ndifference between $@ and \"$@\", which I did fix (by rewriting the former\nto the later).\n\n> I'm not overly concerned since this isn't recommending the technique to\n> end users (and in fact the whole point is to give an alternative), but\n> it may be worth showing a working command in case anybody runs across\n> it.\n\nCompletely agree, and thanks for your review.\n\nThanks,\nTaylor\n"},{"id":"359398","messageId":"20181002020036.GA62117@syl","threadId":"49393","inReplyTo":"20181002015737.GF96979@syl","subject":"Re: [PATCH v3 4/4] transport.c: introduce core.alternateRefsPrefixes","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2018-10-02T02:00:36Z","receivedAt":"2018-10-02T02:00:42Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"On Mon, Oct 01, 2018 at 06:57:37PM -0700, Taylor Blau wrote:\n> On Sat, Sep 29, 2018 at 03:34:26AM -0400, Jeff King wrote:\n> > On Fri, Sep 28, 2018 at 03:05:57PM -0700, Taylor Blau wrote:\n> >\n> > > > > For example, to advertise only tags, a caller using\n> > > > > 'core.alternateRefsCommand' would have to do:\n> > > > >\n> > > > > $ git config core.alternateRefsCommand ' \\\n> > > > > git -C \"$1\" for-each-ref refs/tags --format=\"%(objectname)\"'\n> > > >\n> > > > This has the same \"$@\" issue as the previous one, I think (which only\n> > > > makes your point about it being cumbersome more true!).\n> > >\n> > > Hmm. I'll be curious to how you respond to my other message about the\n> > > same topic. I feel that whatever the outcome there is will affect both\n> > > locations in the same way.\n> >\n> > I think they're separate issues, right? I was just confused on the\n> > earlier patch, but the \"git config\" command you show above is the actual\n> > broken case isn't it?\n>\n> Ah, I certainly had these mixed up on Saturday when I wrote what is\n> quoted here. As I understand it now, you were talking about the\n> difference between $@ and \"$@\", which I did fix (by rewriting the former\n> to the later).\n\nDouble \"ah!\". You were talking about getting the path to the repository\nstuck on the end, which _is_ a problem here. I'll fix that.\n\n> > I'm not overly concerned since this isn't recommending the technique to\n> > end users (and in fact the whole point is to give an alternative), but\n> > it may be worth showing a working command in case anybody runs across\n> > it.\n>\n> Completely agree, and thanks for your review.\n\nThanks,\nTaylor\n"},{"id":"359399","messageId":"cover.1538446826.git.me@ttaylorr.com","threadId":"49393","inReplyTo":"cover.1537466087.git.me@ttaylorr.com","subject":"[PATCH v4 0/4] Filter alternate references","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2018-10-02T02:23:48Z","receivedAt":"2018-10-02T02:23:56Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"Hi,\n\nAttached is the fourth re-roll of a series to teach\n'core.alternateRefsCommand' and 'core.alternateRefsPrefixes' to filter\nrefs from an alternate from being visible to the fork. This is done in\norder to optimize a case described in patch [3/4].\n\nAs always, a range-diff is included below, showing that not much has\nchanged of significance since last round. I mostly focused my efforts on\ntaking Peff's suggestion towards a more straightforward implementation\nof the test setup.\n\nSome extra documentation was written and a couple of commit messages\namended, but no C code has changed since the v2.\n\nThanks again for all of your review.\n\nThanks,\nTaylor\n\nJeff King (1):\n transport: drop refnames from for_each_alternate_ref\n\nTaylor Blau (3):\n transport.c: extract 'fill_alternate_refs_command'\n transport.c: introduce core.alternateRefsCommand\n transport.c: introduce core.alternateRefsPrefixes\n\n Documentation/config.txt | 23 +++++++++++++++++\n builtin/receive-pack.c | 3 +--\n fetch-pack.c | 3 +--\n t/t5410-receive-pack-alternates.sh | 41 ++++++++++++++++++++++++++++++\n transport.c | 38 +++++++++++++++++++++------\n transport.h | 2 +-\n 6 files changed, 97 insertions(+), 13 deletions(-)\n create mode 100755 t/t5410-receive-pack-alternates.sh\n\nRange-diff against v3:\n1: 037273dab0 ! 1: 491f258f50 transport: drop refnames from for_each_alternate_ref\n @@ -14,6 +14,7 @@\n bare minimum.\n\n Signed-off-by: Jeff King \n + Signed-off-by: Taylor Blau \n\n diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c\n --- a/builtin/receive-pack.c\n2: 9479470cb1 = 2: 6119de15f2 transport.c: extract 'fill_alternate_refs_command'\n3: 2dbcd54190 ! 3: aadb27c010 transport.c: introduce core.alternateRefsCommand\n @@ -24,9 +24,7 @@\n\n Let the repository that has alternates configure this command to avoid\n trusting the alternate to provide us a safe command to run in the shell.\n - To behave differently on each alternate (e.g., only list tags from\n - alternate A, only heads from B) provide the path of the alternate as the\n - first argument.\n + To find the alternate, pass its absolute path as the first argument.\n\n Signed-off-by: Taylor Blau \n\n @@ -40,51 +38,41 @@\n +core.alternateRefsCommand::\n +\tWhen advertising tips of available history from an alternate, use the shell to\n +\texecute the specified command instead of linkgit:git-for-each-ref[1]. The\n -+\tfirst argument is the absolute path of the alternate. Output must be of the\n -+\tform: `%(objectname)`, where multiple tips are separated by newlines.\n ++\tfirst argument is the absolute path of the alternate. Output must contain one\n ++\thex object id per line (i.e., the same as produce by `git for-each-ref\n ++\t--format='%(objectname)'`).\n ++\n +This is useful when a repository only wishes to advertise some of its\n -+alternate's references as \".have\"'s. For example, to only advertise branch\n ++alternate's references as `.have`'s. For example, to only advertise branch\n +heads, configure `core.alternateRefsCommand` to the path of a script which runs\n +`git --git-dir=\"$1\" for-each-ref --format='%(objectname)' refs/heads`.\n +++\n ++Note that the configured value is executed in a shell, and thus\n ++linkgit:git-for-each-ref[1] by itself does not work, as scripts have to handle\n ++the path argument specially.\n +\n core.bare::\n \tIf true this repository is assumed to be 'bare' and has no\n \tworking directory associated with it. If this is the case a\n\n - diff --git a/t/t5410-receive-pack.sh b/t/t5410-receive-pack.sh\n + diff --git a/t/t5410-receive-pack-alternates.sh b/t/t5410-receive-pack-alternates.sh\n new file mode 100755\n --- /dev/null\n - +++ b/t/t5410-receive-pack.sh\n + +++ b/t/t5410-receive-pack-alternates.sh\n @@\n +#!/bin/sh\n +\n -+test_description='git receive-pack test'\n ++test_description='git receive-pack with alternate ref filtering'\n +\n +. ./test-lib.sh\n +\n +test_expect_success 'setup' '\n -+\ttest_commit one &&\n -+\tgit update-ref refs/heads/a HEAD &&\n -+\ttest_commit two &&\n -+\tgit update-ref refs/heads/b HEAD &&\n -+\ttest_commit three &&\n -+\tgit update-ref refs/heads/c HEAD &&\n -+\tgit clone --bare . fork &&\n -+\tgit clone fork pusher &&\n -+\t(\n -+\t\tcd fork &&\n -+\t\tgit update-ref --stdin <<-\\EOF &&\n -+\t\tdelete refs/heads/a\n -+\t\tdelete refs/heads/b\n -+\t\tdelete refs/heads/c\n -+\t\tdelete refs/heads/master\n -+\t\tdelete refs/tags/one\n -+\t\tdelete refs/tags/two\n -+\t\tdelete refs/tags/three\n -+\t\tEOF\n -+\t\techo \"../../.git/objects\" >objects/info/alternates\n -+\t)\n ++\ttest_commit base &&\n ++\tgit clone -s --bare . fork &&\n ++\tgit checkout -b public/branch master &&\n ++\ttest_commit public &&\n ++\tgit checkout -b private/branch master &&\n ++\ttest_commit private\n +'\n +\n +extract_haves () {\n @@ -95,11 +83,10 @@\n +\twrite_script fork/alternate-refs <<-\\EOF &&\n +\t\tgit --git-dir=\"$1\" for-each-ref \\\n +\t\t\t--format=\"%(objectname)\" \\\n -+\t\t\trefs/heads/a \\\n -+\t\t\trefs/heads/c\n ++\t\t\trefs/heads/public/\n +\tEOF\n +\ttest_config -C fork core.alternateRefsCommand alternate-refs &&\n -+\tgit rev-parse a c >expect &&\n ++\tgit rev-parse public/branch >expect &&\n +\tprintf \"0000\" | git receive-pack fork >actual &&\n +\textract_haves actual.haves &&\n +\ttest_cmp expect actual.haves\n4: 48eb774c9e ! 4: 0d3521e92a transport.c: introduce core.alternateRefsPrefixes\n @@ -12,7 +12,8 @@\n 'core.alternateRefsCommand' would have to do:\n\n $ git config core.alternateRefsCommand ' \\\n - git -C \"$1\" for-each-ref refs/tags --format=\"%(objectname)\"'\n + f() { git -C \"$1\" for-each-ref \\\n + refs/tags --format=\"%(objectname)\" }; f \"$@\"'\n\n The above is cumbersome to write, so let's introduce a\n \"core.alternateRefsPrefixes\" to address this common case. Instead, the\n @@ -38,8 +39,8 @@\n --- a/Documentation/config.txt\n +++ b/Documentation/config.txt\n @@\n - heads, configure `core.alternateRefsCommand` to the path of a script which runs\n - `git --git-dir=\"$1\" for-each-ref --format='%(objectname)' refs/heads`.\n + linkgit:git-for-each-ref[1] by itself does not work, as scripts have to handle\n + the path argument specially.\n\n +core.alternateRefsPrefixes::\n +\tWhen listing references from an alternate, list only references that begin\n @@ -52,16 +53,16 @@\n \tIf true this repository is assumed to be 'bare' and has no\n \tworking directory associated with it. If this is the case a\n\n - diff --git a/t/t5410-receive-pack.sh b/t/t5410-receive-pack.sh\n - --- a/t/t5410-receive-pack.sh\n - +++ b/t/t5410-receive-pack.sh\n + diff --git a/t/t5410-receive-pack-alternates.sh b/t/t5410-receive-pack-alternates.sh\n + --- a/t/t5410-receive-pack-alternates.sh\n + +++ b/t/t5410-receive-pack-alternates.sh\n @@\n \ttest_cmp expect actual.haves\n '\n\n +test_expect_success 'with core.alternateRefsPrefixes' '\n -+\ttest_config -C fork core.alternateRefsPrefixes \"refs/tags\" &&\n -+\tgit rev-parse one three two >expect &&\n ++\ttest_config -C fork core.alternateRefsPrefixes \"refs/heads/private\" &&\n ++\tgit rev-parse private/branch expect &&\n +\tprintf \"0000\" | git receive-pack fork >actual &&\n +\textract_haves actual.haves &&\n +\ttest_cmp expect actual.haves\n--\n2.19.0.221.g150f307af\n"},{"id":"359400","messageId":"491f258f5080465c6f4feb9958487595368d286e.1538446826.git.me@ttaylorr.com","threadId":"49393","inReplyTo":"cover.1538446826.git.me@ttaylorr.com","subject":"[PATCH v4 1/4] transport: drop refnames from for_each_alternate_ref","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2018-10-02T02:23:53Z","receivedAt":"2018-10-02T02:24:04Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"From: Jeff King \n\nNone of the current callers use the refname parameter we pass to their\ncallbacks. In theory somebody _could_ do so, but it's actually quite\nweird if you think about it: it's a ref in somebody else's repository.\nSo the name has no meaning locally, and in fact there may be duplicates\nif there are multiple alternates.\n\nThe users of this interface really only care about seeing some ref tips,\nsince that promises that the alternate has the full commit graph\nreachable from there. So let's keep the information we pass back to the\nbare minimum.\n\nSigned-off-by: Jeff King \nSigned-off-by: Taylor Blau \n---\n builtin/receive-pack.c | 3 +--\n fetch-pack.c | 3 +--\n transport.c | 6 +++---\n transport.h | 2 +-\n 4 files changed, 6 insertions(+), 8 deletions(-)\n\ndiff --git a/builtin/receive-pack.c b/builtin/receive-pack.c\nindex 4d30001950..6792291f5e 100644\n--- a/builtin/receive-pack.c\n+++ b/builtin/receive-pack.c\n@@ -281,8 +281,7 @@ static int show_ref_cb(const char *path_full, const struct object_id *oid,\n \treturn 0;\n }\n \n-static void show_one_alternate_ref(const char *refname,\n-\t\t\t\t const struct object_id *oid,\n+static void show_one_alternate_ref(const struct object_id *oid,\n \t\t\t\t void *data)\n {\n \tstruct oidset *seen = data;\ndiff --git a/fetch-pack.c b/fetch-pack.c\nindex 75047a4b2a..b643de143b 100644\n--- a/fetch-pack.c\n+++ b/fetch-pack.c\n@@ -76,8 +76,7 @@ struct alternate_object_cache {\n \tsize_t nr, alloc;\n };\n \n-static void cache_one_alternate(const char *refname,\n-\t\t\t\tconst struct object_id *oid,\n+static void cache_one_alternate(const struct object_id *oid,\n \t\t\t\tvoid *vcache)\n {\n \tstruct alternate_object_cache *cache = vcache;\ndiff --git a/transport.c b/transport.c\nindex 1c76d64aba..2e0bc414d0 100644\n--- a/transport.c\n+++ b/transport.c\n@@ -1336,7 +1336,7 @@ static void read_alternate_refs(const char *path,\n \tcmd.git_cmd = 1;\n \targv_array_pushf(&cmd.args, \"--git-dir=%s\", path);\n \targv_array_push(&cmd.args, \"for-each-ref\");\n-\targv_array_push(&cmd.args, \"--format=%(objectname) %(refname)\");\n+\targv_array_push(&cmd.args, \"--format=%(objectname)\");\n \tcmd.env = local_repo_env;\n \tcmd.out = -1;\n \n@@ -1348,13 +1348,13 @@ static void read_alternate_refs(const char *path,\n \t\tstruct object_id oid;\n \n \t\tif (get_oid_hex(line.buf, &oid) ||\n-\t\t line.buf[GIT_SHA1_HEXSZ] != ' ') {\n+\t\t line.buf[GIT_SHA1_HEXSZ]) {\n \t\t\twarning(_(\"invalid line while parsing alternate refs: %s\"),\n \t\t\t\tline.buf);\n \t\t\tbreak;\n \t\t}\n \n-\t\tcb(line.buf + GIT_SHA1_HEXSZ + 1, &oid, data);\n+\t\tcb(&oid, data);\n \t}\n \n \tfclose(fh);\ndiff --git a/transport.h b/transport.h\nindex 01e717c29e..9baeca2d7a 100644\n--- a/transport.h\n+++ b/transport.h\n@@ -261,6 +261,6 @@ int transport_refs_pushed(struct ref *ref);\n void transport_print_push_status(const char *dest, struct ref *refs,\n \t\t int verbose, int porcelain, unsigned int *reject_reasons);\n \n-typedef void alternate_ref_fn(const char *refname, const struct object_id *oid, void *);\n+typedef void alternate_ref_fn(const struct object_id *oid, void *);\n extern void for_each_alternate_ref(alternate_ref_fn, void *);\n #endif\n-- \n2.19.0.221.g150f307af\n\n"},{"id":"359401","messageId":"aadb27c0106d8f1a49dd35e7a040131aecaef2c1.1538446827.git.me@ttaylorr.com","threadId":"49393","inReplyTo":"cover.1538446826.git.me@ttaylorr.com","subject":"[PATCH v4 3/4] transport.c: introduce core.alternateRefsCommand","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2018-10-02T02:23:58Z","receivedAt":"2018-10-02T02:24:04Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"When in a repository containing one or more alternates, Git would\nsometimes like to list references from those alternates. For example,\n'git receive-pack' lists the \"tips\" pointed to by references in those\nalternates as special \".have\" references.\n\nListing \".have\" references is designed to make pushing changes from\nupstream to a fork a lightweight operation, by advertising to the pusher\nthat the fork already has the objects (via its alternate). Thus, the\nclient can avoid sending them.\n\nHowever, when the alternate (upstream, in the previous example) has a\npathologically large number of references, the initial advertisement is\ntoo expensive. In fact, it can dominate any such optimization where the\npusher avoids sending certain objects.\n\nIntroduce \"core.alternateRefsCommand\" in order to provide a facility to\nlimit or filter alternate references. This can be used, for example, to\nfilter out references the alternate does not wish to send (for space\nconcerns, or otherwise) during the initial advertisement.\n\nLet the repository that has alternates configure this command to avoid\ntrusting the alternate to provide us a safe command to run in the shell.\nTo find the alternate, pass its absolute path as the first argument.\n\nSigned-off-by: Taylor Blau \n---\n Documentation/config.txt | 16 +++++++++++++++\n t/t5410-receive-pack-alternates.sh | 33 ++++++++++++++++++++++++++++++\n transport.c | 19 +++++++++++++----\n 3 files changed, 64 insertions(+), 4 deletions(-)\n create mode 100755 t/t5410-receive-pack-alternates.sh\n\ndiff --git a/Documentation/config.txt b/Documentation/config.txt\nindex ad0f4510c3..ac0577d288 100644\n--- a/Documentation/config.txt\n+++ b/Documentation/config.txt\n@@ -616,6 +616,22 @@ core.preferSymlinkRefs::\n \tThis is sometimes needed to work with old scripts that\n \texpect HEAD to be a symbolic link.\n \n+core.alternateRefsCommand::\n+\tWhen advertising tips of available history from an alternate, use the shell to\n+\texecute the specified command instead of linkgit:git-for-each-ref[1]. The\n+\tfirst argument is the absolute path of the alternate. Output must contain one\n+\thex object id per line (i.e., the same as produce by `git for-each-ref\n+\t--format='%(objectname)'`).\n++\n+This is useful when a repository only wishes to advertise some of its\n+alternate's references as `.have`'s. For example, to only advertise branch\n+heads, configure `core.alternateRefsCommand` to the path of a script which runs\n+`git --git-dir=\"$1\" for-each-ref --format='%(objectname)' refs/heads`.\n++\n+Note that the configured value is executed in a shell, and thus\n+linkgit:git-for-each-ref[1] by itself does not work, as scripts have to handle\n+the path argument specially.\n+\n core.bare::\n \tIf true this repository is assumed to be 'bare' and has no\n \tworking directory associated with it. If this is the case a\ndiff --git a/t/t5410-receive-pack-alternates.sh b/t/t5410-receive-pack-alternates.sh\nnew file mode 100755\nindex 0000000000..49d0fe44fb\n--- /dev/null\n+++ b/t/t5410-receive-pack-alternates.sh\n@@ -0,0 +1,33 @@\n+#!/bin/sh\n+\n+test_description='git receive-pack with alternate ref filtering'\n+\n+. ./test-lib.sh\n+\n+test_expect_success 'setup' '\n+\ttest_commit base &&\n+\tgit clone -s --bare . fork &&\n+\tgit checkout -b public/branch master &&\n+\ttest_commit public &&\n+\tgit checkout -b private/branch master &&\n+\ttest_commit private\n+'\n+\n+extract_haves () {\n+\tdepacketize | perl -lne '/^(\\S+) \\.have/ and print $1'\n+}\n+\n+test_expect_success 'with core.alternateRefsCommand' '\n+\twrite_script fork/alternate-refs <<-\\EOF &&\n+\t\tgit --git-dir=\"$1\" for-each-ref \\\n+\t\t\t--format=\"%(objectname)\" \\\n+\t\t\trefs/heads/public/\n+\tEOF\n+\ttest_config -C fork core.alternateRefsCommand alternate-refs &&\n+\tgit rev-parse public/branch >expect &&\n+\tprintf \"0000\" | git receive-pack fork >actual &&\n+\textract_haves actual.haves &&\n+\ttest_cmp expect actual.haves\n+'\n+\n+test_done\ndiff --git a/transport.c b/transport.c\nindex 2825debac5..e271b66603 100644\n--- a/transport.c\n+++ b/transport.c\n@@ -1328,10 +1328,21 @@ char *transport_anonymize_url(const char *url)\n static void fill_alternate_refs_command(struct child_process *cmd,\n \t\t\t\t\tconst char *repo_path)\n {\n-\tcmd->git_cmd = 1;\n-\targv_array_pushf(&cmd->args, \"--git-dir=%s\", repo_path);\n-\targv_array_push(&cmd->args, \"for-each-ref\");\n-\targv_array_push(&cmd->args, \"--format=%(objectname)\");\n+\tconst char *value;\n+\n+\tif (!git_config_get_value(\"core.alternateRefsCommand\", &value)) {\n+\t\tcmd->use_shell = 1;\n+\n+\t\targv_array_push(&cmd->args, value);\n+\t\targv_array_push(&cmd->args, repo_path);\n+\t} else {\n+\t\tcmd->git_cmd = 1;\n+\n+\t\targv_array_pushf(&cmd->args, \"--git-dir=%s\", repo_path);\n+\t\targv_array_push(&cmd->args, \"for-each-ref\");\n+\t\targv_array_push(&cmd->args, \"--format=%(objectname)\");\n+\t}\n+\n \tcmd->env = local_repo_env;\n \tcmd->out = -1;\n }\n-- \n2.19.0.221.g150f307af\n\n"},{"id":"359402","messageId":"6119de15f2ba56e093c60a41990db740f3904d54.1538446827.git.me@ttaylorr.com","threadId":"49393","inReplyTo":"cover.1538446826.git.me@ttaylorr.com","subject":"[PATCH v4 2/4] transport.c: extract 'fill_alternate_refs_command'","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2018-10-02T02:23:56Z","receivedAt":"2018-10-02T02:24:04Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"To list alternate references, 'read_alternate_refs' creates a child\nprocess running 'git for-each-ref' in the alternate's Git directory.\n\nPrepare to run other commands besides 'git for-each-ref' by introducing\nand moving the relevant code from 'read_alternate_refs' to\n'fill_alternate_refs_command'.\n\nSigned-off-by: Taylor Blau \n---\n transport.c | 18 ++++++++++++------\n 1 file changed, 12 insertions(+), 6 deletions(-)\n\ndiff --git a/transport.c b/transport.c\nindex 2e0bc414d0..2825debac5 100644\n--- a/transport.c\n+++ b/transport.c\n@@ -1325,6 +1325,17 @@ char *transport_anonymize_url(const char *url)\n \treturn xstrdup(url);\n }\n \n+static void fill_alternate_refs_command(struct child_process *cmd,\n+\t\t\t\t\tconst char *repo_path)\n+{\n+\tcmd->git_cmd = 1;\n+\targv_array_pushf(&cmd->args, \"--git-dir=%s\", repo_path);\n+\targv_array_push(&cmd->args, \"for-each-ref\");\n+\targv_array_push(&cmd->args, \"--format=%(objectname)\");\n+\tcmd->env = local_repo_env;\n+\tcmd->out = -1;\n+}\n+\n static void read_alternate_refs(const char *path,\n \t\t\t\talternate_ref_fn *cb,\n \t\t\t\tvoid *data)\n@@ -1333,12 +1344,7 @@ static void read_alternate_refs(const char *path,\n \tstruct strbuf line = STRBUF_INIT;\n \tFILE *fh;\n \n-\tcmd.git_cmd = 1;\n-\targv_array_pushf(&cmd.args, \"--git-dir=%s\", path);\n-\targv_array_push(&cmd.args, \"for-each-ref\");\n-\targv_array_push(&cmd.args, \"--format=%(objectname)\");\n-\tcmd.env = local_repo_env;\n-\tcmd.out = -1;\n+\tfill_alternate_refs_command(&cmd, path);\n \n \tif (start_command(&cmd))\n \t\treturn;\n-- \n2.19.0.221.g150f307af\n\n"},{"id":"359403","messageId":"0d3521e92a55377d3e37ecd24ff7667dab50d61d.1538446827.git.me@ttaylorr.com","threadId":"49393","inReplyTo":"cover.1538446826.git.me@ttaylorr.com","subject":"[PATCH v4 4/4] transport.c: introduce core.alternateRefsPrefixes","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2018-10-02T02:24:01Z","receivedAt":"2018-10-02T02:24:13Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"The recently-introduced \"core.alternateRefsCommand\" allows callers to\nspecify with high flexibility the tips that they wish to advertise from\nalternates. This flexibility comes at the cost of some inconvenience\nwhen the caller only wishes to limit the advertisement to one or more\nprefixes.\n\nFor example, to advertise only tags, a caller using\n'core.alternateRefsCommand' would have to do:\n\n $ git config core.alternateRefsCommand ' \\\n f() { git -C \"$1\" for-each-ref \\\n refs/tags --format=\"%(objectname)\" }; f \"$@\"'\n\nThe above is cumbersome to write, so let's introduce a\n\"core.alternateRefsPrefixes\" to address this common case. Instead, the\ncaller can run:\n\n $ git config core.alternateRefsPrefixes 'refs/tags'\n\nWhich will behave identically to the longer example using\n\"core.alternateRefsCommand\".\n\nSince the value of \"core.alternateRefsPrefixes\" is appended to 'git\nfor-each-ref' and then executed, include a \"--\" before taking the\nconfigured value to avoid misinterpreting arguments as flags to 'git\nfor-each-ref'.\n\nIn the case that the caller wishes to specify multiple prefixes, they\nmay separate them by whitespace. If \"core.alternateRefsCommand\" is set,\nit will take precedence over \"core.alternateRefsPrefixes\".\n\nSigned-off-by: Taylor Blau \n---\n Documentation/config.txt | 7 +++++++\n t/t5410-receive-pack-alternates.sh | 8 ++++++++\n transport.c | 5 +++++\n 3 files changed, 20 insertions(+)\n\ndiff --git a/Documentation/config.txt b/Documentation/config.txt\nindex ac0577d288..1dc5eb3cfa 100644\n--- a/Documentation/config.txt\n+++ b/Documentation/config.txt\n@@ -632,6 +632,13 @@ Note that the configured value is executed in a shell, and thus\n linkgit:git-for-each-ref[1] by itself does not work, as scripts have to handle\n the path argument specially.\n \n+core.alternateRefsPrefixes::\n+\tWhen listing references from an alternate, list only references that begin\n+\twith the given prefix. Prefixes match as if they were given as arguments to\n+\tlinkgit:git-for-each-ref[1]. To list multiple prefixes, separate them with\n+\twhitespace. If `core.alternateRefsCommand` is set, setting\n+\t`core.alternateRefsPrefixes` has no effect.\n+\n core.bare::\n \tIf true this repository is assumed to be 'bare' and has no\n \tworking directory associated with it. If this is the case a\ndiff --git a/t/t5410-receive-pack-alternates.sh b/t/t5410-receive-pack-alternates.sh\nindex 49d0fe44fb..94794c35da 100755\n--- a/t/t5410-receive-pack-alternates.sh\n+++ b/t/t5410-receive-pack-alternates.sh\n@@ -30,4 +30,12 @@ test_expect_success 'with core.alternateRefsCommand' '\n \ttest_cmp expect actual.haves\n '\n \n+test_expect_success 'with core.alternateRefsPrefixes' '\n+\ttest_config -C fork core.alternateRefsPrefixes \"refs/heads/private\" &&\n+\tgit rev-parse private/branch expect &&\n+\tprintf \"0000\" | git receive-pack fork >actual &&\n+\textract_haves actual.haves &&\n+\ttest_cmp expect actual.haves\n+'\n+\n test_done\ndiff --git a/transport.c b/transport.c\nindex e271b66603..83474add28 100644\n--- a/transport.c\n+++ b/transport.c\n@@ -1341,6 +1341,11 @@ static void fill_alternate_refs_command(struct child_process *cmd,\n \t\targv_array_pushf(&cmd->args, \"--git-dir=%s\", repo_path);\n \t\targv_array_push(&cmd->args, \"for-each-ref\");\n \t\targv_array_push(&cmd->args, \"--format=%(objectname)\");\n+\n+\t\tif (!git_config_get_value(\"core.alternateRefsPrefixes\", &value)) {\n+\t\t\targv_array_push(&cmd->args, \"--\");\n+\t\t\targv_array_split(&cmd->args, value);\n+\t\t}\n \t}\n \n \tcmd->env = local_repo_env;\n-- \n2.19.0.221.g150f307af\n"},{"id":"359428","messageId":"aeee4d8f-b224-2b1b-40fb-d484dbaabae3@ramsayjones.plus.com","threadId":"49393","inReplyTo":"0d3521e92a55377d3e37ecd24ff7667dab50d61d.1538446827.git.me@ttaylorr.com","subject":"Re: [PATCH v4 4/4] transport.c: introduce core.alternateRefsPrefixes","fromName":"Ramsay Jones","fromEmail":"ramsay@ramsayjones.plus.com","sentAt":"2018-10-02T15:13:13Z","receivedAt":"2018-10-02T15:13:17Z","isPatch":true,"sender":{"key":"ramsay@ramsayjones.plus.com","avatar":"https://avatars.githubusercontent.com/u/33702710?v=4"},"body":"\n\nOn 02/10/18 03:24, Taylor Blau wrote:\n[snip]\n> diff --git a/t/t5410-receive-pack-alternates.sh b/t/t5410-receive-pack-alternates.sh\n> index 49d0fe44fb..94794c35da 100755\n> --- a/t/t5410-receive-pack-alternates.sh\n> +++ b/t/t5410-receive-pack-alternates.sh\n> @@ -30,4 +30,12 @@ test_expect_success 'with core.alternateRefsCommand' '\n> \ttest_cmp expect actual.haves\n> '\n> \n> +test_expect_success 'with core.alternateRefsPrefixes' '\n> +\ttest_config -C fork core.alternateRefsPrefixes \"refs/heads/private\" &&\n> +\tgit rev-parse private/branch expect &&\n\ns/expect/>expect/ ?\n\nATB,\nRamsay Jones\n\n> +\tprintf \"0000\" | git receive-pack fork >actual &&\n> +\textract_haves actual.haves &&\n> +\ttest_cmp expect actual.haves\n> +'\n> +\n> test_done\n> diff --git a/transport.c b/transport.c\n> index e271b66603..83474add28 100644\n> --- a/transport.c\n> +++ b/transport.c\n> @@ -1341,6 +1341,11 @@ static void fill_alternate_refs_command(struct child_process *cmd,\n> \t\targv_array_pushf(&cmd->args, \"--git-dir=%s\", repo_path);\n> \t\targv_array_push(&cmd->args, \"for-each-ref\");\n> \t\targv_array_push(&cmd->args, \"--format=%(objectname)\");\n> +\n> +\t\tif (!git_config_get_value(\"core.alternateRefsPrefixes\", &value)) {\n> +\t\t\targv_array_push(&cmd->args, \"--\");\n> +\t\t\targv_array_split(&cmd->args, value);\n> +\t\t}\n> \t}\n> \n> \tcmd->env = local_repo_env;\n> \n"},{"id":"359484","messageId":"20181002232840.GA31540@syl","threadId":"49393","inReplyTo":"aeee4d8f-b224-2b1b-40fb-d484dbaabae3@ramsayjones.plus.com","subject":"[PATCH v4 4/4] transport.c: introduce core.alternateRefsPrefixes","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2018-10-02T23:28:40Z","receivedAt":"2018-10-02T23:28:54Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"On Tue, Oct 02, 2018 at 04:13:13PM +0100, Ramsay Jones wrote:\n>\n> On 02/10/18 03:24, Taylor Blau wrote:\n> [snip]\n> > diff --git a/t/t5410-receive-pack-alternates.sh b/t/t5410-receive-pack-alternates.sh\n> > index 49d0fe44fb..94794c35da 100755\n> > --- a/t/t5410-receive-pack-alternates.sh\n> > +++ b/t/t5410-receive-pack-alternates.sh\n> > @@ -30,4 +30,12 @@ test_expect_success 'with core.alternateRefsCommand' '\n> > \ttest_cmp expect actual.haves\n> > '\n> >\n> > +test_expect_success 'with core.alternateRefsPrefixes' '\n> > +\ttest_config -C fork core.alternateRefsPrefixes \"refs/heads/private\" &&\n> > +\tgit rev-parse private/branch expect &&\n>\n> s/expect/>expect/ ?\n\nAh, certainly. Thanks for catching my mistake. I've resent 4/4 as below.\n\nJunio -- if you find this re-roll to be acceptable, please queue this\npatch instead of the one that it is in reply to.\n\n-- >8 --\n\nThe recently-introduced \"core.alternateRefsCommand\" allows callers to\nspecify with high flexibility the tips that they wish to advertise from\nalternates. This flexibility comes at the cost of some inconvenience\nwhen the caller only wishes to limit the advertisement to one or more\nprefixes.\n\nFor example, to advertise only tags, a caller using\n'core.alternateRefsCommand' would have to do:\n\n $ git config core.alternateRefsCommand ' \\\n f() { git -C \"$1\" for-each-ref \\\n refs/tags --format=\"%(objectname)\" }; f \"$@\"'\n\nThe above is cumbersome to write, so let's introduce a\n\"core.alternateRefsPrefixes\" to address this common case. Instead, the\ncaller can run:\n\n $ git config core.alternateRefsPrefixes 'refs/tags'\n\nWhich will behave identically to the longer example using\n\"core.alternateRefsCommand\".\n\nSince the value of \"core.alternateRefsPrefixes\" is appended to 'git\nfor-each-ref' and then executed, include a \"--\" before taking the\nconfigured value to avoid misinterpreting arguments as flags to 'git\nfor-each-ref'.\n\nIn the case that the caller wishes to specify multiple prefixes, they\nmay separate them by whitespace. If \"core.alternateRefsCommand\" is set,\nit will take precedence over \"core.alternateRefsPrefixes\".\n\nSigned-off-by: Taylor Blau \n---\n Documentation/config.txt | 7 +++++++\n t/t5410-receive-pack-alternates.sh | 8 ++++++++\n transport.c | 5 +++++\n 3 files changed, 20 insertions(+)\n\ndiff --git a/Documentation/config.txt b/Documentation/config.txt\nindex ac0577d288..1dc5eb3cfa 100644\n--- a/Documentation/config.txt\n+++ b/Documentation/config.txt\n@@ -632,6 +632,13 @@ Note that the configured value is executed in a shell, and thus\n linkgit:git-for-each-ref[1] by itself does not work, as scripts have to handle\n the path argument specially.\n\n+core.alternateRefsPrefixes::\n+\tWhen listing references from an alternate, list only references that begin\n+\twith the given prefix. Prefixes match as if they were given as arguments to\n+\tlinkgit:git-for-each-ref[1]. To list multiple prefixes, separate them with\n+\twhitespace. If `core.alternateRefsCommand` is set, setting\n+\t`core.alternateRefsPrefixes` has no effect.\n+\n core.bare::\n \tIf true this repository is assumed to be 'bare' and has no\n \tworking directory associated with it. If this is the case a\ndiff --git a/t/t5410-receive-pack-alternates.sh b/t/t5410-receive-pack-alternates.sh\nindex 49d0fe44fb..457c20c2a5 100755\n--- a/t/t5410-receive-pack-alternates.sh\n+++ b/t/t5410-receive-pack-alternates.sh\n@@ -30,4 +30,12 @@ test_expect_success 'with core.alternateRefsCommand' '\n \ttest_cmp expect actual.haves\n '\n\n+test_expect_success 'with core.alternateRefsPrefixes' '\n+\ttest_config -C fork core.alternateRefsPrefixes \"refs/heads/private\" &&\n+\tgit rev-parse private/branch >expect &&\n+\tprintf \"0000\" | git receive-pack fork >actual &&\n+\textract_haves actual.haves &&\n+\ttest_cmp expect actual.haves\n+'\n+\n test_done\ndiff --git a/transport.c b/transport.c\nindex e271b66603..83474add28 100644\n--- a/transport.c\n+++ b/transport.c\n@@ -1341,6 +1341,11 @@ static void fill_alternate_refs_command(struct child_process *cmd,\n \t\targv_array_pushf(&cmd->args, \"--git-dir=%s\", repo_path);\n \t\targv_array_push(&cmd->args, \"for-each-ref\");\n \t\targv_array_push(&cmd->args, \"--format=%(objectname)\");\n+\n+\t\tif (!git_config_get_value(\"core.alternateRefsPrefixes\", &value)) {\n+\t\t\targv_array_push(&cmd->args, \"--\");\n+\t\t\targv_array_split(&cmd->args, value);\n+\t\t}\n \t}\n\n \tcmd->env = local_repo_env;\n--\n2.19.0.221.g150f307af\n"},{"id":"359485","messageId":"20181002234056.GA18516@sigill.intra.peff.net","threadId":"49393","inReplyTo":"aadb27c0106d8f1a49dd35e7a040131aecaef2c1.1538446827.git.me@ttaylorr.com","subject":"Re: [PATCH v4 3/4] transport.c: introduce core.alternateRefsCommand","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2018-10-02T23:40:56Z","receivedAt":"2018-10-02T23:41:07Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Mon, Oct 01, 2018 at 07:23:58PM -0700, Taylor Blau wrote:\n\n> +core.alternateRefsCommand::\n> +\tWhen advertising tips of available history from an alternate, use the shell to\n> +\texecute the specified command instead of linkgit:git-for-each-ref[1]. The\n> +\tfirst argument is the absolute path of the alternate. Output must contain one\n> +\thex object id per line (i.e., the same as produce by `git for-each-ref\n> +\t--format='%(objectname)'`).\n> ++\n> +This is useful when a repository only wishes to advertise some of its\n> +alternate's references as `.have`'s. For example, to only advertise branch\n> +heads, configure `core.alternateRefsCommand` to the path of a script which runs\n> +`git --git-dir=\"$1\" for-each-ref --format='%(objectname)' refs/heads`.\n> ++\n> +Note that the configured value is executed in a shell, and thus\n> +linkgit:git-for-each-ref[1] by itself does not work, as scripts have to handle\n> +the path argument specially.\n\nThis last paragraph is trying to fix the wrong-impression that we\ndiscussed in the last round. But I'm not sure it doesn't make things\nmore confusing. ;)\n\nSpecifically, the problem isn't the shell. The issue is that we pass the\nrepo path as an argument to the command. So either:\n\n - it's a real command that we run, in which case git-for-each-ref does\n not take a repo path argument and so doesn't work; or\n\n - it's a shell snippet, in which case the argument is appended to the\n snippet (and here's where you can get into a rabbit hole of\n explaining how our shell invocation works, and we should avoid that)\n\nCan we just say:\n\n Note that you cannot generally put `git for-each-ref` directly into\n the config value, as it does not take a repository path as an argument\n (but you can wrap the command above in a shell script).\n\n> [...]\n\nThe rest of the patch looks good to me, along with the other three\n(modulo the \"expect\" fixup you already sent).\n\n-Peff\n"},{"id":"359600","messageId":"20181004021757.GA67785@syl","threadId":"49393","inReplyTo":"20181002234056.GA18516@sigill.intra.peff.net","subject":"Re: [PATCH v4 3/4] transport.c: introduce core.alternateRefsCommand","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2018-10-04T02:17:57Z","receivedAt":"2018-10-04T02:18:02Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"On Tue, Oct 02, 2018 at 07:40:56PM -0400, Jeff King wrote:\n> On Mon, Oct 01, 2018 at 07:23:58PM -0700, Taylor Blau wrote:\n>\n> > +core.alternateRefsCommand::\n> > +\tWhen advertising tips of available history from an alternate, use the shell to\n> > +\texecute the specified command instead of linkgit:git-for-each-ref[1]. The\n> > +\tfirst argument is the absolute path of the alternate. Output must contain one\n> > +\thex object id per line (i.e., the same as produce by `git for-each-ref\n> > +\t--format='%(objectname)'`).\n> > ++\n> > +This is useful when a repository only wishes to advertise some of its\n> > +alternate's references as `.have`'s. For example, to only advertise branch\n> > +heads, configure `core.alternateRefsCommand` to the path of a script which runs\n> > +`git --git-dir=\"$1\" for-each-ref --format='%(objectname)' refs/heads`.\n> > ++\n> > +Note that the configured value is executed in a shell, and thus\n> > +linkgit:git-for-each-ref[1] by itself does not work, as scripts have to handle\n> > +the path argument specially.\n>\n> This last paragraph is trying to fix the wrong-impression that we\n> discussed in the last round. But I'm not sure it doesn't make things\n> more confusing. ;)\n\nHeh, point taken. I suppose that I won't try to ignore your feedback\nhere!\n\n> Specifically, the problem isn't the shell. The issue is that we pass the\n> repo path as an argument to the command. So either:\n>\n> - it's a real command that we run, in which case git-for-each-ref does\n> not take a repo path argument and so doesn't work; or\n>\n> - it's a shell snippet, in which case the argument is appended to the\n> snippet (and here's where you can get into a rabbit hole of\n> explaining how our shell invocation works, and we should avoid that)\n>\n> Can we just say:\n>\n> Note that you cannot generally put `git for-each-ref` directly into\n> the config value, as it does not take a repository path as an argument\n> (but you can wrap the command above in a shell script).\n>\n> > [...]\n\nYeah, I think that this is certainly the way to go. I took your\nsuggestion as-is, which I think is much clearer than what I wrote.\nThanks!\n\n> The rest of the patch looks good to me, along with the other three\n> (modulo the \"expect\" fixup you already sent).\n\nThanks for your review, as always :-). I certainly appreciate your\npatience with the word-smithing and whatnot.\n\nJunio, I applied Peff's suggestion directly into my local copy, so I'm\nhappy to do either of a couple things, depending on which would be\neasiest for you to pick up. I could either:\n\n 1. Re-send this patch (in addition to 4/4), or\n\n 2. Re-roll the entire series (with this and 4/4 amended to reflect the\n two bits of feedback I've gotten since sending v4).\n\nI imagine that the later will be easier for you to deal with, instead of\nmanually picking up patch amendments, but if you'd like fewer email,\nthat works too :-).\n\nThanks,\nTaylor\n"},{"id":"359841","messageId":"cover.1539021825.git.me@ttaylorr.com","threadId":"49393","inReplyTo":"cover.1537466087.git.me@ttaylorr.com","subject":"[PATCH v5 0/4] Filter alternate references","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2018-10-08T18:09:20Z","receivedAt":"2018-10-08T18:09:26Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"Hi,\n\nAttached is (what I anticipate to be) the final re-roll of my series to\nintroduce 'core.alternateRefsCommand' and 'core.alternateRefsPrefixes'\nin order to limit the \".have\" advertisement when pushing over protocol\nv1 to a repository with configured alternates.\n\nNot much has changed from last time, expect for:\n\n - Taking a documentation suggestion from Peff (in 3/4), and\n\n - Fixing a typo pointed out by Ramsay (in 4/4).\n\nI believe that this series is otherwise ready for queueing, if everyone\nelse feels sufficiently OK about the changes.\n\nThanks in advance for your review.\n\nThanks,\nTaylor\n\nJeff King (1):\n transport: drop refnames from for_each_alternate_ref\n\nTaylor Blau (3):\n transport.c: extract 'fill_alternate_refs_command'\n transport.c: introduce core.alternateRefsCommand\n transport.c: introduce core.alternateRefsPrefixes\n\n Documentation/config.txt | 18 +++++++++++++\n builtin/receive-pack.c | 3 +--\n fetch-pack.c | 3 +--\n t/t5410-receive-pack-alternates.sh | 41 ++++++++++++++++++++++++++++++\n transport.c | 38 +++++++++++++++++++++------\n transport.h | 2 +-\n 6 files changed, 92 insertions(+), 13 deletions(-)\n create mode 100755 t/t5410-receive-pack-alternates.sh\n\nRange-diff against v4:\n1: 76482a7eba = 1: e4947f557b transport: drop refnames from for_each_alternate_ref\n2: 120df009df = 2: 3d77a46c61 transport.c: extract 'fill_alternate_refs_command'\n3: c63864c89a ! 3: 7451b4872a transport.c: introduce core.alternateRefsCommand\n @@ -42,14 +42,9 @@\n +\thex object id per line (i.e., the same as produce by `git for-each-ref\n +\t--format='%(objectname)'`).\n ++\n -+This is useful when a repository only wishes to advertise some of its\n -+alternate's references as `.have`'s. For example, to only advertise branch\n -+heads, configure `core.alternateRefsCommand` to the path of a script which runs\n -+`git --git-dir=\"$1\" for-each-ref --format='%(objectname)' refs/heads`.\n -++\n -+Note that the configured value is executed in a shell, and thus\n -+linkgit:git-for-each-ref[1] by itself does not work, as scripts have to handle\n -+the path argument specially.\n ++Note that you cannot generally put `git for-each-ref` directly into the config\n ++value, as it does not take a repository path as an argument (but you can wrap\n ++the command above in a shell script).\n +\n core.bare::\n \tIf true this repository is assumed to be 'bare' and has no\n4: 0f6cdc7ea4 ! 4: 28cbbe63f7 transport.c: introduce core.alternateRefsPrefixes\n @@ -39,8 +39,8 @@\n --- a/Documentation/config.txt\n +++ b/Documentation/config.txt\n @@\n - linkgit:git-for-each-ref[1] by itself does not work, as scripts have to handle\n - the path argument specially.\n + value, as it does not take a repository path as an argument (but you can wrap\n + the command above in a shell script).\n\n +core.alternateRefsPrefixes::\n +\tWhen listing references from an alternate, list only references that begin\n @@ -62,7 +62,7 @@\n\n +test_expect_success 'with core.alternateRefsPrefixes' '\n +\ttest_config -C fork core.alternateRefsPrefixes \"refs/heads/private\" &&\n -+\tgit rev-parse private/branch expect &&\n ++\tgit rev-parse private/branch >expect &&\n +\tprintf \"0000\" | git receive-pack fork >actual &&\n +\textract_haves actual.haves &&\n +\ttest_cmp expect actual.haves\n--\n2.19.0.221.g150f307af\n"},{"id":"359842","messageId":"e4947f557b2eab84047adde62d3a79e657e664d5.1539021825.git.me@ttaylorr.com","threadId":"49393","inReplyTo":"cover.1539021825.git.me@ttaylorr.com","subject":"[PATCH v5 1/4] transport: drop refnames from for_each_alternate_ref","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2018-10-08T18:09:23Z","receivedAt":"2018-10-08T18:09:28Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"From: Jeff King \n\nNone of the current callers use the refname parameter we pass to their\ncallbacks. In theory somebody _could_ do so, but it's actually quite\nweird if you think about it: it's a ref in somebody else's repository.\nSo the name has no meaning locally, and in fact there may be duplicates\nif there are multiple alternates.\n\nThe users of this interface really only care about seeing some ref tips,\nsince that promises that the alternate has the full commit graph\nreachable from there. So let's keep the information we pass back to the\nbare minimum.\n\nSigned-off-by: Jeff King \nSigned-off-by: Taylor Blau \n---\n builtin/receive-pack.c | 3 +--\n fetch-pack.c | 3 +--\n transport.c | 6 +++---\n transport.h | 2 +-\n 4 files changed, 6 insertions(+), 8 deletions(-)\n\ndiff --git a/builtin/receive-pack.c b/builtin/receive-pack.c\nindex 4d30001950..6792291f5e 100644\n--- a/builtin/receive-pack.c\n+++ b/builtin/receive-pack.c\n@@ -281,8 +281,7 @@ static int show_ref_cb(const char *path_full, const struct object_id *oid,\n \treturn 0;\n }\n \n-static void show_one_alternate_ref(const char *refname,\n-\t\t\t\t const struct object_id *oid,\n+static void show_one_alternate_ref(const struct object_id *oid,\n \t\t\t\t void *data)\n {\n \tstruct oidset *seen = data;\ndiff --git a/fetch-pack.c b/fetch-pack.c\nindex 75047a4b2a..b643de143b 100644\n--- a/fetch-pack.c\n+++ b/fetch-pack.c\n@@ -76,8 +76,7 @@ struct alternate_object_cache {\n \tsize_t nr, alloc;\n };\n \n-static void cache_one_alternate(const char *refname,\n-\t\t\t\tconst struct object_id *oid,\n+static void cache_one_alternate(const struct object_id *oid,\n \t\t\t\tvoid *vcache)\n {\n \tstruct alternate_object_cache *cache = vcache;\ndiff --git a/transport.c b/transport.c\nindex 1c76d64aba..2e0bc414d0 100644\n--- a/transport.c\n+++ b/transport.c\n@@ -1336,7 +1336,7 @@ static void read_alternate_refs(const char *path,\n \tcmd.git_cmd = 1;\n \targv_array_pushf(&cmd.args, \"--git-dir=%s\", path);\n \targv_array_push(&cmd.args, \"for-each-ref\");\n-\targv_array_push(&cmd.args, \"--format=%(objectname) %(refname)\");\n+\targv_array_push(&cmd.args, \"--format=%(objectname)\");\n \tcmd.env = local_repo_env;\n \tcmd.out = -1;\n \n@@ -1348,13 +1348,13 @@ static void read_alternate_refs(const char *path,\n \t\tstruct object_id oid;\n \n \t\tif (get_oid_hex(line.buf, &oid) ||\n-\t\t line.buf[GIT_SHA1_HEXSZ] != ' ') {\n+\t\t line.buf[GIT_SHA1_HEXSZ]) {\n \t\t\twarning(_(\"invalid line while parsing alternate refs: %s\"),\n \t\t\t\tline.buf);\n \t\t\tbreak;\n \t\t}\n \n-\t\tcb(line.buf + GIT_SHA1_HEXSZ + 1, &oid, data);\n+\t\tcb(&oid, data);\n \t}\n \n \tfclose(fh);\ndiff --git a/transport.h b/transport.h\nindex 01e717c29e..9baeca2d7a 100644\n--- a/transport.h\n+++ b/transport.h\n@@ -261,6 +261,6 @@ int transport_refs_pushed(struct ref *ref);\n void transport_print_push_status(const char *dest, struct ref *refs,\n \t\t int verbose, int porcelain, unsigned int *reject_reasons);\n \n-typedef void alternate_ref_fn(const char *refname, const struct object_id *oid, void *);\n+typedef void alternate_ref_fn(const struct object_id *oid, void *);\n extern void for_each_alternate_ref(alternate_ref_fn, void *);\n #endif\n-- \n2.19.0.221.g150f307af\n\n"},{"id":"359843","messageId":"3d77a46c613c1c3d7f1cf161ac12ff6477430958.1539021825.git.me@ttaylorr.com","threadId":"49393","inReplyTo":"cover.1539021825.git.me@ttaylorr.com","subject":"[PATCH v5 2/4] transport.c: extract 'fill_alternate_refs_command'","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2018-10-08T18:09:26Z","receivedAt":"2018-10-08T18:09:30Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"To list alternate references, 'read_alternate_refs' creates a child\nprocess running 'git for-each-ref' in the alternate's Git directory.\n\nPrepare to run other commands besides 'git for-each-ref' by introducing\nand moving the relevant code from 'read_alternate_refs' to\n'fill_alternate_refs_command'.\n\nSigned-off-by: Taylor Blau \n---\n transport.c | 18 ++++++++++++------\n 1 file changed, 12 insertions(+), 6 deletions(-)\n\ndiff --git a/transport.c b/transport.c\nindex 2e0bc414d0..2825debac5 100644\n--- a/transport.c\n+++ b/transport.c\n@@ -1325,6 +1325,17 @@ char *transport_anonymize_url(const char *url)\n \treturn xstrdup(url);\n }\n \n+static void fill_alternate_refs_command(struct child_process *cmd,\n+\t\t\t\t\tconst char *repo_path)\n+{\n+\tcmd->git_cmd = 1;\n+\targv_array_pushf(&cmd->args, \"--git-dir=%s\", repo_path);\n+\targv_array_push(&cmd->args, \"for-each-ref\");\n+\targv_array_push(&cmd->args, \"--format=%(objectname)\");\n+\tcmd->env = local_repo_env;\n+\tcmd->out = -1;\n+}\n+\n static void read_alternate_refs(const char *path,\n \t\t\t\talternate_ref_fn *cb,\n \t\t\t\tvoid *data)\n@@ -1333,12 +1344,7 @@ static void read_alternate_refs(const char *path,\n \tstruct strbuf line = STRBUF_INIT;\n \tFILE *fh;\n \n-\tcmd.git_cmd = 1;\n-\targv_array_pushf(&cmd.args, \"--git-dir=%s\", path);\n-\targv_array_push(&cmd.args, \"for-each-ref\");\n-\targv_array_push(&cmd.args, \"--format=%(objectname)\");\n-\tcmd.env = local_repo_env;\n-\tcmd.out = -1;\n+\tfill_alternate_refs_command(&cmd, path);\n \n \tif (start_command(&cmd))\n \t\treturn;\n-- \n2.19.0.221.g150f307af\n\n"},{"id":"359844","messageId":"7451b4872a0fd66d84cbe492fdfe7a9a8e81eab7.1539021825.git.me@ttaylorr.com","threadId":"49393","inReplyTo":"cover.1539021825.git.me@ttaylorr.com","subject":"[PATCH v5 3/4] transport.c: introduce core.alternateRefsCommand","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2018-10-08T18:09:28Z","receivedAt":"2018-10-08T18:09:32Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"When in a repository containing one or more alternates, Git would\nsometimes like to list references from those alternates. For example,\n'git receive-pack' lists the \"tips\" pointed to by references in those\nalternates as special \".have\" references.\n\nListing \".have\" references is designed to make pushing changes from\nupstream to a fork a lightweight operation, by advertising to the pusher\nthat the fork already has the objects (via its alternate). Thus, the\nclient can avoid sending them.\n\nHowever, when the alternate (upstream, in the previous example) has a\npathologically large number of references, the initial advertisement is\ntoo expensive. In fact, it can dominate any such optimization where the\npusher avoids sending certain objects.\n\nIntroduce \"core.alternateRefsCommand\" in order to provide a facility to\nlimit or filter alternate references. This can be used, for example, to\nfilter out references the alternate does not wish to send (for space\nconcerns, or otherwise) during the initial advertisement.\n\nLet the repository that has alternates configure this command to avoid\ntrusting the alternate to provide us a safe command to run in the shell.\nTo find the alternate, pass its absolute path as the first argument.\n\nSigned-off-by: Taylor Blau \n---\n Documentation/config.txt | 11 ++++++++++\n t/t5410-receive-pack-alternates.sh | 33 ++++++++++++++++++++++++++++++\n transport.c | 19 +++++++++++++----\n 3 files changed, 59 insertions(+), 4 deletions(-)\n create mode 100755 t/t5410-receive-pack-alternates.sh\n\ndiff --git a/Documentation/config.txt b/Documentation/config.txt\nindex ad0f4510c3..c51e82d8a5 100644\n--- a/Documentation/config.txt\n+++ b/Documentation/config.txt\n@@ -616,6 +616,17 @@ core.preferSymlinkRefs::\n \tThis is sometimes needed to work with old scripts that\n \texpect HEAD to be a symbolic link.\n \n+core.alternateRefsCommand::\n+\tWhen advertising tips of available history from an alternate, use the shell to\n+\texecute the specified command instead of linkgit:git-for-each-ref[1]. The\n+\tfirst argument is the absolute path of the alternate. Output must contain one\n+\thex object id per line (i.e., the same as produce by `git for-each-ref\n+\t--format='%(objectname)'`).\n++\n+Note that you cannot generally put `git for-each-ref` directly into the config\n+value, as it does not take a repository path as an argument (but you can wrap\n+the command above in a shell script).\n+\n core.bare::\n \tIf true this repository is assumed to be 'bare' and has no\n \tworking directory associated with it. If this is the case a\ndiff --git a/t/t5410-receive-pack-alternates.sh b/t/t5410-receive-pack-alternates.sh\nnew file mode 100755\nindex 0000000000..49d0fe44fb\n--- /dev/null\n+++ b/t/t5410-receive-pack-alternates.sh\n@@ -0,0 +1,33 @@\n+#!/bin/sh\n+\n+test_description='git receive-pack with alternate ref filtering'\n+\n+. ./test-lib.sh\n+\n+test_expect_success 'setup' '\n+\ttest_commit base &&\n+\tgit clone -s --bare . fork &&\n+\tgit checkout -b public/branch master &&\n+\ttest_commit public &&\n+\tgit checkout -b private/branch master &&\n+\ttest_commit private\n+'\n+\n+extract_haves () {\n+\tdepacketize | perl -lne '/^(\\S+) \\.have/ and print $1'\n+}\n+\n+test_expect_success 'with core.alternateRefsCommand' '\n+\twrite_script fork/alternate-refs <<-\\EOF &&\n+\t\tgit --git-dir=\"$1\" for-each-ref \\\n+\t\t\t--format=\"%(objectname)\" \\\n+\t\t\trefs/heads/public/\n+\tEOF\n+\ttest_config -C fork core.alternateRefsCommand alternate-refs &&\n+\tgit rev-parse public/branch >expect &&\n+\tprintf \"0000\" | git receive-pack fork >actual &&\n+\textract_haves actual.haves &&\n+\ttest_cmp expect actual.haves\n+'\n+\n+test_done\ndiff --git a/transport.c b/transport.c\nindex 2825debac5..e271b66603 100644\n--- a/transport.c\n+++ b/transport.c\n@@ -1328,10 +1328,21 @@ char *transport_anonymize_url(const char *url)\n static void fill_alternate_refs_command(struct child_process *cmd,\n \t\t\t\t\tconst char *repo_path)\n {\n-\tcmd->git_cmd = 1;\n-\targv_array_pushf(&cmd->args, \"--git-dir=%s\", repo_path);\n-\targv_array_push(&cmd->args, \"for-each-ref\");\n-\targv_array_push(&cmd->args, \"--format=%(objectname)\");\n+\tconst char *value;\n+\n+\tif (!git_config_get_value(\"core.alternateRefsCommand\", &value)) {\n+\t\tcmd->use_shell = 1;\n+\n+\t\targv_array_push(&cmd->args, value);\n+\t\targv_array_push(&cmd->args, repo_path);\n+\t} else {\n+\t\tcmd->git_cmd = 1;\n+\n+\t\targv_array_pushf(&cmd->args, \"--git-dir=%s\", repo_path);\n+\t\targv_array_push(&cmd->args, \"for-each-ref\");\n+\t\targv_array_push(&cmd->args, \"--format=%(objectname)\");\n+\t}\n+\n \tcmd->env = local_repo_env;\n \tcmd->out = -1;\n }\n-- \n2.19.0.221.g150f307af\n\n"},{"id":"359845","messageId":"28cbbe63f73b14315f8b1b7c7049b1f81f27057f.1539021825.git.me@ttaylorr.com","threadId":"49393","inReplyTo":"cover.1539021825.git.me@ttaylorr.com","subject":"[PATCH v5 4/4] transport.c: introduce core.alternateRefsPrefixes","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2018-10-08T18:09:30Z","receivedAt":"2018-10-08T18:09:34Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"The recently-introduced \"core.alternateRefsCommand\" allows callers to\nspecify with high flexibility the tips that they wish to advertise from\nalternates. This flexibility comes at the cost of some inconvenience\nwhen the caller only wishes to limit the advertisement to one or more\nprefixes.\n\nFor example, to advertise only tags, a caller using\n'core.alternateRefsCommand' would have to do:\n\n $ git config core.alternateRefsCommand ' \\\n f() { git -C \"$1\" for-each-ref \\\n refs/tags --format=\"%(objectname)\" }; f \"$@\"'\n\nThe above is cumbersome to write, so let's introduce a\n\"core.alternateRefsPrefixes\" to address this common case. Instead, the\ncaller can run:\n\n $ git config core.alternateRefsPrefixes 'refs/tags'\n\nWhich will behave identically to the longer example using\n\"core.alternateRefsCommand\".\n\nSince the value of \"core.alternateRefsPrefixes\" is appended to 'git\nfor-each-ref' and then executed, include a \"--\" before taking the\nconfigured value to avoid misinterpreting arguments as flags to 'git\nfor-each-ref'.\n\nIn the case that the caller wishes to specify multiple prefixes, they\nmay separate them by whitespace. If \"core.alternateRefsCommand\" is set,\nit will take precedence over \"core.alternateRefsPrefixes\".\n\nSigned-off-by: Taylor Blau \n---\n Documentation/config.txt | 7 +++++++\n t/t5410-receive-pack-alternates.sh | 8 ++++++++\n transport.c | 5 +++++\n 3 files changed, 20 insertions(+)\n\ndiff --git a/Documentation/config.txt b/Documentation/config.txt\nindex c51e82d8a5..a133a709f3 100644\n--- a/Documentation/config.txt\n+++ b/Documentation/config.txt\n@@ -627,6 +627,13 @@ Note that you cannot generally put `git for-each-ref` directly into the config\n value, as it does not take a repository path as an argument (but you can wrap\n the command above in a shell script).\n \n+core.alternateRefsPrefixes::\n+\tWhen listing references from an alternate, list only references that begin\n+\twith the given prefix. Prefixes match as if they were given as arguments to\n+\tlinkgit:git-for-each-ref[1]. To list multiple prefixes, separate them with\n+\twhitespace. If `core.alternateRefsCommand` is set, setting\n+\t`core.alternateRefsPrefixes` has no effect.\n+\n core.bare::\n \tIf true this repository is assumed to be 'bare' and has no\n \tworking directory associated with it. If this is the case a\ndiff --git a/t/t5410-receive-pack-alternates.sh b/t/t5410-receive-pack-alternates.sh\nindex 49d0fe44fb..457c20c2a5 100755\n--- a/t/t5410-receive-pack-alternates.sh\n+++ b/t/t5410-receive-pack-alternates.sh\n@@ -30,4 +30,12 @@ test_expect_success 'with core.alternateRefsCommand' '\n \ttest_cmp expect actual.haves\n '\n \n+test_expect_success 'with core.alternateRefsPrefixes' '\n+\ttest_config -C fork core.alternateRefsPrefixes \"refs/heads/private\" &&\n+\tgit rev-parse private/branch >expect &&\n+\tprintf \"0000\" | git receive-pack fork >actual &&\n+\textract_haves actual.haves &&\n+\ttest_cmp expect actual.haves\n+'\n+\n test_done\ndiff --git a/transport.c b/transport.c\nindex e271b66603..83474add28 100644\n--- a/transport.c\n+++ b/transport.c\n@@ -1341,6 +1341,11 @@ static void fill_alternate_refs_command(struct child_process *cmd,\n \t\targv_array_pushf(&cmd->args, \"--git-dir=%s\", repo_path);\n \t\targv_array_push(&cmd->args, \"for-each-ref\");\n \t\targv_array_push(&cmd->args, \"--format=%(objectname)\");\n+\n+\t\tif (!git_config_get_value(\"core.alternateRefsPrefixes\", &value)) {\n+\t\t\targv_array_push(&cmd->args, \"--\");\n+\t\t\targv_array_split(&cmd->args, value);\n+\t\t}\n \t}\n \n \tcmd->env = local_repo_env;\n-- \n2.19.0.221.g150f307af\n"},{"id":"359890","messageId":"20181009030918.GB6250@sigill.intra.peff.net","threadId":"49393","inReplyTo":"cover.1539021825.git.me@ttaylorr.com","subject":"Re: [PATCH v5 0/4] Filter alternate references","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2018-10-09T03:09:18Z","receivedAt":"2018-10-09T03:09:21Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Mon, Oct 08, 2018 at 11:09:20AM -0700, Taylor Blau wrote:\n\n> Attached is (what I anticipate to be) the final re-roll of my series to\n> introduce 'core.alternateRefsCommand' and 'core.alternateRefsPrefixes'\n> in order to limit the \".have\" advertisement when pushing over protocol\n> v1 to a repository with configured alternates.\n\nThanks, this looks good to me!\n\n-Peff\n"},{"id":"359927","messageId":"20181009144954.GA12981@syl","threadId":"49393","inReplyTo":"20181009030918.GB6250@sigill.intra.peff.net","subject":"Re: [PATCH v5 0/4] Filter alternate references","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2018-10-09T14:49:54Z","receivedAt":"2018-10-09T14:49:59Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"On Mon, Oct 08, 2018 at 11:09:18PM -0400, Jeff King wrote:\n> On Mon, Oct 08, 2018 at 11:09:20AM -0700, Taylor Blau wrote:\n>\n> > Attached is (what I anticipate to be) the final re-roll of my series to\n> > introduce 'core.alternateRefsCommand' and 'core.alternateRefsPrefixes'\n> > in order to limit the \".have\" advertisement when pushing over protocol\n> > v1 to a repository with configured alternates.\n>\n> Thanks, this looks good to me!\n\nThanks again for all of your thoughtful review, it is much appreciated\n:-).\n\nThanks,\nTaylor\n"}]}