{"thread":{"id":"52659","subject":"[PATCH] fetch: add --no-update-remote-refs","startedAt":"2020-01-17T15:29:04Z","lastAt":"2020-01-21T19:06:16Z","messageCount":13,"participants":["Derrick Stolee via GitGitGadget","Eric Sunshine","Junio C Hamano","Jeff King","Derrick Stolee"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"389990","messageId":"pull.532.git.1579274939431.gitgitgadget@gmail.com","threadId":"52659","inReplyTo":null,"subject":"[PATCH] fetch: add --no-update-remote-refs","fromName":"Derrick Stolee via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2020-01-17T15:28:59Z","receivedAt":"2020-01-17T15:29:04Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"From: Derrick Stolee <dstolee@microsoft.com>\n\nTo prevent long blocking time during a 'git fetch' call, a user\nmay want to set up a schedule for background 'git fetch' processes.\nHowever, these runs will update the refs/remotes branches, and\nhence the user will not notice when remote refs are updated during\ntheir foreground fetches. In fact, they may _want_ those refs to\nstay put so they can work with the refs from their last foreground\nfetch call.\n\nAdd a --[no-]update-remote-refs option to 'git fetch' which defaults\nto the existing behavior of updating the remote refs. This allows\na user to run\n\n  git fetch <remote> --no-update-remote-refs +refs/heads/*:refs/hidden/*\n\nto populate a custom ref space and download a pack of the new\nreachable objects. This kind of call allows a few things to happen:\n\n1. We download a new pack if refs have updated.\n2. Since the refs/hidden branches exist, GC will not remove the\n   newly-downloaded data.\n3. With fetch.writeCommitGraph enabled, the refs/hidden refs are\n   used to update the commit-graph file.\n\nTo avoid the refs/hidden directory from filling without bound, the\n--prune option can be included. When providing a refspec like this,\nthe --prune option does not delete remote refs and instead only\ndeletes refs in the target refspace.\n\nNote: with the default refpsec, the --prune option will override\nthe --no-update-remote-refs option and will delete the refs that\ndo not exist on the remote.\n\nSigned-off-by: Derrick Stolee <dstolee@microsoft.com>\n---\n    fetch: add --no-update-remote-refs\n    \n    Here is a new feature for git fetch that hopefully is useful to some\n    users. We've been using a patch like this in microsoft/git for about a\n    month now, and I've been testing it locally using the custom refspec\n    mentioned in the commit message. It's quite refreshing to run git fetch\n    --all in my Git repo and see all the branch updates but not actually\n    wait for any pack downloads.\n    \n    There is one question about how --prune and --no-update-remote-refs \n    should interact. Since --prune is not the default, and it works the way\n    I'd like with a non-default refspec, I'm currently proposing allowing it\n    to delete remote refs even if --no-update-remote-refs is provided.\n    \n    Thanks, -Stolee\n\nPublished-As: https://github.com/gitgitgadget/git/releases/tag/pr-532%2Fderrickstolee%2Ffetch-no-update-remote-v1\nFetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-532/derrickstolee/fetch-no-update-remote-v1\nPull-Request: https://github.com/gitgitgadget/git/pull/532\n\n Documentation/fetch-options.txt |  7 +++++++\n builtin/fetch.c                 |  6 ++++++\n t/t5510-fetch.sh                | 24 ++++++++++++++++++++++++\n 3 files changed, 37 insertions(+)\n\ndiff --git a/Documentation/fetch-options.txt b/Documentation/fetch-options.txt\nindex a2f78624a2..0939642dce 100644\n--- a/Documentation/fetch-options.txt\n+++ b/Documentation/fetch-options.txt\n@@ -254,6 +254,13 @@ endif::git-pull[]\n \t'git-pull' the --ff-only option will still check for forced updates\n \tbefore attempting a fast-forward update. See linkgit:git-config[1].\n \n+--no-update-remote-refs::\n+\tBy default, git updates the `refs/remotes/` refspace with the refs\n+\tadvertised by the remotes during a `git fetch` command. With this\n+\toption, those refs will be ignored. If the `--prune` option is\n+\tspecified and the default refpsec is used, then a ref that does not\n+\tappear in the remote will still be deleted from refs/remotes.\n+\n -4::\n --ipv4::\n \tUse IPv4 addresses only, ignoring IPv6 addresses.\ndiff --git a/builtin/fetch.c b/builtin/fetch.c\nindex b4c6d921d0..bf8000adaf 100644\n--- a/builtin/fetch.c\n+++ b/builtin/fetch.c\n@@ -78,6 +78,7 @@ static struct list_objects_filter_options filter_options;\n static struct string_list server_options = STRING_LIST_INIT_DUP;\n static struct string_list negotiation_tip = STRING_LIST_INIT_NODUP;\n static int fetch_write_commit_graph = -1;\n+static int update_remote_refs = 1;\n \n static int git_fetch_config(const char *k, const char *v, void *cb)\n {\n@@ -201,6 +202,8 @@ static struct option builtin_fetch_options[] = {\n \t\t N_(\"check for forced-updates on all updated branches\")),\n \tOPT_BOOL(0, \"write-commit-graph\", &fetch_write_commit_graph,\n \t\t N_(\"write the commit-graph after fetching\")),\n+\tOPT_BOOL(0, \"update-remote-refs\", &update_remote_refs,\n+\t\t N_(\"update the refs/remotes/ refspace\")),\n \tOPT_END()\n };\n \n@@ -746,6 +749,9 @@ static int update_local_ref(struct ref *ref,\n \tconst char *pretty_ref = prettify_refname(ref->name);\n \tint fast_forward = 0;\n \n+\tif (!update_remote_refs && starts_with(ref->name, \"refs/remotes/\"))\n+\t\treturn 0;\n+\n \ttype = oid_object_info(the_repository, &ref->new_oid, NULL);\n \tif (type < 0)\n \t\tdie(_(\"object %s not found\"), oid_to_hex(&ref->new_oid));\ndiff --git a/t/t5510-fetch.sh b/t/t5510-fetch.sh\nindex 4b60282689..35b50b2047 100755\n--- a/t/t5510-fetch.sh\n+++ b/t/t5510-fetch.sh\n@@ -174,6 +174,30 @@ test_expect_success 'fetch --prune --tags with refspec prunes based on refspec'\n \tgit rev-parse sometag\n '\n \n+test_expect_success 'fetch --no-update-remote-refs keeps existing refs' '\n+\tcd \"$TRASH_DIRECTORY\" &&\n+\tgit clone \"$D\" remote-refs &&\n+\tgit -C remote-refs rev-parse remotes/origin/master >old &&\n+\tgit -C remote-refs update-ref refs/remotes/origin/master master~1 &&\n+\tgit -C remote-refs rev-parse remotes/origin/master >new &&\n+\tgit -C remote-refs fetch --no-update-remote-refs origin &&\n+\tgit -C remote-refs rev-parse remotes/origin/master >actual &&\n+\ttest_cmp new actual &&\n+\tgit -C remote-refs fetch origin &&\n+\tgit -C remote-refs rev-parse remotes/origin/master >actual &&\n+\ttest_cmp old actual\n+'\n+\n+test_expect_success 'fetch --no-update-remote-refs --prune with refspec' '\n+\tgit -C remote-refs update-ref refs/remotes/origin/foo/otherbranch master &&\n+\tgit -C remote-refs update-ref refs/hidden/foo/otherbranch master &&\n+\tgit -C remote-refs fetch --prune --no-update-remote-refs origin +refs/heads/*:refs/hidden/* &&\n+\tgit -C remote-refs rev-parse remotes/origin/foo/otherbranch &&\n+\ttest_must_fail git -C remote-refs rev-parse refs/hidden/foo/otherbranch &&\n+\tgit -C remote-refs fetch --prune --no-update-remote-refs origin &&\n+\ttest_must_fail git -C remote-refs rev-parse remotes/origin/foo/otherbranch\n+'\n+\n test_expect_success 'fetch tags when there is no tags' '\n \n     cd \"$D\" &&\n\nbase-commit: d0654dc308b0ba76dd8ed7bbb33c8d8f7aacd783\n-- \ngitgitgadget\n"},{"id":"389997","messageId":"CAPig+cTK86FPm1kk-P=dvegu+FTzV3k5Z12Etz_jpE=JdyztDQ@mail.gmail.com","threadId":"52659","inReplyTo":"pull.532.git.1579274939431.gitgitgadget@gmail.com","subject":"Re: [PATCH] fetch: add --no-update-remote-refs","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2020-01-17T16:23:06Z","receivedAt":"2020-01-17T16:23:23Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Fri, Jan 17, 2020 at 10:29 AM Derrick Stolee via GitGitGadget\n<gitgitgadget@gmail.com> wrote:\n> diff --git a/t/t5510-fetch.sh b/t/t5510-fetch.sh\n> @@ -174,6 +174,30 @@ test_expect_success 'fetch --prune --tags with refspec prunes based on refspec'\n> +test_expect_success 'fetch --no-update-remote-refs keeps existing refs' '\n> +       cd \"$TRASH_DIRECTORY\" &&\n\nWhy does the prologue of this test use `cd \"$TRASH_DIRECTORY\"` when\nall the other tests use `cd \"$D\"`?\n\n> +       git clone \"$D\" remote-refs &&\n> +       git -C remote-refs rev-parse remotes/origin/master >old &&\n> +       git -C remote-refs update-ref refs/remotes/origin/master master~1 &&\n> +       git -C remote-refs rev-parse remotes/origin/master >new &&\n> +       git -C remote-refs fetch --no-update-remote-refs origin &&\n> +       git -C remote-refs rev-parse remotes/origin/master >actual &&\n> +       test_cmp new actual &&\n> +       git -C remote-refs fetch origin &&\n> +       git -C remote-refs rev-parse remotes/origin/master >actual &&\n> +       test_cmp old actual\n> +'\n\nI wouldn't expect a re-roll just for this (nor do I insist upon such a\nchange), but this is one of those cases when -C hurts, rather than\nhelps, readability, due to the amount of noise it adds to nearly every\nline of the test. Using a subshell makes for less noisy code:\n\n    git clone \"$D\" remote-refs &&\n    (\n        cd remote-refs &&\n        git rev-parse remotes/origin/master >old &&\n        git update-ref refs/remotes/origin/master master~1 &&\n        git rev-parse remotes/origin/master >new &&\n        git fetch --no-update-remote-refs origin &&\n        git rev-parse remotes/origin/master >actual &&\n        test_cmp new actual &&\n        git fetch origin &&\n        git rev-parse remotes/origin/master >actual &&\n        test_cmp old actual\n    )\n\n> +test_expect_success 'fetch --no-update-remote-refs --prune with refspec' '\n> +       git -C remote-refs update-ref refs/remotes/origin/foo/otherbranch master &&\n> +       git -C remote-refs update-ref refs/hidden/foo/otherbranch master &&\n> +       git -C remote-refs fetch --prune --no-update-remote-refs origin +refs/heads/*:refs/hidden/* &&\n> +       git -C remote-refs rev-parse remotes/origin/foo/otherbranch &&\n> +       test_must_fail git -C remote-refs rev-parse refs/hidden/foo/otherbranch &&\n> +       git -C remote-refs fetch --prune --no-update-remote-refs origin &&\n> +       test_must_fail git -C remote-refs rev-parse remotes/origin/foo/otherbranch\n> +'\n\nDitto.\n"},{"id":"390007","messageId":"xmqqa76lew1l.fsf@gitster-ct.c.googlers.com","threadId":"52659","inReplyTo":"pull.532.git.1579274939431.gitgitgadget@gmail.com","subject":"Re: [PATCH] fetch: add --no-update-remote-refs","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2020-01-17T19:13:10Z","receivedAt":"2020-01-17T19:13:21Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Derrick Stolee via GitGitGadget\" <gitgitgadget@gmail.com> writes:\n\n> From: Derrick Stolee <dstolee@microsoft.com>\n>\n> To prevent long blocking time during a 'git fetch' call, a user\n> may want to set up a schedule for background 'git fetch' processes.\n> However, these runs will update the refs/remotes branches, and\n> hence the user will not notice when remote refs are updated during\n> their foreground fetches. In fact, they may _want_ those refs to\n> stay put so they can work with the refs from their last foreground\n> fetch call.\n\nI've always hated anything that makes the remote-tracking refs\n\"float\" and surprise end users.  I even hated that 'git push' that\npretends as if we immediately turned around and fetched from the\nremote we just pushed when it was introduced, even though I gave up\nby now.\n\nSo I am OK in principle to make it more difficult to update\nrefs/remotes/* while the end-user is looking the other way, but I\nhad to wonder why \"git fetch\" is even being done if it is done to\nsilently update/catch-up remote-tracking branches automatically in\nthe first place.\n\nThis is more like a \"preload\" option---without updating the end-user\nvisible set of remote-tracking branches, you can make the data\navailable earlier so that the actual \"fetch\" the end-user runs in\norder to update the remote-tracking branches can complete faster and\nbecome ready to be used more quickly.  \n\nWhich makes sense.\n\n> Add a --[no-]update-remote-refs option to 'git fetch' which defaults\n> to the existing behavior of updating the remote refs. This allows\n> a user to run\n>\n>   git fetch <remote> --no-update-remote-refs +refs/heads/*:refs/hidden/*\n>\n> to populate a custom ref space and download a pack of the new\n> reachable objects.\n\nHmph.  I have to wonder if this should have been the default.  That\nis, when refs/heads/X on the remote is configured to be copied to\nrefs/remotes/origin/X on this side, and an explicit refspec says it\nshould go some other place (i.e. refs/hidden/X), shouldn't that\nautomatically bypass configured +refs/heads/*:refs/remotes/origin/*\nrefspec?  In any case, it is too late to change that now.\n\n> This kind of call allows a few things to happen:\n>\n> 1. We download a new pack if refs have updated.\n> 2. Since the refs/hidden branches exist, GC will not remove the\n>    newly-downloaded data.\n\nCaution.  Since you didn't make it \"refs/hidden/<remote>/*\", you\njust made the data you fetched the same way with this shiny new\n\"--no-update-remote-tracking-branches\" option from another remote\nunanchored and susceptible to GCs.\n\n> 3. With fetch.writeCommitGraph enabled, the refs/hidden refs are\n>    used to update the commit-graph file.\n\nI have a moderately strong suspicion that it would be better to make\nthis \"--ignore-configured-refspecs\" and implemented without special\ncasign the \"refs/remotes/\" hierarchy like the code does by\nhardcoding.\n\nI also wonder if auto-following of tags should be disabled at the\nsame time.  I have no good argument either way (yet).\n\nThanks.\n"},{"id":"390008","messageId":"20200117192016.GB11358@coredump.intra.peff.net","threadId":"52659","inReplyTo":"pull.532.git.1579274939431.gitgitgadget@gmail.com","subject":"Re: [PATCH] fetch: add --no-update-remote-refs","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2020-01-17T19:20:16Z","receivedAt":"2020-01-17T19:20:19Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Fri, Jan 17, 2020 at 03:28:59PM +0000, Derrick Stolee via GitGitGadget wrote:\n\n> To prevent long blocking time during a 'git fetch' call, a user\n> may want to set up a schedule for background 'git fetch' processes.\n> However, these runs will update the refs/remotes branches, and\n> hence the user will not notice when remote refs are updated during\n> their foreground fetches. In fact, they may _want_ those refs to\n> stay put so they can work with the refs from their last foreground\n> fetch call.\n> \n> Add a --[no-]update-remote-refs option to 'git fetch' which defaults\n> to the existing behavior of updating the remote refs. This allows\n> a user to run\n> \n>   git fetch <remote> --no-update-remote-refs +refs/heads/*:refs/hidden/*\n> \n> to populate a custom ref space and download a pack of the new\n> reachable objects. This kind of call allows a few things to happen:\n> \n> 1. We download a new pack if refs have updated.\n> 2. Since the refs/hidden branches exist, GC will not remove the\n>    newly-downloaded data.\n> 3. With fetch.writeCommitGraph enabled, the refs/hidden refs are\n>    used to update the commit-graph file.\n> \n> To avoid the refs/hidden directory from filling without bound, the\n> --prune option can be included. When providing a refspec like this,\n> the --prune option does not delete remote refs and instead only\n> deletes refs in the target refspace.\n\nSo refs/hidden is basically a parallel namespace that is exactly\nidentical to refs/remotes/origin? It seems like a very roundabout way of\nsolving the problem. Which, AFAICT, is just that you want \"git fetch\" to\nprint out the set of updates since the last manual fetch.  I suppose\nthis also doesn't update what people see when they refer to\n\"origin/master\" until they run such a fetch.\n\nI don't know. Part of me wants to say this is overly complicated. If you\nwant to see the difference in refs between two states, then we should\nhave some tool for showing that (even if it's git-fetch) without having\nto have this weird parallel ref structure. OTOH, I guess any such tool\nwould need to save the ref state, which is equivalent to having all\nthese parallel refs.\n\n> +--no-update-remote-refs::\n> +\tBy default, git updates the `refs/remotes/` refspace with the refs\n> +\tadvertised by the remotes during a `git fetch` command. With this\n> +\toption, those refs will be ignored. If the `--prune` option is\n> +\tspecified and the default refpsec is used, then a ref that does not\n> +\tappear in the remote will still be deleted from refs/remotes.\n\nThat \"by default\" isn't exactly how it works. It's not updating the\nadvertised refs, but rather applying the configured refspec\nopportunistically when we are actually fetching one of those refs. The\nidea being that the remote tracking refs should always reflect our\nlatest notion of what's on the remote. This is due to f269048754 (fetch:\nopportunistically update tracking refs, 2013-05-11).\n\nSo I think a cleaner implementation would be to prevent that ref mapping\nfrom kicking in in the first place, rather than hacking it into\nupdate_local_ref() as you have here. And it would avoid making\nassumptions that \"refs/remotes\" is the only thing we'd see in such a\nconfigured refspec. E.g., I have a configured refspec for gitster.git\nwith refs/notes/amlog:refs/notes/amlog, and that _would_ still update\neven with your new option.\n\nBut I think there's already a way to do that: the --refmap option added\nby c5558f80c3 (fetch: allow explicit --refmap to override configuration,\n2014-05-29). Using an empty refmap like:\n\n  git fetch --refmap= <remote> refs/heads/*:refs/hidden/*\n\nshould do what you want. It suppresses the use of the configured\nrefspecs, so we don't find any opportunistic mappings to make.\n\n-Peff\n"},{"id":"390009","messageId":"20200117192610.GC11358@coredump.intra.peff.net","threadId":"52659","inReplyTo":"xmqqa76lew1l.fsf@gitster-ct.c.googlers.com","subject":"Re: [PATCH] fetch: add --no-update-remote-refs","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2020-01-17T19:26:10Z","receivedAt":"2020-01-17T19:26:12Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Fri, Jan 17, 2020 at 11:13:10AM -0800, Junio C Hamano wrote:\n\n> > Add a --[no-]update-remote-refs option to 'git fetch' which defaults\n> > to the existing behavior of updating the remote refs. This allows\n> > a user to run\n> >\n> >   git fetch <remote> --no-update-remote-refs +refs/heads/*:refs/hidden/*\n> >\n> > to populate a custom ref space and download a pack of the new\n> > reachable objects.\n> \n> Hmph.  I have to wonder if this should have been the default.  That\n> is, when refs/heads/X on the remote is configured to be copied to\n> refs/remotes/origin/X on this side, and an explicit refspec says it\n> should go some other place (i.e. refs/hidden/X), shouldn't that\n> automatically bypass configured +refs/heads/*:refs/remotes/origin/*\n> refspec?  In any case, it is too late to change that now.\n\nIt used to be the default. You can blame 2013-me. ;)\n\nBefore that, though, we had people complaining the other way (\"I just\nfetched from the remote, but my tracking ref is stale!\").\n\n> > 3. With fetch.writeCommitGraph enabled, the refs/hidden refs are\n> >    used to update the commit-graph file.\n> \n> I have a moderately strong suspicion that it would be better to make\n> this \"--ignore-configured-refspecs\" and implemented without special\n> casign the \"refs/remotes/\" hierarchy like the code does by\n> hardcoding.\n\nYeah, I just independently wrote something similar. Your \"--refmap\"\noption can accomplish that already.\n\n> I also wonder if auto-following of tags should be disabled at the\n> same time.  I have no good argument either way (yet).\n\nI _didn't_ think of tag auto-following in my other response. That's a\ngood point. I think he'd probably just want to use \"--no-tags\" for the\nbackground fetch. You'd end up having to fetch the tag objects\nthemselves during the \"real\" fetch, but presumably they're very small\ncompared to the rest of history.\n\nYou could also do:\n\n  git fetch --refmap= <remote> +refs/heads/*:refs/hidden/heads/* +refs/tags/*:refs/hidden/tags/*\n\nto get everything in your pre-fetch (though you actually get more than\nthe real fetch would need, since by default it won't grab tags which\ndon't point to otherwise-reachable history).\n\n-Peff\n"},{"id":"390107","messageId":"84ea103d-8bb0-6043-bc51-9a83866d4e87@gmail.com","threadId":"52659","inReplyTo":"xmqqa76lew1l.fsf@gitster-ct.c.googlers.com","subject":"Re: [PATCH] fetch: add --no-update-remote-refs","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2020-01-20T14:44:19Z","receivedAt":"2020-01-20T14:44:26Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 1/17/2020 2:13 PM, Junio C Hamano wrote:\n> \"Derrick Stolee via GitGitGadget\" <gitgitgadget@gmail.com> writes:\n> \n>> From: Derrick Stolee <dstolee@microsoft.com>\n>>\n>> To prevent long blocking time during a 'git fetch' call, a user\n>> may want to set up a schedule for background 'git fetch' processes.\n>> However, these runs will update the refs/remotes branches, and\n>> hence the user will not notice when remote refs are updated during\n>> their foreground fetches. In fact, they may _want_ those refs to\n>> stay put so they can work with the refs from their last foreground\n>> fetch call.\n> \n> I've always hated anything that makes the remote-tracking refs\n> \"float\" and surprise end users.  I even hated that 'git push' that\n> pretends as if we immediately turned around and fetched from the\n> remote we just pushed when it was introduced, even though I gave up\n> by now.\n> \n> So I am OK in principle to make it more difficult to update\n> refs/remotes/* while the end-user is looking the other way, but I\n> had to wonder why \"git fetch\" is even being done if it is done to\n> silently update/catch-up remote-tracking branches automatically in\n> the first place.\n>\n> This is more like a \"preload\" option---without updating the end-user\n> visible set of remote-tracking branches, you can make the data\n> available earlier so that the actual \"fetch\" the end-user runs in\n> order to update the remote-tracking branches can complete faster and\n> become ready to be used more quickly.  \n> \n> Which makes sense.\n\nYes, we get the pack data earlier, and that's the primary cost of\nthe fetch command. We can also update the commit-graph using the\nhidden refs.\n\n>> Add a --[no-]update-remote-refs option to 'git fetch' which defaults\n>> to the existing behavior of updating the remote refs. This allows\n>> a user to run\n>>\n>>   git fetch <remote> --no-update-remote-refs +refs/heads/*:refs/hidden/*\n>>\n>> to populate a custom ref space and download a pack of the new\n>> reachable objects.\n> \n> Hmph.  I have to wonder if this should have been the default.  That\n> is, when refs/heads/X on the remote is configured to be copied to\n> refs/remotes/origin/X on this side, and an explicit refspec says it\n> should go some other place (i.e. refs/hidden/X), shouldn't that\n> automatically bypass configured +refs/heads/*:refs/remotes/origin/*\n> refspec?  In any case, it is too late to change that now.\n> \n>> This kind of call allows a few things to happen:\n>>\n>> 1. We download a new pack if refs have updated.\n>> 2. Since the refs/hidden branches exist, GC will not remove the\n>>    newly-downloaded data.\n> \n> Caution.  Since you didn't make it \"refs/hidden/<remote>/*\", you\n> just made the data you fetched the same way with this shiny new\n> \"--no-update-remote-tracking-branches\" option from another remote\n> unanchored and susceptible to GCs.\n\nYou're right. I neglected to say \"refs/hidden/<remote>/*\" in my\nmessage, but it _is_ something I've been doing in my background\nfetches.\n\n>> 3. With fetch.writeCommitGraph enabled, the refs/hidden refs are\n>>    used to update the commit-graph file.\n> \n> I have a moderately strong suspicion that it would be better to make\n> this \"--ignore-configured-refspecs\" and implemented without special\n> casign the \"refs/remotes/\" hierarchy like the code does by\n> hardcoding.\n\nBased on this and Peff's response, I think you are pointing me in\na better direction with this. It should make the change less of a\nhack and also make it more general to users with custom refspecs.\n\n> I also wonder if auto-following of tags should be disabled at the\n> same time.  I have no good argument either way (yet).\n\nWould ignoring the configured refspecs stop auto-following tags?\nI'll take a look and see. Otherwise, I'll add --no-tags to my\nbackground fetch command.\n\nThanks!\n-Stolee\n"},{"id":"390143","messageId":"4e444a99-6785-1d11-d8cf-87903c3b3760@gmail.com","threadId":"52659","inReplyTo":"20200117192016.GB11358@coredump.intra.peff.net","subject":"Re: [PATCH] fetch: add --no-update-remote-refs","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2020-01-21T00:57:43Z","receivedAt":"2020-01-21T00:57:47Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 1/17/2020 2:20 PM, Jeff King wrote:\n> On Fri, Jan 17, 2020 at 03:28:59PM +0000, Derrick Stolee via GitGitGadget wrote:\n>> Add a --[no-]update-remote-refs option to 'git fetch' which defaults\n>> to the existing behavior of updating the remote refs. This allows\n>> a user to run\n>>\n>>   git fetch <remote> --no-update-remote-refs +refs/heads/*:refs/hidden/*\n>>\n>> to populate a custom ref space and download a pack of the new\n>> reachable objects. This kind of call allows a few things to happen:\n> But I think there's already a way to do that: the --refmap option added\n> by c5558f80c3 (fetch: allow explicit --refmap to override configuration,\n> 2014-05-29). Using an empty refmap like:\n> \n>   git fetch --refmap= <remote> refs/heads/*:refs/hidden/*\n> \n> should do what you want. It suppresses the use of the configured\n> refspecs, so we don't find any opportunistic mappings to make.\n\nYou're absolutely right. There are details at [1].\n\nThe tricky thing is that the documentation makes it look like you\nneed a value there, and if you supply a value without a refspec\nargument, an error is given:\n\n$ git fetch origin --refmap=\"+refs/heads/*:refs/hidden/origin/*\"\nfatal: --refmap option is only meaningful with command-line refspec(s).\n\nI think I'll follow up with a documentation patch to point out\nthat an empty --refmap value will ignore the configured refspec\nand rely only on the provided refspec argument.\n\nThanks!\n-Stolee\n\n[1] https://git-scm.com/docs/git-fetch#_configured_remote_tracking_branches_a_id_crtb_a\n"},{"id":"390144","messageId":"pull.532.v2.git.1579570692766.gitgitgadget@gmail.com","threadId":"52659","inReplyTo":"pull.532.git.1579274939431.gitgitgadget@gmail.com","subject":"[PATCH v2] fetch: document and test --refmap=\"\"","fromName":"Derrick Stolee via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2020-01-21T01:38:12Z","receivedAt":"2020-01-21T01:38:19Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"From: Derrick Stolee <dstolee@microsoft.com>\n\nTo prevent long blocking time during a 'git fetch' call, a user\nmay want to set up a schedule for background 'git fetch' processes.\nHowever, these runs will update the refs/remotes branches due to\nthe default refspec set in the config when Git adds a remote.\nHence the user will not notice when remote refs are updated during\ntheir foreground fetches. In fact, they may _want_ those refs to\nstay put so they can work with the refs from their last foreground\nfetch call.\n\nThis can be accomplished by overriding the configured refspec using\n'--refmap=' along with a custom refspec:\n\n  git fetch <remote> --refmap= +refs/heads/*:refs/hidden/<remote>/*\n\nto populate a custom ref space and download a pack of the new\nreachable objects. This kind of call allows a few things to happen:\n\n1. We download a new pack if refs have updated.\n2. Since the refs/hidden branches exist, GC will not remove the\n   newly-downloaded data.\n3. With fetch.writeCommitGraph enabled, the refs/hidden refs are\n   used to update the commit-graph file.\n\nTo avoid the refs/hidden directory from filling without bound, the\n--prune option can be included. When providing a refspec like this,\nthe --prune option does not delete remote refs and instead only\ndeletes refs in the target refspace.\n\nUpdate the documentation to clarify how '--refmap=\"\"' works and\ncreate tests to guarantee this behavior remains in the future.\n\nSigned-off-by: Derrick Stolee <dstolee@microsoft.com>\n---\n    fetch: document and test '--refmap=\"\"'\n    \n    Thanks for all the feedback leading to absolutely no code change. It's\n    good we already have the flexibility for this. I'm a bit embarrassed\n    that I did not discover this, so perhaps this doc change and new tests\n    will help clarify the behavior.\n    \n    Thanks, -Stolee\n\nPublished-As: https://github.com/gitgitgadget/git/releases/tag/pr-532%2Fderrickstolee%2Ffetch-no-update-remote-v2\nFetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-532/derrickstolee/fetch-no-update-remote-v2\nPull-Request: https://github.com/gitgitgadget/git/pull/532\n\nRange-diff vs v1:\n\n 1:  26f13ea5f5 ! 1:  51c6221a31 fetch: add --no-update-remote-refs\n     @@ -1,20 +1,20 @@\n      Author: Derrick Stolee <dstolee@microsoft.com>\n      \n     -    fetch: add --no-update-remote-refs\n     +    fetch: document and test --refmap=\"\"\n      \n          To prevent long blocking time during a 'git fetch' call, a user\n          may want to set up a schedule for background 'git fetch' processes.\n     -    However, these runs will update the refs/remotes branches, and\n     -    hence the user will not notice when remote refs are updated during\n     +    However, these runs will update the refs/remotes branches due to\n     +    the default refspec set in the config when Git adds a remote.\n     +    Hence the user will not notice when remote refs are updated during\n          their foreground fetches. In fact, they may _want_ those refs to\n          stay put so they can work with the refs from their last foreground\n          fetch call.\n      \n     -    Add a --[no-]update-remote-refs option to 'git fetch' which defaults\n     -    to the existing behavior of updating the remote refs. This allows\n     -    a user to run\n     +    This can be accomplished by overriding the configured refspec using\n     +    '--refmap=' along with a custom refspec:\n      \n     -      git fetch <remote> --no-update-remote-refs +refs/heads/*:refs/hidden/*\n     +      git fetch <remote> --refmap= +refs/heads/*:refs/hidden/<remote>/*\n      \n          to populate a custom ref space and download a pack of the new\n          reachable objects. This kind of call allows a few things to happen:\n     @@ -30,9 +30,8 @@\n          the --prune option does not delete remote refs and instead only\n          deletes refs in the target refspace.\n      \n     -    Note: with the default refpsec, the --prune option will override\n     -    the --no-update-remote-refs option and will delete the refs that\n     -    do not exist on the remote.\n     +    Update the documentation to clarify how '--refmap=\"\"' works and\n     +    create tests to guarantee this behavior remains in the future.\n      \n          Signed-off-by: Derrick Stolee <dstolee@microsoft.com>\n      \n     @@ -40,50 +39,17 @@\n       --- a/Documentation/fetch-options.txt\n       +++ b/Documentation/fetch-options.txt\n      @@\n     - \t'git-pull' the --ff-only option will still check for forced updates\n     - \tbefore attempting a fast-forward update. See linkgit:git-config[1].\n     + \tspecified refspec (can be given more than once) to map the\n     + \trefs to remote-tracking branches, instead of the values of\n     + \t`remote.*.fetch` configuration variables for the remote\n     +-\trepository.  See section on \"Configured Remote-tracking\n     ++\trepository.  Providing an empty `<refspec>` to the\n     ++\t`--refmap` option causes Git to ignore the configured\n     ++\trefspecs and rely entirely on the refspecs supplied as\n     ++\tcommand-line arguments. See section on \"Configured Remote-tracking\n     + \tBranches\" for details.\n       \n     -+--no-update-remote-refs::\n     -+\tBy default, git updates the `refs/remotes/` refspace with the refs\n     -+\tadvertised by the remotes during a `git fetch` command. With this\n     -+\toption, those refs will be ignored. If the `--prune` option is\n     -+\tspecified and the default refpsec is used, then a ref that does not\n     -+\tappear in the remote will still be deleted from refs/remotes.\n     -+\n     - -4::\n     - --ipv4::\n     - \tUse IPv4 addresses only, ignoring IPv6 addresses.\n     -\n     - diff --git a/builtin/fetch.c b/builtin/fetch.c\n     - --- a/builtin/fetch.c\n     - +++ b/builtin/fetch.c\n     -@@\n     - static struct string_list server_options = STRING_LIST_INIT_DUP;\n     - static struct string_list negotiation_tip = STRING_LIST_INIT_NODUP;\n     - static int fetch_write_commit_graph = -1;\n     -+static int update_remote_refs = 1;\n     - \n     - static int git_fetch_config(const char *k, const char *v, void *cb)\n     - {\n     -@@\n     - \t\t N_(\"check for forced-updates on all updated branches\")),\n     - \tOPT_BOOL(0, \"write-commit-graph\", &fetch_write_commit_graph,\n     - \t\t N_(\"write the commit-graph after fetching\")),\n     -+\tOPT_BOOL(0, \"update-remote-refs\", &update_remote_refs,\n     -+\t\t N_(\"update the refs/remotes/ refspace\")),\n     - \tOPT_END()\n     - };\n     - \n     -@@\n     - \tconst char *pretty_ref = prettify_refname(ref->name);\n     - \tint fast_forward = 0;\n     - \n     -+\tif (!update_remote_refs && starts_with(ref->name, \"refs/remotes/\"))\n     -+\t\treturn 0;\n     -+\n     - \ttype = oid_object_info(the_repository, &ref->new_oid, NULL);\n     - \tif (type < 0)\n     - \t\tdie(_(\"object %s not found\"), oid_to_hex(&ref->new_oid));\n     + -t::\n      \n       diff --git a/t/t5510-fetch.sh b/t/t5510-fetch.sh\n       --- a/t/t5510-fetch.sh\n     @@ -92,13 +58,13 @@\n       \tgit rev-parse sometag\n       '\n       \n     -+test_expect_success 'fetch --no-update-remote-refs keeps existing refs' '\n     ++test_expect_success '--refmap=\"\" ignores configured refspec' '\n      +\tcd \"$TRASH_DIRECTORY\" &&\n      +\tgit clone \"$D\" remote-refs &&\n      +\tgit -C remote-refs rev-parse remotes/origin/master >old &&\n      +\tgit -C remote-refs update-ref refs/remotes/origin/master master~1 &&\n      +\tgit -C remote-refs rev-parse remotes/origin/master >new &&\n     -+\tgit -C remote-refs fetch --no-update-remote-refs origin &&\n     ++\tgit -C remote-refs fetch --refmap= origin \"+refs/heads/*:refs/hidden/origin/*\" &&\n      +\tgit -C remote-refs rev-parse remotes/origin/master >actual &&\n      +\ttest_cmp new actual &&\n      +\tgit -C remote-refs fetch origin &&\n     @@ -106,13 +72,13 @@\n      +\ttest_cmp old actual\n      +'\n      +\n     -+test_expect_success 'fetch --no-update-remote-refs --prune with refspec' '\n     ++test_expect_success '--refmap=\"\" and --prune' '\n      +\tgit -C remote-refs update-ref refs/remotes/origin/foo/otherbranch master &&\n      +\tgit -C remote-refs update-ref refs/hidden/foo/otherbranch master &&\n     -+\tgit -C remote-refs fetch --prune --no-update-remote-refs origin +refs/heads/*:refs/hidden/* &&\n     ++\tgit -C remote-refs fetch --prune --refmap=\"\" origin +refs/heads/*:refs/hidden/* &&\n      +\tgit -C remote-refs rev-parse remotes/origin/foo/otherbranch &&\n      +\ttest_must_fail git -C remote-refs rev-parse refs/hidden/foo/otherbranch &&\n     -+\tgit -C remote-refs fetch --prune --no-update-remote-refs origin &&\n     ++\tgit -C remote-refs fetch --prune origin &&\n      +\ttest_must_fail git -C remote-refs rev-parse remotes/origin/foo/otherbranch\n      +'\n      +\n\n\n Documentation/fetch-options.txt |  5 ++++-\n t/t5510-fetch.sh                | 24 ++++++++++++++++++++++++\n 2 files changed, 28 insertions(+), 1 deletion(-)\n\ndiff --git a/Documentation/fetch-options.txt b/Documentation/fetch-options.txt\nindex a2f78624a2..a115a1ae0e 100644\n--- a/Documentation/fetch-options.txt\n+++ b/Documentation/fetch-options.txt\n@@ -139,7 +139,10 @@ ifndef::git-pull[]\n \tspecified refspec (can be given more than once) to map the\n \trefs to remote-tracking branches, instead of the values of\n \t`remote.*.fetch` configuration variables for the remote\n-\trepository.  See section on \"Configured Remote-tracking\n+\trepository.  Providing an empty `<refspec>` to the\n+\t`--refmap` option causes Git to ignore the configured\n+\trefspecs and rely entirely on the refspecs supplied as\n+\tcommand-line arguments. See section on \"Configured Remote-tracking\n \tBranches\" for details.\n \n -t::\ndiff --git a/t/t5510-fetch.sh b/t/t5510-fetch.sh\nindex 4b60282689..5f8f1c287f 100755\n--- a/t/t5510-fetch.sh\n+++ b/t/t5510-fetch.sh\n@@ -174,6 +174,30 @@ test_expect_success 'fetch --prune --tags with refspec prunes based on refspec'\n \tgit rev-parse sometag\n '\n \n+test_expect_success '--refmap=\"\" ignores configured refspec' '\n+\tcd \"$TRASH_DIRECTORY\" &&\n+\tgit clone \"$D\" remote-refs &&\n+\tgit -C remote-refs rev-parse remotes/origin/master >old &&\n+\tgit -C remote-refs update-ref refs/remotes/origin/master master~1 &&\n+\tgit -C remote-refs rev-parse remotes/origin/master >new &&\n+\tgit -C remote-refs fetch --refmap= origin \"+refs/heads/*:refs/hidden/origin/*\" &&\n+\tgit -C remote-refs rev-parse remotes/origin/master >actual &&\n+\ttest_cmp new actual &&\n+\tgit -C remote-refs fetch origin &&\n+\tgit -C remote-refs rev-parse remotes/origin/master >actual &&\n+\ttest_cmp old actual\n+'\n+\n+test_expect_success '--refmap=\"\" and --prune' '\n+\tgit -C remote-refs update-ref refs/remotes/origin/foo/otherbranch master &&\n+\tgit -C remote-refs update-ref refs/hidden/foo/otherbranch master &&\n+\tgit -C remote-refs fetch --prune --refmap=\"\" origin +refs/heads/*:refs/hidden/* &&\n+\tgit -C remote-refs rev-parse remotes/origin/foo/otherbranch &&\n+\ttest_must_fail git -C remote-refs rev-parse refs/hidden/foo/otherbranch &&\n+\tgit -C remote-refs fetch --prune origin &&\n+\ttest_must_fail git -C remote-refs rev-parse remotes/origin/foo/otherbranch\n+'\n+\n test_expect_success 'fetch tags when there is no tags' '\n \n     cd \"$D\" &&\n\nbase-commit: d0654dc308b0ba76dd8ed7bbb33c8d8f7aacd783\n-- \ngitgitgadget\n"},{"id":"390174","messageId":"20200121162433.GA6215@coredump.intra.peff.net","threadId":"52659","inReplyTo":"pull.532.v2.git.1579570692766.gitgitgadget@gmail.com","subject":"Re: [PATCH v2] fetch: document and test --refmap=\"\"","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2020-01-21T16:24:33Z","receivedAt":"2020-01-21T16:24:36Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Tue, Jan 21, 2020 at 01:38:12AM +0000, Derrick Stolee via GitGitGadget wrote:\n\n> Update the documentation to clarify how '--refmap=\"\"' works and\n> create tests to guarantee this behavior remains in the future.\n\nYeah, this looks like a good solution to me.\n\n> This can be accomplished by overriding the configured refspec using\n> '--refmap=' along with a custom refspec:\n> \n>   git fetch <remote> --refmap= +refs/heads/*:refs/hidden/<remote>/*\n\nThis isn't strictly related to your patch, but since the rationale here\ndescribes the concept of a background job and people might end up using\nit as a reference, do you want to add in --no-tags to help them out?\n\n>     Thanks for all the feedback leading to absolutely no code change. It's\n>     good we already have the flexibility for this. I'm a bit embarrassed\n>     that I did not discover this, so perhaps this doc change and new tests\n>     will help clarify the behavior.\n\nIf it makes you feel better, I only found --refmap because I was the one\nwho implemented the original \"update based on refspecs\" logic, and while\nlooking for that commit with \"git log --grep=opportunistic\" I stumbled\nonto Junio's commit adding --refmap, which referenced mine. Maybe this\nalso works as a good case study in why we should write good commit\nmessages and refer to related work. :)\n\nAnyway, I wasn't at all sure that a blank --refmap= would do what you\nwant until I tried it. But it was always intended to work that way. From\nc5558f80c3 (fetch: allow explicit --refmap to override configuration,\n2014-05-29):\n\n  +static int parse_refmap_arg(const struct option *opt, const char *arg, int unset)\n  +{\n  +       ALLOC_GROW(refmap_array, refmap_nr + 1, refmap_alloc);\n  +\n  +       /*\n  +        * \"git fetch --refmap='' origin foo\"\n  +        * can be used to tell the command not to store anywhere\n  +        */\n  +       if (*arg)\n  +               refmap_array[refmap_nr++] = arg;\n  +       return 0;\n  +}\n\nAt first I thought the comment was wrong, since we don't actually\nincrement refmap_nr. But the ALLOC_GROW makes refmap_array non-NULL,\nwhich is what triggers the \"do not use configured refspecs\" logic.\n\nThis code switched to refspec_append() in e4cffacc80 (fetch: convert\nrefmap to use struct refspec, 2018-05-16), and I think we actually do\npush an empty string onto the list. Which then triggers the \"do not use\nconfigured refspecs\" logic, but doesn't match anything itself. I'm not\nsure whether that behavior was entirely planned, or just what the code\nhappens to do. So it's doubly useful to add a test here covering the\nexpected behavior.\n\n>  Documentation/fetch-options.txt |  5 ++++-\n>  t/t5510-fetch.sh                | 24 ++++++++++++++++++++++++\n>  2 files changed, 28 insertions(+), 1 deletion(-)\n\nThe patch looks good to me.\n\n-Peff\n"},{"id":"390184","messageId":"a87b9fc2-dae4-7e39-5aab-243ba9679531@gmail.com","threadId":"52659","inReplyTo":"20200121162433.GA6215@coredump.intra.peff.net","subject":"Re: [PATCH v2] fetch: document and test --refmap=\"\"","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2020-01-21T18:01:47Z","receivedAt":"2020-01-21T18:01:53Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 1/21/2020 11:24 AM, Jeff King wrote:\n> On Tue, Jan 21, 2020 at 01:38:12AM +0000, Derrick Stolee via GitGitGadget wrote:\n> \n>> Update the documentation to clarify how '--refmap=\"\"' works and\n>> create tests to guarantee this behavior remains in the future.\n> \n> Yeah, this looks like a good solution to me.\n> \n>> This can be accomplished by overriding the configured refspec using\n>> '--refmap=' along with a custom refspec:\n>>\n>>   git fetch <remote> --refmap= +refs/heads/*:refs/hidden/<remote>/*\n> \n> This isn't strictly related to your patch, but since the rationale here\n> describes the concept of a background job and people might end up using\n> it as a reference, do you want to add in --no-tags to help them out?\n\nThat's a good idea. I keep forgetting about that. It's interesting that\ntags are fetched even though my refpsec does not include refs/tags.\n\n>>     Thanks for all the feedback leading to absolutely no code change. It's\n>>     good we already have the flexibility for this. I'm a bit embarrassed\n>>     that I did not discover this, so perhaps this doc change and new tests\n>>     will help clarify the behavior.\n\n> Anyway, I wasn't at all sure that a blank --refmap= would do what you\n> want until I tried it. But it was always intended to work that way. From\n> c5558f80c3 (fetch: allow explicit --refmap to override configuration,\n> 2014-05-29):\n> \n>   +static int parse_refmap_arg(const struct option *opt, const char *arg, int unset)\n>   +{\n>   +       ALLOC_GROW(refmap_array, refmap_nr + 1, refmap_alloc);\n>   +\n>   +       /*\n>   +        * \"git fetch --refmap='' origin foo\"\n>   +        * can be used to tell the command not to store anywhere\n>   +        */\n>   +       if (*arg)\n>   +               refmap_array[refmap_nr++] = arg;\n>   +       return 0;\n>   +}\n> \n> At first I thought the comment was wrong, since we don't actually\n> increment refmap_nr. But the ALLOC_GROW makes refmap_array non-NULL,\n> which is what triggers the \"do not use configured refspecs\" logic.\n\nThis works due to a subtle arrangement of things, like a non-NULL\nbut \"empty\" array. That justifies the test even more.\n \n> This code switched to refspec_append() in e4cffacc80 (fetch: convert\n> refmap to use struct refspec, 2018-05-16), and I think we actually do\n> push an empty string onto the list. Which then triggers the \"do not use\n> configured refspecs\" logic, but doesn't match anything itself. I'm not\n> sure whether that behavior was entirely planned, or just what the code\n> happens to do. So it's doubly useful to add a test here covering the\n> expected behavior.\n\nExcellent. Glad I'm not just adding test bloat for now reason.\n\n-Stolee\n"},{"id":"390187","messageId":"xmqq5zh4brf4.fsf@gitster-ct.c.googlers.com","threadId":"52659","inReplyTo":"20200121162433.GA6215@coredump.intra.peff.net","subject":"Re: [PATCH v2] fetch: document and test --refmap=\"\"","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2020-01-21T18:22:39Z","receivedAt":"2020-01-21T18:22:44Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jeff King <peff@peff.net> writes:\n\n> On Tue, Jan 21, 2020 at 01:38:12AM +0000, Derrick Stolee via GitGitGadget wrote:\n>\n>> Update the documentation to clarify how '--refmap=\"\"' works and\n>> create tests to guarantee this behavior remains in the future.\n>\n> Yeah, this looks like a good solution to me.\n>\n>> This can be accomplished by overriding the configured refspec using\n>> '--refmap=' along with a custom refspec:\n>> \n>>   git fetch <remote> --refmap= +refs/heads/*:refs/hidden/<remote>/*\n>\n> This isn't strictly related to your patch, but since the rationale here\n> describes the concept of a background job and people might end up using\n> it as a reference, do you want to add in --no-tags to help them out?\n\nAlso, is the order of the arguments correct?  I would have expected\nto see <remote> after all dashed options and before refspecs.\n\n> If it makes you feel better, I only found --refmap because I was the one\n> who implemented the original \"update based on refspecs\" logic, and while\n> looking for that commit with \"git log --grep=opportunistic\" I stumbled\n> onto Junio's commit adding --refmap, which referenced mine. Maybe this\n> also works as a good case study in why we should write good commit\n> messages and refer to related work. :)\n\n;-)\n\n> Anyway, I wasn't at all sure that a blank --refmap= would do what you\n> want until I tried it. But it was always intended to work that way. From\n> c5558f80c3 (fetch: allow explicit --refmap to override configuration,\n> 2014-05-29):\n>\n>   +static int parse_refmap_arg(const struct option *opt, const char *arg, int unset)\n>   +{\n>   +       ALLOC_GROW(refmap_array, refmap_nr + 1, refmap_alloc);\n>   +\n>   +       /*\n>   +        * \"git fetch --refmap='' origin foo\"\n>   +        * can be used to tell the command not to store anywhere\n>   +        */\n>   +       if (*arg)\n>   +               refmap_array[refmap_nr++] = arg;\n>   +       return 0;\n>   +}\n\nGood analysis, and also the answer to my previous question is also\nthere ;-)\n\n"},{"id":"390188","messageId":"xmqq1rrsbrc7.fsf@gitster-ct.c.googlers.com","threadId":"52659","inReplyTo":"pull.532.v2.git.1579570692766.gitgitgadget@gmail.com","subject":"Re: [PATCH v2] fetch: document and test --refmap=\"\"","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2020-01-21T18:24:24Z","receivedAt":"2020-01-21T18:24:28Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Derrick Stolee via GitGitGadget\" <gitgitgadget@gmail.com> writes:\n\n> From: Derrick Stolee <dstolee@microsoft.com>\n>\n> To prevent long blocking time during a 'git fetch' call, a user\n> may want to set up a schedule for background 'git fetch' processes.\n> However, these runs will update the refs/remotes branches due to\n> the default refspec set in the config when Git adds a remote.\n> Hence the user will not notice when remote refs are updated during\n> their foreground fetches. In fact, they may _want_ those refs to\n> stay put so they can work with the refs from their last foreground\n> fetch call.\n>\n> This can be accomplished by overriding the configured refspec using\n> '--refmap=' along with a custom refspec:\n>\n>   git fetch <remote> --refmap= +refs/heads/*:refs/hidden/<remote>/*\n>\n> to populate a custom ref space and download a pack of the new\n> reachable objects. This kind of call allows a few things to happen:\n>\n> 1. We download a new pack if refs have updated.\n> 2. Since the refs/hidden branches exist, GC will not remove the\n>    newly-downloaded data.\n> 3. With fetch.writeCommitGraph enabled, the refs/hidden refs are\n>    used to update the commit-graph file.\n>\n> To avoid the refs/hidden directory from filling without bound, the\n> --prune option can be included. When providing a refspec like this,\n> the --prune option does not delete remote refs and instead only\n> deletes refs in the target refspace.\n>\n> Update the documentation to clarify how '--refmap=\"\"' works and\n> create tests to guarantee this behavior remains in the future.\n>\n> Signed-off-by: Derrick Stolee <dstolee@microsoft.com>\n> ---\n\nThanks.  Looks quite good.\n\nWill queue.\n"},{"id":"390192","messageId":"20200121190613.GB11027@coredump.intra.peff.net","threadId":"52659","inReplyTo":"a87b9fc2-dae4-7e39-5aab-243ba9679531@gmail.com","subject":"Re: [PATCH v2] fetch: document and test --refmap=\"\"","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2020-01-21T19:06:13Z","receivedAt":"2020-01-21T19:06:16Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Tue, Jan 21, 2020 at 01:01:47PM -0500, Derrick Stolee wrote:\n\n> > This isn't strictly related to your patch, but since the rationale here\n> > describes the concept of a background job and people might end up using\n> > it as a reference, do you want to add in --no-tags to help them out?\n> \n> That's a good idea. I keep forgetting about that. It's interesting that\n> tags are fetched even though my refpsec does not include refs/tags.\n\nYeah, because tag-following is an extra thing outside of the refspecs. I\nthink it would be nice if there were a way to specify a \"following\"\nrefspec, something like:\n\n  ~refs/tags/*:refs/tags/*\n\n(where \"~\" is just a character I picked out of the blue that I think\ndoesn't have any other meaning there now). And then we could specify it\nin the config alongside other refspecs, override it with a refmap, etc.\n\nAs a bonus, it would also give us a stepping stone towards being able to\ndo remote-specific tags like:\n\n  [remote \"origin\"]\n  url = ...\n  fetch = +refs/heads/*:refs/remotes/origin/heads/*\n  fetch = ~refs/tags/*:refs/remotes/origin/tags/*\n\nBut I'm sure there are a lot of backwards-compatibility gotchas.\n\n-Peff\n"}]}