From: Harald Nordgren via GitGitGadget Date: Tue, 29 Sep 2026 09:19:54 GMT Subject: [PATCH v4 0/4] fetch: avoid fetching every branch of a new remote in a shallow repo Message-ID: In-Reply-To: Avoid fetching every branch of a new remote in a shallow repo. Changes in v4: * Removed the automatic default-branch fetch. A fresh remote fetches nothing until you track a branch explicitly. * Fixed fetch report showing "new ref HEAD" instead of "new branch " * Reworded remote..refmap docs and commit message. Changes in v3: * Replace the special ":"/"+:" fetch refspec with remote..refmap, reusing git's existing --refmap mechanism instead of inventing new refspec syntax. * Split the change into 4 commits. Changes in v2: * Replaced the opt-in fetch.shallow config entirely with a new special fetch refspec (+:) that git remote add now defaults new remotes to in a shallow repository. The new refspec fetches whichever branches any local branch tracks at that remote, plus the remote's default branch. Harald Nordgren (4): fetch: add remote..refmap fetch: infer branches to fetch from a refmap-only remote remote: add "git remote add --limited-fetch" remote: default to --limited-fetch in a shallow repository Documentation/config/remote.adoc | 8 +++ Documentation/fetch-options.adoc | 5 ++ Documentation/git-remote.adoc | 14 +++- builtin/fetch.c | 60 +++++++++++++--- builtin/remote.c | 31 ++++++-- remote.c | 41 ++++++++++- remote.h | 9 +++ t/meson.build | 1 + t/t5505-remote.sh | 76 ++++++++++++++++++++ t/t5510-fetch.sh | 17 +++++ t/t5585-fetch-refmap.sh | 119 +++++++++++++++++++++++++++++++ 11 files changed, 363 insertions(+), 18 deletions(-) create mode 100755 t/t5585-fetch-refmap.sh base-commit: a018953688f1b10bddf91bff8747068f5f4746a4 Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-2412%2FHaraldNordgren%2Ffetch-shallow-narrow-refspec-v4 Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-2412/HaraldNordgren/fetch-shallow-narrow-refspec-v4 Pull-Request: https://github.com/git/git/pull/2412 Range-diff vs v3: 1: b04c00b974 ! 1: d48a7004e4 fetch: add remote..refmap @@ Commit message Add a per-remote config variable, remote..refmap, that provides the default value for --refmap the same way remote..fetch - already provides the default refspecs to fetch. It only takes effect - when there is something explicit to fetch, on the command line or via - remote..fetch, matching how --refmap itself already behaves. + already provides the default refspecs to fetch. Like --refmap itself, + it only maps refs that are actually being fetched, so it has nothing + to do when there is nothing explicit to fetch, on the command line or + via remote..fetch. Signed-off-by: Harald Nordgren @@ Documentation/config/remote.adoc: remote..fetch:: +remote..refmap:: + The default value of the `--refmap` option for linkgit:git-fetch[1]. -+ Only takes effect when the fetch names what to fetch explicitly, -+ either on the command line or via `remote..fetch`. See the -+ `--refmap` entry in linkgit:git-fetch[1]. ++ Used to map remote refs being fetched to remote-tracking refs to ++ store. See the `--refmap` entry in linkgit:git-fetch[1]. + remote..push:: The default set of "refspec" for linkgit:git-push[1]. See 2: 4ec508a223 ! 2: 45b26e2bb2 fetch: infer branches to fetch from a refmap-only remote @@ Commit message says where to put fetched refs, not what to fetch. Make that case infer what to fetch: the local branches whose - @{upstream} is already on that remote, plus the remote's default - branch, which is always included so it is available even before - anything is set up to track it. This lets a remote be configured to - fetch only the branches actually in use, without listing them by - hand in remote..fetch, and without needing to touch the - command line every time. + @{upstream} is already on that remote. This lets a remote be + configured to fetch only the branches actually in use, without + listing them by hand in remote..fetch, and without needing to + touch the command line every time. Signed-off-by: Harald Nordgren ## Documentation/config/remote.adoc ## @@ Documentation/config/remote.adoc: remote..fetch:: - remote..refmap:: The default value of the `--refmap` option for linkgit:git-fetch[1]. -- Only takes effect when the fetch names what to fetch explicitly, -- either on the command line or via `remote..fetch`. See the -- `--refmap` entry in linkgit:git-fetch[1]. -+ If `remote..fetch` is not set either, a refspec-less fetch -+ infers what to fetch from local branches built on this remote, -+ instead of fetching every branch it has. See the `--refmap` entry -+ in linkgit:git-fetch[1]. + Used to map remote refs being fetched to remote-tracking refs to +- store. See the `--refmap` entry in linkgit:git-fetch[1]. ++ store. If `remote..fetch` is not set either, a refspec-less ++ fetch infers what to fetch from local branches built on this ++ remote, instead of fetching every branch it has. See the ++ `--refmap` entry in linkgit:git-fetch[1]. remote..push:: The default set of "refspec" for linkgit:git-push[1]. See @@ Documentation/fetch-options.adoc: endif::git-pull[] +When a refmap is active (from `--refmap` or `remote..refmap`) but +there is nothing to fetch, neither on the command line nor from +`remote..fetch`, Git infers what to fetch from the local branches -+whose `@{upstream}` is on that remote, plus the remote's default branch, -+which is always included so that it is available even before anything -+is set up to track it. ++whose `@{upstream}` is on that remote. `-t`:: `--tags`:: ## builtin/fetch.c ## -@@ builtin/fetch.c: static void filter_prefetch_refspec(struct refspec *rs) - static struct ref *get_ref_map(struct remote *remote, - const struct ref *remote_refs, - struct refspec *rs, -- int tags, int *autotags) -+ int tags, int *autotags, -+ char **bootstrap_head_branch) - { - int i; - struct ref *rm; @@ builtin/fetch.c: static struct ref *get_ref_map(struct remote *remote, struct ref **tail = &ref_map; struct refspec *effective_refmap = @@ builtin/fetch.c: static struct ref *get_ref_map(struct remote *remote, + effective_refmap && effective_refmap->nr) { + struct string_list tracked = STRING_LIST_INIT_DUP; + struct string_list_item *item; -+ struct ref *head; -+ char *default_branch = NULL; -+ char *default_branch_dst = NULL; + + branches_tracking_remote(remote, &tracked); + for_each_string_list_item(item, &tracked) + refspec_append(&inferred_rs, item->string); -+ -+ /* Always fetch the default branch too, as "HEAD". */ -+ head = get_remote_ref(remote_refs, "HEAD"); -+ if (head && head->symref && *head->symref) -+ default_branch = xstrdup(head->symref); -+ free_one_ref(head); -+ -+ for (i = 0; default_branch && !default_branch_dst && -+ !string_list_has_string(&tracked, default_branch) && -+ i < effective_refmap->nr; i++) { -+ struct refspec_item *map = &effective_refmap->items[i]; -+ -+ if (map->pattern) -+ match_refname_with_pattern(map->src, default_branch, -+ map->dst, &default_branch_dst); -+ else if (!strcmp(map->src, default_branch)) -+ default_branch_dst = xstrdup(map->dst); -+ } -+ -+ if (default_branch_dst) { -+ struct refspec_item head_item = { .force = 1 }; -+ -+ head_item.src = xstrdup("HEAD"); -+ head_item.dst = default_branch_dst; -+ get_fetch_map(remote_refs, &head_item, &tail, 1); -+ free(head_item.src); -+ } -+ -+ if (default_branch && bootstrap_head_branch) { -+ const char *branch_name = default_branch; -+ skip_prefix(branch_name, "refs/heads/", &branch_name); -+ *bootstrap_head_branch = xstrdup(branch_name); -+ } -+ -+ free(default_branch); -+ free(default_branch_dst); + string_list_clear(&tracked, 0); + + rs = &inferred_rs; @@ builtin/fetch.c: static struct ref *get_ref_map(struct remote *remote, return ref_map; } -@@ builtin/fetch.c: static void warn_set_head(const char *remote, const char *head_name, - } - - static int set_head(const struct ref *remote_refs, struct remote *remote, -- int follow_remote_head) -+ int follow_remote_head, const char *known_head_branch) - { - int result = 0, create_only, baremirror, was_detached; - struct strbuf b_head = STRBUF_INIT, b_remote_head = STRBUF_INIT, - b_local_head = STRBUF_INIT; - const char *no_warn_branch = remote->no_warn_branch; - char *head_name = NULL; -- struct ref *ref, *matches; -+ struct ref *ref, *matches = NULL; - struct ref *fetch_map = NULL, **fetch_map_tail = &fetch_map; - struct refspec_item refspec = { - .force = 0, -@@ builtin/fetch.c: static int set_head(const struct ref *remote_refs, struct remote *remote, - struct string_list heads = STRING_LIST_INIT_DUP; - struct ref_store *refs = get_main_ref_store(the_repository); - -- get_fetch_map(remote_refs, &refspec, &fetch_map_tail, 0); -- matches = guess_remote_head(find_ref_by_name(remote_refs, "HEAD"), -- fetch_map, REMOTE_GUESS_HEAD_ALL); -- for (ref = matches; ref; ref = ref->next) { -- string_list_append(&heads, strip_refshead(ref->name)); -- } -+ if (known_head_branch) { -+ head_name = xstrdup(known_head_branch); -+ } else { -+ get_fetch_map(remote_refs, &refspec, &fetch_map_tail, 0); -+ matches = guess_remote_head(find_ref_by_name(remote_refs, "HEAD"), -+ fetch_map, REMOTE_GUESS_HEAD_ALL); -+ for (ref = matches; ref; ref = ref->next) { -+ string_list_append(&heads, strip_refshead(ref->name)); -+ } - -- if (!heads.nr) -- result = 1; -- else if (heads.nr > 1) -- result = 1; -- else -- head_name = xstrdup(heads.items[0].string); -+ if (!heads.nr) -+ result = 1; -+ else if (heads.nr > 1) -+ result = 1; -+ else -+ head_name = xstrdup(heads.items[0].string); -+ } - - if (!head_name) - goto cleanup; -@@ builtin/fetch.c: static int do_fetch(struct transport *transport, - struct strmap rejected_refs = STRMAP_INIT; - int summary_width = 0; - int follow_remote_head; -+ char *bootstrap_head_branch = NULL; - - if (tags == TAGS_DEFAULT) { - if (transport->remote->fetch_tags == 2) @@ builtin/fetch.c: static int do_fetch(struct transport *transport, refspec_ref_prefixes(rs, &transport_ls_refs_options.ref_prefixes); } else { @@ builtin/fetch.c: static int do_fetch(struct transport *transport, + strvec_push(&transport_ls_refs_options.ref_prefixes, + item->string); + string_list_clear(&tracked, 0); -+ strvec_push(&transport_ls_refs_options.ref_prefixes, "HEAD"); + } else if (transport->remote->fetch.nr) { refspec_ref_prefixes(&transport->remote->fetch, &transport_ls_refs_options.ref_prefixes); @@ builtin/fetch.c: static int do_fetch(struct transport *transport, for (i = 0; i < branch->merge_nr; i++) { strvec_push(&transport_ls_refs_options.ref_prefixes, @@ builtin/fetch.c: static int do_fetch(struct transport *transport, - transport_ls_refs_options_release(&transport_ls_refs_options); ref_map = get_ref_map(transport->remote, remote_refs, rs, -- tags, &autotags); -+ tags, &autotags, &bootstrap_head_branch); + tags, &autotags); + if (!update_head_ok) check_not_current_branch(ref_map); -@@ builtin/fetch.c: static int do_fetch(struct transport *transport, - * Way too many cases where this can go wrong so let's just - * ignore errors and fail silently for now. - */ -- set_head(remote_refs, transport->remote, follow_remote_head); -+ set_head(remote_refs, transport->remote, follow_remote_head, -+ bootstrap_head_branch); - } - - cleanup: -+ free(bootstrap_head_branch); - /* - * When using batched updates, we want to commit the non-rejected - * updates and also handle the rejections. ## remote.c ## @@ remote.c: int branch_merge_matches(struct branch *branch, @@ t/t5585-fetch-refmap.sh (new) + +When a remote has a refmap configured but no fetch refspec, a +refspec-less fetch infers what to fetch from the local branches whose -+@{upstream} is on that remote, plus the default branch of that remote, -+which is always included so that it is available even before anything -+is set up to track it. ++@{upstream} is on that remote. +' + +GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main @@ t/t5585-fetch-refmap.sh (new) + ) +' + -+test_expect_success 'fetching the new remote does not need every branch it has' ' ++test_expect_success 'a bare fetch needs nothing until a branch is tracked' ' + ( + cd client && + git fetch upstream && + git for-each-ref --format="%(refname)" refs/remotes/upstream >actual && -+ cat >expect <<-\EOF && -+ refs/remotes/upstream/HEAD -+ refs/remotes/upstream/main -+ EOF -+ test_cmp expect actual ++ test_must_be_empty actual + ) +' + -+test_expect_success 'set-upstream-to now resolves right after that first fetch' ' ++test_expect_success 'an explicit one-time fetch lets a branch be tracked' ' + ( + cd client && -+ git branch --set-upstream-to=upstream && ++ git fetch upstream main && ++ git branch --set-upstream-to=upstream/main && + test_cmp_config upstream branch.main.remote && + test_cmp_config refs/heads/main branch.main.merge + ) 3: 604e584956 ! 3: e2f072c254 remote: add "git remote add --limited-fetch" @@ Documentation/git-remote.adoc: the `refs/remotes//` namespace, a refspec t +With `--limited-fetch` option, instead of a `remote..fetch` refspec +that tracks all branches, `remote..refmap` is set up so that a +refspec-less `git fetch ` only fetches branches our local branches -+are built on, plus the remote's default branch. See the `--refmap` entry -+in linkgit:git-fetch[1] for details. ++are built on. See the `--refmap` entry in linkgit:git-fetch[1] for ++details. ++ With `-m ` option, a symbolic-ref `refs/remotes//HEAD` is set up to point at remote's __ branch. See also the set-head command. 4: 31462e9445 ! 4: 43b9711a2c remote: default to --limited-fetch in a shallow repository @@ Documentation/git-remote.adoc: Add a remote named __ for the repository at @@ Documentation/git-remote.adoc: With `--limited-fetch` option, instead of a `remote..fetch` refspec that tracks all branches, `remote..refmap` is set up so that a refspec-less `git fetch ` only fetches branches our local branches - are built on, plus the remote's default branch. See the `--refmap` entry --in linkgit:git-fetch[1] for details. -+in linkgit:git-fetch[1] for details. `--no-limited-fetch` explicitly -+disables this, overriding the shallow-repository default described above. + are built on. See the `--refmap` entry in linkgit:git-fetch[1] for +-details. ++details. `--no-limited-fetch` explicitly disables this, overriding the ++shallow-repository default described above. + With `-m ` option, a symbolic-ref `refs/remotes//HEAD` is set up to point at remote's __ branch. See also the set-head command. @@ t/t5505-remote.sh: test_expect_success 'filters are listed by git remote -v only + test_cmp_config "+refs/heads/*:refs/remotes/upstream/*" \ + remote.upstream.refmap && + test_must_fail git config get remote.upstream.fetch && ++ git fetch upstream main && ++ git branch --set-upstream-to=upstream/main && ++ test_cmp_config upstream branch.main.remote && ++ test_cmp_config refs/heads/main branch.main.merge && + git fetch upstream && + git for-each-ref --format="%(refname)" refs/remotes/upstream >actual && + cat >expect <<-\EOF && + refs/remotes/upstream/HEAD + refs/remotes/upstream/main + EOF -+ test_cmp expect actual && -+ git branch --set-upstream-to=upstream && -+ test_cmp_config upstream branch.main.remote && -+ test_cmp_config refs/heads/main branch.main.merge ++ test_cmp expect actual + ) +' + -- gitgitgadget