From: Shubham Kanodia via GitGitGadget Date: Thu, 05 Sep 2024 02:52:05 GMT Subject: [PATCH v2] remote: prefetch config Message-ID: In-Reply-To: From: Shubham Kanodia Large repositories often contain numerous branches and refs, many of which individual users may not need. This commit introduces a new configuration option (`remote..prefetch`) to allow users to specify which remotes to prefetch during the maintenance task. Key behaviors: 1. If `remote..prefetch` is unset or true, running `git-maintenance` will prefetch all refs for the remote. 2. If `remote..prefetch` is set to false, the remote will be ignored for prefetching. In a future change, we could also allow restricting the refs that are prefetched per remote using the `prefetchref` config option per remote. Both of these options in unison would allow users to optimize their prefetch operations, reducing network traffic and disk usage. Signed-off-by: Shubham Kanodia --- remote: prefetch config Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1779%2Fpastelsky%2Fsk%2Fmaintenance-prefetch-remote-v2 Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1779/pastelsky/sk/maintenance-prefetch-remote-v2 Pull-Request: https://github.com/gitgitgadget/git/pull/1779 Range-diff vs v1: 1: 667553b114b ! 1: 1d58b782e22 remote: prefetch config @@ builtin/gc.c: static int fetch_remote(struct remote *remote, void *cbdata) if (remote->skip_default_update) return 0; -+ if (remote->prefetch == 0) ++ if (!remote->prefetch) + return 0; + child.git_cmd = 1; @@ remote.h: struct remote { struct refspec fetch; + /* -+ * This setting for whether to prefetch from a remote ++ * The setting for whether to prefetch from a remote + * when a fetch is invoked with a prefetch flag. + * -1 = unset + * 0 = don't prefetch from this remote @@ t/t7900-maintenance.sh: test_expect_success 'prefetch multiple remotes' ' + # Run maintenance prefetch task + GIT_TRACE2_EVENT="$(pwd)/prefetch.txt" git maintenance run --task=prefetch 2>/dev/null && + -+ # Check that remote1 was not fetched (prefetch=false) -+ test_subcommand ! git fetch remote1 --prefetch --prune --no-tags \ -+ --no-write-fetch-head --recurse-submodules=no --quiet \ -+ .fetch:: The default set of "refspec" for linkgit:git-fetch[1]. See linkgit:git-fetch[1]. +remote..prefetch:: + If false, refs from the remote would not be prefetched for + the prefetch task in linkgit:git-maintenance[1]. If not set, + the value is assumed to be true. + remote..push:: The default set of "refspec" for linkgit:git-push[1]. See linkgit:git-push[1]. diff --git a/Documentation/git-maintenance.txt b/Documentation/git-maintenance.txt index 51d0f7e94b6..2fd38706ea2 100644 --- a/Documentation/git-maintenance.txt +++ b/Documentation/git-maintenance.txt @@ -97,9 +97,10 @@ commit-graph:: prefetch:: The `prefetch` task updates the object directory with the latest - objects from all registered remotes. For each remote, a `git fetch` - command is run. The configured refspec is modified to place all - requested refs within `refs/prefetch/`. Also, tags are not updated. + objects from all registered remotes unless they've disabled prefetch + using `remote..prefetch` set to `false`. For each such remote, + a `git fetch` command is run. The configured refspec is modified to place + all requested refs within `refs/prefetch/`. Also, tags are not updated. + This is done to avoid disrupting the remote-tracking branches. The end users expect these refs to stay unmoved unless they initiate a fetch. However, diff --git a/builtin/gc.c b/builtin/gc.c index 427faf1cfe1..2ca3a3e7d6a 100644 --- a/builtin/gc.c +++ b/builtin/gc.c @@ -1027,6 +1027,9 @@ static int fetch_remote(struct remote *remote, void *cbdata) if (remote->skip_default_update) return 0; + if (!remote->prefetch) + return 0; + child.git_cmd = 1; strvec_pushl(&child.args, "fetch", remote->name, "--prefetch", "--prune", "--no-tags", diff --git a/remote.c b/remote.c index 8f3dee13186..05edb3a5f40 100644 --- a/remote.c +++ b/remote.c @@ -140,6 +140,7 @@ static struct remote *make_remote(struct remote_state *remote_state, CALLOC_ARRAY(ret, 1); ret->prune = -1; /* unspecified */ ret->prune_tags = -1; /* unspecified */ + ret->prefetch = -1; /* unspecified */ ret->name = xstrndup(name, len); refspec_init(&ret->push, REFSPEC_PUSH); refspec_init(&ret->fetch, REFSPEC_FETCH); @@ -456,6 +457,8 @@ static int handle_config(const char *key, const char *value, remote->prune = git_config_bool(key, value); else if (!strcmp(subkey, "prunetags")) remote->prune_tags = git_config_bool(key, value); + else if (!strcmp(subkey, "prefetch")) + remote->prefetch = git_config_bool(key, value); else if (!strcmp(subkey, "url")) { if (!value) return config_error_nonbool(key); diff --git a/remote.h b/remote.h index b901b56746d..4522fdec354 100644 --- a/remote.h +++ b/remote.h @@ -77,6 +77,15 @@ struct remote { struct refspec fetch; + /* + * The setting for whether to prefetch from a remote + * when a fetch is invoked with a prefetch flag. + * -1 = unset + * 0 = don't prefetch from this remote + * 1 = prefetch from this remote + */ + int prefetch; + /* * The setting for whether to fetch tags (as a separate rule from the * configured refspecs); diff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh index abae7a97546..7484e1f1d46 100755 --- a/t/t7900-maintenance.sh +++ b/t/t7900-maintenance.sh @@ -245,6 +245,48 @@ test_expect_success 'prefetch multiple remotes' ' test_subcommand git fetch remote2 $fetchargs /dev/null && + + # Check that if remotes were prefetched properly + fetchargs="--prefetch --prune --no-tags --no-write-fetch-head --recurse-submodules=no --quiet" && + test_subcommand ! git fetch remote1 $fetchargs