Re: [PATCH 2/9] maintenance: add prefetch task
- From
Emily Shaffer <emilyshaffer@google.com>
- Date
- Aug 12, 2020, 23:10 UTC
- Message-ID
- <20200812231036.GH2965447@google.com>
- In-Reply-To
- <85118ed5f19468d5051dd1579e35cae3c3114d24.1596731425.git.gitgitgadget@gmail.com>
On Thu, Aug 06, 2020 at 04:30:17PM +0000, Derrick Stolee via GitGitGadget wrote:
Show 6 quoted lines
> > 3. By adding a new refspec "+refs/heads/*:refs/prefetch/<remote>/*" > we can ensure that we actually load the new values somewhere in > our refspace while not updating refs/heads or refs/remotes. By > storing these refs here, the commit-graph job will update the > commit-graph with the commits from these hidden refs.
[emily] How does the content of refs/prefetch/* get used? [jrnieder] How does the content of refs/prefetch/* get cleaned up? [jonathantan] refs/prefetch/* will get used in negotiation so it is useful to keep these.
> > 4. --prune will delete the refs/prefetch/<remote> refs that no > longer appear on the remote.
[jrnieder] this is what cleans up refs/prefetch/* later :)
Show 7 quoted lines
> +static int fetch_remote(const char *remote, struct maintenance_opts *opts)
> +{
> + struct child_process child = CHILD_PROCESS_INIT;
> +
> + child.git_cmd = 1;
> + strvec_pushl(&child.args, "fetch", remote, "--prune", "--no-tags",
> + "--no-write-fetch-head", "--refmap=", NULL);[jonathantan] It would be good to pass --recurse-submodules=no. [jrnieder] Since we are specifying the refmap here, if we were to recurse into submodules, we'd have to be careful about making sure refmap gets propagated correctly.
> +static int fill_each_remote(struct remote *remote, void *cbdata)
[jrnieder] Since this is a callback that happens for each remote, maybe this should be named to indicate it only fills one remote at a time instead. Nit :)
Show 22 quoted lines
> +{
> + struct string_list *remotes = (struct string_list *)cbdata;
> +
> + string_list_append(remotes, remote->name);
> + return 0;
> +}
> +
> +static int maintenance_task_prefetch(struct maintenance_opts *opts)
> +{
> + int result = 0;
> + struct string_list_item *item;
> + struct string_list remotes = STRING_LIST_INIT_DUP;
> +
> + if (for_each_remote(fill_each_remote, &remotes)) {
> + error(_("failed to fill remotes"));
> + result = 1;
> + goto cleanup;
> + }
> +
> + for (item = remotes.items;
> + item && item < remotes.items + remotes.nr;
> + item++)Is there a reason not to use for_each_string_list_item() instead? This would be more brief and also easier to read (less thinking about what the loop is doing).
Show 7 quoted lines
> + result |= fetch_remote(item->string, opts); > + > +cleanup: > + string_list_clear(&remotes, 0); > + return result; > +} > +
> enum maintenance_task_label {
> + TASK_PREFETCH,[jrnieder] Nit: Is there a sort order for these? Should we establish an order early on (e.g. alphabetical)?
> TASK_GC, > TASK_COMMIT_GRAPH,
Show 12 quoted lines
> +test_expect_success 'prefetch multiple remotes' ' > + git clone . clone1 && > + git clone . clone2 && > + git remote add remote1 "file://$(pwd)/clone1" && > + git remote add remote2 "file://$(pwd)/clone2" && > + git -C clone1 switch -c one && > + git -C clone2 switch -c two && > + test_commit -C clone1 one && > + test_commit -C clone2 two && > + GIT_TRACE2_EVENT="$(pwd)/run-prefetch.txt" git maintenance run --task=prefetch 2>/dev/null && > + fetchargs="--prune --no-tags --no-write-fetch-head --refmap= --quiet" && > + test_subcommand git fetch remote1 $fetchargs +refs/heads/\\*:refs/prefetch/remote1/\\* <run-prefetch.txt &&
[jrnieder] In practice, why were all the \\ needed? Trying to figure out where Git is using a shell that would need the * escaped and finding it hard to reason about.
> + test_subcommand git fetch remote2 $fetchargs +refs/heads/\\*:refs/prefetch/remote2/\\* <run-prefetch.txt && > + test_path_is_missing .git/refs/remotes && > + test_cmp clone1/.git/refs/heads/one .git/refs/prefetch/remote1/one && > + test_cmp clone2/.git/refs/heads/two .git/refs/prefetch/remote2/two &&
[jrnieder] Should we use test_cmp_rev instead to make compatible with packed-refs?
> + git log prefetch/remote1/one && > + git log prefetch/remote2/two
[jonathantan] Why do we use 'git log' to check? I'm a little confused about what's going on; if you just want to check that the refs are present you could use 'git rev-parse' instead?