git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v4 3/4] maintenance: use 'git fetch --prefetch'

From
Derrick Stolee via GitGitGadget <gitgitgadget@gmail.com>
Date
Apr 16, 2021, 12:49 UTC
Message-ID
<565ed8a1892946a75c1116b0f351725e64fa90ba.1618577399.git.gitgitgadget@gmail.com>
In-Reply-To
<pull.924.v4.git.1618577399.gitgitgadget@gmail.com>
From: Derrick Stolee <dstolee@microsoft.com>

The 'prefetch' maintenance task previously forced the following refspec for each remote:

	+refs/heads/*:refs/prefetch/<remote>/*

If a user has specified a more strict refspec for the remote, then this prefetch task downloads more objects than necessary.

The previous change introduced the '--prefetch' option to 'git fetch' which manipulates the remote's refspec to place all resulting refs into refs/prefetch/, with further partitioning based on the destinations of those refspecs.

Update the documentation to be more generic about the destination refs. Do not mention custom refspecs explicitly, as that does not need to be highlighted in this documentation. The important part of placing refs in refs/prefetch/ remains.

Reported-by: Tom Saeger <tom.saeger@oracle.com>
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
---
 Documentation/git-maintenance.txt |  6 ++----
 builtin/gc.c                      |  7 +++----
 t/t7900-maintenance.sh            | 14 +++++++-------
 3 files changed, 12 insertions(+), 15 deletions(-)
diff --git a/Documentation/git-maintenance.txt b/Documentation/git-maintenance.txt
index 80ddd33ceba0..1e738ad39832 100644
--- a/Documentation/git-maintenance.txt
+++ b/Documentation/git-maintenance.txt
@@ -92,10 +92,8 @@ commit-graph::
 prefetch::
 	The `prefetch` task updates the object directory with the latest
 	objects from all registered remotes. For each remote, a `git fetch`
-	command is run. The refmap is custom to avoid updating local or remote
-	branches (those in `refs/heads` or `refs/remotes`). Instead, the
-	remote refs are stored in `refs/prefetch/<remote>/`. Also, tags are
-	not updated.
+	command is run. The configured refspec is modified to place all
+	requested refs within `refs/prefetch/`. Also, tags are not updated.
 +
 This is done to avoid disrupting the remote-tracking branches. The end users
 expect these refs to stay unmoved unless they initiate a fetch.  With prefetch
diff --git a/builtin/gc.c b/builtin/gc.c
index fa8128de9ae1..9d35f7da50d8 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -879,15 +879,14 @@ static int fetch_remote(struct remote *remote, void *cbdata)
 	struct child_process child = CHILD_PROCESS_INIT;
 
 	child.git_cmd = 1;
-	strvec_pushl(&child.args, "fetch", remote->name, "--prune", "--no-tags",
+	strvec_pushl(&child.args, "fetch", remote->name,
+		     "--prefetch", "--prune", "--no-tags",
 		     "--no-write-fetch-head", "--recurse-submodules=no",
-		     "--refmap=", NULL);
+		     NULL);
 
 	if (opts->quiet)
 		strvec_push(&child.args, "--quiet");
 
-	strvec_pushf(&child.args, "+refs/heads/*:refs/prefetch/%s/*", remote->name);
-
 	return !!run_command(&child);
 }
 
diff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh
index 2412d8c5c006..eadb800c08cc 100755
--- a/t/t7900-maintenance.sh
+++ b/t/t7900-maintenance.sh
@@ -141,15 +141,15 @@ test_expect_success 'prefetch multiple remotes' '
 	test_commit -C clone1 one &&
 	test_commit -C clone2 two &&
 	GIT_TRACE2_EVENT="$(pwd)/run-prefetch.txt" git maintenance run --task=prefetch 2>/dev/null &&
-	fetchargs="--prune --no-tags --no-write-fetch-head --recurse-submodules=no --refmap= --quiet" &&
-	test_subcommand git fetch remote1 $fetchargs +refs/heads/\\*:refs/prefetch/remote1/\\* <run-prefetch.txt &&
-	test_subcommand git fetch remote2 $fetchargs +refs/heads/\\*:refs/prefetch/remote2/\\* <run-prefetch.txt &&
+	fetchargs="--prefetch --prune --no-tags --no-write-fetch-head --recurse-submodules=no --quiet" &&
+	test_subcommand git fetch remote1 $fetchargs <run-prefetch.txt &&
+	test_subcommand git fetch remote2 $fetchargs <run-prefetch.txt &&
 	test_path_is_missing .git/refs/remotes &&
-	git log prefetch/remote1/one &&
-	git log prefetch/remote2/two &&
+	git log prefetch/remotes/remote1/one &&
+	git log prefetch/remotes/remote2/two &&
 	git fetch --all &&
-	test_cmp_rev refs/remotes/remote1/one refs/prefetch/remote1/one &&
-	test_cmp_rev refs/remotes/remote2/two refs/prefetch/remote2/two &&
+	test_cmp_rev refs/remotes/remote1/one refs/prefetch/remotes/remote1/one &&
+	test_cmp_rev refs/remotes/remote2/two refs/prefetch/remotes/remote2/two &&
 
 	test_cmp_config refs/prefetch/ log.excludedecoration &&
 	git log --oneline --decorate --all >log &&
-- 
gitgitgadget
Previous: Tom SaegerNext: Derrick Stolee via GitGitGadget
Message 68 of 72 in “Maintenance: adapt custom refspecs”
  1. 0/5 Maintenance: adapt custom refspecsDerrick Stolee via GitGitGadget, Apr 5, 2021
  2. 2/5 test-lib: use exact match for test_subcommandDerrick Stolee via GitGitGadget, Apr 5, 2021
  3. Eric SunshineApr 5, 2021
  4. Junio C HamanoApr 5, 2021
  5. 1/5 maintenance: simplify prefetch logicDerrick Stolee via GitGitGadget, Apr 5, 2021
  6. Tom SaegerApr 5, 2021
  7. 3/5 refspec: output a refspec itemDerrick Stolee via GitGitGadget, Apr 5, 2021
  8. Tom SaegerApr 5, 2021
  9. Eric SunshineApr 5, 2021
  10. Junio C HamanoApr 5, 2021
  11. Derrick StoleeApr 6, 2021
  12. Eric SunshineApr 6, 2021
  13. Derrick StoleeApr 6, 2021
  14. Ævar Arnfjörð BjarmasonApr 7, 2021
  15. Derrick StoleeApr 7, 2021
  16. Ævar Arnfjörð BjarmasonApr 7, 2021
  17. Junio C HamanoApr 7, 2021
  18. Ævar Arnfjörð BjarmasonApr 7, 2021
  19. Junio C HamanoApr 8, 2021
  20. 4/5 test-tool: test refspec input/outputDerrick Stolee via GitGitGadget, Apr 5, 2021
  21. Eric SunshineApr 5, 2021
  22. Derrick StoleeApr 6, 2021
  23. Ævar Arnfjörð BjarmasonApr 7, 2021
  24. 5/5 maintenance: allow custom refspecs during prefetchDerrick Stolee via GitGitGadget, Apr 5, 2021
  25. Tom SaegerApr 5, 2021
  26. Derrick StoleeApr 6, 2021
  27. Ævar Arnfjörð BjarmasonApr 7, 2021
  28. Ævar Arnfjörð BjarmasonApr 7, 2021
  29. Derrick StoleeApr 9, 2021
  30. Ævar Arnfjörð BjarmasonApr 9, 2021
  31. Derrick StoleeApr 10, 2021
  32. Ævar Arnfjörð BjarmasonApr 10, 2021
  33. Ævar Arnfjörð BjarmasonApr 7, 2021
  34. 0/5 Maintenance: adapt custom refspecsDerrick Stolee via GitGitGadget, Apr 6, 2021
  35. 1/5 maintenance: simplify prefetch logicDerrick Stolee via GitGitGadget, Apr 6, 2021
  36. Emily ShafferApr 7, 2021
  37. Derrick StoleeApr 9, 2021
  38. 2/5 test-lib: use exact match for test_subcommandDerrick Stolee via GitGitGadget, Apr 6, 2021
  39. 4/5 test-tool: test refspec input/outputDerrick Stolee via GitGitGadget, Apr 6, 2021
  40. Josh SteadmonApr 7, 2021
  41. Emily ShafferApr 7, 2021
  42. 3/5 refspec: output a refspec itemDerrick Stolee via GitGitGadget, Apr 6, 2021
  43. 5/5 maintenance: allow custom refspecs during prefetchDerrick Stolee via GitGitGadget, Apr 6, 2021
  44. Tom SaegerApr 6, 2021
  45. Derrick StoleeApr 6, 2021
  46. Josh SteadmonApr 7, 2021
  47. Emily ShafferApr 7, 2021
  48. Jonathan TanApr 8, 2021
  49. 0/3 Maintenance: adapt custom refspecsDerrick Stolee via GitGitGadget, Apr 10, 2021
  50. 2/3 fetch: add --prefetch optionDerrick Stolee via GitGitGadget, Apr 10, 2021
  51. Ramsay JonesApr 11, 2021
  52. Derrick StoleeApr 12, 2021
  53. 3/3 maintenance: use 'git fetch --prefetch'Derrick Stolee via GitGitGadget, Apr 10, 2021
  54. 1/3 maintenance: simplify prefetch logicDerrick Stolee via GitGitGadget, Apr 10, 2021
  55. Tom SaegerApr 12, 2021
  56. Derrick StoleeApr 12, 2021
  57. Junio C HamanoApr 11, 2021
  58. Tom SaegerApr 12, 2021
  59. Tom SaegerApr 12, 2021
  60. Tom SaegerApr 12, 2021
  61. Derrick StoleeApr 12, 2021
  62. 0/4 Maintenance: adapt custom refspecsDerrick Stolee via GitGitGadget, Apr 16, 2021
  63. 1/4 maintenance: simplify prefetch logicDerrick Stolee via GitGitGadget, Apr 16, 2021
  64. Tom SaegerApr 16, 2021
  65. 2/4 fetch: add --prefetch optionDerrick Stolee via GitGitGadget, Apr 16, 2021
  66. Tom SaegerApr 16, 2021
  67. Tom SaegerApr 16, 2021
  68. 3/4 maintenance: use 'git fetch --prefetch'Derrick Stolee via GitGitGadget, Apr 16, 2021
  69. 4/4 maintenance: respect remote.*.skipFetchAllDerrick Stolee via GitGitGadget, Apr 16, 2021
  70. Ævar Arnfjörð BjarmasonApr 16, 2021
  71. Tom SaegerApr 16, 2021
  72. Tom SaegerApr 16, 2021

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.