git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[RFC/PATCH 11/11] WIP fetch: add a --fetch-prune option and fetch.pruneTags config

From
Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Date
Jan 19, 2018, 00:00 UTC
Message-ID
<20180119000027.28898-12-avarab@gmail.com>
In-Reply-To
<20180119000027.28898-1-avarab@gmail.com>

[WIP: This doesn't (yet) work as advertised, see further WIP note below]

Add a --fetch-prune option to git-fetch along with fetch.pruneTags config option. This allows for doing:

    git fetch origin -p -P
Or simply:
    git config fetch.prune true &&
    git config fetch.pruneTags true &&
    git fetch origin
Instead of the much more verbose:
    git fetch --prune origin 'refs/tags/*:refs/tags/*' '+refs/heads/*:refs/remotes/origin/*'

A use-case of git which is a pain to support now is frequent pulling from a repo which is having both its branches *and* tags deleted regularly. At work we create deployment tags in the repo for each rollout, and there's *lots* of those, so they're archived within weeks for performance reasons.

Without this change it's hard to centrally configure such repos in /etc/gitconfig (on servers that are only used for working with them). You need to set fetch.prune=true globally, and then for each repo:

    git -C {} config --replace-all remote.origin.fetch "refs/tags/*:refs/tags/*" "^refs/tags/*:refs/tags/*$"

Now I can simply set fetch.pruneTags=true in /etc/gitconfig as well, and users running "git pull" will automatically get the pruning semantics we want.

See my "Re: [BUG] git remote prune removes local tags, depending on fetch config" (87po6ahx87.fsf@evledraar.gmail.com; https://public-inbox.org/git/87po6ahx87.fsf@evledraar.gmail.com/) for more details.

[WIP: This doesn't actually work, since the implementation is to just pretend that 'refs/tags/*:refs/tags/*' was supplied on the command-line, we'll consequently stop fetching any branches, since we don't use the default refspec found in .git/config anymore

I'm not familiar enough with this code and the refspec machinery to know if this argument injection is crazy, and how to tell it to *also* do whatever it's doing by default]

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 Documentation/config.txt        | 15 +++++++++++++++
 Documentation/fetch-options.txt | 15 ++++++++++++++-
 builtin/fetch.c                 | 37 ++++++++++++++++++++++++++++++++++---
 remote.c                        |  2 ++
 remote.h                        |  1 +
 t/t5510-fetch.sh                |  1 +
 6 files changed, 67 insertions(+), 4 deletions(-)
diff --git a/Documentation/config.txt b/Documentation/config.txt
index 0f27af5760..476175ef8d 100644
--- a/Documentation/config.txt
+++ b/Documentation/config.txt
@@ -1401,6 +1401,14 @@ fetch.prune::
 	option was given on the command line.  See also `remote.<name>.prune`
 	and the PRUNING section of linkgit:git-fetch[1].
 
+fetch.pruneTags::
+	If true, fetch will automatically behave as if the
+	`refs/tags/*:refs/tags/*` refspec was provided when pruning,
+	if not set already. This allows for setting both this option
+	and `fetch.prune` to maintain a 1=1 mapping to upstrem
+	refs. See also `remote.<name>.pruneTags` and the PRUNING
+	section of linkgit:git-fetch[1].
+
 fetch.output::
 	Control how ref update status is printed. Valid values are
 	`full` and `compact`. Default value is `full`. See section
@@ -2945,6 +2953,13 @@ remote.<name>.prune::
 	remove any remote-tracking references that no longer exist on the
 	remote (as if the `--prune` option was given on the command line).
 	Overrides `fetch.prune` settings, if any.
+
+remote.<name>.pruneTags::
+	When set to true, fetching from this remote by default will also
+	remove any local tags that no longer exist on the remote (as
+	if the `--prune` option was given on the command line in
+	conjunction with the `refs/tags/*:refs/tags/*` refspec).
+	Overrides `fetch.prune` settings, if any.
 +
 See also `remote.<name>.prune` and the PRUNING section of
 linkgit:git-fetch[1].
diff --git a/Documentation/fetch-options.txt b/Documentation/fetch-options.txt
index 9f5c85ad96..dc13bed741 100644
--- a/Documentation/fetch-options.txt
+++ b/Documentation/fetch-options.txt
@@ -73,7 +73,20 @@ ifndef::git-pull[]
 	are fetched due to an explicit refspec (either on the command
 	line or in the remote configuration, for example if the remote
 	was cloned with the --mirror option), then they are also
-	subject to pruning.
+	subject to pruning. Supplying `--prune-tags` is a shorthand for
+	providing the tag refspec.
++
+See the PRUNING section below for more details.
+
+-P::
+--prune-tags::
+	Before fetching, remove any local tags that no longer exist on
+	the remote if `--prune` is enabled. This option should be used
+	more carefully, unlike `--prune` it will remove any local
+	references (local tags) that have been created. This option is
+	merely a shorthand for providing the explicit tag refspec
+	along with `--prune`, see the discussion about that in its
+	documentation.
 +
 See the PRUNING section below for more details.
 
diff --git a/builtin/fetch.c b/builtin/fetch.c
index b34665db9e..57669a000c 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -38,6 +38,10 @@ static int fetch_prune_config = -1; /* unspecified */
 static int prune = -1; /* unspecified */
 #define PRUNE_BY_DEFAULT 0 /* do we prune by default? */
 
+static int fetch_prune_tags_config = -1; /* unspecified */
+static int prune_tags = -1; /* unspecified */
+#define PRUNE_TAGS_BY_DEFAULT 0 /* do we prune tags by default? */
+
 static int all, append, dry_run, force, keep, multiple, update_head_ok, verbosity, deepen_relative;
 static int progress = -1;
 static int tags = TAGS_DEFAULT, unshallow, update_shallow, deepen;
@@ -64,6 +68,11 @@ static int git_fetch_config(const char *k, const char *v, void *cb)
 		return 0;
 	}
 
+	if (!strcmp(k, "fetch.prunetags")) {
+		fetch_prune_tags_config = git_config_bool(k, v);
+		return 0;
+	}
+
 	if (!strcmp(k, "submodule.recurse")) {
 		int r = git_config_bool(k, v) ?
 			RECURSE_SUBMODULES_ON : RECURSE_SUBMODULES_OFF;
@@ -126,6 +135,8 @@ static struct option builtin_fetch_options[] = {
 		    N_("number of submodules fetched in parallel")),
 	OPT_BOOL('p', "prune", &prune,
 		 N_("prune remote-tracking branches no longer on remote")),
+	OPT_BOOL('P', "prune-tags", &prune_tags,
+		 N_("prune local tags not found on remote")),
 	{ OPTION_CALLBACK, 0, "recurse-submodules", &recurse_submodules, N_("on-demand"),
 		    N_("control recursive fetching of submodules"),
 		    PARSE_OPT_OPTARG, option_fetch_parse_recurse_submodules },
@@ -1212,6 +1223,8 @@ static void add_options_to_argv(struct argv_array *argv)
 		argv_array_push(argv, "--dry-run");
 	if (prune != -1)
 		argv_array_push(argv, prune ? "--prune" : "--no-prune");
+	if (prune_tags != -1)
+		argv_array_push(argv, prune_tags ? "--prune-tags" : "--no-prune-tags");
 	if (update_head_ok)
 		argv_array_push(argv, "--update-head-ok");
 	if (force)
@@ -1270,6 +1283,7 @@ static int fetch_one(struct remote *remote, int argc, const char **argv)
 	static const char **refs = NULL;
 	struct refspec *refspec;
 	int ref_nr = 0;
+	int j = 0;
 	int exit_code;
 
 	if (!remote)
@@ -1288,10 +1302,27 @@ static int fetch_one(struct remote *remote, int argc, const char **argv)
 			prune = PRUNE_BY_DEFAULT;
 	}
 
+	if (prune_tags < 0) {
+		/* no command line request */
+		if (0 <= gtransport->remote->prune_tags)
+			prune_tags = gtransport->remote->prune_tags;
+		else if (0 <= fetch_prune_tags_config)
+			prune_tags = fetch_prune_tags_config;
+		else
+			prune_tags = PRUNE_TAGS_BY_DEFAULT;
+	}
+
+	if (argc > 0 || prune_tags) {
+		size_t nr_alloc = st_add3(argc, prune_tags, 1);
+		refs = xcalloc(nr_alloc, sizeof(const char *));
+		if (prune_tags) {
+			refs[j++] = xstrfmt("refs/tags/*:refs/tags/*");
+			ref_nr++;
+		}
+	}
+
 	if (argc > 0) {
-		int j = 0;
 		int i;
-		refs = xcalloc(st_add(argc, 1), sizeof(const char *));
 		for (i = 0; i < argc; i++) {
 			if (!strcmp(argv[i], "tag")) {
 				i++;
@@ -1301,8 +1332,8 @@ static int fetch_one(struct remote *remote, int argc, const char **argv)
 						    argv[i], argv[i]);
 			} else
 				refs[j++] = argv[i];
+			ref_nr++;
 		}
-		ref_nr = j;
 	}
 
 	sigchain_push_common(unlock_pack_on_signal);
diff --git a/remote.c b/remote.c
index 4e93753e19..9eb6d24009 100644
--- a/remote.c
+++ b/remote.c
@@ -391,6 +391,8 @@ static int handle_config(const char *key, const char *value, void *cb)
 		remote->skip_default_update = git_config_bool(key, value);
 	else if (!strcmp(subkey, "prune"))
 		remote->prune = git_config_bool(key, value);
+	else if (!strcmp(subkey, "prunetags"))
+		remote->prune_tags = git_config_bool(key, value);
 	else if (!strcmp(subkey, "url")) {
 		const char *v;
 		if (git_config_string(&v, key, value))
diff --git a/remote.h b/remote.h
index 1f6611be21..04d7994cbf 100644
--- a/remote.h
+++ b/remote.h
@@ -47,6 +47,7 @@ struct remote {
 	int skip_default_update;
 	int mirror;
 	int prune;
+	int prune_tags;
 
 	const char *receivepack;
 	const char *uploadpack;
diff --git a/t/t5510-fetch.sh b/t/t5510-fetch.sh
index 18280df4fc..1a822861e0 100755
--- a/t/t5510-fetch.sh
+++ b/t/t5510-fetch.sh
@@ -617,6 +617,7 @@ test_configured_prune () {
 test_configured_prune unset unset unset unset kept   kept   ""
 test_configured_prune unset unset unset unset kept   kept   "--no-prune"
 test_configured_prune unset unset unset unset pruned kept   "--prune"
+test_configured_prune unset unset unset unset pruned pruned "--prune --prune-tags" # WIP: broken, see commit msg
 test_configured_prune unset unset unset unset kept   pruned \
 	"--prune origin 'refs/tags/*:refs/tags/*'"
 test_configured_prune unset unset unset unset pruned pruned \

-- 
2.15.1.424.g9478a66081
Previous: Ævar Arnfjörð BjarmasonNext: Ævar Arnfjörð Bjarmason
Message 17 of 118 in “[BUG] git remote prune removes local tags, depending on fetch config”
  1. Michael GiuffridaJan 15, 2018
  2. Ævar Arnfjörð BjarmasonJan 16, 2018
  3. Michael GiuffridaJan 16, 2018
  4. Ævar Arnfjörð BjarmasonJan 16, 2018
  5. 00/11 document & test fetch pruning + WIP fetch.pruneTagsÆvar Arnfjörð Bjarmason, Jan 19, 2018
  6. 01/11 fetch tests: refactor in preparation for testing tag pruningÆvar Arnfjörð Bjarmason, Jan 19, 2018
  7. 02/11 fetch tests: arrange arguments for future readabilityÆvar Arnfjörð Bjarmason, Jan 19, 2018
  8. 03/11 fetch tests: add a tag to be deleted to the pruning testsÆvar Arnfjörð Bjarmason, Jan 19, 2018
  9. 04/11 fetch tests: double quote a variable for interpolationÆvar Arnfjörð Bjarmason, Jan 19, 2018
  10. 05/11 fetch tests: test --prune and refspec interactionÆvar Arnfjörð Bjarmason, Jan 19, 2018
  11. 06/11 git fetch doc: add a new section to explain the ins & outs of pruningÆvar Arnfjörð Bjarmason, Jan 19, 2018
  12. Eric SunshineJan 19, 2018
  13. 07/11 git remote doc: correct dangerous lies about what prune doesÆvar Arnfjörð Bjarmason, Jan 19, 2018
  14. 08/11 git-fetch & config doc: link to the new PRUNING sectionÆvar Arnfjörð Bjarmason, Jan 19, 2018
  15. 09/11 fetch: don't redundantly null something calloc() gave usÆvar Arnfjörð Bjarmason, Jan 19, 2018
  16. 10/11 fetch tests: add scaffolding for the new fetch.pruneTagsÆvar Arnfjörð Bjarmason, Jan 19, 2018
  17. 11/11 WIP fetch: add a --fetch-prune option and fetch.pruneTags configÆvar Arnfjörð Bjarmason, Jan 19, 2018
  18. 00/12 document & test fetch pruning & add fetch.pruneTagsÆvar Arnfjörð Bjarmason, Jan 21, 2018
  19. 01/12 fetch tests: refactor in preparation for testing tag pruningÆvar Arnfjörð Bjarmason, Jan 21, 2018
  20. 02/12 fetch tests: arrange arguments for future readabilityÆvar Arnfjörð Bjarmason, Jan 21, 2018
  21. 03/12 fetch tests: add a tag to be deleted to the pruning testsÆvar Arnfjörð Bjarmason, Jan 21, 2018
  22. 04/12 fetch tests: double quote a variable for interpolationÆvar Arnfjörð Bjarmason, Jan 21, 2018
  23. Junio C HamanoJan 22, 2018
  24. Ævar Arnfjörð BjarmasonJan 22, 2018
  25. 05/12 fetch tests: test --prune and refspec interactionÆvar Arnfjörð Bjarmason, Jan 21, 2018
  26. 06/12 git fetch doc: add a new section to explain the ins & outs of pruningÆvar Arnfjörð Bjarmason, Jan 21, 2018
  27. 07/12 git remote doc: correct dangerous lies about what prune doesÆvar Arnfjörð Bjarmason, Jan 21, 2018
  28. Junio C HamanoJan 22, 2018
  29. Ævar Arnfjörð BjarmasonJan 22, 2018
  30. 08/12 git-fetch & config doc: link to the new PRUNING sectionÆvar Arnfjörð Bjarmason, Jan 21, 2018
  31. 09/12 fetch: don't redundantly NULL something calloc() gave usÆvar Arnfjörð Bjarmason, Jan 21, 2018
  32. Ævar Arnfjörð BjarmasonJan 21, 2018
  33. 10/12 fetch: stop accessing "remote" variable indirectlyÆvar Arnfjörð Bjarmason, Jan 21, 2018
  34. 11/12 fetch tests: add scaffolding for the new fetch.pruneTagsÆvar Arnfjörð Bjarmason, Jan 21, 2018
  35. 12/12 fetch: add a --fetch-prune option and fetch.pruneTags configÆvar Arnfjörð Bjarmason, Jan 21, 2018
  36. Junio C HamanoJan 22, 2018
  37. Ævar Arnfjörð BjarmasonJan 22, 2018
  38. 00/11 document & test fetch pruning & add fetch.pruneTagsÆvar Arnfjörð Bjarmason, Jan 23, 2018
  39. 01/11 fetch: don't redundantly NULL something calloc() gave usÆvar Arnfjörð Bjarmason, Jan 23, 2018
  40. 03/11 fetch tests: refactor in preparation for testing tag pruningÆvar Arnfjörð Bjarmason, Jan 23, 2018
  41. 02/11 fetch: stop accessing "remote" variable indirectlyÆvar Arnfjörð Bjarmason, Jan 23, 2018
  42. Junio C HamanoJan 24, 2018
  43. 06/11 fetch tests: test --prune and refspec interactionÆvar Arnfjörð Bjarmason, Jan 23, 2018
  44. 07/11 git fetch doc: add a new section to explain the ins & outs of pruningÆvar Arnfjörð Bjarmason, Jan 23, 2018
  45. 09/11 git-fetch & config doc: link to the new PRUNING sectionÆvar Arnfjörð Bjarmason, Jan 23, 2018
  46. 10/11 fetch tests: add scaffolding for the new fetch.pruneTagsÆvar Arnfjörð Bjarmason, Jan 23, 2018
  47. 11/11 fetch: add a --fetch-prune option and fetch.pruneTags configÆvar Arnfjörð Bjarmason, Jan 23, 2018
  48. Junio C HamanoJan 24, 2018
  49. Ævar Arnfjörð BjarmasonJan 24, 2018
  50. Junio C HamanoJan 24, 2018
  51. 05/11 fetch tests: add a tag to be deleted to the pruning testsÆvar Arnfjörð Bjarmason, Jan 23, 2018
  52. 08/11 git remote doc: correct dangerous lies about what prune doesÆvar Arnfjörð Bjarmason, Jan 23, 2018
  53. 04/11 fetch tests: re-arrange arguments for future readabilityÆvar Arnfjörð Bjarmason, Jan 23, 2018
  54. Junio C HamanoJan 24, 2018
  55. Ævar Arnfjörð BjarmasonJan 24, 2018
  56. Ævar Arnfjörð BjarmasonFeb 6, 2018
  57. 00/17 document & test fetch pruning & add fetch.pruneTagsÆvar Arnfjörð Bjarmason, Feb 8, 2018
  58. 01/17 fetch: don't redundantly NULL something calloc() gave usÆvar Arnfjörð Bjarmason, Feb 8, 2018
  59. Eric SunshineFeb 9, 2018
  60. Ævar Arnfjörð BjarmasonFeb 9, 2018
  61. 02/17 fetch: trivially refactor assignment to ref_nrÆvar Arnfjörð Bjarmason, Feb 8, 2018
  62. Eric SunshineFeb 9, 2018
  63. 03/17 fetch: stop accessing "remote" variable indirectlyÆvar Arnfjörð Bjarmason, Feb 8, 2018
  64. 05/17 fetch tests: refactor in preparation for testing tag pruningÆvar Arnfjörð Bjarmason, Feb 8, 2018
  65. 07/17 fetch tests: add a tag to be deleted to the pruning testsÆvar Arnfjörð Bjarmason, Feb 8, 2018
  66. 11/17 fetch tests: fetch <url> <spec> as well as fetch [<remote>]Ævar Arnfjörð Bjarmason, Feb 8, 2018
  67. Eric SunshineFeb 9, 2018
  68. Ævar Arnfjörð BjarmasonFeb 9, 2018
  69. Jeff KingFeb 9, 2018
  70. Eric SunshineFeb 9, 2018
  71. Junio C HamanoFeb 9, 2018
  72. Junio C HamanoFeb 9, 2018
  73. 12/17 git fetch doc: add a new section to explain the ins & outs of pruningÆvar Arnfjörð Bjarmason, Feb 8, 2018
  74. Eric SunshineFeb 9, 2018
  75. 13/17 git remote doc: correct dangerous lies about what prune doesÆvar Arnfjörð Bjarmason, Feb 8, 2018
  76. Eric SunshineFeb 9, 2018
  77. 16/17 fetch: add a --fetch-prune option and fetch.pruneTags configÆvar Arnfjörð Bjarmason, Feb 8, 2018
  78. Eric SunshineFeb 9, 2018
  79. Ævar Arnfjörð BjarmasonFeb 9, 2018
  80. 15/17 fetch tests: add scaffolding for the new fetch.pruneTagsÆvar Arnfjörð Bjarmason, Feb 8, 2018
  81. 14/17 git-fetch & config doc: link to the new PRUNING sectionÆvar Arnfjörð Bjarmason, Feb 8, 2018
  82. 17/17 fetch: make the --fetch-prune work with <url>Ævar Arnfjörð Bjarmason, Feb 8, 2018
  83. Eric SunshineFeb 9, 2018
  84. Ævar Arnfjörð BjarmasonFeb 9, 2018
  85. 10/17 fetch tests: expand case/esac for later changeÆvar Arnfjörð Bjarmason, Feb 8, 2018
  86. 09/17 fetch tests: double quote a variable for interpolationÆvar Arnfjörð Bjarmason, Feb 8, 2018
  87. 06/17 fetch tests: re-arrange arguments for future readabilityÆvar Arnfjörð Bjarmason, Feb 8, 2018
  88. 08/17 fetch tests: test --prune and refspec interactionÆvar Arnfjörð Bjarmason, Feb 8, 2018
  89. 04/17 remote: add a macro for "refs/tags/*:refs/tags/*"Ævar Arnfjörð Bjarmason, Feb 8, 2018
  90. Ævar Arnfjörð BjarmasonFeb 8, 2018
  91. Junio C HamanoFeb 8, 2018
  92. Ævar Arnfjörð BjarmasonFeb 9, 2018
  93. 00/17 document & test fetch pruning & add fetch.pruneTagsÆvar Arnfjörð Bjarmason, Feb 9, 2018
  94. 01/17 fetch: don't redundantly NULL something calloc() gave usÆvar Arnfjörð Bjarmason, Feb 9, 2018
  95. 02/17 fetch: trivially refactor assignment to ref_nrÆvar Arnfjörð Bjarmason, Feb 9, 2018
  96. 03/17 fetch: stop accessing "remote" variable indirectlyÆvar Arnfjörð Bjarmason, Feb 9, 2018
  97. 04/17 remote: add a macro for "refs/tags/*:refs/tags/*"Ævar Arnfjörð Bjarmason, Feb 9, 2018
  98. 05/17 fetch tests: refactor in preparation for testing tag pruningÆvar Arnfjörð Bjarmason, Feb 9, 2018
  99. 08/17 fetch tests: test --prune and refspec interactionÆvar Arnfjörð Bjarmason, Feb 9, 2018
  100. 09/17 fetch tests: double quote a variable for interpolationÆvar Arnfjörð Bjarmason, Feb 9, 2018
  101. 10/17 fetch tests: expand case/esac for later changeÆvar Arnfjörð Bjarmason, Feb 9, 2018
  102. 11/17 fetch tests: fetch <url> <spec> as well as fetch [<remote>]Ævar Arnfjörð Bjarmason, Feb 9, 2018
  103. 13/17 git remote doc: correct dangerous lies about what prune doesÆvar Arnfjörð Bjarmason, Feb 9, 2018
  104. 12/17 git fetch doc: add a new section to explain the ins & outs of pruningÆvar Arnfjörð Bjarmason, Feb 9, 2018
  105. 06/17 fetch tests: re-arrange arguments for future readabilityÆvar Arnfjörð Bjarmason, Feb 9, 2018
  106. 07/17 fetch tests: add a tag to be deleted to the pruning testsÆvar Arnfjörð Bjarmason, Feb 9, 2018
  107. 14/17 git-fetch & config doc: link to the new PRUNING sectionÆvar Arnfjörð Bjarmason, Feb 9, 2018
  108. 15/17 fetch tests: add scaffolding for the new fetch.pruneTagsÆvar Arnfjörð Bjarmason, Feb 9, 2018
  109. 16/17 fetch: add a --prune-tags option and fetch.pruneTags configÆvar Arnfjörð Bjarmason, Feb 9, 2018
  110. 17/17 fetch: make the --prune-tags work with <url>Ævar Arnfjörð Bjarmason, Feb 9, 2018
  111. Junio C HamanoFeb 22, 2018
  112. Ævar Arnfjörð BjarmasonFeb 22, 2018
  113. Junio C HamanoFeb 22, 2018
  114. Ævar Arnfjörð BjarmasonFeb 23, 2018
  115. Junio C HamanoFeb 23, 2018
  116. Ævar Arnfjörð BjarmasonFeb 24, 2018
  117. Andreas SchwabJan 16, 2018
  118. Kevin DaudtJan 18, 2018

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.