git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v3 0/3] builtin/fetch: skip unnecessary tasks when using --negotiate-only

From
Glen Choo <chooglen@google.com>
Date
Dec 22, 2021, 00:11 UTC
Message-ID
<20211222001134.28933-1-chooglen@google.com>
In-Reply-To
<20211217000235.68996-1-chooglen@google.com>

cmd_fetch() performs certain tasks with the assumption that objects are fetched, but `git fetch --negotiate-only` does not fetch objects, as its name implies. This is results in behavior that is unnecessary at best, and incorrect at worst:

* Submodules are updated if enabled by recurse.submodules=true, but
  negotiation fetch doesn't actually update the repo, so this doesn't
  make sense (introduced in [1]).
* Commit graphs will be written if enabled by
  fetch.writeCommitGraph=true. But according to
  Documentation/config/fetch.txt [2], this should only be done if a
  pack-file is downloaded.
* gc is run, but according to [3], we only do this because we expect
  `git fetch` to introduce objects.

Make `git fetch --negotiate-only` handle these tasks more rigorously by doing the following:

* Make cmd_fetch() return early if we know for certain that objects will
  not be fetched
* Disable submodule recursion and die() if a user explicitly asks for it

[1] 7dce19d374 (fetch/pull: Add the --recurse-submodules option, 2010-11-12) [2] 50f26bd035 (fetch: add fetch.writeCommitGraph config setting, 2019-09-02) [3] 131b8fcbfb (fetch: run gc --auto after fetching, 2013-01-26)

Changes since v2:
* added a prepatory patch that introduces a "goto cleanup"
* drop an unnecessary line move (as suggested by Junio)
* check for user-given --recurse-submodules and die() (as suggested by
  Jonathan and Junio)
* update --negotiate-only's documentation 
Changes since v1:
* added more context to commit message
* added a NEEDSWORK comment 
Glen Choo (3):
  builtin/fetch: use goto cleanup in cmd_fetch()
  builtin/fetch: skip unnecessary tasks when using --negotiate-only
  builtin/fetch: die on --negotiate-only and --recurse-submodules
 Documentation/fetch-options.txt |  1 +
 builtin/fetch.c                 | 41 +++++++++++++++++++++++++++++----
 t/t5516-fetch-push.sh           | 12 ++++++++++
 t/t5702-protocol-v2.sh          | 17 ++++++++++++++
 4 files changed, 67 insertions(+), 4 deletions(-)
Range-diff against v2:
-:  ---------- > 1:  97e2cba633 builtin/fetch: use goto cleanup in cmd_fetch()
1:  9d18270a41 ! 2:  3a3a04b649 builtin/fetch: skip unnecessary tasks when using --negotiate-only
    @@ Commit message
         return early whenever objects are not fetched, but for now this only
         considers --negotiate-only.
     
    +    A similar optimization would be to skip irrelevant tasks in `git fetch
    +    --dry-run`. This optimization was not done in this commit because a dry
    +    run will actually fetch objects; we'd presumably still want to recurse
    +    into submodules and run gc.
    +
         [1] 7dce19d374 (fetch/pull: Add the --recurse-submodules option, 2010-11-12)
         [2] 50f26bd035 (fetch: add fetch.writeCommitGraph config setting, 2019-09-02)
         [3] 131b8fcbfb (fetch: run gc --auto after fetching, 2013-01-26)
    @@ builtin/fetch.c: int cmd_fetch(int argc, const char **argv, const char *prefix)
     +		 * submodules, so there is no need to read .gitmodules.
     +		 */
     +		recurse_submodules = RECURSE_SUBMODULES_OFF;
    -+		if (!negotiation_tip.nr)
    -+			die(_("--negotiate-only needs one or more --negotiate-tip=*"));
     +	}
     +
      	if (recurse_submodules != RECURSE_SUBMODULES_OFF) {
      		int *sfjc = submodule_fetch_jobs_config == -1
      			    ? &submodule_fetch_jobs_config : NULL;
    -@@ builtin/fetch.c: int cmd_fetch(int argc, const char **argv, const char *prefix)
    - 		fetch_config_from_gitmodules(sfjc, rs);
    - 	}
    - 
    --	if (negotiate_only && !negotiation_tip.nr)
    --		die(_("--negotiate-only needs one or more --negotiate-tip=*"));
    --
    - 	if (deepen_relative) {
    - 		if (deepen_relative < 0)
    - 			die(_("Negative depth in --deepen is not supported"));
     @@ builtin/fetch.c: int cmd_fetch(int argc, const char **argv, const char *prefix)
      		result = fetch_multiple(&list, max_children);
      	}
      
    -+	string_list_clear(&list, 0);
    -+
     +	/*
     +	 * Skip irrelevant tasks because we know objects were not
     +	 * fetched.
    @@ builtin/fetch.c: int cmd_fetch(int argc, const char **argv, const char *prefix)
     +	 * of them.
     +	 */
     +	if (negotiate_only)
    -+		return result;
    ++		goto cleanup;
     +
      	if (!result && (recurse_submodules != RECURSE_SUBMODULES_OFF)) {
      		struct strvec options = STRVEC_INIT;
      		int max_children = max_jobs;
    -@@ builtin/fetch.c: int cmd_fetch(int argc, const char **argv, const char *prefix)
    - 		strvec_clear(&options);
    - 	}
    - 
    --	string_list_clear(&list, 0);
    --
    - 	prepare_repo_settings(the_repository);
    - 	if (fetch_write_commit_graph > 0 ||
    - 	    (fetch_write_commit_graph < 0 &&
     
      ## t/t5516-fetch-push.sh ##
     @@ t/t5516-fetch-push.sh: test_expect_success 'push with negotiation proceeds anyway even if negotiation f
-:  ---------- > 3:  97792b5214 builtin/fetch: die on --negotiate-only and --recurse-submodules
base-commit: abe6bb3905392d5eb6b01fa6e54d7e784e0522aa
-- 
2.33.GIT
Previous: Glen ChooNext: Glen Choo
Message 14 of 50 in “builtin/fetch: skip unnecessary tasks when using --negotiate-only”
  1. builtin/fetch: skip unnecessary tasks when using --negotiate-onlyGlen Choo, Dec 7, 2021
  2. Jonathan TanDec 9, 2021
  3. Glen ChooDec 9, 2021
  4. Jonathan TanDec 13, 2021
  5. Glen ChooDec 16, 2021
  6. builtin/fetch: skip unnecessary tasks when using --negotiate-onlyGlen Choo, Dec 17, 2021
  7. Junio C HamanoDec 17, 2021
  8. Glen ChooDec 20, 2021
  9. Junio C HamanoDec 20, 2021
  10. Glen ChooDec 20, 2021
  11. Junio C HamanoDec 20, 2021
  12. Glen ChooDec 21, 2021
  13. Glen ChooDec 21, 2021
  14. 0/3 builtin/fetch: skip unnecessary tasks when using --negotiate-onlyGlen Choo, Dec 22, 2021
  15. 1/3 builtin/fetch: use goto cleanup in cmd_fetch()Glen Choo, Dec 22, 2021
  16. 2/3 builtin/fetch: skip unnecessary tasks when using --negotiate-onlyGlen Choo, Dec 22, 2021
  17. Junio C HamanoDec 22, 2021
  18. Glen ChooDec 22, 2021
  19. Junio C HamanoDec 22, 2021
  20. Glen ChooDec 22, 2021
  21. 3/3 builtin/fetch: die on --negotiate-only and --recurse-submodulesGlen Choo, Dec 22, 2021
  22. Junio C HamanoDec 22, 2021
  23. Jonathan TanDec 23, 2021
  24. 0/3 fetch: skip unnecessary tasks when using --negotiate-onlyGlen Choo, Jan 13, 2022
  25. 1/3 fetch: use goto cleanup in cmd_fetch()Glen Choo, Jan 13, 2022
  26. 2/3 fetch: skip tasks related to fetching objectsGlen Choo, Jan 13, 2022
  27. 3/3 fetch --negotiate-only: do not update submodulesGlen Choo, Jan 13, 2022
  28. Junio C HamanoJan 13, 2022
  29. 0/3 fetch: skip unnecessary tasks when using --negotiate-onlyGlen Choo, Jan 18, 2022
  30. 1/3 fetch: use goto cleanup in cmd_fetch()Glen Choo, Jan 18, 2022
  31. 2/3 fetch: skip tasks related to fetching objectsGlen Choo, Jan 18, 2022
  32. 3/3 fetch --negotiate-only: do not update submodulesGlen Choo, Jan 18, 2022
  33. Junio C HamanoJan 18, 2022
  34. Glen ChooJan 18, 2022
  35. Junio C HamanoJan 19, 2022
  36. 0/3 fetch: skip unnecessary tasks when using --negotiate-onlyGlen Choo, Jan 19, 2022
  37. 1/3 fetch: use goto cleanup in cmd_fetch()Glen Choo, Jan 19, 2022
  38. 2/3 fetch: skip tasks related to fetching objectsGlen Choo, Jan 19, 2022
  39. 3/3 fetch --negotiate-only: do not update submodulesGlen Choo, Jan 19, 2022
  40. Jiang XinJan 20, 2022
  41. Glen ChooJan 20, 2022
  42. 0/3 fetch: skip unnecessary tasks when using --negotiate-onlyGlen Choo, Jan 20, 2022
  43. 1/3 fetch: use goto cleanup in cmd_fetch()Glen Choo, Jan 20, 2022
  44. 2/3 fetch: skip tasks related to fetching objectsGlen Choo, Jan 20, 2022
  45. 3/3 fetch --negotiate-only: do not update submodulesGlen Choo, Jan 20, 2022
  46. Junio C HamanoJan 20, 2022
  47. Glen ChooJan 20, 2022
  48. Re* [PATCH v7 0/3] fetch: skip unnecessary tasks when using --negotiate-onlyJunio C Hamano, Jan 20, 2022
  49. Glen ChooJan 20, 2022
  50. Jiang XinJan 21, 2022

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.