git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v3 8/8] fetch: introduce machine-parseable "porcelain" output format

From
Glen Choo <chooglen@google.com>
Date
May 8, 2023, 23:42 UTC
Message-ID
<kl6lild21jbd.fsf@chooglen-macbookpro.roam.corp.google.com>
In-Reply-To
<e132d9494ee7d8c4eb3a7a82b6420743c8c7a9b0.1683113177.git.ps@pks.im>
This version looks great! I only have minor comments.
Patrick Steinhardt <ps@pks.im> writes:
Show 17 quoted lines
> A notable ommission here is that the output format does not include the
> remote from which a reference was fetched, which might be important
> information especially in the context of multi-remote fetches. But as
> such a format would require us to print the remote for every single
> reference update due to parallelizable fetches it feels wasteful for the
> most likely usecase, which is when fetching from a single remote.
>
> In a similar spirit, a second restriction is that this cannot be used
> with `--recurse-submodules`. This is because any reference updates would
> be ambiguous without also printing the repository in which the update
> happens.
>
> Considering that both multi-remote and submodule fetches are rather
> niche and likely not going to be useful for the majority of usecases
> these omissions feel acceptable. If usecases for either of these come up
> in the future though it is easy enough to add a new "porcelain-v2"
> format that adds this information.

As a point of clarification, I think these options aren't niche in themselves, but they are more user-facing, so using them _in conjunction with_ --porcelain is probably pretty niche, so I think this is okay for now.

Show 34 quoted lines
> --- a/Documentation/fetch-options.txt
> +++ b/Documentation/fetch-options.txt
> @@ -78,6 +78,12 @@ linkgit:git-config[1].
>  --dry-run::
>  	Show what would be done, without making any changes.
>  
> +--porcelain::
> +	Print the output to standard output in an easy-to-parse format for
> +	scripts. See section OUTPUT in linkgit:git-fetch[1] for details.
> ++
> +This is incompatible with `--recurse-submodules=[yes|on-demand]`.
> +
>  ifndef::git-pull[]
>  --[no-]write-fetch-head::
>  	Write the list of remote refs fetched in the `FETCH_HEAD`
> diff --git a/Documentation/git-fetch.txt b/Documentation/git-fetch.txt
> index fba66f1460..f123139c58 100644
> --- a/Documentation/git-fetch.txt
> +++ b/Documentation/git-fetch.txt
> @@ -204,6 +204,15 @@ representing the status of a single ref. Each line is of the form:
>   <flag> <summary> <from> -> <to> [<reason>]
>  -------------------------------
>  
> +When using `--porcelain`, the output format is intended to be
> +machine-parseable. In contrast to the human-readable output formats it
> +thus prints to standard output instead of standard error. Each line is
> +of the form:
> +
> +-------------------------------
> +<flag> <old-object-id> <new-object-id> <local-reference>
> +-------------------------------
> +
>  The status of up-to-date refs is shown only if the --verbose option is
>  used.
I hadn't commented on the docs before, but they are very clear. Thanks!
Show 17 quoted lines
> @@ -1830,6 +1857,7 @@ struct parallel_fetch_state {
>  	const char **argv;
>  	struct string_list *remotes;
>  	int next, result;
> +	enum display_format format;
>  };
>  
>  static int fetch_next_remote(struct child_process *cp,
> @@ -1849,7 +1877,7 @@ static int fetch_next_remote(struct child_process *cp,
>  	strvec_push(&cp->args, remote);
>  	cp->git_cmd = 1;
>  
> -	if (verbosity >= 0)
> +	if (verbosity >= 0 && state->format != DISPLAY_FORMAT_PORCELAIN)
>  		printf(_("Fetching %s\n"), remote);
>  
>  	return 1;

Here and elsewhere, I wonder if it's clearer to name the variable "porcelain" and separate it from "enum display_format". Then we can check "porcelain" directly instead of using "format == DISPLAY_FORMAT_PORCELAIN" as a proxy...

Show 19 quoted lines
> +static int opt_parse_porcelain(const struct option *opt, const char *arg, int unset)
> +{
> +	enum display_format *format = opt->value;
> +	*format = DISPLAY_FORMAT_PORCELAIN;
> +	return 0;
> +}
> +
>  int cmd_fetch(int argc, const char **argv, const char *prefix)
>  {
>  	const char *bundle_uri;
> @@ -2104,6 +2140,8 @@ int cmd_fetch(int argc, const char **argv, const char *prefix)
>  			    PARSE_OPT_OPTARG, option_fetch_parse_recurse_submodules),
>  		OPT_BOOL(0, "dry-run", &dry_run,
>  			 N_("dry run")),
> +		OPT_CALLBACK_F(0, "porcelain", &display_format, NULL, N_("machine-readable output"),
> +			       PARSE_OPT_NOARG|PARSE_OPT_NONEG, opt_parse_porcelain),
>  		OPT_BOOL(0, "write-fetch-head", &write_fetch_head,
>  			 N_("write fetched references to the FETCH_HEAD file")),
>  		OPT_BOOL('k', "keep", &keep, N_("keep downloaded pack")),

e.g. since we are reusing the "display_format" variable, we need to make sure we parse "--porcelain" after we read "fetch.output". I double-checked to make sure we were doing the right thing, though it would be nice to not have to worry about those sorts of things. This shouldn't hold up the series though.

Show 15 quoted lines
> +test_expect_success 'fetch porcelain with multiple remotes' '
> +	test_when_finished "rm -rf porcelain" &&
> +
> +	git clone . porcelain &&
> +	git -C porcelain remote add second-remote "$PWD" &&
> +	git -C porcelain fetch second-remote &&
> +
> +	test_commit --no-tag multi-commit &&
> +	old_commit=$(git rev-parse HEAD~) &&
> +	new_commit=$(git rev-parse HEAD) &&
> +
> +	cat >expect <<-EOF &&
> +	  $old_commit $new_commit refs/remotes/origin/force-updated
> +	  $old_commit $new_commit refs/remotes/second-remote/force-updated
> +	EOF

The only thing in this test that relies on the previous test is that HEAD is pointing to "force-updated", and it's hard to tell where HEAD is since the previous test is so long. Could we create a new branch instead?

Previous: Patrick SteinhardtNext: Patrick Steinhardt
Message 69 of 120 in “fetch: introduce machine-parseable output”
  1. 0/8 fetch: introduce machine-parseable outputPatrick Steinhardt, Apr 19, 2023
  2. 1/8 fetch: split out tests for output formatPatrick Steinhardt, Apr 19, 2023
  3. 2/8 fetch: add a test to exercise invalid output formatsPatrick Steinhardt, Apr 19, 2023
  4. 3/8 fetch: fix missing from-reference when fetching HEAD:fooPatrick Steinhardt, Apr 19, 2023
  5. Jacob KellerApr 26, 2023
  6. Patrick SteinhardtApr 27, 2023
  7. Jacob KellerApr 26, 2023
  8. Patrick SteinhardtApr 27, 2023
  9. Glen ChooApr 26, 2023
  10. Patrick SteinhardtApr 27, 2023
  11. 4/8 fetch: introduce `display_format` enumPatrick Steinhardt, Apr 19, 2023
  12. 5/8 fetch: move display format parsing into main functionPatrick Steinhardt, Apr 19, 2023
  13. 6/8 fetch: move option related variables into main functionPatrick Steinhardt, Apr 19, 2023
  14. 7/8 fetch: introduce new `--output-format` optionPatrick Steinhardt, Apr 19, 2023
  15. Glen ChooApr 26, 2023
  16. Patrick SteinhardtApr 27, 2023
  17. 8/8 fetch: introduce machine-parseable "porcelain" output formatPatrick Steinhardt, Apr 19, 2023
  18. Glen ChooApr 26, 2023
  19. Patrick SteinhardtApr 27, 2023
  20. Glen ChooApr 27, 2023
  21. Patrick SteinhardtApr 28, 2023
  22. Glen ChooApr 28, 2023
  23. Felipe ContrerasMay 2, 2023
  24. Felipe ContrerasApr 24, 2023
  25. Patrick SteinhardtApr 25, 2023
  26. Jacob KellerApr 26, 2023
  27. Junio C HamanoApr 26, 2023
  28. Jacob KellerApr 26, 2023
  29. Patrick SteinhardtApr 27, 2023
  30. Jacob KellerApr 27, 2023
  31. Glen ChooApr 27, 2023
  32. Junio C HamanoApr 26, 2023
  33. Glen ChooApr 26, 2023
  34. Glen ChooApr 26, 2023
  35. Jacob KellerApr 26, 2023
  36. 0/8 fetch: introduce machine-parseable outputPatrick Steinhardt, Apr 27, 2023
  37. 1/8 fetch: split out tests for output formatPatrick Steinhardt, Apr 27, 2023
  38. SZEDER GáborApr 29, 2023
  39. Patrick SteinhardtMay 3, 2023
  40. 2/8 fetch: add a test to exercise invalid output formatsPatrick Steinhardt, Apr 27, 2023
  41. 3/8 fetch: fix missing from-reference when fetching HEAD:fooPatrick Steinhardt, Apr 27, 2023
  42. Glen ChooApr 27, 2023
  43. Jacob KellerApr 27, 2023
  44. 4/8 fetch: introduce `display_format` enumPatrick Steinhardt, Apr 27, 2023
  45. 5/8 fetch: move display format parsing into main functionPatrick Steinhardt, Apr 27, 2023
  46. 6/8 fetch: move option related variables into main functionPatrick Steinhardt, Apr 27, 2023
  47. Junio C HamanoApr 27, 2023
  48. 7/8 fetch: introduce new `--output-format` optionPatrick Steinhardt, Apr 27, 2023
  49. Junio C HamanoApr 27, 2023
  50. Glen ChooApr 28, 2023
  51. Patrick SteinhardtMay 3, 2023
  52. Glen ChooApr 28, 2023
  53. Patrick SteinhardtMay 3, 2023
  54. Patrick SteinhardtMay 3, 2023
  55. 8/8 fetch: introduce machine-parseable "porcelain" output formatPatrick Steinhardt, Apr 27, 2023
  56. Jacob KellerApr 27, 2023
  57. Glen ChooApr 28, 2023
  58. 0/8 fetch: introduce machine-parseable outputPatrick Steinhardt, May 3, 2023
  59. 1/8 fetch: fix `--no-recurse-submodules` with multi-remote fetchesPatrick Steinhardt, May 3, 2023
  60. Glen ChooMay 8, 2023
  61. Patrick SteinhardtMay 9, 2023
  62. 2/8 fetch: split out tests for output formatPatrick Steinhardt, May 3, 2023
  63. 3/8 fetch: add a test to exercise invalid output formatsPatrick Steinhardt, May 3, 2023
  64. 4/8 fetch: fix missing from-reference when fetching HEAD:fooPatrick Steinhardt, May 3, 2023
  65. 5/8 fetch: introduce `display_format` enumPatrick Steinhardt, May 3, 2023
  66. 6/8 fetch: move display format parsing into main functionPatrick Steinhardt, May 3, 2023
  67. 7/8 fetch: move option related variables into main functionPatrick Steinhardt, May 3, 2023
  68. 8/8 fetch: introduce machine-parseable "porcelain" output formatPatrick Steinhardt, May 3, 2023
  69. Glen ChooMay 8, 2023
  70. Patrick SteinhardtMay 9, 2023
  71. Glen ChooMay 9, 2023
  72. Junio C HamanoMay 3, 2023
  73. Junio C HamanoMay 3, 2023
  74. Patrick SteinhardtMay 4, 2023
  75. Glen ChooMay 9, 2023
  76. Patrick SteinhardtMay 9, 2023
  77. 0/8 fetch: introduce machine-parseable outputPatrick Steinhardt, May 9, 2023
  78. 1/8 fetch: fix `--no-recurse-submodules` with multi-remote fetchesPatrick Steinhardt, May 9, 2023
  79. Junio C HamanoMay 9, 2023
  80. Glen ChooMay 9, 2023
  81. Patrick SteinhardtMay 10, 2023
  82. 2/8 fetch: split out tests for output formatPatrick Steinhardt, May 9, 2023
  83. 3/8 fetch: add a test to exercise invalid output formatsPatrick Steinhardt, May 9, 2023
  84. Junio C HamanoMay 9, 2023
  85. Patrick SteinhardtMay 10, 2023
  86. 4/8 fetch: fix missing from-reference when fetching HEAD:fooPatrick Steinhardt, May 9, 2023
  87. Junio C HamanoMay 9, 2023
  88. Patrick SteinhardtMay 10, 2023
  89. 5/8 fetch: introduce `display_format` enumPatrick Steinhardt, May 9, 2023
  90. Junio C HamanoMay 9, 2023
  91. Patrick SteinhardtMay 10, 2023
  92. 6/8 fetch: move display format parsing into main functionPatrick Steinhardt, May 9, 2023
  93. Junio C HamanoMay 9, 2023
  94. Glen ChooMay 9, 2023
  95. Patrick SteinhardtMay 10, 2023
  96. 7/8 fetch: move option related variables into main functionPatrick Steinhardt, May 9, 2023
  97. 8/8 fetch: introduce machine-parseable "porcelain" output formatPatrick Steinhardt, May 9, 2023
  98. Junio C HamanoMay 9, 2023
  99. Patrick SteinhardtMay 10, 2023
  100. 0/9 fetch: introduce machine-parseable outputPatrick Steinhardt, May 10, 2023
  101. 1/9 fetch: fix `--no-recurse-submodules` with multi-remote fetchesPatrick Steinhardt, May 10, 2023
  102. 2/9 fetch: split out tests for output formatPatrick Steinhardt, May 10, 2023
  103. 3/9 fetch: add a test to exercise invalid output formatsPatrick Steinhardt, May 10, 2023
  104. 4/9 fetch: print left-hand side when fetching HEAD:fooPatrick Steinhardt, May 10, 2023
  105. Glen ChooMay 12, 2023
  106. Jeff KingMay 13, 2023
  107. Patrick SteinhardtMay 15, 2023
  108. 5/9 fetch: refactor calculation of the display table widthPatrick Steinhardt, May 10, 2023
  109. Glen ChooMay 12, 2023
  110. 6/9 fetch: introduce `display_format` enumPatrick Steinhardt, May 10, 2023
  111. 7/9 fetch: lift up parsing of "fetch.output" config variablePatrick Steinhardt, May 10, 2023
  112. 8/9 fetch: move option related variables into main functionPatrick Steinhardt, May 10, 2023
  113. 9/9 fetch: introduce machine-parseable "porcelain" output formatPatrick Steinhardt, May 10, 2023
  114. Glen ChooMay 12, 2023
  115. Junio C HamanoMay 10, 2023
  116. Patrick SteinhardtMay 11, 2023
  117. Junio C HamanoMay 11, 2023
  118. Felipe ContrerasMay 11, 2023
  119. Glen ChooMay 12, 2023
  120. Patrick SteinhardtMay 12, 2023

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.