[PATCH v4 0/8] fetch: introduce machine-parseable output

Hi,

this is the fourth version of my patch series to introduce a machine-parseable output format for git-fetch(1). It applies on top of e9dffbc7f1 (Merge branch 'ps/fetch-ref-update-reporting', 2023-04-06).

There's only a small set of change compared to v3:

    - Patch 1/8: The test added by this change has been significantly
      simplified.

    - Patch 8/8: Reworded the commit message to clarify that parallel
      fetches and `--recurse-submodules` aren't niche on their own, but
      are likely niche combined with `--porcelain` given that they are
      user-facing while `--porcelain` is script-facing.

    - Patch 8/8: Clarified that `--porcelain` takes precedence over the
      `fetch.output` config option in the user-facing documentation.
      I've also added a test to verify that this is the case.

    - Patch 8/8: Amended one test to not depend on the state that its
      preceding test cretaes.

Thanks to Glen for the feedback.

Patrick

Patrick Steinhardt (8):
  fetch: fix `--no-recurse-submodules` with multi-remote fetches
  fetch: split out tests for output format
  fetch: add a test to exercise invalid output formats
  fetch: fix missing from-reference when fetching HEAD:foo
  fetch: introduce `display_format` enum
  fetch: move display format parsing into main function
  fetch: move option related variables into main function
  fetch: introduce machine-parseable "porcelain" output format

 Documentation/fetch-options.txt |   7 +
 Documentation/git-fetch.txt     |   9 +
 builtin/fetch.c                 | 437 +++++++++++++++++++-------------
 t/t5510-fetch.sh                |  53 ----
 t/t5526-fetch-submodules.sh     |  19 ++
 t/t5574-fetch-output.sh         | 251 ++++++++++++++++++
 6 files changed, 552 insertions(+), 224 deletions(-)
 create mode 100755 t/t5574-fetch-output.sh

Range-diff against v3:
1:  4b2b0cfe15 ! 1:  d82b42ed34 fetch: fix `--no-recurse-submodules` with multi-remote fetches
    @@ t/t5526-fetch-submodules.sh: test_expect_success 'fetch --all with --recurse-sub
      '
      
     +test_expect_success "fetch --all with --no-recurse-submodules only fetches superproject" '
    -+	test_when_finished "git -C downstream remote remove second" &&
    ++	test_when_finished "rm -rf src_clone" &&
     +
    -+	# We need to add a second remote, otherwise --all falls back to the
    -+	# normal fetch-one case.
    -+	git -C downstream remote add second .. &&
    -+	git -C downstream fetch --all &&
    ++	git clone --recurse-submodules src src_clone &&
    ++	(
    ++		cd src_clone &&
    ++		git remote add secondary ../src &&
    ++		git config submodule.recurse true &&
    ++		git config fetch.parallel 0 &&
    ++		git fetch --all --no-recurse-submodules 2>../actual
    ++	) &&
     +
    -+	add_submodule_commits &&
    -+	add_superproject_commits &&
    -+	old_commit=$(git rev-parse --short HEAD~) &&
    -+	new_commit=$(git rev-parse --short HEAD) &&
    -+
    -+	git -C downstream fetch --all --no-recurse-submodules >actual.out 2>actual.err &&
    -+
    -+	cat >expect.out <<-EOF &&
    -+	Fetching origin
    -+	Fetching second
    ++	cat >expect <<-EOF &&
    ++	From ../src
    ++	 * [new branch]      master     -> secondary/master
     +	EOF
    -+
    -+	cat >expect.err <<-EOF &&
    -+	From $(test-tool path-utils real_path .)/.
    -+	   $old_commit..$new_commit  super      -> origin/super
    -+	From ..
    -+	   $old_commit..$new_commit  super      -> second/super
    -+	EOF
    -+
    -+	test_cmp expect.out actual.out &&
    -+	test_cmp expect.err actual.err
    ++	test_cmp expect actual
     +'
     +
      test_done
2:  6ebc7450ba = 2:  33112dc51a fetch: split out tests for output format
3:  78479922ac = 3:  006ea93afb fetch: add a test to exercise invalid output formats
4:  46e1266ab0 = 4:  e599ea6d33 fetch: fix missing from-reference when fetching HEAD:foo
5:  acc0f7f520 = 5:  80ac00b0c4 fetch: introduce `display_format` enum
6:  cd23440128 = 6:  826b8b7bc0 fetch: move display format parsing into main function
7:  edbc31013f = 7:  20f2e061d6 fetch: move option related variables into main function
8:  e132d9494e ! 8:  24ae381950 fetch: introduce machine-parseable "porcelain" output format
    @@ Commit message
         be ambiguous without also printing the repository in which the update
         happens.
     
    -    Considering that both multi-remote and submodule fetches are rather
    -    niche and likely not going to be useful for the majority of usecases
    -    these omissions feel acceptable. If usecases for either of these come up
    -    in the future though it is easy enough to add a new "porcelain-v2"
    -    format that adds this information.
    +    Considering that both multi-remote and submodule fetches are user-facing
    +    features, using them in conjunction with `--porcelain` that is intended
    +    for scripting purposes is likely not going to be useful in the majority
    +    of cases. With that in mind these restrictions feel acceptable. If
    +    usecases for either of these come up in the future though it is easy
    +    enough to add a new "porcelain-v2" format that adds this information.
     
         Signed-off-by: Patrick Steinhardt <ps@pks.im>
     
    @@ Documentation/fetch-options.txt: linkgit:git-config[1].
     +	Print the output to standard output in an easy-to-parse format for
     +	scripts. See section OUTPUT in linkgit:git-fetch[1] for details.
     ++
    -+This is incompatible with `--recurse-submodules=[yes|on-demand]`.
    ++This is incompatible with `--recurse-submodules=[yes|on-demand]` and takes
    ++precedence over the `fetch.output` config option.
     +
      ifndef::git-pull[]
      --[no-]write-fetch-head::
    @@ t/t5574-fetch-output.sh: test_expect_success 'fetch compact output' '
     +test_expect_success 'fetch porcelain with multiple remotes' '
     +	test_when_finished "rm -rf porcelain" &&
     +
    ++	git switch --create multiple-remotes &&
     +	git clone . porcelain &&
     +	git -C porcelain remote add second-remote "$PWD" &&
     +	git -C porcelain fetch second-remote &&
    @@ t/t5574-fetch-output.sh: test_expect_success 'fetch compact output' '
     +	new_commit=$(git rev-parse HEAD) &&
     +
     +	cat >expect <<-EOF &&
    -+	  $old_commit $new_commit refs/remotes/origin/force-updated
    -+	  $old_commit $new_commit refs/remotes/second-remote/force-updated
    ++	  $old_commit $new_commit refs/remotes/origin/multiple-remotes
    ++	  $old_commit $new_commit refs/remotes/second-remote/multiple-remotes
     +	EOF
     +
     +	git -C porcelain fetch --porcelain --all >actual 2>stderr &&
    @@ t/t5574-fetch-output.sh: test_expect_success 'fetch compact output' '
     +	test_must_fail git -C porcelain fetch --porcelain --recurse-submodules=on-demand 2>stderr &&
     +	test_cmp expect stderr
     +'
    ++
    ++test_expect_success 'fetch porcelain overrides fetch.output config' '
    ++	test_when_finished "rm -rf porcelain" &&
    ++
    ++	git switch --create config-override &&
    ++	git clone . porcelain &&
    ++	test_commit new-commit &&
    ++	old_commit=$(git rev-parse HEAD~) &&
    ++	new_commit=$(git rev-parse HEAD) &&
    ++
    ++	cat >expect <<-EOF &&
    ++	  $old_commit $new_commit refs/remotes/origin/config-override
    ++	* $ZERO_OID $new_commit refs/tags/new-commit
    ++	EOF
    ++
    ++	git -C porcelain -c fetch.output=compact fetch --porcelain >stdout 2>stderr &&
    ++	test_must_be_empty stderr &&
    ++	test_cmp expect stdout
    ++'
     +
      test_expect_success 'fetch output with HEAD' '
      	test_when_finished "rm -rf head" &&