git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v3 0/4] Filter alternate references

From
Taylor Blau <me@ttaylorr.com>
Date
Sep 28, 2018, 04:25 UTC
Message-ID
<cover.1538108385.git.me@ttaylorr.com>
In-Reply-To
<cover.1537466087.git.me@ttaylorr.com>
Hi,

Attached is the third re-roll of mine and Peff's series to introduce 'core.alternateRefsCommand', and 'core.alternateRefsPrefixes' to filter the initial ".have" advertisement when an alternate has a pathologically large number of references.

A range-diff against v2 is included below, but the major changes between the two revisions are as follows:

  1. Documentation and testing clean-up, per helpful input from Junio,
     Peff, and brian carlson.
  2. Included also is a preparatory patch from Peff, to change the
     requirement that we provide refnames for alternate references. We
     no longer allow this, and the first commit sent makes that such
     change.

I imagine that we may hit one more re-roll, depending on the outcome of this review. The series has not fundamentally changed since v2, so I think that we are at a point of stasis there. Anything that is left outstanding from v3 should hopefully be similarly-not-earth-shattering ;-).

Thanks in advance for your review.

Thanks, Taylor

Jeff King (1):
  transport: drop refnames from for_each_alternate_ref
Taylor Blau (3):
  transport.c: extract 'fill_alternate_refs_command'
  transport.c: introduce core.alternateRefsCommand
  transport.c: introduce core.alternateRefsPrefixes
 Documentation/config.txt | 18 +++++++++++++
 builtin/receive-pack.c   |  3 +--
 fetch-pack.c             |  3 +--
 t/t5410-receive-pack.sh  | 57 ++++++++++++++++++++++++++++++++++++++++
 transport.c              | 38 +++++++++++++++++++++------
 transport.h              |  2 +-
 6 files changed, 108 insertions(+), 13 deletions(-)
 create mode 100755 t/t5410-receive-pack.sh
Range-diff against v2:
-:  ---------- > 1:  037273dab0 transport: drop refnames from for_each_alternate_ref
1:  6e3a58afe7 ! 2:  9479470cb1 transport.c: extract 'fill_alternate_refs_command'
    @@ -24,7 +24,7 @@
     +	cmd->git_cmd = 1;
     +	argv_array_pushf(&cmd->args, "--git-dir=%s", repo_path);
     +	argv_array_push(&cmd->args, "for-each-ref");
    -+	argv_array_push(&cmd->args, "--format=%(objectname) %(refname)");
    ++	argv_array_push(&cmd->args, "--format=%(objectname)");
     +	cmd->env = local_repo_env;
     +	cmd->out = -1;
     +}
    @@ -39,7 +39,7 @@
     -	cmd.git_cmd = 1;
     -	argv_array_pushf(&cmd.args, "--git-dir=%s", path);
     -	argv_array_push(&cmd.args, "for-each-ref");
    --	argv_array_push(&cmd.args, "--format=%(objectname) %(refname)");
    +-	argv_array_push(&cmd.args, "--format=%(objectname)");
     -	cmd.env = local_repo_env;
     -	cmd.out = -1;
     +	fill_alternate_refs_command(&cmd, path);
2:  9797f52551 ! 3:  2dbcd54190 transport.c: introduce core.alternateRefsCommand
    @@ -3,24 +3,24 @@
         transport.c: introduce core.alternateRefsCommand
         When in a repository containing one or more alternates, Git would
    -    sometimes like to list references from its alternates. For example, 'git
    -    receive-pack' list the objects pointed to by alternate references as
    -    special ".have" references.
    +    sometimes like to list references from those alternates. For example,
    +    'git receive-pack' lists the "tips" pointed to by references in those
    +    alternates as special ".have" references.
         Listing ".have" references is designed to make pushing changes from
         upstream to a fork a lightweight operation, by advertising to the pusher
         that the fork already has the objects (via its alternate). Thus, the
         client can avoid sending them.
    -    However, when the alternate has a pathologically large number of
    -    references, the initial advertisement is too expensive. In fact, it can
    -    dominate any such optimization where the pusher avoids sending certain
    -    objects.
    +    However, when the alternate (upstream, in the previous example) has a
    +    pathologically large number of references, the initial advertisement is
    +    too expensive. In fact, it can dominate any such optimization where the
    +    pusher avoids sending certain objects.
         Introduce "core.alternateRefsCommand" in order to provide a facility to
         limit or filter alternate references. This can be used, for example, to
    -    filter out "uninteresting" references from the initial advertisement in
    -    the above scenario.
    +    filter out references the alternate does not wish to send (for space
    +    concerns, or otherwise) during the initial advertisement.
         Let the repository that has alternates configure this command to avoid
         trusting the alternate to provide us a safe command to run in the shell.
    @@ -38,15 +38,15 @@
      	expect HEAD to be a symbolic link.
     +core.alternateRefsCommand::
    -+	When listing references from an alternate (e.g., in the case of ".have"), use
    -+	the shell to execute the specified command instead of
    -+	linkgit:git-for-each-ref[1]. The first argument is the path of the alternate.
    -+	Output must be of the form: `%(objectname) SPC %(refname)`.
    ++	When advertising tips of available history from an alternate, use the shell to
    ++	execute the specified command instead of linkgit:git-for-each-ref[1]. The
    ++	first argument is the absolute path of the alternate. Output must be of the
    ++	form: `%(objectname)`, where multiple tips are separated by newlines.
     ++
     +This is useful when a repository only wishes to advertise some of its
     +alternate's references as ".have"'s. For example, to only advertise branch
     +heads, configure `core.alternateRefsCommand` to the path of a script which runs
    -+`git --git-dir="$1" for-each-ref refs/heads`.
    ++`git --git-dir="$1" for-each-ref --format='%(objectname)' refs/heads`.
     +
      core.bare::
      	If true this repository is assumed to be 'bare' and has no
    @@ -74,8 +74,7 @@
     +	git clone fork pusher &&
     +	(
     +		cd fork &&
    -+		git config receive.advertisealternates true &&
    -+		cat <<-EOF | git update-ref --stdin &&
    ++		git update-ref --stdin <<-\EOF &&
     +		delete refs/heads/a
     +		delete refs/heads/b
     +		delete refs/heads/c
    @@ -88,23 +87,19 @@
     +	)
     +'
     +
    -+expect_haves () {
    -+	printf "%s .have\n" $(git rev-parse $@) >expect
    -+}
    -+
     +extract_haves () {
    -+	depacketize - | grep '\.have' | sed -e 's/\\0.*$//g'
    ++	depacketize | perl -lne '/^(\S+) \.have/ and print $1'
     +}
     +
     +test_expect_success 'with core.alternateRefsCommand' '
     +	write_script fork/alternate-refs <<-\EOF &&
     +		git --git-dir="$1" for-each-ref \
    -+			--format="%(objectname) %(refname)" \
    ++			--format="%(objectname)" \
     +			refs/heads/a \
     +			refs/heads/c
     +	EOF
     +	test_config -C fork core.alternateRefsCommand alternate-refs &&
    -+	expect_haves a c >expect &&
    ++	git rev-parse a c >expect &&
     +	printf "0000" | git receive-pack fork >actual &&
     +	extract_haves <actual >actual.haves &&
     +	test_cmp expect actual.haves
    @@ -122,7 +117,7 @@
     -	cmd->git_cmd = 1;
     -	argv_array_pushf(&cmd->args, "--git-dir=%s", repo_path);
     -	argv_array_push(&cmd->args, "for-each-ref");
    --	argv_array_push(&cmd->args, "--format=%(objectname) %(refname)");
    +-	argv_array_push(&cmd->args, "--format=%(objectname)");
     +	const char *value;
     +
     +	if (!git_config_get_value("core.alternateRefsCommand", &value)) {
    @@ -135,7 +130,7 @@
     +
     +		argv_array_pushf(&cmd->args, "--git-dir=%s", repo_path);
     +		argv_array_push(&cmd->args, "for-each-ref");
    -+		argv_array_push(&cmd->args, "--format=%(objectname) %(refname)");
    ++		argv_array_push(&cmd->args, "--format=%(objectname)");
     +	}
     +
      	cmd->env = local_repo_env;
3:  6e8f65a16d ! 4:  48eb774c9e transport.c: introduce core.alternateRefsPrefixes
    @@ -12,9 +12,7 @@
         'core.alternateRefsCommand' would have to do:
           $ git config core.alternateRefsCommand ' \
    -          git -C "$1" for-each-ref refs/tags \
    -          --format="%(objectname) %(refname)" \
    -        '
    +          git -C "$1" for-each-ref refs/tags --format="%(objectname)"'
         The above is cumbersome to write, so let's introduce a
         "core.alternateRefsPrefixes" to address this common case. Instead, the
    @@ -41,7 +39,7 @@
      +++ b/Documentation/config.txt
     @@
      heads, configure `core.alternateRefsCommand` to the path of a script which runs
    - `git --git-dir="$1" for-each-ref refs/heads`.
    + `git --git-dir="$1" for-each-ref --format='%(objectname)' refs/heads`.
     +core.alternateRefsPrefixes::
     +	When listing references from an alternate, list only references that begin
    @@ -63,7 +61,7 @@
     +test_expect_success 'with core.alternateRefsPrefixes' '
     +	test_config -C fork core.alternateRefsPrefixes "refs/tags" &&
    -+	expect_haves one three two >expect &&
    ++	git rev-parse one three two >expect &&
     +	printf "0000" | git receive-pack fork >actual &&
     +	extract_haves <actual >actual.haves &&
     +	test_cmp expect actual.haves
    @@ -77,7 +75,7 @@
     @@
      		argv_array_pushf(&cmd->args, "--git-dir=%s", repo_path);
      		argv_array_push(&cmd->args, "for-each-ref");
    - 		argv_array_push(&cmd->args, "--format=%(objectname) %(refname)");
    + 		argv_array_push(&cmd->args, "--format=%(objectname)");
     +
     +		if (!git_config_get_value("core.alternateRefsPrefixes", &value)) {
     +			argv_array_push(&cmd->args, "--");
--
2.19.0.221.g150f307af
Previous: Jeff KingNext: Jeff King
Message 62 of 94 in “Filter alternate references”
  1. 0/3 Filter alternate referencesTaylor Blau, Sep 20, 2018
  2. 1/3 transport.c: extract 'fill_alternate_refs_command'Taylor Blau, Sep 20, 2018
  3. 2/3 transport.c: introduce core.alternateRefsCommandTaylor Blau, Sep 20, 2018
  4. Jeff KingSep 20, 2018
  5. Taylor BlauSep 20, 2018
  6. Jeff KingSep 20, 2018
  7. Junio C HamanoSep 21, 2018
  8. Taylor BlauSep 21, 2018
  9. Taylor BlauSep 21, 2018
  10. Junio C HamanoSep 21, 2018
  11. Taylor BlauSep 26, 2018
  12. 3/3 transport.c: introduce core.alternateRefsPrefixesTaylor Blau, Sep 20, 2018
  13. Jeff KingSep 20, 2018
  14. Taylor BlauSep 20, 2018
  15. Eric SunshineSep 21, 2018
  16. Taylor BlauSep 21, 2018
  17. Junio C HamanoSep 21, 2018
  18. Taylor BlauSep 21, 2018
  19. Junio C HamanoSep 21, 2018
  20. Stefan BellerSep 20, 2018
  21. Taylor BlauSep 20, 2018
  22. Jeff KingSep 20, 2018
  23. Jeff KingSep 20, 2018
  24. 0/3 Filter alternate referencesTaylor Blau, Sep 21, 2018
  25. 1/3 transport.c: extract 'fill_alternate_refs_command'Taylor Blau, Sep 21, 2018
  26. 2/3 transport.c: introduce core.alternateRefsCommandTaylor Blau, Sep 21, 2018
  27. Eric SunshineSep 21, 2018
  28. Taylor BlauSep 26, 2018
  29. Junio C HamanoSep 21, 2018
  30. Jeff KingSep 21, 2018
  31. Junio C HamanoSep 21, 2018
  32. Jeff KingSep 21, 2018
  33. Taylor BlauSep 26, 2018
  34. Jeff KingSep 26, 2018
  35. Eric SunshineSep 21, 2018
  36. brian m. carlsonSep 22, 2018
  37. Jeff KingSep 22, 2018
  38. brian m. carlsonSep 23, 2018
  39. Taylor BlauSep 26, 2018
  40. Jeff KingSep 26, 2018
  41. Taylor BlauSep 26, 2018
  42. Jeff KingSep 26, 2018
  43. Taylor BlauSep 28, 2018
  44. 3/3 transport.c: introduce core.alternateRefsPrefixesTaylor Blau, Sep 21, 2018
  45. Junio C HamanoSep 21, 2018
  46. Jeff KingSep 21, 2018
  47. Junio C HamanoSep 21, 2018
  48. Jeff KingSep 21, 2018
  49. Stefan BellerSep 21, 2018
  50. Junio C HamanoSep 24, 2018
  51. Jeff KingSep 24, 2018
  52. Junio C HamanoSep 24, 2018
  53. Jeff KingSep 24, 2018
  54. Jeff KingSep 24, 2018
  55. Junio C HamanoSep 24, 2018
  56. Jeff KingSep 24, 2018
  57. Junio C HamanoSep 25, 2018
  58. Taylor BlauSep 25, 2018
  59. Junio C HamanoSep 25, 2018
  60. Taylor BlauSep 26, 2018
  61. Jeff KingSep 26, 2018
  62. 0/4 Filter alternate referencesTaylor Blau, Sep 28, 2018
  63. 1/4 transport: drop refnames from for_each_alternate_refJeff King, Sep 28, 2018
  64. Jeff KingSep 28, 2018
  65. Taylor BlauSep 28, 2018
  66. 2/4 transport.c: extract 'fill_alternate_refs_command'Taylor Blau, Sep 28, 2018
  67. Jeff KingSep 28, 2018
  68. 3/4 transport.c: introduce core.alternateRefsCommandTaylor Blau, Sep 28, 2018
  69. Jeff KingSep 28, 2018
  70. Taylor BlauSep 28, 2018
  71. Jeff KingSep 29, 2018
  72. Taylor BlauOct 2, 2018
  73. 4/4 transport.c: introduce core.alternateRefsPrefixesTaylor Blau, Sep 28, 2018
  74. Jeff KingSep 28, 2018
  75. Taylor BlauSep 28, 2018
  76. Jeff KingSep 29, 2018
  77. Taylor BlauOct 2, 2018
  78. Taylor BlauOct 2, 2018
  79. 0/4 Filter alternate referencesTaylor Blau, Oct 2, 2018
  80. 1/4 transport: drop refnames from for_each_alternate_refTaylor Blau, Oct 2, 2018
  81. 3/4 transport.c: introduce core.alternateRefsCommandTaylor Blau, Oct 2, 2018
  82. Jeff KingOct 2, 2018
  83. Taylor BlauOct 4, 2018
  84. 2/4 transport.c: extract 'fill_alternate_refs_command'Taylor Blau, Oct 2, 2018
  85. 4/4 transport.c: introduce core.alternateRefsPrefixesTaylor Blau, Oct 2, 2018
  86. Ramsay JonesOct 2, 2018
  87. 4/4 transport.c: introduce core.alternateRefsPrefixesTaylor Blau, Oct 2, 2018
  88. 0/4 Filter alternate referencesTaylor Blau, Oct 8, 2018
  89. 1/4 transport: drop refnames from for_each_alternate_refTaylor Blau, Oct 8, 2018
  90. 2/4 transport.c: extract 'fill_alternate_refs_command'Taylor Blau, Oct 8, 2018
  91. 3/4 transport.c: introduce core.alternateRefsCommandTaylor Blau, Oct 8, 2018
  92. 4/4 transport.c: introduce core.alternateRefsPrefixesTaylor Blau, Oct 8, 2018
  93. Jeff KingOct 9, 2018
  94. Taylor BlauOct 9, 2018

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.