git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v2 2/3] revision: add new parameter to specify all visible refs

From
Patrick Steinhardt <ps@pks.im>
Date
Nov 7, 2022, 08:20 UTC
Message-ID
<Y2i/uJPm5KxqAdkE@ncase>
In-Reply-To
<Y2ZbDXYb1jGqSfTj@coredump.intra.peff.net>
On Sat, Nov 05, 2022 at 08:46:05AM -0400, Jeff King wrote:
Show 47 quoted lines
> On Thu, Nov 03, 2022 at 03:37:32PM +0100, Patrick Steinhardt wrote:
> 
> > Users can optionally hide refs from remote users in git-upload-pack(1),
> > git-receive-pack(1) and others via the `transfer.hideRefs`, but there is
> > not an easy way to obtain the list of all visible or hidden refs right
> > now. We'll require just that though for a performance improvement in our
> > connectivity check.
> > 
> > Add a new pseudo-ref `--visible-refs=` that pretends as if all refs have
> > been added to the command line that are not hidden. The pseudo-ref
> > requiers either one of "transfer", "uploadpack" or "receive" as argument
> > to pay attention to `transfer.hideRefs`, `uploadpack.hideRefs` or
> > `receive.hideRefs`, respectively.
> 
> Thanks for re-working this. I think it's a sensible path forward for the
> problem you're facing.
> 
> There were two parts of the implementation that surprised me a bit.
> These might just be nits, but because this is a new user-facing plumbing
> option that will be hard to change later, we should make sure it fits in
> with the adjacent features.
> 
> The two things I saw were:
> 
>   1. The mutual-exclusion selection between "transfer", "uploadpack",
>      and "receive" is not how those options work in their respective
>      programs. The "transfer.hideRefs" variable is respected for either
>      program. So whichever program you are running, it will always look
>      at both "transfer" and itself ("uploadpack" or "receive"). Another
>      way to think of it is that the "section" argument to
>      parse_hide_refs_config() is not a config section so much as a
>      profile. And the profiles are:
> 
>        - uploadpack: respect transfer.hideRefs and uploadpack.hideRefs
>        - receive: respect transfer.hideRefs and receive.hideRefs
> 
>      So it does not make sense to ask for "transfer" as a section; each
>      of the modes is already looking at transfer.hideRefs.
> 
>      In theory if this option was "look just at $section.hideRefs", it
>      could be more flexible to separate out the two. But that makes it
>      more of a pain to use (for normal use, you have to specify both
>      "transfer" and "receive"). And that is not what your patch
>      implements anyway; because it relies on parse_hide_refs_config(),
>      it is always adding in "transfer" under the hood (which is why your
>      final patch is correct to just say "--visible-refs=receive" without
>      specifying "transfer" as well).

Yup, I'm aware of this. And as you say, the current implementation already handles this alright for both `receive` and `uploadpack` as we rely on `parse_hide_refs_config()`, which knows to look at both `transfer.hideRefs` and `$section.hideRefs`. But I don't see a reason why we shouldn't allow users to ask "What is the set of hidden refs that are shared by `uploadpack` and `receive`?", which is exactly what `--visible-refs=transfer` does.

The implementation is not really explicit about this as we cheat a little bit here by passing "transfer" as a section to the parsing function. So what it does right now is to basically check for the same section twice, once via the hard-coded "transfer.hideRefs" and once for the "$section.hideRefs" with `section == "transfer"`. But I didn't see much of a point in making this more explicit.

I might update the commit message and/or documentation though to point this out.

Show 37 quoted lines
>   2. Your "--visible-refs" implies "--all", because it's really "all
>      refs minus the hidden ones". That's convenient for the intended
>      caller, but not as flexible as it could be. If it were instead
>      treated the way "--exclude" is, as a modifier for the next
>      iteration option, then you do a few extra things:
> 
>        a. Combine multiple exclusions in a single iteration:
> 
>             git rev-list --exclude-hidden=receive \
> 	                 --exclude-hidden=upload \
> 			 --all
> 
>           That excludes both types in a single iteration. Whereas if you
> 	  did:
> 
> 	    git rev-list --visible-refs=receive \
> 	                 --visible-refs=upload
> 
> 	  that will do _two_ iterations, and end up with the union of
> 	  the sets. Equivalent to:
> 
> 	    git rev-list --exclude-hidden=receive --all \
> 	                 --exclude-hidden=upload  --all
> 
>        b. Do exclusions on smaller sets than --all:
> 
>             git rev-list --exclude-hidden=receive \
> 	                 --branches
> 
> 	  which would show just the branches that we'd advertise.
> 
>      Now I don't have a particular use case for either of those things.
>      But they're plausible things to want in the long run, and they fit
>      in nicely with the existing ref-selection scheme of rev-list. They
>      do make your call from check_connected() slightly longer, but it is
>      pretty negligible. It's "--exclude-hidden=receive --all" instead of
>      "--visible-refs=hidden".

Fair enough. I guess that the usecase where you e.g. only hide a subset of branches via `hideRefs` is going to be rare, so in most cases you don't gain much by modelling this so that you can `--exclude-hidden --branches`. But as you rightfully point out, modelling it that way fits neatly with the existing `--exclude` switch and is overall more flexible. So there's not much of a reason to not do so.

I'll send a v3 and include your suggestion, thanks.
Patrick
Previous: Jeff KingNext: Jeff King
Message 27 of 88 in “receive-pack: use advertised reference tips to inform connectivity check”
  1. 0/2 receive-pack: use advertised reference tips to inform connectivity checkPatrick Steinhardt, Oct 28, 2022
  2. 1/2 connected: allow supplying different view of reachable objectsPatrick Steinhardt, Oct 28, 2022
  3. Ævar Arnfjörð BjarmasonOct 28, 2022
  4. Junio C HamanoOct 28, 2022
  5. Taylor BlauOct 30, 2022
  6. Patrick SteinhardtOct 31, 2022
  7. Taylor BlauNov 1, 2022
  8. 2/2 receive-pack: use advertised reference tips to inform connectivity checkPatrick Steinhardt, Oct 28, 2022
  9. Ævar Arnfjörð BjarmasonOct 28, 2022
  10. Patrick SteinhardtOct 31, 2022
  11. Ævar Arnfjörð BjarmasonOct 31, 2022
  12. Taylor BlauOct 30, 2022
  13. Patrick SteinhardtOct 31, 2022
  14. Taylor BlauNov 1, 2022
  15. Patrick SteinhardtNov 1, 2022
  16. Patrick SteinhardtNov 1, 2022
  17. Taylor BlauNov 2, 2022
  18. Jeff KingNov 1, 2022
  19. Junio C HamanoOct 28, 2022
  20. Taylor BlauNov 1, 2022
  21. Jeff KingNov 1, 2022
  22. Patrick SteinhardtNov 1, 2022
  23. 0/3 receive-pack: only use visible refs for connectivity checkPatrick Steinhardt, Nov 3, 2022
  24. 1/3 refs: get rid of global list of hidden refsPatrick Steinhardt, Nov 3, 2022
  25. 2/3 revision: add new parameter to specify all visible refsPatrick Steinhardt, Nov 3, 2022
  26. Jeff KingNov 5, 2022
  27. Patrick SteinhardtNov 7, 2022
  28. Jeff KingNov 8, 2022
  29. Jeff KingNov 5, 2022
  30. 3/3 receive-pack: only use visible refs for connectivity checkPatrick Steinhardt, Nov 3, 2022
  31. Taylor BlauNov 5, 2022
  32. Jeff KingNov 5, 2022
  33. Jeff KingNov 5, 2022
  34. 0/6 receive-pack: only use visible refs for connectivity checkPatrick Steinhardt, Nov 7, 2022
  35. 2/6 revision: move together exclusion-related functionsPatrick Steinhardt, Nov 7, 2022
  36. 1/6 refs: get rid of global list of hidden refsPatrick Steinhardt, Nov 7, 2022
  37. 6/6 receive-pack: only use visible refs for connectivity checkPatrick Steinhardt, Nov 7, 2022
  38. 3/6 revision: introduce struct to handle exclusionsPatrick Steinhardt, Nov 7, 2022
  39. Ævar Arnfjörð BjarmasonNov 7, 2022
  40. Patrick SteinhardtNov 8, 2022
  41. 5/6 revparse: add `--exclude-hidden=` optionPatrick Steinhardt, Nov 7, 2022
  42. Jeff KingNov 8, 2022
  43. 4/6 revision: add new parameter to exclude hidden refsPatrick Steinhardt, Nov 7, 2022
  44. Ævar Arnfjörð BjarmasonNov 7, 2022
  45. Ævar Arnfjörð BjarmasonNov 7, 2022
  46. Patrick SteinhardtNov 8, 2022
  47. Patrick SteinhardtNov 8, 2022
  48. Taylor BlauNov 8, 2022
  49. Patrick SteinhardtNov 8, 2022
  50. Jeff KingNov 8, 2022
  51. Taylor BlauNov 8, 2022
  52. 0/6 receive-pack: only use visible refs for connectivity checkPatrick Steinhardt, Nov 8, 2022
  53. 1/6 refs: get rid of global list of hidden refsPatrick Steinhardt, Nov 8, 2022
  54. Ævar Arnfjörð BjarmasonNov 8, 2022
  55. Patrick SteinhardtNov 8, 2022
  56. Jeff KingNov 8, 2022
  57. 2/6 revision: move together exclusion-related functionsPatrick Steinhardt, Nov 8, 2022
  58. 5/6 rev-parse: add `--exclude-hidden=` optionPatrick Steinhardt, Nov 8, 2022
  59. 3/6 revision: introduce struct to handle exclusionsPatrick Steinhardt, Nov 8, 2022
  60. 4/6 revision: add new parameter to exclude hidden refsPatrick Steinhardt, Nov 8, 2022
  61. Jeff KingNov 8, 2022
  62. Taylor BlauNov 8, 2022
  63. Patrick SteinhardtNov 11, 2022
  64. 6/6 receive-pack: only use visible refs for connectivity checkPatrick Steinhardt, Nov 8, 2022
  65. 0/7 receive-pack: only use visible refs for connectivity checkPatrick Steinhardt, Nov 11, 2022
  66. 1/7 refs: fix memory leak when parsing hideRefs configPatrick Steinhardt, Nov 11, 2022
  67. 2/7 refs: get rid of global list of hidden refsPatrick Steinhardt, Nov 11, 2022
  68. 3/7 revision: move together exclusion-related functionsPatrick Steinhardt, Nov 11, 2022
  69. 4/7 revision: introduce struct to handle exclusionsPatrick Steinhardt, Nov 11, 2022
  70. 5/7 revision: add new parameter to exclude hidden refsPatrick Steinhardt, Nov 11, 2022
  71. 6/7 rev-parse: add `--exclude-hidden=` optionPatrick Steinhardt, Nov 11, 2022
  72. 7/7 receive-pack: only use visible refs for connectivity checkPatrick Steinhardt, Nov 11, 2022
  73. Taylor BlauNov 11, 2022
  74. Jeff KingNov 15, 2022
  75. Taylor BlauNov 16, 2022
  76. Jeff KingNov 16, 2022
  77. Taylor BlauNov 16, 2022
  78. Patrick SteinhardtNov 17, 2022
  79. 0/7 receive-pack: only use visible refs for connectivity checkPatrick Steinhardt, Nov 17, 2022
  80. 1/7 refs: fix memory leak when parsing hideRefs configPatrick Steinhardt, Nov 17, 2022
  81. 2/7 refs: get rid of global list of hidden refsPatrick Steinhardt, Nov 17, 2022
  82. 3/7 revision: move together exclusion-related functionsPatrick Steinhardt, Nov 17, 2022
  83. 4/7 revision: introduce struct to handle exclusionsPatrick Steinhardt, Nov 17, 2022
  84. 5/7 revision: add new parameter to exclude hidden refsPatrick Steinhardt, Nov 17, 2022
  85. 6/7 rev-parse: add `--exclude-hidden=` optionPatrick Steinhardt, Nov 17, 2022
  86. 7/7 receive-pack: only use visible refs for connectivity checkPatrick Steinhardt, Nov 17, 2022
  87. Jeff KingNov 17, 2022
  88. Taylor BlauNov 17, 2022

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.