git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v2 1/5] promisor-remote: factor out lazy_fetch_objects()

From
Christian Couder <christian.couder@gmail.com>
Date
Sep 8, 2026, 17:11 UTC
Message-ID
<CAP8UFD3DUAYpBpfcrub6CPJ0AHFvJGEi52=eXYwrBT+YXpu7PA@mail.gmail.com>
In-Reply-To
<xmqqjypsoami.fsf@gitster.g>
On Fri, Aug 14, 2026 at 7:49 PM Junio C Hamano <gitster@pobox.com> wrote:
Show 6 quoted lines
> Perhaps writing it this way would make it easier to tell what is
> going on.  We try the preferred ones first, and then fall back to
> the other ones.
>
>         return (try_promisor_remotes(..., true) ||
>                 try_promisor_remotes(..., false));
Yes, this is used in v3.
Show 29 quoted lines
> But more importantly, I wonder if keeping the list of missing object
> names in memory will later turn out to be problematic in real-life
> applications.  Without knowing much about how the current code for
> bulk dehydrating promisor objects is structured, I expected an API
> that looks more like:
>
>  - bulk_download_begin(): performs the early part of
>    fetch_objects(), sets up connections to the promisor remote(s),
>    and calls start_command() on the child process.
>
>  - bulk_download_this(): after calling the _begin() function above,
>    it runs around and collects missing objects that it needs to do
>    its work.  For each such missing object it discovers, this
>    function is called, which sends the object name down the
>    '--stdin' file descriptor.
>
>  - bulk_download_done(): tells the child process that we are done
>    feeding object names.
>
> but that is not what I am seeing.  I guess the current arrangement
> cannot be avoided, because we are going to fetch from more than one
> promisor remote.  Under such constraints, the way to deal with a
> massive number of missing objects will not be "streaming" like I
> imagined above, but needs to be done differently, like spooling to a
> file or something silly like that.
>
> In any case, except that this avoids checking the environment
> variable multiple times, I can see that it is a no-op refactoring of
> the existing code.

Yeah, I prefer to avoid working on a big refactoring in this area until we have evidence showing that there is a bottleneck here.

> Nice and cleanly done.
Thanks.
Previous: Junio C HamanoNext: Christian Couder
Message 58 of 70 in “Introduce a 'fromAccepted' option to GIT_NO_LAZY_FETCH”
  1. 0/3 Introduce a 'fromAccepted' option to GIT_NO_LAZY_FETCHChristian Couder, Jul 10, 2026
  2. 1/3 promisor-remote: factor out lazy_fetch_objects()Christian Couder, Jul 10, 2026
  3. 2/3 promisor-remote: introduce enum allow_lazy_fetchChristian Couder, Jul 10, 2026
  4. 3/3 promisor-remote: teach 'fromAccepted' to GIT_NO_LAZY_FETCHChristian Couder, Jul 10, 2026
  5. brian m. carlsonJul 10, 2026
  6. Christian CouderJul 12, 2026
  7. 0/5 Introduce 'uploadpack.lazyFetchTrusted'Christian Couder, Aug 7, 2026
  8. 1/5 promisor-remote: factor out lazy_fetch_objects()Christian Couder, Aug 7, 2026
  9. Christian CouderAug 7, 2026
  10. 2/5 setup: extract path_allowlist_apply()Christian Couder, Aug 7, 2026
  11. 4/5 upload-pack: read uploadpack.lazyFetchTrustedChristian Couder, Aug 7, 2026
  12. 5/5 builtin/upload-pack: set GIT_NO_LAZY_FETCH to 0 on trusted repoChristian Couder, Aug 7, 2026
  13. 3/5 setup: add 'allow_dot' arg to path_allowlist_apply()Christian Couder, Aug 7, 2026
  14. Junio C HamanoAug 7, 2026
  15. Christian CouderAug 10, 2026
  16. Junio C HamanoAug 11, 2026
  17. 0/5 Introduce 'uploadpack.lazyFetchTrusted'Christian Couder, Aug 13, 2026
  18. Junio C HamanoAug 13, 2026
  19. Christian CouderAug 14, 2026
  20. Junio C HamanoAug 14, 2026
  21. 0/5 Introduce 'uploadpack.lazyFetchTrusted'Christian Couder, Sep 8, 2026
  22. 1/5 promisor-remote: factor out lazy_fetch_objects()Christian Couder, Sep 8, 2026
  23. Junio C HamanoSep 8, 2026
  24. Christian CouderSep 28, 2026
  25. 2/5 setup: extract path_allowlist_apply()Christian Couder, Sep 8, 2026
  26. Junio C HamanoSep 8, 2026
  27. Christian CouderSep 28, 2026
  28. 3/5 upload-pack: read uploadpack.lazyFetchTrustedChristian Couder, Sep 8, 2026
  29. 4/5 promisor-remote: prevent infinite recursion when lazy fetchingChristian Couder, Sep 8, 2026
  30. Junio C HamanoSep 8, 2026
  31. Christian CouderSep 9, 2026
  32. Junio C HamanoSep 9, 2026
  33. Christian CouderSep 28, 2026
  34. 5/5 builtin/upload-pack: set GIT_NO_LAZY_FETCH to 0 on trusted repoChristian Couder, Sep 8, 2026
  35. Junio C HamanoSep 8, 2026
  36. Christian CouderSep 28, 2026
  37. 0/5 Introduce 'uploadpack.lazyFetchTrusted'Christian Couder, Sep 28, 2026
  38. 1/5 promisor-remote: factor out lazy_fetch_objects()Christian Couder, Sep 28, 2026
  39. 2/5 setup: extract path_allowlist_apply()Christian Couder, Sep 28, 2026
  40. Junio C HamanoSep 29, 2026
  41. Christian CouderOct 2, 2026
  42. 3/5 upload-pack: read uploadpack.lazyFetchTrustedChristian Couder, Sep 28, 2026
  43. 4/5 promisor-remote: prevent infinite recursion when lazy fetchingChristian Couder, Sep 28, 2026
  44. 5/5 builtin/upload-pack: don't disable lazy fetching on trusted repoChristian Couder, Sep 28, 2026
  45. Junio C HamanoSep 29, 2026
  46. Christian CouderOct 2, 2026
  47. Christian CouderOct 2, 2026
  48. 0/5 Introduce 'uploadpack.lazyFetchTrusted'Christian Couder, Oct 2, 2026
  49. 1/5 promisor-remote: factor out lazy_fetch_objects()Christian Couder, Oct 2, 2026
  50. 2/5 setup: extract path_allowlist_apply()Christian Couder, Oct 2, 2026
  51. 3/5 upload-pack: read uploadpack.lazyFetchTrustedChristian Couder, Oct 2, 2026
  52. 4/5 promisor-remote: prevent infinite recursion when lazy fetchingChristian Couder, Oct 2, 2026
  53. 5/5 builtin/upload-pack: don't disable lazy fetching on trusted repoChristian Couder, Oct 2, 2026
  54. Junio C HamanoOct 5, 2026
  55. Christian CouderOct 6, 2026
  56. 1/5 promisor-remote: factor out lazy_fetch_objects()Christian Couder, Aug 13, 2026
  57. Junio C HamanoAug 14, 2026
  58. Christian CouderSep 8, 2026
  59. 2/5 setup: extract path_allowlist_apply()Christian Couder, Aug 13, 2026
  60. Junio C HamanoAug 14, 2026
  61. Christian CouderSep 8, 2026
  62. Junio C HamanoSep 8, 2026
  63. 3/5 setup: add 'allow_dot' arg to path_allowlist_apply()Christian Couder, Aug 13, 2026
  64. Junio C HamanoAug 14, 2026
  65. Christian CouderSep 8, 2026
  66. 4/5 upload-pack: read uploadpack.lazyFetchTrustedChristian Couder, Aug 13, 2026
  67. Junio C HamanoAug 14, 2026
  68. 5/5 builtin/upload-pack: set GIT_NO_LAZY_FETCH to 0 on trusted repoChristian Couder, Aug 13, 2026
  69. Junio C HamanoAug 14, 2026
  70. Christian CouderSep 8, 2026

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.