git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v3 4/4] doc: add large-object-promisors.adoc to the docs build

From
Ramsay Jones <ramsay@ramsayjones.plus.com>
Date
Oct 17, 2025, 16:37 UTC
Message-ID
<e009f8c7-9809-4ea7-ba6a-8351282de8ca@ramsayjones.plus.com>
In-Reply-To
<20251016200301.1595204-5-ramsay@ramsayjones.plus.com>

Sorry, I meant to add Christian on CC:, since he wrote this document in commit 5040f9f164, but I totally forgot. :( Sorry about that.

On 16/10/2025 9:03 pm, Ramsay Jones wrote:
Show 299 quoted lines
> Commit 5040f9f164 ("doc: add technical design doc for large object
> promisors", 2025-02-18) added the large object promisors document
> as a technical document (with a '.txt' extension). The merge commit
> 2c6fd30198 ("Merge branch 'cc/lop-remote'", 2025-03-05) seems to
> have renamed the file with an '.adoc' extension.
> 
> Despite the '.adoc' extension, this document was not being formatted
> by asciidoc(tor) as part of the docs build. In order to do so, add
> the document to the make and meson build files.
> 
> Having added the document to the build, asciidoc and asciidoctor find
> (slightly different) problems with the syntax of the input document.
> 
> The first set of warnings (only issued by asciidoc) relate to some
> 'section title out of sequence: expected level 3, got level 4'. This
> document uses 'setext' style of section headers, using a series of
> underline characters, where the character used denotes the level of
> the title. From document title to level 5 (see [1]), these characters
> are =, -, ~, ^, +. This does not seem to fit the error message, which
> implies that those characters denote levels 0 -> 4. Replacing the headings
> underlined with '+' by the '^' character eliminates these warnings.
> 
> The second set of warnings (only issued by asciidoctor) relate to some
> headings which seem to use both arabic and roman numerals as part of
> a single 'list' sequence. This elicited either 'unterminated listing
> block' or (for example) 'list item index: expected I, got II' warnings.
> In order not to mix arabic and roman numerals, remove the numeral from
> the '0) Non goals' heading.  Similarly, the remaining roman numeral
> entries had the ')' removed and turned into regular headings with I, II,
> III ... at the beginning.
> 
> [1] https://asciidoctor.org/docs/asciidoc-recommended-practices/
> 
> Signed-off-by: Ramsay Jones <ramsay@ramsayjones.plus.com>
> ---
>  Documentation/Makefile                        |  1 +
>  .../technical/large-object-promisors.adoc     | 64 +++++++++----------
>  Documentation/technical/meson.build           |  1 +
>  3 files changed, 34 insertions(+), 32 deletions(-)
> 
> diff --git a/Documentation/Makefile b/Documentation/Makefile
> index a3fbd29744..a3ba25e659 100644
> --- a/Documentation/Makefile
> +++ b/Documentation/Makefile
> @@ -122,6 +122,7 @@ TECH_DOCS += technical/bundle-uri
>  TECH_DOCS += technical/commit-graph
>  TECH_DOCS += technical/directory-rename-detection
>  TECH_DOCS += technical/hash-function-transition
> +TECH_DOCS += technical/large-object-promisors
>  TECH_DOCS += technical/long-running-process-protocol
>  TECH_DOCS += technical/multi-pack-index
>  TECH_DOCS += technical/packfile-uri
> diff --git a/Documentation/technical/large-object-promisors.adoc b/Documentation/technical/large-object-promisors.adoc
> index dea8dafa66..2aa815e023 100644
> --- a/Documentation/technical/large-object-promisors.adoc
> +++ b/Documentation/technical/large-object-promisors.adoc
> @@ -34,8 +34,8 @@ a new object representation for large blobs as discussed in:
>  
>  https://lore.kernel.org/git/xmqqbkdometi.fsf@gitster.g/
>  
> -0) Non goals
> -------------
> +Non goals
> +---------
>  
>  - We will not discuss those client side improvements here, as they
>    would require changes in different parts of Git than this effort.
> @@ -90,8 +90,8 @@ later in this document:
>      even more to host content with larger blobs or more large blobs
>      than currently.
>  
> -I) Issues with the current situation
> -------------------------------------
> +I Issues with the current situation
> +-----------------------------------
>  
>  - Some statistics made on GitLab repos have shown that more than 75%
>    of the disk space is used by blobs that are larger than 1MB and
> @@ -138,8 +138,8 @@ I) Issues with the current situation
>    complaining that these tools require significant effort to set up,
>    learn and use correctly.
>  
> -II) Main features of the "Large Object Promisors" solution
> -----------------------------------------------------------
> +II Main features of the "Large Object Promisors" solution
> +---------------------------------------------------------
>  
>  The main features below should give a rough overview of how the
>  solution may work. Details about needed elements can be found in
> @@ -166,7 +166,7 @@ format. They should be used along with main remotes that contain the
>  other objects.
>  
>  Note 1
> -++++++
> +^^^^^^
>  
>  To clarify, a LOP is a normal promisor remote, except that:
>  
> @@ -178,7 +178,7 @@ To clarify, a LOP is a normal promisor remote, except that:
>    itself.
>  
>  Note 2
> -++++++
> +^^^^^^
>  
>  Git already makes it possible for a main remote to also be a promisor
>  remote storing both regular objects and large blobs for a client that
> @@ -186,13 +186,13 @@ clones from it with a filter on blob size. But here we explicitly want
>  to avoid that.
>  
>  Rationale
> -+++++++++
> +^^^^^^^^^
>  
>  LOPs aim to be good at handling large blobs while main remotes are
>  already good at handling other objects.
>  
>  Implementation
> -++++++++++++++
> +^^^^^^^^^^^^^^
>  
>  Git already has support for multiple promisor remotes, see
>  link:partial-clone.html#using-many-promisor-remotes[the partial clone documentation].
> @@ -213,19 +213,19 @@ remote helper (see linkgit:gitremote-helpers[7]) which makes the
>  underlying object storage appear like a remote to Git.
>  
>  Note
> -++++
> +^^^^
>  
>  A LOP can be a promisor remote accessed using a remote helper by
>  both some clients and the main remote.
>  
>  Rationale
> -+++++++++
> +^^^^^^^^^
>  
>  This looks like the simplest way to create LOPs that can cheaply
>  handle many large blobs.
>  
>  Implementation
> -++++++++++++++
> +^^^^^^^^^^^^^^
>  
>  Remote helpers are quite easy to write as shell scripts, but it might
>  be more efficient and maintainable to write them using other languages
> @@ -247,7 +247,7 @@ The underlying object storage that a LOP uses could also serve as
>  storage for large files handled by Git LFS.
>  
>  Rationale
> -+++++++++
> +^^^^^^^^^
>  
>  This would simplify the server side if it wants to both use a LOP and
>  act as a Git LFS server.
> @@ -259,7 +259,7 @@ On the server side, a main remote should have a way to offload to a
>  LOP all its blobs with a size over a configurable threshold.
>  
>  Rationale
> -+++++++++
> +^^^^^^^^^
>  
>  This makes it easy to set things up and to clean things up. For
>  example, an admin could use this to manually convert a repo not using
> @@ -268,7 +268,7 @@ some users would sometimes push large blobs, a cron job could use this
>  to regularly make sure the large blobs are moved to the LOP.
>  
>  Implementation
> -++++++++++++++
> +^^^^^^^^^^^^^^
>  
>  Using something based on `git repack --filter=...` to separate the
>  blobs we want to offload from the other Git objects could be a good
> @@ -284,13 +284,13 @@ should have ways to prevent oversize blobs to be fetched, and also
>  perhaps pushed, into it.
>  
>  Rationale
> -+++++++++
> +^^^^^^^^^
>  
>  A main remote containing many oversize blobs would defeat the purpose
>  of LOPs.
>  
>  Implementation
> -++++++++++++++
> +^^^^^^^^^^^^^^
>  
>  The way to offload to a LOP discussed in 4) above can be used to
>  regularly offload oversize blobs. About preventing oversize blobs from
> @@ -326,18 +326,18 @@ large blobs directly from the LOP and the server would not need to
>  fetch those blobs from the LOP to be able to serve the client.
>  
>  Note
> -++++
> +^^^^
>  
>  For fetches instead of clones, a protocol negotiation might not always
>  happen, see the "What about fetches?" FAQ entry below for details.
>  
>  Rationale
> -+++++++++
> +^^^^^^^^^
>  
>  Security, configurability and efficiency of setting things up.
>  
>  Implementation
> -++++++++++++++
> +^^^^^^^^^^^^^^
>  
>  A "promisor-remote" protocol v2 capability looks like a good way to
>  implement this. The way the client and server use this capability
> @@ -356,7 +356,7 @@ the client should be able to offload some large blobs it has fetched,
>  but might not need anymore, to the LOP.
>  
>  Note
> -++++
> +^^^^
>  
>  It might depend on the context if it should be OK or not for clients
>  to offload large blobs they have created, instead of fetched, directly
> @@ -367,13 +367,13 @@ This should be discussed and refined when we get closer to
>  implementing this feature.
>  
>  Rationale
> -+++++++++
> +^^^^^^^^^
>  
>  On the client, the easiest way to deal with unneeded large blobs is to
>  offload them.
>  
>  Implementation
> -++++++++++++++
> +^^^^^^^^^^^^^^
>  
>  This is very similar to what 4) above is about, except on the client
>  side instead of the server side. So a good solution to 4) could likely
> @@ -385,8 +385,8 @@ when cloning (see 6) above). Also if the large blobs were fetched from
>  a LOP, it is likely, and can easily be confirmed, that the LOP still
>  has them, so that they can just be removed from the client.
>  
> -III) Benefits of using LOPs
> ----------------------------
> +III Benefits of using LOPs
> +--------------------------
>  
>  Many benefits are related to the issues discussed in "I) Issues with
>  the current situation" above:
> @@ -406,8 +406,8 @@ the current situation" above:
>  
>  - Reduced storage needs on the client side.
>  
> -IV) FAQ
> --------
> +IV FAQ
> +------
>  
>  What about using multiple LOPs on the server and client side?
>  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> @@ -533,7 +533,7 @@ some objects it already knows about but doesn't have because they are
>  on a promisor remote.
>  
>  Regular fetch
> -+++++++++++++
> +^^^^^^^^^^^^^
>  
>  In a regular fetch, the client will contact the main remote and a
>  protocol negotiation will happen between them. It's a good thing that
> @@ -551,7 +551,7 @@ new fetch will happen in the same way as the previous clone or fetch,
>  using, or not using, the same LOP(s) as last time.
>  
>  "Backfill" or "lazy" fetch
> -++++++++++++++++++++++++++
> +^^^^^^^^^^^^^^^^^^^^^^^^^^
>  
>  When there is a backfill fetch, the client doesn't necessarily contact
>  the main remote first. It will try to fetch from its promisor remotes
> @@ -576,8 +576,8 @@ from the client when it fetches from them. The client could get the
>  token when performing a protocol negotiation with the main remote (see
>  section II.6 above).
>  
> -V) Future improvements
> -----------------------
> +V Future improvements
> +---------------------
>  
>  It is expected that at the beginning using LOPs will be mostly worth
>  it either in a corporate context where the Git version that clients
> diff --git a/Documentation/technical/meson.build b/Documentation/technical/meson.build
> index a13aafcfbb..34b5ebe5c3 100644
> --- a/Documentation/technical/meson.build
> +++ b/Documentation/technical/meson.build
> @@ -13,6 +13,7 @@ articles = [
>    'commit-graph.adoc',
>    'directory-rename-detection.adoc',
>    'hash-function-transition.adoc',
> +  'large-object-promisors.adoc',
>    'long-running-process-protocol.adoc',
>    'multi-pack-index.adoc',
>    'packfile-uri.adoc',
Previous: Ramsay JonesNext: Junio C Hamano
Message 23 of 25 in “technical docs in make build”
  1. 0/4 technical docs in make buildRamsay Jones, Oct 2, 2025
  2. 1/4 doc: add some missing technical documentsRamsay Jones, Oct 2, 2025
  3. Patrick SteinhardtOct 8, 2025
  4. Junio C HamanoOct 8, 2025
  5. Ramsay JonesOct 8, 2025
  6. Junio C HamanoOct 8, 2025
  7. Ramsay JonesOct 8, 2025
  8. 2/4 doc: remembering-renames.adoc: fix asciidoc warningsRamsay Jones, Oct 2, 2025
  9. Elijah NewrenOct 8, 2025
  10. Ramsay JonesOct 8, 2025
  11. 3/4 doc: sparse-checkout.adoc: fix asciidoc warningsRamsay Jones, Oct 2, 2025
  12. Kristoffer HaugsbakkOct 7, 2025
  13. Ramsay JonesOct 7, 2025
  14. Elijah NewrenOct 8, 2025
  15. Ramsay JonesOct 8, 2025
  16. 4/4 doc: commit-graph.adoc: fix up some formattingRamsay Jones, Oct 2, 2025
  17. Ramsay JonesOct 2, 2025
  18. 0/4 technical docs in make buildRamsay Jones, Oct 16, 2025
  19. 1/4 doc: remembering-renames.adoc: fix asciidoc warningsRamsay Jones, Oct 16, 2025
  20. 2/4 doc: sparse-checkout.adoc: fix asciidoc warningsRamsay Jones, Oct 16, 2025
  21. 3/4 doc: commit-graph.adoc: fix up some formattingRamsay Jones, Oct 16, 2025
  22. 4/4 doc: add large-object-promisors.adoc to the docs buildRamsay Jones, Oct 16, 2025
  23. Ramsay JonesOct 17, 2025
  24. Junio C HamanoOct 23, 2025
  25. Ramsay JonesOct 23, 2025

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.