git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v2 00/13] odb: stop registering in-memory sources

From
Karthik Nayak <karthik.188@gmail.com>
Date
Sep 4, 2026, 22:45 UTC
Message-ID
<CAOLa=ZQVRJwRqapa8p+fdW2VL652vh5=TJ_ToNGXXM1z=aiXBg@mail.gmail.com>
In-Reply-To
<20260902-pks-odb-registering-in-memory-sources-v2-0-c6ca12fdea4d@pks.im>
Patrick Steinhardt <ps@pks.im> writes:
Show 50 quoted lines
> Hi,
>
> the object database has a list of sources that is used for two
> different purposes:
>
>   - We use it to track the list of alternates.
>
>   - We use it to track temporary in-memory sources that we create for
>     various purposes. Most importantly, this is used to link object
>     database sources from submodules into the main store.
>
> This dual-use is quite awkward, as it mixes two different levels of
> concerns and thus as a consequence makes both harder to reason about.
> It's also a source of bugs: we make assumptions about the ordering of
> sources all over the place, and we furthermore assume in other places
> that the sources only contain alternates in the first place. I don't
> think this surfaces in the form of real bugs, but I've long disliked
> this dual-use.
>
> Furthermore, we want to migrate handling of alternates into the "files"
> backend itself in a subsequent patch series. This is most importantly to
> fix a performance regression by making the backend own all of its
> alternates, but it also fixes a couple of longer-standing design issues
> that I've been struggling with [1].
>
> Most importantly though: this whole machinery is not even needed at all.
> A couple years ago we have already refactored our codebase so that
> submodule sources don't even have to be linked into the main object
> database anymore. And all the other use cases where we link sources into
> the main object database can be trivially converted, too.
>
> So this patch series does exactly that: it removes the mechanism to link
> ad-hoc sources into the object database entirely. This ensures that the
> list of sources is exactly the list of alternates, and that makes it
> easier to move them into the "files" backend in a subsequent patch
> series.
>
> There is one exception though: creating transactions still creates a
> temporary quarantine directory. This mechanism is left as-is for now,
> but as it's an implementation detail of the "files" backend anyway
> that's not conflicting with our above stated goals.
>
> This series is built on top of 1630431f32 (The 21st batch, 2026-08-31)
> with ty/repository-fetch-if-missing at 508ec9837c (repository: move
> fetch_if_missing into struct repository, 2026-08-15) merged into it.
> There's still two merge conflicts, but these are trivial to resolve: in
> "odb.c" and "odb.h" you simply remove both ours and theirs, and in
> "builtin/multi-pack-index.c" you only need to munge the parameters a
> bit.
>

This was a bit of a dense read for me, mostly as I'm not too familiar with the code here. So I can not so confidently say that the series looks good :)

Show 9 quoted lines
> Changes in v2:
>   - Adapt `cache_tree_fully_valid()` to take a `struct index_state` as
>     input instead of taking both a repository and a cache tree, as
>     suggested by Junio.
>   - Link to v1: https://patch.msgid.link/20260901-pks-odb-registering-in-memory-sources-v1-0-97a312d5fa25@pks.im
>
> Thanks!
>
> Patrick
[snip]
Previous: Patrick Steinhardt
Message 23 of 23 in “odb: stop registering in-memory sources”
  1. 00/13 odb: stop registering in-memory sourcesPatrick Steinhardt, Sep 2, 2026
  2. 01/13 cache-tree: drop `the_repository` in `cache_tree_fully_valid()`Patrick Steinhardt, Sep 2, 2026
  3. 02/13 cache-tree: remove dependency on `the_repository`Patrick Steinhardt, Sep 2, 2026
  4. Karthik NayakSep 4, 2026
  5. Patrick SteinhardtSep 7, 2026
  6. 03/13 submodule-config: remove uses of `the_repository`Patrick Steinhardt, Sep 2, 2026
  7. 04/13 submodule-config: stop using `the_hash_algo`Patrick Steinhardt, Sep 2, 2026
  8. 05/13 submodule-config: stop registering submodule sourcesPatrick Steinhardt, Sep 2, 2026
  9. Justin ToblerSep 6, 2026
  10. Patrick SteinhardtSep 7, 2026
  11. Justin ToblerSep 8, 2026
  12. Patrick SteinhardtSep 9, 2026
  13. 06/13 builtin/grep: stop registering submodule ODB as sourcePatrick Steinhardt, Sep 2, 2026
  14. 07/13 odb: remove infrastructure to register submodule sourcesPatrick Steinhardt, Sep 2, 2026
  15. Karthik NayakSep 4, 2026
  16. 08/13 tmp-objdir: drop unused function to register alternatePatrick Steinhardt, Sep 2, 2026
  17. 09/13 odb/packed: fix memory leaks when freeing sourcePatrick Steinhardt, Sep 2, 2026
  18. 10/13 builtin/multi-pack-index: refuse unknown sources with "--object-dir="Patrick Steinhardt, Sep 2, 2026
  19. Justin ToblerSep 8, 2026
  20. 11/13 t/helper: adapt read-midx to not link ad-hoc source anymorePatrick Steinhardt, Sep 2, 2026
  21. 12/13 t/helper: stop registering alternates in "ref-store" commandPatrick Steinhardt, Sep 2, 2026
  22. 13/13 odb: remove the ability to link sources ad-hocPatrick Steinhardt, Sep 2, 2026
  23. Karthik NayakSep 4, 2026

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.