git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v2 00/18] odb: make loose object source a proper `struct odb_source`

From
Patrick Steinhardt <ps@pks.im>
Date
Jun 1, 2026, 08:20 UTC
Message-ID
<20260601-b4-pks-odb-source-loose-v2-0-90ff159430af@pks.im>
In-Reply-To
<20260521-b4-pks-odb-source-loose-v1-0-6553b399be2d@pks.im>
Hi,

this patch series converts the loose object source into a proper `struct odb_source` so that it can be used via our generic interfaces.

The patch series is relatively straight-forward, as the source basically already exists as such and the interfaces already match. So for most of the part we are just moving around some code and converting functions that were previously called directly into callbacks.

I guess the only part that needs some attention is that there is some confusion at first with the `struct odb_source_loose::source` parent pointer that initially points at the owning `struct odb_source_files`. This relationship doesn't make much sense, as a loose source can totally exist standalone without the files source.

We're thus getting rid of this relationship in this series, too. I found it quite hard to reason about which pointer one is holding at any point in time though, doubly so because the parent pointer was named "source", which is rather generic. The second commit thus renames the pointer to `files` and converts it into `struct odb_source_files` to make the transition cleaner, but the whole pointer will be dropped at the end of this series.

The series is built on top of aec3f58750 (Sync with 'maint', 2026-05-21) with ps/odb-in-memory at d2902a4549 (t/unit-tests: add tests for the in-memory object source, 2026-04-10) merged into it.

Changes in v2:
  - Some smaller typo fixes.
  - Link to v1: https://patch.msgid.link/20260521-b4-pks-odb-source-loose-v1-0-6553b399be2d@pks.im
Thanks!
Patrick
---
Patrick Steinhardt (18):
      odb/source-loose: move loose source into "odb/" subsystem
      odb/source-loose: store pointer to "files" instead of generic source
      odb/source-loose: start converting to a proper `struct odb_source`
      odb/source-loose: wire up `reprepare()` callback
      odb/source-loose: wire up `close()` callback
      odb/source-loose: wire up `read_object_info()` callback
      odb/source-loose: wire up `read_object_stream()` callback
      odb/source-loose: wire up `for_each_object()` callback
      odb/source-loose: wire up `find_abbrev_len()` callback
      odb/source-loose: wire up `count_objects()` callback
      odb/source-loose: drop `odb_source_loose_has_object()`
      odb/source-loose: wire up `freshen_object()` callback
      loose: refactor object map to operate on `struct odb_source_loose`
      odb/source-loose: wire up `write_object()` callback
      object-file: refactor writing objects to use loose source
      odb/source-loose: wire up `write_object_stream()` callback
      odb/source-loose: stub out remaining callbacks
      odb/source-loose: drop pointer to the "files" source
 Makefile               |   1 +
 builtin/cat-file.c     |   5 +-
 builtin/gc.c           |   6 +-
 builtin/pack-objects.c |  12 +-
 http-walker.c          |   3 +-
 http.c                 |   6 +-
 loose.c                |  45 ++-
 loose.h                |   4 +-
 meson.build            |   1 +
 object-file.c          | 796 ++++---------------------------------------------
 object-file.h          | 149 ++++-----
 odb/source-files.c     |  28 +-
 odb/source-loose.c     | 736 +++++++++++++++++++++++++++++++++++++++++++++
 odb/source-loose.h     |  48 +++
 odb/source.h           |   3 +
 15 files changed, 973 insertions(+), 870 deletions(-)
Range-diff versus v1:
 1:  f25aaf0889 =  1:  7c97c1687c odb/source-loose: move loose source into "odb/" subsystem
 2:  0bfebeb0da =  2:  1e1e267b39 odb/source-loose: store pointer to "files" instead of generic source
 3:  35787e6ca6 !  3:  847cb523ee odb/source-loose: start converting to a proper `struct odb_source`
    @@ odb/source-loose.h: struct odb_source_loose {
      
     +/*
     + * Cast the given object database source to the loose backend. This will cause
    -+ * a BUG in case the source uses doesn't use this backend.
    ++ * a BUG in case the source doesn't use this backend.
     + */
     +static inline struct odb_source_loose *odb_source_loose_downcast(struct odb_source *source)
     +{
 4:  392962c177 =  4:  af543598ee odb/source-loose: wire up `reprepare()` callback
 5:  b4102668c3 =  5:  884f573f89 odb/source-loose: wire up `close()` callback
 6:  63da6e4abb =  6:  de85ffb4a9 odb/source-loose: wire up `read_object_info()` callback
 7:  12b0c5c32d =  7:  522aaa9c3d odb/source-loose: wire up `read_object_stream()` callback
 8:  8df176e282 =  8:  75cf3f4428 odb/source-loose: wire up `for_each_object()` callback
 9:  6199ae90e0 =  9:  87c1c9ae5e odb/source-loose: wire up `find_abbrev_len()` callback
10:  d0b1ef48d4 = 10:  f6405c8070 odb/source-loose: wire up `count_objects()` callback
11:  0476d8b0c4 = 11:  0e8d6b6487 odb/source-loose: drop `odb_source_loose_has_object()`
12:  27bb7b0724 = 12:  58cc626dd1 odb/source-loose: wire up `freshen_object()` callback
13:  f8ce6a169d = 13:  51a22e7400 loose: refactor object map to operate on `struct odb_source_loose`
14:  7ab570b776 = 14:  a9a88d6200 odb/source-loose: wire up `write_object()` callback
15:  8c9240aaa0 = 15:  9236d2fd26 object-file: refactor writing objects to use loose source
16:  de69621fa1 ! 16:  6316efb890 odb/source-loose: wire up `write_object_stream()` callback
    @@ object-file.h: int index_path(struct index_state *istate, struct object_id *oid,
      
     -int odb_source_loose_write_stream(struct odb_source_loose *loose,
     +/*
    -+ * Write the given stream into the loose object source. The only difference to
    -+ * the generic implementation of this function is that we don't perform an
    ++ * Write the given stream into the loose object source. The only difference
    ++ * from the generic implementation of this function is that we don't perform an
     + * object existence check here.
     + *
     + * TODO: We should stop exposing this function altogether and move it into
17:  f2d45e1a56 = 17:  789ec50474 odb/source-loose: stub out remaining callbacks
18:  070052fc22 = 18:  0a64d23377 odb/source-loose: drop pointer to the "files" source

--- base-commit: 072edab49f312c80561b2899f03f361f74fc38e4 change-id: 20260413-b4-pks-odb-source-loose-4900c8ca91db

Previous: Patrick SteinhardtNext: Patrick Steinhardt
Message 23 of 46 in “odb: make loose object source a proper `struct odb_source`”
  1. 00/18 odb: make loose object source a proper `struct odb_source`Patrick Steinhardt, May 21, 2026
  2. 01/18 odb/source-loose: move loose source into "odb/" subsystemPatrick Steinhardt, May 21, 2026
  3. 02/18 odb/source-loose: store pointer to "files" instead of generic sourcePatrick Steinhardt, May 21, 2026
  4. 03/18 odb/source-loose: start converting to a proper `struct odb_source`Patrick Steinhardt, May 21, 2026
  5. Junio C HamanoMay 21, 2026
  6. 04/18 odb/source-loose: wire up `reprepare()` callbackPatrick Steinhardt, May 21, 2026
  7. 05/18 odb/source-loose: wire up `close()` callbackPatrick Steinhardt, May 21, 2026
  8. 06/18 odb/source-loose: wire up `read_object_info()` callbackPatrick Steinhardt, May 21, 2026
  9. 07/18 odb/source-loose: wire up `read_object_stream()` callbackPatrick Steinhardt, May 21, 2026
  10. 08/18 odb/source-loose: wire up `for_each_object()` callbackPatrick Steinhardt, May 21, 2026
  11. 09/18 odb/source-loose: wire up `find_abbrev_len()` callbackPatrick Steinhardt, May 21, 2026
  12. 10/18 odb/source-loose: wire up `count_objects()` callbackPatrick Steinhardt, May 21, 2026
  13. 11/18 odb/source-loose: drop `odb_source_loose_has_object()`Patrick Steinhardt, May 21, 2026
  14. 12/18 odb/source-loose: wire up `freshen_object()` callbackPatrick Steinhardt, May 21, 2026
  15. 13/18 loose: refactor object map to operate on `struct odb_source_loose`Patrick Steinhardt, May 21, 2026
  16. 14/18 odb/source-loose: wire up `write_object()` callbackPatrick Steinhardt, May 21, 2026
  17. 15/18 object-file: refactor writing objects to use loose sourcePatrick Steinhardt, May 21, 2026
  18. 16/18 odb/source-loose: wire up `write_object_stream()` callbackPatrick Steinhardt, May 21, 2026
  19. Junio C HamanoMay 21, 2026
  20. Patrick SteinhardtMay 22, 2026
  21. 17/18 odb/source-loose: stub out remaining callbacksPatrick Steinhardt, May 21, 2026
  22. 18/18 odb/source-loose: drop pointer to the "files" sourcePatrick Steinhardt, May 21, 2026
  23. 00/18 odb: make loose object source a proper `struct odb_source`Patrick Steinhardt, Jun 1, 2026
  24. 01/18 odb/source-loose: move loose source into "odb/" subsystemPatrick Steinhardt, Jun 1, 2026
  25. Karthik NayakJun 3, 2026
  26. Patrick SteinhardtJun 3, 2026
  27. Karthik NayakJun 3, 2026
  28. 02/18 odb/source-loose: store pointer to "files" instead of generic sourcePatrick Steinhardt, Jun 1, 2026
  29. 03/18 odb/source-loose: start converting to a proper `struct odb_source`Patrick Steinhardt, Jun 1, 2026
  30. 04/18 odb/source-loose: wire up `reprepare()` callbackPatrick Steinhardt, Jun 1, 2026
  31. 05/18 odb/source-loose: wire up `close()` callbackPatrick Steinhardt, Jun 1, 2026
  32. 06/18 odb/source-loose: wire up `read_object_info()` callbackPatrick Steinhardt, Jun 1, 2026
  33. 07/18 odb/source-loose: wire up `read_object_stream()` callbackPatrick Steinhardt, Jun 1, 2026
  34. 08/18 odb/source-loose: wire up `for_each_object()` callbackPatrick Steinhardt, Jun 1, 2026
  35. 09/18 odb/source-loose: wire up `find_abbrev_len()` callbackPatrick Steinhardt, Jun 1, 2026
  36. 10/18 odb/source-loose: wire up `count_objects()` callbackPatrick Steinhardt, Jun 1, 2026
  37. 11/18 odb/source-loose: drop `odb_source_loose_has_object()`Patrick Steinhardt, Jun 1, 2026
  38. 12/18 odb/source-loose: wire up `freshen_object()` callbackPatrick Steinhardt, Jun 1, 2026
  39. 13/18 loose: refactor object map to operate on `struct odb_source_loose`Patrick Steinhardt, Jun 1, 2026
  40. 14/18 odb/source-loose: wire up `write_object()` callbackPatrick Steinhardt, Jun 1, 2026
  41. 15/18 object-file: refactor writing objects to use loose sourcePatrick Steinhardt, Jun 1, 2026
  42. 16/18 odb/source-loose: wire up `write_object_stream()` callbackPatrick Steinhardt, Jun 1, 2026
  43. 17/18 odb/source-loose: stub out remaining callbacksPatrick Steinhardt, Jun 1, 2026
  44. 18/18 odb/source-loose: drop pointer to the "files" sourcePatrick Steinhardt, Jun 1, 2026
  45. Junio C HamanoJun 1, 2026
  46. Junio C HamanoJun 1, 2026

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.