git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v3 00/17] odb: make packed object source a proper `struct odb_source`

From
Patrick Steinhardt <ps@pks.im>
Date
Jun 17, 2026, 06:39 UTC
Message-ID
<20260617-pks-odb-source-packed-v3-0-b5c7583cd795@pks.im>
In-Reply-To
<20260604-pks-odb-source-packed-v1-0-2e7ab31b4b5c@pks.im>
Hi,

this patch series converts the "packed" source into a proper `struct odb_source`. It's thus the equivalent to [1], which did the same thing for the "loose" source.

This series here is unfortunately a bit bigger, mostly because I'm also renaming `struct packfile_store` to `struct odb_source_packed`. Back when I introduced the packfile store I didn't yet have the full vision of how the final layout will look like, so I didn't have the foresight yet to call it `struct odb_source_packed`. But now that the layout has materialized I think it's sensible to adjust its naming to match all the other sources that we have.

Also: I don't have anything else in the pipeline anymore that moves
around large pieces of our code in the vicinity of the object database.
So after this series got merged, subsequent changes should be of a more
incremental nature.

This series is built on top of 9ac3f193c0 (The 11th batch, 2026-06-02) with ps/odb-source-loose at ef4778bcba (odb/source-loose: drop pointer to the "files" source, 2026-06-01) merged into it.

Changes in v3:
  - Add some more explanations to the commit messages.
  - Link to v2: https://patch.msgid.link/20260609-pks-odb-source-packed-v2-0-839089132c8b@pks.im
Changes in v2:
  - Split out `struct packfile_list` into a separate code unit to fix a
    cyclic dependency between "packfile.h" and "odb/souurce-packed.h".
  - Fix an extraneous newline.
  - Link to v1: https://patch.msgid.link/20260604-pks-odb-source-packed-v1-0-2e7ab31b4b5c@pks.im
Thanks!
Patrick
[1]: <20260521-b4-pks-odb-source-loose-v1-0-6553b399be2d@pks.im>
---
Patrick Steinhardt (17):
      packfile: rename `struct packfile_store` to `odb_source_packed`
      packfile: split out packfile list logic
      packfile: move packed source into "odb/" subsystem
      odb/source-packed: store pointer to "files" instead of generic source
      odb/source-packed: start converting to a proper `struct odb_source`
      odb/source-packed: wire up `close()` callback
      odb/source-packed: wire up `reprepare()` callback
      packfile: use higher-level interface to implement `has_object_pack()`
      odb/source-packed: wire up `read_object_info()` callback
      odb/source-packed: wire up `read_object_stream()` callback
      odb/source-packed: wire up `for_each_object()` callback
      odb/source-packed: wire up `count_objects()` callback
      odb/source-packed: wire up `find_abbrev_len()` callback
      odb/source-packed: wire up `freshen_object()` callback
      odb/source-packed: stub out remaining functions
      midx: refactor interfaces to work on "packed" source
      odb/source-packed: drop pointer to "files" parent source
 Makefile                   |   2 +
 builtin/cat-file.c         |   4 +-
 builtin/grep.c             |   2 +-
 builtin/multi-pack-index.c |  29 +-
 builtin/pack-objects.c     |   7 +-
 builtin/repack.c           |   8 +-
 commit-graph.c             |   4 +-
 meson.build                |   2 +
 midx-write.c               |  34 +-
 midx.c                     | 118 +++----
 midx.h                     |  30 +-
 odb/source-files.c         |  20 +-
 odb/source-files.h         |   4 +-
 odb/source-packed.c        | 764 +++++++++++++++++++++++++++++++++++++++++++
 odb/source-packed.h        |  94 ++++++
 odb/source.h               |   3 +
 pack-bitmap.c              |   8 +-
 pack-revindex.c            |   6 +-
 packfile-list.c            |  86 +++++
 packfile-list.h            |  28 ++
 packfile.c                 | 784 +--------------------------------------------
 packfile.h                 | 180 +----------
 repack-geometry.c          |   3 +-
 repack-midx.c              |   9 +-
 repack.c                   |   6 +-
 t/helper/test-read-midx.c  |   7 +-
 26 files changed, 1163 insertions(+), 1079 deletions(-)
Range-diff versus v2:
 1:  9f917fe0cd =  1:  9c4cb4106c packfile: rename `struct packfile_store` to `odb_source_packed`
 2:  eb6740c34c =  2:  198b137c84 packfile: split out packfile list logic
 3:  5c363883f1 =  3:  fa91889861 packfile: move packed source into "odb/" subsystem
 4:  caa19f59bb !  4:  5f0da3a334 odb/source-packed: store pointer to "files" instead of generic source
    @@ Commit message
     
         Make this relationship more explicit by storing a pointer to the "files"
         source instead of storing a pointer to a generic `struct odb_source`.
    -    This will help make subsequent steps a bit clearer.
    +    This will help make subsequent steps a bit clearer by making it more
    +    obvious whether we're using the generic "base" source or the owning
    +    "files" source.
     
         Note that this is a temporary step, only. At the end of this series
    -    we will have dropped the parent pointer completely.
    +    we will have dropped the "files" pointer completely.
     
         Signed-off-by: Patrick Steinhardt <ps@pks.im>
     
 5:  8eb3cb17a1 !  5:  c9b1e1da26 odb/source-packed: start converting to a proper `struct odb_source`
    @@ Commit message
         odb_source`, as it's missing all of the callback implementations. These
         will be wired up in subsequent commits.
     
    +    Further note that we're also registering a `chdir_notify` callback to
    +    reparent our path. This wasn't previously necessary (and still isn't at
    +    this point in time) because all paths are taken from the owning "files"
    +    source, and that source already handles the reparenting for us. But a
    +    subsequent commit will change that so that we're using the path of the
    +    "packed" source, and once that happens we'll need it to be updated when
    +    changing the working directory.
    +
         Signed-off-by: Patrick Steinhardt <ps@pks.im>
     
      ## odb/source-files.c ##
 6:  0b81cf21c8 =  6:  350cc18b4c odb/source-packed: wire up `close()` callback
 7:  3024f236eb =  7:  b42a633423 odb/source-packed: wire up `reprepare()` callback
 8:  e3553cae61 =  8:  2eda992ff9 packfile: use higher-level interface to implement `has_object_pack()`
 9:  0d4d888c66 =  9:  81201eca00 odb/source-packed: wire up `read_object_info()` callback
10:  0a81bc811f = 10:  2f6ea598a5 odb/source-packed: wire up `read_object_stream()` callback
11:  fc4b2477b5 = 11:  154ded231a odb/source-packed: wire up `for_each_object()` callback
12:  58c7c4c6c1 = 12:  a35c357358 odb/source-packed: wire up `count_objects()` callback
13:  b81e053716 = 13:  134c51a0cd odb/source-packed: wire up `find_abbrev_len()` callback
14:  424b9ee1b2 = 14:  542694cff2 odb/source-packed: wire up `freshen_object()` callback
15:  f91f6355fe = 15:  fa222d0cda odb/source-packed: stub out remaining functions
16:  877721f2ea = 16:  cee3806dc7 midx: refactor interfaces to work on "packed" source
17:  5779eab630 = 17:  200fbcf4bb odb/source-packed: drop pointer to "files" parent source

--- base-commit: 06d49cec508464ced5d42541890ce5d749542a61 change-id: 20260602-pks-odb-source-packed-3826c352f059

Previous: Patrick SteinhardtNext: Patrick Steinhardt
Message 63 of 85 in “odb: make packed object source a proper `struct odb_source`”
  1. 00/16 odb: make packed object source a proper `struct odb_source`Patrick Steinhardt, Jun 4, 2026
  2. 01/16 packfile: rename `struct packfile_store` to `odb_source_packed`Patrick Steinhardt, Jun 4, 2026
  3. Karthik NayakJun 5, 2026
  4. Patrick SteinhardtJun 8, 2026
  5. 02/16 packfile: move packed source into "odb/" subsystemPatrick Steinhardt, Jun 4, 2026
  6. Karthik NayakJun 8, 2026
  7. Patrick SteinhardtJun 9, 2026
  8. 03/16 odb/source-packed: store pointer to "files" instead of generic sourcePatrick Steinhardt, Jun 4, 2026
  9. 04/16 odb/source-packed: start converting to a proper `struct odb_source`Patrick Steinhardt, Jun 4, 2026
  10. Karthik NayakJun 8, 2026
  11. Patrick SteinhardtJun 9, 2026
  12. Patrick SteinhardtJun 9, 2026
  13. 05/16 odb/source-packed: wire up `close()` callbackPatrick Steinhardt, Jun 4, 2026
  14. Karthik NayakJun 8, 2026
  15. 06/16 odb/source-packed: wire up `reprepare()` callbackPatrick Steinhardt, Jun 4, 2026
  16. 07/16 packfile: use higher-level interface to implement `has_object_pack()`Patrick Steinhardt, Jun 4, 2026
  17. Karthik NayakJun 8, 2026
  18. 08/16 odb/source-packed: wire up `read_object_info()` callbackPatrick Steinhardt, Jun 4, 2026
  19. 09/16 odb/source-packed: wire up `read_object_stream()` callbackPatrick Steinhardt, Jun 4, 2026
  20. 10/16 odb/source-packed: wire up `for_each_object()` callbackPatrick Steinhardt, Jun 4, 2026
  21. 11/16 odb/source-packed: wire up `count_objects()` callbackPatrick Steinhardt, Jun 4, 2026
  22. Karthik NayakJun 8, 2026
  23. Patrick SteinhardtJun 9, 2026
  24. 12/16 odb/source-packed: wire up `find_abbrev_len()` callbackPatrick Steinhardt, Jun 4, 2026
  25. 13/16 odb/source-packed: wire up `freshen_object()` callbackPatrick Steinhardt, Jun 4, 2026
  26. 14/16 odb/source-packed: stub out remaining functionsPatrick Steinhardt, Jun 4, 2026
  27. 15/16 midx: refactor interfaces to work on "packed" sourcePatrick Steinhardt, Jun 4, 2026
  28. 16/16 odb/source-packed: drop pointer to "files" parent sourcePatrick Steinhardt, Jun 4, 2026
  29. Karthik NayakJun 8, 2026
  30. Patrick SteinhardtJun 9, 2026
  31. 00/17 odb: make packed object source a proper `struct odb_source`Patrick Steinhardt, Jun 9, 2026
  32. 01/17 packfile: rename `struct packfile_store` to `odb_source_packed`Patrick Steinhardt, Jun 9, 2026
  33. Justin ToblerJun 16, 2026
  34. 02/17 packfile: split out packfile list logicPatrick Steinhardt, Jun 9, 2026
  35. 03/17 packfile: move packed source into "odb/" subsystemPatrick Steinhardt, Jun 9, 2026
  36. 04/17 odb/source-packed: store pointer to "files" instead of generic sourcePatrick Steinhardt, Jun 9, 2026
  37. Justin ToblerJun 16, 2026
  38. Justin ToblerJun 16, 2026
  39. Patrick SteinhardtJun 17, 2026
  40. 05/17 odb/source-packed: start converting to a proper `struct odb_source`Patrick Steinhardt, Jun 9, 2026
  41. Justin ToblerJun 16, 2026
  42. Patrick SteinhardtJun 17, 2026
  43. 06/17 odb/source-packed: wire up `close()` callbackPatrick Steinhardt, Jun 9, 2026
  44. 07/17 odb/source-packed: wire up `reprepare()` callbackPatrick Steinhardt, Jun 9, 2026
  45. Justin ToblerJun 16, 2026
  46. Patrick SteinhardtJun 17, 2026
  47. 08/17 packfile: use higher-level interface to implement `has_object_pack()`Patrick Steinhardt, Jun 9, 2026
  48. 09/17 odb/source-packed: wire up `read_object_info()` callbackPatrick Steinhardt, Jun 9, 2026
  49. 10/17 odb/source-packed: wire up `read_object_stream()` callbackPatrick Steinhardt, Jun 9, 2026
  50. 11/17 odb/source-packed: wire up `for_each_object()` callbackPatrick Steinhardt, Jun 9, 2026
  51. Justin ToblerJun 16, 2026
  52. Patrick SteinhardtJun 17, 2026
  53. 12/17 odb/source-packed: wire up `count_objects()` callbackPatrick Steinhardt, Jun 9, 2026
  54. 13/17 odb/source-packed: wire up `find_abbrev_len()` callbackPatrick Steinhardt, Jun 9, 2026
  55. 14/17 odb/source-packed: wire up `freshen_object()` callbackPatrick Steinhardt, Jun 9, 2026
  56. 15/17 odb/source-packed: stub out remaining functionsPatrick Steinhardt, Jun 9, 2026
  57. 16/17 midx: refactor interfaces to work on "packed" sourcePatrick Steinhardt, Jun 9, 2026
  58. Justin ToblerJun 16, 2026
  59. Patrick SteinhardtJun 17, 2026
  60. 17/17 odb/source-packed: drop pointer to "files" parent sourcePatrick Steinhardt, Jun 9, 2026
  61. Justin ToblerJun 16, 2026
  62. Patrick SteinhardtJun 17, 2026
  63. 00/17 odb: make packed object source a proper `struct odb_source`Patrick Steinhardt, Jun 17, 2026
  64. 01/17 packfile: rename `struct packfile_store` to `odb_source_packed`Patrick Steinhardt, Jun 17, 2026
  65. 02/17 packfile: split out packfile list logicPatrick Steinhardt, Jun 17, 2026
  66. 03/17 packfile: move packed source into "odb/" subsystemPatrick Steinhardt, Jun 17, 2026
  67. 04/17 odb/source-packed: store pointer to "files" instead of generic sourcePatrick Steinhardt, Jun 17, 2026
  68. 05/17 odb/source-packed: start converting to a proper `struct odb_source`Patrick Steinhardt, Jun 17, 2026
  69. 06/17 odb/source-packed: wire up `close()` callbackPatrick Steinhardt, Jun 17, 2026
  70. 07/17 odb/source-packed: wire up `reprepare()` callbackPatrick Steinhardt, Jun 17, 2026
  71. 08/17 packfile: use higher-level interface to implement `has_object_pack()`Patrick Steinhardt, Jun 17, 2026
  72. 09/17 odb/source-packed: wire up `read_object_info()` callbackPatrick Steinhardt, Jun 17, 2026
  73. 10/17 odb/source-packed: wire up `read_object_stream()` callbackPatrick Steinhardt, Jun 17, 2026
  74. 11/17 odb/source-packed: wire up `for_each_object()` callbackPatrick Steinhardt, Jun 17, 2026
  75. 12/17 odb/source-packed: wire up `count_objects()` callbackPatrick Steinhardt, Jun 17, 2026
  76. 13/17 odb/source-packed: wire up `find_abbrev_len()` callbackPatrick Steinhardt, Jun 17, 2026
  77. 14/17 odb/source-packed: wire up `freshen_object()` callbackPatrick Steinhardt, Jun 17, 2026
  78. 15/17 odb/source-packed: stub out remaining functionsPatrick Steinhardt, Jun 17, 2026
  79. Junio C HamanoJun 18, 2026
  80. Patrick SteinhardtJun 19, 2026
  81. Junio C HamanoJun 19, 2026
  82. 16/17 midx: refactor interfaces to work on "packed" sourcePatrick Steinhardt, Jun 17, 2026
  83. 17/17 odb/source-packed: drop pointer to "files" parent sourcePatrick Steinhardt, Jun 17, 2026
  84. Justin ToblerJun 17, 2026
  85. Junio C HamanoJun 17, 2026

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.