[PATCH 00/18] Refactor object read streams to work via object sources

Hi,

the `git_istream` data structure can be used to read objects from the object database in a streaming fashion. This is used for example to read large files that one doesn't want to load into memory in full.

In the current architecture, all the logic to handle these streams is fully self-contained in "streaming.c". It contains the logic to set up streams for loose, packed, in-memory and filtered objects. This doesn't really play all that well with pluggable object databases, as it should be the responsibility of the object database source itself to handle the logic.

This patch series thus revamps our object read streams: instead of being entirely contained in "streaming.c", the format-specific streams are now created by the ODB sources. This allows each source itself to decide whether and, if so, how to make objects streamable.

This overall requires quite a bit of refactoring, but I think that the end result is an easier-to-understand infrastructure that is an improvement even without pluggable object databases.

This series is built on top of v2.52.0 with ps/object-source-loose at 3e5e360888 (object-file: refactor writing objects via a stream, 2025-11-03) merged into it.

Thanks!

Patrick

---
Patrick Steinhardt (18):
      streaming: rename `git_istream` into `odb_read_stream`
      streaming: drop the `open()` callback function
      streaming: propagate final object type via the stream
      streaming: explicitly pass packfile info when streaming a packed object
      streaming: allocate stream inside the backend-specific logic
      streaming: create structure for in-core object streams
      streaming: create structure for loose object streams
      streaming: create structure for packed object streams
      streaming: create structure for filtered object streams
      streaming: move zlib stream into backends
      packfile: introduce function to read object info from a store
      streaming: rely on object sources to create object stream
      streaming: get rid of `the_repository`
      streaming: make the `odb_read_stream` definition public
      streaming: move logic to read loose objects streams into backend
      streaming: move logic to read packed objects streams into backend
      streaming: refactor interface to be object-database-centric
      streaming: move into object database subsystem

 Makefile               |   2 +-
 archive-tar.c          |  10 +-
 archive-zip.c          |  16 +-
 builtin/cat-file.c     |   4 +-
 builtin/fsck.c         |   5 +-
 builtin/index-pack.c   |  12 +-
 builtin/log.c          |   6 +-
 builtin/pack-objects.c |  20 +-
 entry.c                |   4 +-
 meson.build            |   2 +-
 object-file.c          | 179 ++++++++++++++--
 object-file.h          |  42 +---
 odb.c                  |  29 +--
 odb/streaming.c        | 299 ++++++++++++++++++++++++++
 odb/streaming.h        |  70 ++++++
 packfile.c             | 199 ++++++++++++++++--
 packfile.h             |  17 +-
 parallel-checkout.c    |   5 +-
 streaming.c            | 561 -------------------------------------------------
 streaming.h            |  21 --
 20 files changed, 784 insertions(+), 719 deletions(-)

--- base-commit: 899e578b5b7c020aec806bd694adf2563f62843c change-id: 20251107-b4-pks-odb-read-stream-7ea7f0e0a8f4