git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v2 18/19] streaming: move into object database subsystem

From
Junio C Hamano <gitster@pobox.com>
Date
Nov 23, 2025, 02:20 UTC
Message-ID
<xmqq5xb132xn.fsf@gitster.g>
In-Reply-To
<20251121-b4-pks-odb-read-stream-v2-18-ca8534963150@pks.im>
Patrick Steinhardt <ps@pks.im> writes:
> The "streaming" terminology is somewhat generic, so it may not be
> immediately obvious that "streaming.{c,h}" is specific to the object
> database. Rectify this by moving it into the "odb/" directory so that it
> can be immediately attributed to the object subsystem.
I do not have an objection against this move.  Looking good.
Show 270 quoted lines
>
> Signed-off-by: Patrick Steinhardt <ps@pks.im>
> ---
>  Makefile                       | 2 +-
>  archive-tar.c                  | 2 +-
>  archive-zip.c                  | 2 +-
>  builtin/cat-file.c             | 2 +-
>  builtin/fsck.c                 | 2 +-
>  builtin/index-pack.c           | 2 +-
>  builtin/log.c                  | 2 +-
>  builtin/pack-objects.c         | 2 +-
>  entry.c                        | 2 +-
>  meson.build                    | 2 +-
>  object-file.c                  | 2 +-
>  streaming.c => odb/streaming.c | 2 +-
>  streaming.h => odb/streaming.h | 0
>  packfile.c                     | 2 +-
>  parallel-checkout.c            | 2 +-
>  15 files changed, 14 insertions(+), 14 deletions(-)
>
> diff --git a/Makefile b/Makefile
> index 7e0f77e298..6d8dcc4622 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -1201,6 +1201,7 @@ LIB_OBJS += object-file.o
>  LIB_OBJS += object-name.o
>  LIB_OBJS += object.o
>  LIB_OBJS += odb.o
> +LIB_OBJS += odb/streaming.o
>  LIB_OBJS += oid-array.o
>  LIB_OBJS += oidmap.o
>  LIB_OBJS += oidset.o
> @@ -1294,7 +1295,6 @@ LIB_OBJS += split-index.o
>  LIB_OBJS += stable-qsort.o
>  LIB_OBJS += statinfo.o
>  LIB_OBJS += strbuf.o
> -LIB_OBJS += streaming.o
>  LIB_OBJS += string-list.o
>  LIB_OBJS += strmap.o
>  LIB_OBJS += strvec.o
> diff --git a/archive-tar.c b/archive-tar.c
> index 4133e09ca1..74499c311f 100644
> --- a/archive-tar.c
> +++ b/archive-tar.c
> @@ -12,8 +12,8 @@
>  #include "tar.h"
>  #include "archive.h"
>  #include "odb.h"
> +#include "odb/streaming.h"
>  #include "strbuf.h"
> -#include "streaming.h"
>  #include "run-command.h"
>  #include "write-or-die.h"
>  
> diff --git a/archive-zip.c b/archive-zip.c
> index ff57f4f884..2b645f28ef 100644
> --- a/archive-zip.c
> +++ b/archive-zip.c
> @@ -10,9 +10,9 @@
>  #include "gettext.h"
>  #include "git-zlib.h"
>  #include "hex.h"
> -#include "streaming.h"
>  #include "utf8.h"
>  #include "odb.h"
> +#include "odb/streaming.h"
>  #include "strbuf.h"
>  #include "userdiff.h"
>  #include "write-or-die.h"
> diff --git a/builtin/cat-file.c b/builtin/cat-file.c
> index 120d626d66..505ddaa12f 100644
> --- a/builtin/cat-file.c
> +++ b/builtin/cat-file.c
> @@ -18,13 +18,13 @@
>  #include "list-objects-filter-options.h"
>  #include "parse-options.h"
>  #include "userdiff.h"
> -#include "streaming.h"
>  #include "oid-array.h"
>  #include "packfile.h"
>  #include "pack-bitmap.h"
>  #include "object-file.h"
>  #include "object-name.h"
>  #include "odb.h"
> +#include "odb/streaming.h"
>  #include "replace-object.h"
>  #include "promisor-remote.h"
>  #include "mailmap.h"
> diff --git a/builtin/fsck.c b/builtin/fsck.c
> index 1a348d43c2..c7d2eea287 100644
> --- a/builtin/fsck.c
> +++ b/builtin/fsck.c
> @@ -13,11 +13,11 @@
>  #include "fsck.h"
>  #include "parse-options.h"
>  #include "progress.h"
> -#include "streaming.h"
>  #include "packfile.h"
>  #include "object-file.h"
>  #include "object-name.h"
>  #include "odb.h"
> +#include "odb/streaming.h"
>  #include "path.h"
>  #include "read-cache-ll.h"
>  #include "replace-object.h"
> diff --git a/builtin/index-pack.c b/builtin/index-pack.c
> index 67221dbe6a..6403edd3a6 100644
> --- a/builtin/index-pack.c
> +++ b/builtin/index-pack.c
> @@ -16,12 +16,12 @@
>  #include "progress.h"
>  #include "fsck.h"
>  #include "strbuf.h"
> -#include "streaming.h"
>  #include "thread-utils.h"
>  #include "packfile.h"
>  #include "pack-revindex.h"
>  #include "object-file.h"
>  #include "odb.h"
> +#include "odb/streaming.h"
>  #include "oid-array.h"
>  #include "oidset.h"
>  #include "path.h"
> diff --git a/builtin/log.c b/builtin/log.c
> index e7b83a6e00..d4cf9c59c8 100644
> --- a/builtin/log.c
> +++ b/builtin/log.c
> @@ -16,6 +16,7 @@
>  #include "refs.h"
>  #include "object-name.h"
>  #include "odb.h"
> +#include "odb/streaming.h"
>  #include "pager.h"
>  #include "color.h"
>  #include "commit.h"
> @@ -35,7 +36,6 @@
>  #include "parse-options.h"
>  #include "line-log.h"
>  #include "branch.h"
> -#include "streaming.h"
>  #include "version.h"
>  #include "mailmap.h"
>  #include "progress.h"
> diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c
> index adf267c59d..f6c01bc4e0 100644
> --- a/builtin/pack-objects.c
> +++ b/builtin/pack-objects.c
> @@ -22,7 +22,6 @@
>  #include "pack-objects.h"
>  #include "progress.h"
>  #include "refs.h"
> -#include "streaming.h"
>  #include "thread-utils.h"
>  #include "pack-bitmap.h"
>  #include "delta-islands.h"
> @@ -33,6 +32,7 @@
>  #include "packfile.h"
>  #include "object-file.h"
>  #include "odb.h"
> +#include "odb/streaming.h"
>  #include "replace-object.h"
>  #include "dir.h"
>  #include "midx.h"
> diff --git a/entry.c b/entry.c
> index 38dfe670f7..7817aee362 100644
> --- a/entry.c
> +++ b/entry.c
> @@ -2,13 +2,13 @@
>  
>  #include "git-compat-util.h"
>  #include "odb.h"
> +#include "odb/streaming.h"
>  #include "dir.h"
>  #include "environment.h"
>  #include "gettext.h"
>  #include "hex.h"
>  #include "name-hash.h"
>  #include "sparse-index.h"
> -#include "streaming.h"
>  #include "submodule.h"
>  #include "symlinks.h"
>  #include "progress.h"
> diff --git a/meson.build b/meson.build
> index 1f95a06edb..fc82929b37 100644
> --- a/meson.build
> +++ b/meson.build
> @@ -397,6 +397,7 @@ libgit_sources = [
>    'object-name.c',
>    'object.c',
>    'odb.c',
> +  'odb/streaming.c',
>    'oid-array.c',
>    'oidmap.c',
>    'oidset.c',
> @@ -490,7 +491,6 @@ libgit_sources = [
>    'stable-qsort.c',
>    'statinfo.c',
>    'strbuf.c',
> -  'streaming.c',
>    'string-list.c',
>    'strmap.c',
>    'strvec.c',
> diff --git a/object-file.c b/object-file.c
> index c6d2f2d953..4b46cf5b71 100644
> --- a/object-file.c
> +++ b/object-file.c
> @@ -20,13 +20,13 @@
>  #include "object-file-convert.h"
>  #include "object-file.h"
>  #include "odb.h"
> +#include "odb/streaming.h"
>  #include "oidtree.h"
>  #include "pack.h"
>  #include "packfile.h"
>  #include "path.h"
>  #include "read-cache-ll.h"
>  #include "setup.h"
> -#include "streaming.h"
>  #include "tempfile.h"
>  #include "tmp-objdir.h"
>  
> diff --git a/streaming.c b/odb/streaming.c
> similarity index 99%
> rename from streaming.c
> rename to odb/streaming.c
> index 3ac1a0c40f..a7ee50dc34 100644
> --- a/streaming.c
> +++ b/odb/streaming.c
> @@ -5,10 +5,10 @@
>  #include "git-compat-util.h"
>  #include "convert.h"
>  #include "environment.h"
> -#include "streaming.h"
>  #include "repository.h"
>  #include "object-file.h"
>  #include "odb.h"
> +#include "odb/streaming.h"
>  #include "replace-object.h"
>  #include "packfile.h"
>  
> diff --git a/streaming.h b/odb/streaming.h
> similarity index 100%
> rename from streaming.h
> rename to odb/streaming.h
> diff --git a/packfile.c b/packfile.c
> index ad56ce0b90..7a16aaa90d 100644
> --- a/packfile.c
> +++ b/packfile.c
> @@ -20,7 +20,7 @@
>  #include "tree.h"
>  #include "object-file.h"
>  #include "odb.h"
> -#include "streaming.h"
> +#include "odb/streaming.h"
>  #include "midx.h"
>  #include "commit-graph.h"
>  #include "pack-revindex.h"
> diff --git a/parallel-checkout.c b/parallel-checkout.c
> index 1cb6701b92..0bf4bd6d4a 100644
> --- a/parallel-checkout.c
> +++ b/parallel-checkout.c
> @@ -13,7 +13,7 @@
>  #include "read-cache-ll.h"
>  #include "run-command.h"
>  #include "sigchain.h"
> -#include "streaming.h"
> +#include "odb/streaming.h"
>  #include "symlinks.h"
>  #include "thread-utils.h"
>  #include "trace2.h"
Previous: Patrick SteinhardtNext: Patrick Steinhardt
Message 64 of 85 in “Refactor object read streams to work via object sources”
  1. 00/18 Refactor object read streams to work via object sourcesPatrick Steinhardt, Nov 19, 2025
  2. 01/18 streaming: rename `git_istream` into `odb_read_stream`Patrick Steinhardt, Nov 19, 2025
  3. Justin ToblerNov 19, 2025
  4. Junio C HamanoNov 19, 2025
  5. Patrick SteinhardtNov 21, 2025
  6. 02/18 streaming: drop the `open()` callback functionPatrick Steinhardt, Nov 19, 2025
  7. Karthik NayakNov 19, 2025
  8. Justin ToblerNov 19, 2025
  9. Patrick SteinhardtNov 21, 2025
  10. 03/18 streaming: propagate final object type via the streamPatrick Steinhardt, Nov 19, 2025
  11. Justin ToblerNov 19, 2025
  12. Patrick SteinhardtNov 21, 2025
  13. 04/18 streaming: explicitly pass packfile info when streaming a packed objectPatrick Steinhardt, Nov 19, 2025
  14. 05/18 streaming: allocate stream inside the backend-specific logicPatrick Steinhardt, Nov 19, 2025
  15. Karthik NayakNov 19, 2025
  16. Patrick SteinhardtNov 21, 2025
  17. 06/18 streaming: create structure for in-core object streamsPatrick Steinhardt, Nov 19, 2025
  18. Karthik NayakNov 19, 2025
  19. Patrick SteinhardtNov 21, 2025
  20. 07/18 streaming: create structure for loose object streamsPatrick Steinhardt, Nov 19, 2025
  21. 08/18 streaming: create structure for packed object streamsPatrick Steinhardt, Nov 19, 2025
  22. 09/18 streaming: create structure for filtered object streamsPatrick Steinhardt, Nov 19, 2025
  23. 10/18 streaming: move zlib stream into backendsPatrick Steinhardt, Nov 19, 2025
  24. 11/18 packfile: introduce function to read object info from a storePatrick Steinhardt, Nov 19, 2025
  25. Karthik NayakNov 19, 2025
  26. Patrick SteinhardtNov 21, 2025
  27. 12/18 streaming: rely on object sources to create object streamPatrick Steinhardt, Nov 19, 2025
  28. Karthik NayakNov 19, 2025
  29. 13/18 streaming: get rid of `the_repository`Patrick Steinhardt, Nov 19, 2025
  30. 14/18 streaming: make the `odb_read_stream` definition publicPatrick Steinhardt, Nov 19, 2025
  31. Karthik NayakNov 19, 2025
  32. Patrick SteinhardtNov 21, 2025
  33. 15/18 streaming: move logic to read loose objects streams into backendPatrick Steinhardt, Nov 19, 2025
  34. 16/18 streaming: move logic to read packed objects streams into backendPatrick Steinhardt, Nov 19, 2025
  35. 17/18 streaming: refactor interface to be object-database-centricPatrick Steinhardt, Nov 19, 2025
  36. 18/18 streaming: move into object database subsystemPatrick Steinhardt, Nov 19, 2025
  37. 00/19 Refactor object read streams to work via object sourcesPatrick Steinhardt, Nov 21, 2025
  38. 01/19 streaming: rename `git_istream` into `odb_read_stream`Patrick Steinhardt, Nov 21, 2025
  39. 02/19 streaming: drop the `open()` callback functionPatrick Steinhardt, Nov 21, 2025
  40. Junio C HamanoNov 21, 2025
  41. Patrick SteinhardtNov 23, 2025
  42. 03/19 streaming: propagate final object type via the streamPatrick Steinhardt, Nov 21, 2025
  43. 04/19 streaming: explicitly pass packfile info when streaming a packed objectPatrick Steinhardt, Nov 21, 2025
  44. 05/19 streaming: allocate stream inside the backend-specific logicPatrick Steinhardt, Nov 21, 2025
  45. 06/19 streaming: create structure for in-core object streamsPatrick Steinhardt, Nov 21, 2025
  46. 07/19 streaming: create structure for loose object streamsPatrick Steinhardt, Nov 21, 2025
  47. 08/19 streaming: create structure for packed object streamsPatrick Steinhardt, Nov 21, 2025
  48. 09/19 streaming: create structure for filtered object streamsPatrick Steinhardt, Nov 21, 2025
  49. 10/19 streaming: move zlib stream into backendsPatrick Steinhardt, Nov 21, 2025
  50. 11/19 packfile: introduce function to read object info from a storePatrick Steinhardt, Nov 21, 2025
  51. 12/19 streaming: rely on object sources to create object streamPatrick Steinhardt, Nov 21, 2025
  52. Junio C HamanoNov 21, 2025
  53. Patrick SteinhardtNov 23, 2025
  54. 13/19 streaming: get rid of `the_repository`Patrick Steinhardt, Nov 21, 2025
  55. Junio C HamanoNov 21, 2025
  56. Patrick SteinhardtNov 23, 2025
  57. 14/19 streaming: make the `odb_read_stream` definition publicPatrick Steinhardt, Nov 21, 2025
  58. 15/19 streaming: move logic to read loose objects streams into backendPatrick Steinhardt, Nov 21, 2025
  59. 16/19 streaming: move logic to read packed objects streams into backendPatrick Steinhardt, Nov 21, 2025
  60. 17/19 streaming: refactor interface to be object-database-centricPatrick Steinhardt, Nov 21, 2025
  61. Junio C HamanoNov 22, 2025
  62. Patrick SteinhardtNov 23, 2025
  63. 18/19 streaming: move into object database subsystemPatrick Steinhardt, Nov 21, 2025
  64. Junio C HamanoNov 23, 2025
  65. 19/19 streaming: drop redundant type and size pointersPatrick Steinhardt, Nov 21, 2025
  66. 00/19 Refactor object read streams to work via object sourcesPatrick Steinhardt, Nov 23, 2025
  67. 01/19 streaming: rename `git_istream` into `odb_read_stream`Patrick Steinhardt, Nov 23, 2025
  68. 02/19 streaming: drop the `open()` callback functionPatrick Steinhardt, Nov 23, 2025
  69. 03/19 streaming: propagate final object type via the streamPatrick Steinhardt, Nov 23, 2025
  70. 04/19 streaming: explicitly pass packfile info when streaming a packed objectPatrick Steinhardt, Nov 23, 2025
  71. 05/19 streaming: allocate stream inside the backend-specific logicPatrick Steinhardt, Nov 23, 2025
  72. 06/19 streaming: create structure for in-core object streamsPatrick Steinhardt, Nov 23, 2025
  73. 07/19 streaming: create structure for loose object streamsPatrick Steinhardt, Nov 23, 2025
  74. 08/19 streaming: create structure for packed object streamsPatrick Steinhardt, Nov 23, 2025
  75. 09/19 streaming: create structure for filtered object streamsPatrick Steinhardt, Nov 23, 2025
  76. 10/19 streaming: move zlib stream into backendsPatrick Steinhardt, Nov 23, 2025
  77. 11/19 packfile: introduce function to read object info from a storePatrick Steinhardt, Nov 23, 2025
  78. 12/19 streaming: rely on object sources to create object streamPatrick Steinhardt, Nov 23, 2025
  79. 13/19 streaming: get rid of `the_repository`Patrick Steinhardt, Nov 23, 2025
  80. 14/19 streaming: make the `odb_read_stream` definition publicPatrick Steinhardt, Nov 23, 2025
  81. 15/19 streaming: move logic to read loose objects streams into backendPatrick Steinhardt, Nov 23, 2025
  82. 16/19 streaming: move logic to read packed objects streams into backendPatrick Steinhardt, Nov 23, 2025
  83. 17/19 streaming: refactor interface to be object-database-centricPatrick Steinhardt, Nov 23, 2025
  84. 18/19 streaming: move into object database subsystemPatrick Steinhardt, Nov 23, 2025
  85. 19/19 streaming: drop redundant type and size pointersPatrick Steinhardt, Nov 23, 2025

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.