git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v2 02/19] streaming: drop the `open()` callback function

From
Patrick Steinhardt <ps@pks.im>
Date
Nov 23, 2025, 18:59 UTC
Message-ID
<aSNZklOl98TYRTUg@pks.im>
In-Reply-To
<xmqqqztr45t5.fsf@gitster.g>
On Fri, Nov 21, 2025 at 10:08:22AM -0800, Junio C Hamano wrote:
Show 22 quoted lines
> Patrick Steinhardt <ps@pks.im> writes:
> 
> > When creating a read stream we first populate the structure with the
> > open callback function and then subsequently call the function. This
> > layout is somewhat weird though:
> >
> >   - The structure needs to be allocated and partially populated with the
> >     open function before we can properly initialize it.
> 
> It is unclear what are left for delayed initialization from this
> description.
> 
> >   - We never use the `open()` callback after having opened it initially.
> 
> I was not sure what this means in v1 and it still is not clear to
> me.  Naively the above reads as if it is somehow desirable if we can
> call open() after we have already called it on an object.  The flow
> being a caller (e.g., stream_blob_to_fd()) first ask open_istream(),
> which calls the open method after figuring out which backend knows
> about the object and how to open a stream on it, I am not sure what
> you want your second and subsequent uses of the open() calklbacks
> do.  Puzzled.

I actually mean the opposite, so exactly what you describe: why do we store the `open()` callback in a member variable of the stream if it's only ever called a single time, only, and is never called a second time thereafter?

I'll rephrase this.
Show 25 quoted lines
> > Instead, drop the callback entirely and refactor `istream_source()` so
> > that we open the streams immediately. This unblocks a subsequent step,
> > where we'll also start to allocate the structure in the source-specific
> > logic.
> 
> Because I do not think these open methods specific to each storage
> mechanism cascades into each other, open-coding the logic to
> dispatch into these open() methods in istream_source() itself,
> instead of setting the method there and then have the caller call
> it, is a perfectly fine simplification, I think.
> 
> > @@ -478,19 +477,14 @@ struct odb_read_stream *open_istream(struct repository *r,
> >  {
> >  	struct odb_read_stream *st = xmalloc(sizeof(*st));
> >  	const struct object_id *real = lookup_replace_object(r, oid);
> > -	int ret = istream_source(st, r, real, type);
> > +	int ret;
> >  
> > +	ret = istream_source(st, r, real, type);
> >  	if (ret) {
> >  		free(st);
> >  		return NULL;
> >  	}
> 
> A patch noise?
Will drop this line.
Thanks!
Patrick
Previous: Junio C HamanoNext: Patrick Steinhardt
Message 41 of 85 in “Refactor object read streams to work via object sources”
  1. 00/18 Refactor object read streams to work via object sourcesPatrick Steinhardt, Nov 19, 2025
  2. 01/18 streaming: rename `git_istream` into `odb_read_stream`Patrick Steinhardt, Nov 19, 2025
  3. Justin ToblerNov 19, 2025
  4. Junio C HamanoNov 19, 2025
  5. Patrick SteinhardtNov 21, 2025
  6. 02/18 streaming: drop the `open()` callback functionPatrick Steinhardt, Nov 19, 2025
  7. Karthik NayakNov 19, 2025
  8. Justin ToblerNov 19, 2025
  9. Patrick SteinhardtNov 21, 2025
  10. 03/18 streaming: propagate final object type via the streamPatrick Steinhardt, Nov 19, 2025
  11. Justin ToblerNov 19, 2025
  12. Patrick SteinhardtNov 21, 2025
  13. 04/18 streaming: explicitly pass packfile info when streaming a packed objectPatrick Steinhardt, Nov 19, 2025
  14. 05/18 streaming: allocate stream inside the backend-specific logicPatrick Steinhardt, Nov 19, 2025
  15. Karthik NayakNov 19, 2025
  16. Patrick SteinhardtNov 21, 2025
  17. 06/18 streaming: create structure for in-core object streamsPatrick Steinhardt, Nov 19, 2025
  18. Karthik NayakNov 19, 2025
  19. Patrick SteinhardtNov 21, 2025
  20. 07/18 streaming: create structure for loose object streamsPatrick Steinhardt, Nov 19, 2025
  21. 08/18 streaming: create structure for packed object streamsPatrick Steinhardt, Nov 19, 2025
  22. 09/18 streaming: create structure for filtered object streamsPatrick Steinhardt, Nov 19, 2025
  23. 10/18 streaming: move zlib stream into backendsPatrick Steinhardt, Nov 19, 2025
  24. 11/18 packfile: introduce function to read object info from a storePatrick Steinhardt, Nov 19, 2025
  25. Karthik NayakNov 19, 2025
  26. Patrick SteinhardtNov 21, 2025
  27. 12/18 streaming: rely on object sources to create object streamPatrick Steinhardt, Nov 19, 2025
  28. Karthik NayakNov 19, 2025
  29. 13/18 streaming: get rid of `the_repository`Patrick Steinhardt, Nov 19, 2025
  30. 14/18 streaming: make the `odb_read_stream` definition publicPatrick Steinhardt, Nov 19, 2025
  31. Karthik NayakNov 19, 2025
  32. Patrick SteinhardtNov 21, 2025
  33. 15/18 streaming: move logic to read loose objects streams into backendPatrick Steinhardt, Nov 19, 2025
  34. 16/18 streaming: move logic to read packed objects streams into backendPatrick Steinhardt, Nov 19, 2025
  35. 17/18 streaming: refactor interface to be object-database-centricPatrick Steinhardt, Nov 19, 2025
  36. 18/18 streaming: move into object database subsystemPatrick Steinhardt, Nov 19, 2025
  37. 00/19 Refactor object read streams to work via object sourcesPatrick Steinhardt, Nov 21, 2025
  38. 01/19 streaming: rename `git_istream` into `odb_read_stream`Patrick Steinhardt, Nov 21, 2025
  39. 02/19 streaming: drop the `open()` callback functionPatrick Steinhardt, Nov 21, 2025
  40. Junio C HamanoNov 21, 2025
  41. Patrick SteinhardtNov 23, 2025
  42. 03/19 streaming: propagate final object type via the streamPatrick Steinhardt, Nov 21, 2025
  43. 04/19 streaming: explicitly pass packfile info when streaming a packed objectPatrick Steinhardt, Nov 21, 2025
  44. 05/19 streaming: allocate stream inside the backend-specific logicPatrick Steinhardt, Nov 21, 2025
  45. 06/19 streaming: create structure for in-core object streamsPatrick Steinhardt, Nov 21, 2025
  46. 07/19 streaming: create structure for loose object streamsPatrick Steinhardt, Nov 21, 2025
  47. 08/19 streaming: create structure for packed object streamsPatrick Steinhardt, Nov 21, 2025
  48. 09/19 streaming: create structure for filtered object streamsPatrick Steinhardt, Nov 21, 2025
  49. 10/19 streaming: move zlib stream into backendsPatrick Steinhardt, Nov 21, 2025
  50. 11/19 packfile: introduce function to read object info from a storePatrick Steinhardt, Nov 21, 2025
  51. 12/19 streaming: rely on object sources to create object streamPatrick Steinhardt, Nov 21, 2025
  52. Junio C HamanoNov 21, 2025
  53. Patrick SteinhardtNov 23, 2025
  54. 13/19 streaming: get rid of `the_repository`Patrick Steinhardt, Nov 21, 2025
  55. Junio C HamanoNov 21, 2025
  56. Patrick SteinhardtNov 23, 2025
  57. 14/19 streaming: make the `odb_read_stream` definition publicPatrick Steinhardt, Nov 21, 2025
  58. 15/19 streaming: move logic to read loose objects streams into backendPatrick Steinhardt, Nov 21, 2025
  59. 16/19 streaming: move logic to read packed objects streams into backendPatrick Steinhardt, Nov 21, 2025
  60. 17/19 streaming: refactor interface to be object-database-centricPatrick Steinhardt, Nov 21, 2025
  61. Junio C HamanoNov 22, 2025
  62. Patrick SteinhardtNov 23, 2025
  63. 18/19 streaming: move into object database subsystemPatrick Steinhardt, Nov 21, 2025
  64. Junio C HamanoNov 23, 2025
  65. 19/19 streaming: drop redundant type and size pointersPatrick Steinhardt, Nov 21, 2025
  66. 00/19 Refactor object read streams to work via object sourcesPatrick Steinhardt, Nov 23, 2025
  67. 01/19 streaming: rename `git_istream` into `odb_read_stream`Patrick Steinhardt, Nov 23, 2025
  68. 02/19 streaming: drop the `open()` callback functionPatrick Steinhardt, Nov 23, 2025
  69. 03/19 streaming: propagate final object type via the streamPatrick Steinhardt, Nov 23, 2025
  70. 04/19 streaming: explicitly pass packfile info when streaming a packed objectPatrick Steinhardt, Nov 23, 2025
  71. 05/19 streaming: allocate stream inside the backend-specific logicPatrick Steinhardt, Nov 23, 2025
  72. 06/19 streaming: create structure for in-core object streamsPatrick Steinhardt, Nov 23, 2025
  73. 07/19 streaming: create structure for loose object streamsPatrick Steinhardt, Nov 23, 2025
  74. 08/19 streaming: create structure for packed object streamsPatrick Steinhardt, Nov 23, 2025
  75. 09/19 streaming: create structure for filtered object streamsPatrick Steinhardt, Nov 23, 2025
  76. 10/19 streaming: move zlib stream into backendsPatrick Steinhardt, Nov 23, 2025
  77. 11/19 packfile: introduce function to read object info from a storePatrick Steinhardt, Nov 23, 2025
  78. 12/19 streaming: rely on object sources to create object streamPatrick Steinhardt, Nov 23, 2025
  79. 13/19 streaming: get rid of `the_repository`Patrick Steinhardt, Nov 23, 2025
  80. 14/19 streaming: make the `odb_read_stream` definition publicPatrick Steinhardt, Nov 23, 2025
  81. 15/19 streaming: move logic to read loose objects streams into backendPatrick Steinhardt, Nov 23, 2025
  82. 16/19 streaming: move logic to read packed objects streams into backendPatrick Steinhardt, Nov 23, 2025
  83. 17/19 streaming: refactor interface to be object-database-centricPatrick Steinhardt, Nov 23, 2025
  84. 18/19 streaming: move into object database subsystemPatrick Steinhardt, Nov 23, 2025
  85. 19/19 streaming: drop redundant type and size pointersPatrick Steinhardt, Nov 23, 2025

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.