git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v3 1/6] odb: introduce interface to generate packfiles

From
Patrick Steinhardt <ps@pks.im>
Date
Aug 21, 2026, 06:28 UTC
Message-ID
<aofwHhFeeWgh_3FY@pks.im>
In-Reply-To
<xmqqik54soy0.fsf@gitster.g>
On Thu, Aug 20, 2026 at 10:04:55AM -0700, Junio C Hamano wrote:
Show 41 quoted lines
> Patrick Steinhardt <ps@pks.im> writes:
> > diff --git a/odb.c b/odb.c
> > index caf1d0f542..cd9d5b48bc 100644
> > --- a/odb.c
> > +++ b/odb.c
> > @@ -1046,6 +1046,27 @@ bool odb_optimize_required(struct object_database *odb,
> >  	return odb_source_optimize_required(odb->sources, opts);
> >  }
> >  
> > +void odb_generate_pack_options_release(struct odb_generate_pack_options *opts)
> > +{
> > +	oid_array_clear(&opts->wants);
> > +	oid_array_clear(&opts->haves);
> > +	oid_array_clear(&opts->shallows);
> > +}
> > +
> > +int odb_generate_pack(struct object_database *odb,
> > +		      struct odb_pack_generator **out,
> > +		      const struct odb_generate_pack_options *opts)
> > +{
> > +	if (!odb->sources->generate_pack)
> > +		return error(_("primary object source does not support generating packfiles"));
> > +	return odb_source_generate_pack(odb->sources, out, opts);
> > +}
> 
> Perhaps a stupid question but the opts->pack_fd is documented:
> 
> > +struct odb_generate_pack_options {
> > ...
> > +	/*
> > +	 * File descriptor that the generated pack shall be written to. If set
> > +	 * to `-1`, a pipe will be created and exposed via the pack generator's
> > +	 * `out` field. If set to `0`, the pack will be written to the standard
> > +	 * output stream. Otherwise, the provided descriptor will be written to
> > +	 * and is consumed by the generator.
> > +	 */
> > +	int pack_fd;
> > +
> 
> Here I assume that "and is consumed by" refers to "generator writes
> into it and then closes it when it is done"?
Yes.
Show 5 quoted lines
> odb_source_generate_pack() delegate to source->generate_pack(),
> which I presume goes to odb_source_files_generate_pack(), which in
> turn assigns opts->pack_fd to cp->out and calls start_command(cp) to
> run pack-objects.  The file descriptor is closed when the process
> finishes.
Exactly.
> What happens if the odb->sources[0] does not support .generate_pack?

If it does not support generating packs then Git would crash as this is a non-optional callback. All sources that could be our primary source though do support it, and the expectation is that any future backends would know how to implement it, too.

> Should opts->pack_fd be "consumed" here to avoid leaking it, or we
> do not have to worry about it because the caller will soon exit
> itself?

So this case here should not ever happen -- if we don't have the callback, then there's nothing that can even set `pack_fd` and we should die.

Patrick
Previous: Junio C HamanoNext: Patrick Steinhardt
Message 36 of 58 in “odb: make packfile generation pluggable”
  1. 0/5 odb: make packfile generation pluggablePatrick Steinhardt, Aug 7, 2026
  2. 1/5 odb: introduce interface to generate packfilesPatrick Steinhardt, Aug 7, 2026
  3. 2/5 upload-pack: generate packfiles via the object databasePatrick Steinhardt, Aug 7, 2026
  4. 3/5 send-pack: generate packfiles via the object databasePatrick Steinhardt, Aug 7, 2026
  5. 4/5 builtin/bundle: refactor option handling for progress meterPatrick Steinhardt, Aug 7, 2026
  6. 5/5 bundle: generate packfiles via the object databasePatrick Steinhardt, Aug 7, 2026
  7. Junio C HamanoAug 13, 2026
  8. Patrick SteinhardtAug 14, 2026
  9. Junio C HamanoAug 7, 2026
  10. Patrick SteinhardtAug 10, 2026
  11. Taylor BlauAug 12, 2026
  12. Patrick SteinhardtAug 13, 2026
  13. Junio C HamanoAug 13, 2026
  14. 0/6 odb: make packfile generation pluggablePatrick Steinhardt, Aug 17, 2026
  15. 1/6 odb: introduce interface to generate packfilesPatrick Steinhardt, Aug 17, 2026
  16. Elijah NewrenAug 19, 2026
  17. Patrick SteinhardtAug 20, 2026
  18. Elijah NewrenAug 21, 2026
  19. Patrick SteinhardtAug 21, 2026
  20. 2/6 upload-pack: generate packfiles via the object databasePatrick Steinhardt, Aug 17, 2026
  21. 3/6 send-pack: generate packfiles via the object databasePatrick Steinhardt, Aug 17, 2026
  22. 4/6 builtin/bundle: refactor option handling for progress meterPatrick Steinhardt, Aug 17, 2026
  23. 5/6 bundle: get (mostly) rid of `the_repository`Patrick Steinhardt, Aug 17, 2026
  24. Junio C HamanoAug 17, 2026
  25. Patrick SteinhardtAug 18, 2026
  26. Elijah NewrenAug 21, 2026
  27. Patrick SteinhardtAug 21, 2026
  28. 6/6 bundle: generate packfiles via the object databasePatrick Steinhardt, Aug 17, 2026
  29. Justin ToblerAug 19, 2026
  30. Patrick SteinhardtAug 20, 2026
  31. 0/6 odb: make packfile generation pluggablePatrick Steinhardt, Aug 20, 2026
  32. 1/6 odb: introduce interface to generate packfilesPatrick Steinhardt, Aug 20, 2026
  33. Karthik NayakAug 20, 2026
  34. Patrick SteinhardtAug 20, 2026
  35. Junio C HamanoAug 20, 2026
  36. Patrick SteinhardtAug 21, 2026
  37. 2/6 upload-pack: generate packfiles via the object databasePatrick Steinhardt, Aug 20, 2026
  38. Karthik NayakAug 20, 2026
  39. Patrick SteinhardtAug 20, 2026
  40. Karthik NayakAug 20, 2026
  41. 3/6 send-pack: generate packfiles via the object databasePatrick Steinhardt, Aug 20, 2026
  42. 4/6 builtin/bundle: refactor option handling for progress meterPatrick Steinhardt, Aug 20, 2026
  43. Karthik NayakAug 20, 2026
  44. 5/6 bundle: get (mostly) rid of `the_repository`Patrick Steinhardt, Aug 20, 2026
  45. 6/6 bundle: generate packfiles via the object databasePatrick Steinhardt, Aug 20, 2026
  46. Karthik NayakAug 20, 2026
  47. Patrick SteinhardtAug 20, 2026
  48. Karthik NayakAug 20, 2026
  49. Patrick SteinhardtAug 20, 2026
  50. Elijah NewrenAug 21, 2026
  51. 0/6 odb: make packfile generation pluggablePatrick Steinhardt, Aug 21, 2026
  52. 1/6 odb: introduce interface to generate packfilesPatrick Steinhardt, Aug 21, 2026
  53. 2/6 upload-pack: generate packfiles via the object databasePatrick Steinhardt, Aug 21, 2026
  54. 3/6 send-pack: generate packfiles via the object databasePatrick Steinhardt, Aug 21, 2026
  55. 4/6 builtin/bundle: refactor option handling for progress meterPatrick Steinhardt, Aug 21, 2026
  56. 5/6 bundle: get (mostly) rid of `the_repository`Patrick Steinhardt, Aug 21, 2026
  57. 6/6 bundle: generate packfiles via the object databasePatrick Steinhardt, Aug 21, 2026
  58. Karthik NayakAug 21, 2026

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.