git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v2 7/7] odb/transaction: add transaction interface to write packfiles

From
Justin Tobler <jltobler@gmail.com>
Date
Aug 10, 2026, 19:29 UTC
Message-ID
<anojlSEahYWzFOBV@denethor>
In-Reply-To
<xmqqa4qu91vi.fsf@gitster.g>
On 26/08/09 06:54PM, Junio C Hamano wrote:
Show 29 quoted lines
> Justin Tobler <jltobler@gmail.com> writes:
> 
> > In git-receive-pack(1), the incoming packfile is written to the ODB via
> > `unpack()`, which spawns git-index-pack(1) or git-unpack-objects(1)
> > directly. With pluggable object databases, an alternative backend may
> > need to handle writing packfile data differently though.
> >
> > Introduce `odb_transaction_write_pack()` as a generic interface to
> > handle writing a packfile to a transaction and use the logic from
> > `unpack()` as the "files" backend implementation. Note that a packfile
> > written via git-index-pack(1) is kept in place by a ".keep" lockfile
> > that must be retained until references are updated. To faciliate this in
> > an ODB backend agnostic manner, the "files" transaction backend takes
> > ownership of these lockfiles and removes them post-commit through its
> > release callback.
> > ...
> > +static int odb_transaction_files_write_pack(struct odb_transaction *base,
> > +					    int pack_fd, struct strbuf *err_msg,
> > +					    const struct odb_transaction_write_pack_opts *opts)
> > +{
> > + ...
> > +		odb_source_prepare(repo->objects->sources,
> > +				   ODB_PREPARE_FLUSH_CACHES);
> > +	}
> 
> This assumes that we are working with the first entry in the
> 'repo->objects->sources' linked list.  Should we not use the source
> actually associated with the current transaction (I am guessing that
> it is 'base->source' but I may be wrong)?

Currently tracking of the transaction's ODB source relies on reading the first source in this list which is rather awkward in my opinion and rather fragile. The ODB source specified by `base->source` here is actually the main ODB source that the transaction is created against so it can't be used here.

In a future series, my plan is to stop reordering the source list when starting a transaction and instead track the source separately which should simplify some of this. It may be a good idea for the ODB transaction itself to track its own source regardless though and may be something nice to add as a part of this series. I'll explore this in my next version of the series.

Thanks, -Justin

Previous: Junio C HamanoNext: Junio C Hamano
Message 34 of 80 in “builtin/receive-pack: support pluggable packfile writes”
  1. 0/6 builtin/receive-pack: support pluggable packfile writesJustin Tobler, Aug 6, 2026
  2. 1/6 odb/transaction: add transaction release interfaceJustin Tobler, Aug 6, 2026
  3. Patrick SteinhardtAug 7, 2026
  4. Justin ToblerAug 7, 2026
  5. 2/6 builtin/receive-pack: pass shallow file explicitlyJustin Tobler, Aug 6, 2026
  6. Patrick SteinhardtAug 7, 2026
  7. 4/6 builtin/receive-pack: report unpack errors via strbufJustin Tobler, Aug 6, 2026
  8. Patrick SteinhardtAug 7, 2026
  9. Justin ToblerAug 7, 2026
  10. Justin ToblerAug 9, 2026
  11. Patrick SteinhardtAug 10, 2026
  12. 3/6 builtin/receive-pack: lift global state out of unpack()Justin Tobler, Aug 6, 2026
  13. Patrick SteinhardtAug 7, 2026
  14. Justin ToblerAug 7, 2026
  15. 5/6 builtin/receive-pack: explicitly pass packfile fdJustin Tobler, Aug 6, 2026
  16. 6/6 odb/transaction: add transaction interface to write packfilesJustin Tobler, Aug 6, 2026
  17. Patrick SteinhardtAug 7, 2026
  18. Justin ToblerAug 7, 2026
  19. 0/7 builtin/receive-pack: support pluggable packfile writesJustin Tobler, Aug 9, 2026
  20. 1/7 odb/transaction: add transaction finalize interfaceJustin Tobler, Aug 9, 2026
  21. Junio C HamanoAug 10, 2026
  22. Justin ToblerAug 10, 2026
  23. 2/7 builtin/receive-pack: pass shallow file explicitlyJustin Tobler, Aug 9, 2026
  24. 3/7 builtin/receive-pack: read unpack limit config lazilyJustin Tobler, Aug 9, 2026
  25. Patrick SteinhardtAug 10, 2026
  26. Justin ToblerAug 10, 2026
  27. Junio C HamanoAug 10, 2026
  28. Justin ToblerAug 10, 2026
  29. 4/7 builtin/receive-pack: lift global state out of unpack()Justin Tobler, Aug 9, 2026
  30. 5/7 builtin/receive-pack: report unpack errors via strbufJustin Tobler, Aug 9, 2026
  31. 6/7 builtin/receive-pack: explicitly pass packfile fdJustin Tobler, Aug 9, 2026
  32. 7/7 odb/transaction: add transaction interface to write packfilesJustin Tobler, Aug 9, 2026
  33. Junio C HamanoAug 10, 2026
  34. Justin ToblerAug 10, 2026
  35. Junio C HamanoAug 10, 2026
  36. Justin ToblerAug 10, 2026
  37. 0/9 builtin/receive-pack: support pluggable packfile writesJustin Tobler, Aug 11, 2026
  38. 1/9 builtin/receive-pack: properly clean up keep filesJustin Tobler, Aug 11, 2026
  39. Patrick SteinhardtAug 12, 2026
  40. Justin ToblerAug 13, 2026
  41. Patrick SteinhardtAug 14, 2026
  42. 2/9 odb/transaction: add transaction finalize interfaceJustin Tobler, Aug 11, 2026
  43. Patrick SteinhardtAug 12, 2026
  44. 3/9 builtin/receive-pack: pass shallow file explicitlyJustin Tobler, Aug 11, 2026
  45. 4/9 builtin/receive-pack: read unpack limit config lazilyJustin Tobler, Aug 11, 2026
  46. 5/9 builtin/receive-pack: lift global state out of unpack()Justin Tobler, Aug 11, 2026
  47. 6/9 builtin/receive-pack: report unpack errors via strbufJustin Tobler, Aug 11, 2026
  48. 8/9 odb: return temporary ODB source when setJustin Tobler, Aug 11, 2026
  49. Patrick SteinhardtAug 12, 2026
  50. 7/9 builtin/receive-pack: explicitly pass packfile fdJustin Tobler, Aug 11, 2026
  51. 9/9 odb/transaction: add transaction interface to write packfilesJustin Tobler, Aug 11, 2026
  52. Patrick SteinhardtAug 14, 2026
  53. Justin ToblerAug 14, 2026
  54. Patrick SteinhardtAug 17, 2026
  55. 0/9 builtin/receive-pack: support pluggable packfile writesJustin Tobler, Aug 19, 2026
  56. 1/9 builtin/receive-pack: properly clean up keep filesJustin Tobler, Aug 19, 2026
  57. Patrick SteinhardtAug 20, 2026
  58. Justin ToblerAug 20, 2026
  59. 2/9 odb/transaction: add transaction finalize interfaceJustin Tobler, Aug 19, 2026
  60. Patrick SteinhardtAug 20, 2026
  61. 3/9 builtin/receive-pack: pass shallow file explicitlyJustin Tobler, Aug 19, 2026
  62. 4/9 builtin/receive-pack: read unpack limit config lazilyJustin Tobler, Aug 19, 2026
  63. 5/9 builtin/receive-pack: lift global state out of unpack()Justin Tobler, Aug 19, 2026
  64. 6/9 builtin/receive-pack: report unpack errors via strbufJustin Tobler, Aug 19, 2026
  65. 7/9 builtin/receive-pack: explicitly pass packfile fdJustin Tobler, Aug 19, 2026
  66. 8/9 odb: return temporary ODB source when setJustin Tobler, Aug 19, 2026
  67. 9/9 odb/transaction: add transaction interface to write packfilesJustin Tobler, Aug 19, 2026
  68. Patrick SteinhardtAug 20, 2026
  69. 0/9 builtin/receive-pack: support pluggable packfile writesJustin Tobler, Aug 20, 2026
  70. 1/9 builtin/receive-pack: properly clean up keep filesJustin Tobler, Aug 20, 2026
  71. 2/9 odb/transaction: add transaction finalize interfaceJustin Tobler, Aug 20, 2026
  72. 3/9 builtin/receive-pack: pass shallow file explicitlyJustin Tobler, Aug 20, 2026
  73. 4/9 builtin/receive-pack: read unpack limit config lazilyJustin Tobler, Aug 20, 2026
  74. 5/9 builtin/receive-pack: lift global state out of unpack()Justin Tobler, Aug 20, 2026
  75. 6/9 builtin/receive-pack: report unpack errors via strbufJustin Tobler, Aug 20, 2026
  76. 8/9 odb: return temporary ODB source when setJustin Tobler, Aug 20, 2026
  77. 9/9 odb/transaction: add transaction interface to write packfilesJustin Tobler, Aug 20, 2026
  78. Junio C HamanoAug 21, 2026
  79. 7/9 builtin/receive-pack: explicitly pass packfile fdJustin Tobler, Aug 20, 2026
  80. Patrick SteinhardtAug 21, 2026

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.