git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 0/6] builtin/receive-pack: support pluggable packfile writes

From
Justin Tobler <jltobler@gmail.com>
Date
Aug 6, 2026, 21:38 UTC
Message-ID
<20260806213859.816157-1-jltobler@gmail.com>
Greetings,

With bdee7b3013 (builtin/receive-pack: stage incoming objects via ODB transactions, 2026-07-10), git-receive-pack(1) started using the ODB transaction interfaces to stage incoming objects. While this brought the command closer to being ODB backend agnostic, the underlying git-index-pack(1) and git-unpack-objects(1) processes used to actually write the objects to the transaction are still fundamentally tied to the "files" backend.

This series aims to address this by introducing a generic `odb_transaction_write_pack()` transaction interface to handle writing the incoming packfile to the transaction. The existing logic in git-receive-pack(1) that spawns the child processes to write the packfile becomes the "files" backend implementation of this interface.

As part of this series, the first patch also introduces the `odb_transaction_release()` transaction interface. This is done to decouple freeing the transaction from committing it and is used later in the series to allow the post-commit cleanup of the ".keep" lockfile to be deffered until after references have been updated. I'm a bit uncertain as to whether the "release" part of the transaction lifecyle is really the appropriate spot for such logic though. An alternative could be to introduce a separate post-commit transaction interface that would exist to remove any lockfiles after reference updates have been performed. I am not certain such an explicit transaction interface is also the best route either. In this version, I've opted to keep it simple for now and tie the lockfile cleanup to transaction release, but I am open to change based on feedback. :)

Most of the other patches are just structural refactorings to prepare git-receive-pack(1) to eventually use `odb_transaction_write_pack()`. The final patch makes the switch in git-receive-pack(1).

Thanks for the review, -Justin

Justin Tobler (6):
  odb/transaction: add transaction release interface
  builtin/receive-pack: pass shallow file explicitly
  builtin/receive-pack: lift global state out of unpack()
  builtin/receive-pack: report unpack errors via strbuf
  builtin/receive-pack: explicitly pass packfile fd
  odb/transaction: add transaction interface to write packfiles
 builtin/add.c            |   3 +-
 builtin/receive-pack.c   | 192 ++++++++++-----------------------------
 builtin/unpack-objects.c |   1 +
 builtin/update-index.c   |   2 +
 cache-tree.c             |   4 +-
 object-file.c            | 153 ++++++++++++++++++++++++++++++-
 odb/transaction.c        |  19 +++-
 odb/transaction.h        |  77 ++++++++++++++++
 read-cache.c             |   4 +-
 9 files changed, 305 insertions(+), 150 deletions(-)
base-commit: 2c78326f810173a4f3aefd8021f1e07575412481
-- 
2.55.0.424.g13c7afec21
Next: Justin Tobler
Message 1 of 80 in “builtin/receive-pack: support pluggable packfile writes”
  1. 0/6 builtin/receive-pack: support pluggable packfile writesJustin Tobler, Aug 6, 2026
  2. 1/6 odb/transaction: add transaction release interfaceJustin Tobler, Aug 6, 2026
  3. Patrick SteinhardtAug 7, 2026
  4. Justin ToblerAug 7, 2026
  5. 2/6 builtin/receive-pack: pass shallow file explicitlyJustin Tobler, Aug 6, 2026
  6. Patrick SteinhardtAug 7, 2026
  7. 4/6 builtin/receive-pack: report unpack errors via strbufJustin Tobler, Aug 6, 2026
  8. Patrick SteinhardtAug 7, 2026
  9. Justin ToblerAug 7, 2026
  10. Justin ToblerAug 9, 2026
  11. Patrick SteinhardtAug 10, 2026
  12. 3/6 builtin/receive-pack: lift global state out of unpack()Justin Tobler, Aug 6, 2026
  13. Patrick SteinhardtAug 7, 2026
  14. Justin ToblerAug 7, 2026
  15. 5/6 builtin/receive-pack: explicitly pass packfile fdJustin Tobler, Aug 6, 2026
  16. 6/6 odb/transaction: add transaction interface to write packfilesJustin Tobler, Aug 6, 2026
  17. Patrick SteinhardtAug 7, 2026
  18. Justin ToblerAug 7, 2026
  19. 0/7 builtin/receive-pack: support pluggable packfile writesJustin Tobler, Aug 9, 2026
  20. 1/7 odb/transaction: add transaction finalize interfaceJustin Tobler, Aug 9, 2026
  21. Junio C HamanoAug 10, 2026
  22. Justin ToblerAug 10, 2026
  23. 2/7 builtin/receive-pack: pass shallow file explicitlyJustin Tobler, Aug 9, 2026
  24. 3/7 builtin/receive-pack: read unpack limit config lazilyJustin Tobler, Aug 9, 2026
  25. Patrick SteinhardtAug 10, 2026
  26. Justin ToblerAug 10, 2026
  27. Junio C HamanoAug 10, 2026
  28. Justin ToblerAug 10, 2026
  29. 4/7 builtin/receive-pack: lift global state out of unpack()Justin Tobler, Aug 9, 2026
  30. 5/7 builtin/receive-pack: report unpack errors via strbufJustin Tobler, Aug 9, 2026
  31. 6/7 builtin/receive-pack: explicitly pass packfile fdJustin Tobler, Aug 9, 2026
  32. 7/7 odb/transaction: add transaction interface to write packfilesJustin Tobler, Aug 9, 2026
  33. Junio C HamanoAug 10, 2026
  34. Justin ToblerAug 10, 2026
  35. Junio C HamanoAug 10, 2026
  36. Justin ToblerAug 10, 2026
  37. 0/9 builtin/receive-pack: support pluggable packfile writesJustin Tobler, Aug 11, 2026
  38. 1/9 builtin/receive-pack: properly clean up keep filesJustin Tobler, Aug 11, 2026
  39. Patrick SteinhardtAug 12, 2026
  40. Justin ToblerAug 13, 2026
  41. Patrick SteinhardtAug 14, 2026
  42. 2/9 odb/transaction: add transaction finalize interfaceJustin Tobler, Aug 11, 2026
  43. Patrick SteinhardtAug 12, 2026
  44. 3/9 builtin/receive-pack: pass shallow file explicitlyJustin Tobler, Aug 11, 2026
  45. 4/9 builtin/receive-pack: read unpack limit config lazilyJustin Tobler, Aug 11, 2026
  46. 5/9 builtin/receive-pack: lift global state out of unpack()Justin Tobler, Aug 11, 2026
  47. 6/9 builtin/receive-pack: report unpack errors via strbufJustin Tobler, Aug 11, 2026
  48. 8/9 odb: return temporary ODB source when setJustin Tobler, Aug 11, 2026
  49. Patrick SteinhardtAug 12, 2026
  50. 7/9 builtin/receive-pack: explicitly pass packfile fdJustin Tobler, Aug 11, 2026
  51. 9/9 odb/transaction: add transaction interface to write packfilesJustin Tobler, Aug 11, 2026
  52. Patrick SteinhardtAug 14, 2026
  53. Justin ToblerAug 14, 2026
  54. Patrick SteinhardtAug 17, 2026
  55. 0/9 builtin/receive-pack: support pluggable packfile writesJustin Tobler, Aug 19, 2026
  56. 1/9 builtin/receive-pack: properly clean up keep filesJustin Tobler, Aug 19, 2026
  57. Patrick SteinhardtAug 20, 2026
  58. Justin ToblerAug 20, 2026
  59. 2/9 odb/transaction: add transaction finalize interfaceJustin Tobler, Aug 19, 2026
  60. Patrick SteinhardtAug 20, 2026
  61. 3/9 builtin/receive-pack: pass shallow file explicitlyJustin Tobler, Aug 19, 2026
  62. 4/9 builtin/receive-pack: read unpack limit config lazilyJustin Tobler, Aug 19, 2026
  63. 5/9 builtin/receive-pack: lift global state out of unpack()Justin Tobler, Aug 19, 2026
  64. 6/9 builtin/receive-pack: report unpack errors via strbufJustin Tobler, Aug 19, 2026
  65. 7/9 builtin/receive-pack: explicitly pass packfile fdJustin Tobler, Aug 19, 2026
  66. 8/9 odb: return temporary ODB source when setJustin Tobler, Aug 19, 2026
  67. 9/9 odb/transaction: add transaction interface to write packfilesJustin Tobler, Aug 19, 2026
  68. Patrick SteinhardtAug 20, 2026
  69. 0/9 builtin/receive-pack: support pluggable packfile writesJustin Tobler, Aug 20, 2026
  70. 1/9 builtin/receive-pack: properly clean up keep filesJustin Tobler, Aug 20, 2026
  71. 2/9 odb/transaction: add transaction finalize interfaceJustin Tobler, Aug 20, 2026
  72. 3/9 builtin/receive-pack: pass shallow file explicitlyJustin Tobler, Aug 20, 2026
  73. 4/9 builtin/receive-pack: read unpack limit config lazilyJustin Tobler, Aug 20, 2026
  74. 5/9 builtin/receive-pack: lift global state out of unpack()Justin Tobler, Aug 20, 2026
  75. 6/9 builtin/receive-pack: report unpack errors via strbufJustin Tobler, Aug 20, 2026
  76. 8/9 odb: return temporary ODB source when setJustin Tobler, Aug 20, 2026
  77. 9/9 odb/transaction: add transaction interface to write packfilesJustin Tobler, Aug 20, 2026
  78. Junio C HamanoAug 21, 2026
  79. 7/9 builtin/receive-pack: explicitly pass packfile fdJustin Tobler, Aug 20, 2026
  80. Patrick SteinhardtAug 21, 2026

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.