git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v4 0/8] reftable: a couple of improvements for libgit2

From
Patrick Steinhardt <ps@pks.im>
Date
Aug 13, 2025, 06:25 UTC
Message-ID
<20250813-pks-reftable-fixes-for-libgit2-v4-0-42b5544c8e2a@pks.im>
In-Reply-To
<20250801-pks-reftable-fixes-for-libgit2-v1-0-f446e1c33cb9@pks.im>
Hi,

this small patch series contains a couple of improvements I required for libgit2. With those changes libgit2 is now able to run its full test suite with reftable-enabled repositories. I still need to invest a bit of work to make it memory-leak free and compile on Windows, but overall I think that support for reftables is almost ready.

Changes in v2:
  - Another commit that fixes handling of outdated stacks when doing
    compaction. This issue is hit in libgit2, which has a test that
    performs writes with a couple dozen concurrent threads.
  - Add a link to past discussions around `{0}` vs `{{0}}` as provided
    by Eric.
  - Link to v1: https://lore.kernel.org/r/20250801-pks-reftable-fixes-for-libgit2-v1-0-f446e1c33cb9@pks.im
Changes in v3:
  - Adapt `reftable_stack_init_addition()` to `memset()` the addition
    instead of using `{{0}}`.
  - Add another commit on top that makes the `flock` interface more
    robust by not relying on `errno`.
  - Add another commit to fix races with concurrent writers caused by
    the backend not passing `REFTABLE_STACK_NEW_ADDITION_RELOAD`.
  - Link to v2: https://lore.kernel.org/r/20250804-pks-reftable-fixes-for-libgit2-v2-0-fef06209a984@pks.im
Changes in v4:
  - Adjust stale commit message.
  - Fix a typo while at it.
  - Link to v3: https://lore.kernel.org/r/20250812-pks-reftable-fixes-for-libgit2-v3-0-cf3b2267867e@pks.im
Thanks!
Patrick
---
Patrick Steinhardt (8):
      reftable/writer: fix type used for number of records
      reftable/writer: drop Git-specific `QSORT()` macro
      reftable/stack: reorder code to avoid forward declarations
      reftable/stack: fix compiler warning due to missing braces
      reftable/stack: allow passing flags to `reftable_stack_add()`
      reftable/stack: handle outdated stacks when compacting
      reftable: don't second-guess errors from flock interface
      refs/reftable: always reload stacks when creating lock
 refs/reftable-backend.c         |  23 ++-
 reftable/reftable-stack.h       |   9 +-
 reftable/reftable-writer.h      |   4 +-
 reftable/stack.c                | 439 +++++++++++++++++++---------------------
 reftable/system.c               |   2 +-
 reftable/system.h               |   4 +-
 reftable/writer.c               |  23 ++-
 t/unit-tests/t-reftable-stack.c |  50 ++---
 8 files changed, 275 insertions(+), 279 deletions(-)
Range-diff versus v3:
1:  afb757e6c1 = 1:  631293a69e reftable/writer: fix type used for number of records
2:  75dd5f6cb6 = 2:  a583b9f51c reftable/writer: drop Git-specific `QSORT()` macro
3:  6badd0ae26 = 3:  7153526268 reftable/stack: reorder code to avoid forward declarations
4:  9f5aa7f808 = 4:  7372cbe037 reftable/stack: fix compiler warning due to missing braces
5:  d4fea00eb2 ! 5:  57810b984e reftable/stack: allow passing flags to `reftable_stack_add()`
    @@ Commit message
         stack.
     
         Add a `flags` field to plug this gap and pass it through accordingly.
    -    For now this new flag won't be used by us, but it will be used by
    -    libgit2.
    +    This flag will be used in a subsequent patch and by libgit2.
     
         Signed-off-by: Patrick Steinhardt <ps@pks.im>
     
6:  d2cdd89280 = 6:  211d3f646e reftable/stack: handle outdated stacks when compacting
7:  635dd20dcd ! 7:  f8d541af00 reftable: don't second-guess errors from flock interface
    @@ reftable/system.h: struct reftable_flock {
       * we block indefinitely.
       *
     - * Retrun 0 on success, a reftable error code on error.
    -+ * Retrun 0 on success, a reftable error code on error. Specifically,
    ++ * Return 0 on success, a reftable error code on error. Specifically,
     + * `REFTABLE_LOCK_ERROR` should be returned in case the target path is already
     + * locked.
       */
8:  4396793747 = 8:  912d00a4a8 refs/reftable: always reload stacks when creating lock

--- base-commit: e813a0200a7121b97fec535f0d0b460b0a33356c change-id: 20250801-pks-reftable-fixes-for-libgit2-562b959a5603

Previous: Junio C HamanoNext: Patrick Steinhardt
Message 43 of 52 in “reftable: a couple of improvements for libgit2”
  1. 0/5 reftable: a couple of improvements for libgit2Patrick Steinhardt, Aug 1, 2025
  2. 1/5 reftable/writer: fix type used for number of recordsPatrick Steinhardt, Aug 1, 2025
  3. 2/5 reftable/writer: drop Git-specific `QSORT()` macroPatrick Steinhardt, Aug 1, 2025
  4. 3/5 reftable/stack: fix compiler warning due to missing bracesPatrick Steinhardt, Aug 1, 2025
  5. Eric SunshineAug 1, 2025
  6. Patrick SteinhardtAug 4, 2025
  7. Junio C HamanoAug 4, 2025
  8. Patrick SteinhardtAug 5, 2025
  9. Carlo ArenasAug 12, 2025
  10. Patrick SteinhardtAug 12, 2025
  11. Junio C HamanoAug 12, 2025
  12. 4/5 reftable/stack: reorder code to avoid forward declarationsPatrick Steinhardt, Aug 1, 2025
  13. 5/5 reftable/stack: allow passing flags to `reftable_stack_add()`Patrick Steinhardt, Aug 1, 2025
  14. 0/6 reftable: a couple of improvements for libgit2Patrick Steinhardt, Aug 4, 2025
  15. 1/6 reftable/writer: fix type used for number of recordsPatrick Steinhardt, Aug 4, 2025
  16. 2/6 reftable/writer: drop Git-specific `QSORT()` macroPatrick Steinhardt, Aug 4, 2025
  17. 3/6 reftable/stack: fix compiler warning due to missing bracesPatrick Steinhardt, Aug 4, 2025
  18. 4/6 reftable/stack: reorder code to avoid forward declarationsPatrick Steinhardt, Aug 4, 2025
  19. Justin ToblerAug 11, 2025
  20. 5/6 reftable/stack: allow passing flags to `reftable_stack_add()`Patrick Steinhardt, Aug 4, 2025
  21. Justin ToblerAug 11, 2025
  22. Patrick SteinhardtAug 12, 2025
  23. 6/6 reftable/stack: handle outdated stacks when compactingPatrick Steinhardt, Aug 4, 2025
  24. Justin ToblerAug 11, 2025
  25. Patrick SteinhardtAug 12, 2025
  26. 0/8 reftable: a couple of improvements for libgit2Patrick Steinhardt, Aug 12, 2025
  27. 1/8 reftable/writer: fix type used for number of recordsPatrick Steinhardt, Aug 12, 2025
  28. 2/8 reftable/writer: drop Git-specific `QSORT()` macroPatrick Steinhardt, Aug 12, 2025
  29. 3/8 reftable/stack: reorder code to avoid forward declarationsPatrick Steinhardt, Aug 12, 2025
  30. 4/8 reftable/stack: fix compiler warning due to missing bracesPatrick Steinhardt, Aug 12, 2025
  31. Justin ToblerAug 12, 2025
  32. 5/8 reftable/stack: allow passing flags to `reftable_stack_add()`Patrick Steinhardt, Aug 12, 2025
  33. Justin ToblerAug 12, 2025
  34. Patrick SteinhardtAug 13, 2025
  35. 6/8 reftable/stack: handle outdated stacks when compactingPatrick Steinhardt, Aug 12, 2025
  36. 7/8 reftable: don't second-guess errors from flock interfacePatrick Steinhardt, Aug 12, 2025
  37. Justin ToblerAug 12, 2025
  38. 8/8 refs/reftable: always reload stacks when creating lockPatrick Steinhardt, Aug 12, 2025
  39. Justin ToblerAug 12, 2025
  40. Carlo ArenasAug 12, 2025
  41. Patrick SteinhardtAug 13, 2025
  42. Junio C HamanoAug 13, 2025
  43. 0/8 reftable: a couple of improvements for libgit2Patrick Steinhardt, Aug 13, 2025
  44. 1/8 reftable/writer: fix type used for number of recordsPatrick Steinhardt, Aug 13, 2025
  45. 2/8 reftable/writer: drop Git-specific `QSORT()` macroPatrick Steinhardt, Aug 13, 2025
  46. 3/8 reftable/stack: reorder code to avoid forward declarationsPatrick Steinhardt, Aug 13, 2025
  47. 4/8 reftable/stack: fix compiler warning due to missing bracesPatrick Steinhardt, Aug 13, 2025
  48. 5/8 reftable/stack: allow passing flags to `reftable_stack_add()`Patrick Steinhardt, Aug 13, 2025
  49. 6/8 reftable/stack: handle outdated stacks when compactingPatrick Steinhardt, Aug 13, 2025
  50. 7/8 reftable: don't second-guess errors from flock interfacePatrick Steinhardt, Aug 13, 2025
  51. 8/8 refs/reftable: always reload stacks when creating lockPatrick Steinhardt, Aug 13, 2025
  52. Justin ToblerAug 13, 2025

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.