git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 0/3] Simple reftable backend

From
Han-Wen Nienhuys via GitGitGadget <gitgitgadget@gmail.com>
Date
Sep 18, 2023, 17:59 UTC
Message-ID
<pull.1574.git.git.1695059978.gitgitgadget@gmail.com>

This series comes from a conversation with Patrick Steinhardt at Gitlab, who have an interest in a more scalable ref storage system.

I unfortunately don't have business reasons to spend much time on this project anymore, and the original, sweeping migration has too many open questions.

I thought of an alternate approach, and I wanted to show it as input to discussions at the contributor summit.

I think the first part ("refs: push lock management into packed backend") is a good improvement overall, and could be landed separately without much tweaking.

The second part ("refs: alternate reftable ref backend implementation") is something open for discussion: the alternative "packed storage" is based on reftable, but it could conceivably be a different type of database/file format too. I think it's attractive to use reftable here, because over time more data (symrefs, reflog) could be moved into reftable.

Han-Wen Nienhuys (3):
  refs: push lock management into packed backend
  refs: move is_packed_transaction_needed out of packed-backend.c
  refs: alternate reftable ref backend implementation
 Makefile                        |    1 +
 config.mak.uname                |    2 +-
 contrib/workdir/git-new-workdir |    2 +-
 refs.c                          |    8 +
 refs/debug.c                    |   15 +
 refs/files-backend.c            |  182 +++-
 refs/packed-backend.c           |  150 +--
 refs/packed-backend.h           |   19 -
 refs/refs-internal.h            |    6 +
 refs/reftable-backend.c         | 1745 +++++++++++++++++++++++++++++++
 refs/reftable-backend.h         |    8 +
 11 files changed, 1941 insertions(+), 197 deletions(-)
 create mode 100644 refs/reftable-backend.c
 create mode 100644 refs/reftable-backend.h
base-commit: bda494f4043963b9ec9a1ecd4b19b7d1cd9a0518
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-1574%2Fhanwen%2Fsimple-reftable-backend-v1
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-1574/hanwen/simple-reftable-backend-v1
Pull-Request: https://github.com/git/git/pull/1574
-- 
gitgitgadget
Next: Han-Wen Nienhuys via GitGitGadget
Message 1 of 15 in “Simple reftable backend”
  1. 0/3 Simple reftable backendHan-Wen Nienhuys via GitGitGadget, Sep 18, 2023
  2. 1/3 refs: push lock management into packed backendHan-Wen Nienhuys via GitGitGadget, Sep 18, 2023
  3. Junio C HamanoSep 18, 2023
  4. Han-Wen NienhuysSep 19, 2023
  5. 2/3 refs: move is_packed_transaction_needed out of packed-backend.cHan-Wen Nienhuys via GitGitGadget, Sep 18, 2023
  6. 3/3 refs: alternate reftable ref backend implementationHan-Wen Nienhuys via GitGitGadget, Sep 18, 2023
  7. Junio C HamanoSep 18, 2023
  8. 0/6 RFC: simple reftable backendHan-Wen Nienhuys via GitGitGadget, Sep 20, 2023
  9. 1/6 refs: construct transaction using a _begin callbackHan-Wen Nienhuys via GitGitGadget, Sep 20, 2023
  10. 2/6 refs: wrap transaction in a debug-specific transactionHan-Wen Nienhuys via GitGitGadget, Sep 20, 2023
  11. 3/6 refs: push lock management into packed backendHan-Wen Nienhuys via GitGitGadget, Sep 20, 2023
  12. 4/6 refs: move is_packed_transaction_needed out of packed-backend.cHan-Wen Nienhuys via GitGitGadget, Sep 20, 2023
  13. 6/6 refs: always try to do packed transactions for reftableHan-Wen Nienhuys via GitGitGadget, Sep 20, 2023
  14. 5/6 refs: alternate reftable ref backend implementationHan-Wen Nienhuys via GitGitGadget, Sep 20, 2023
  15. Patrick SteinhardtSep 21, 2023

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.