[PATCH v6 0/9] refs: fix migration of reflog entries

Hi,

after the announcement that "reftable" will become the default backend in Git 3.0 I've revived the efforts to implement this backend in libgit2. I'm happy to report that this implementation is almost done by now: out of 3000 tests only four are failing now.

For two of these tests I have been completely puzzled why those are failing, as everything really looked perfectly fine in libgit2. As it turned out, the bug wasn't in libgit2 though, but in Git. Namely, the way we migrate reflog entries between storage formats is broken in two ways:

  - The identity we write into the reflog entries is wrong.

  - The old commit ID of reflog entries is always set to all-zeroes.
    This is what caused the libgit2 tests to fail, as I used `git refs
    migrate` to convert test repositories to use reftables.

This patch series fixes both of these issues. Furthermore, it also adds a new `git reflog write` subcommand to write new reflog entries for a specific reference. This command was helpful to reproduce some test constellations in libgit2.

Changes in v2:
  - !!! The base of this topic has changed so that it sits on top of
    v2.50.1. This is done so that we can backport this change to older
    release tracks.
  - A couple of typo fixes and clarifications for commit messages.
  - Reorder sections in git-reflog(1) manpage according to the
    reordering we have in the synopsis.
  - Add a section for the new `write` command.
  - Improve test coverage for the `git reflog write` command.
  - Avoid `cat`ing a file into a Bash loop.
  - Remove a stale comment.
  - Make `ref_update_expects_existing_old_ref()` a bit more straight
    forward.
  - Link to v1: https://lore.kernel.org/r/20250722-pks-reflog-append-v1-0-183e5949de16@pks.im

Changes in v3:
  - `git reflog write` now requires fully-qualified refnames.
  - A new commit that plugs one part of the race around splitting of
    reflogs for HEAD in the "files" backend.
  - Link to v2: https://lore.kernel.org/r/20250725-pks-reflog-append-v2-0-e4e7cbe3f578@pks.im

Changes in v4:
  - Improve one of the tests to use an existing abbreviated object ID
    instead of a non-existing one to make sure that we indeed fail due
    to the abbreviation.
  - Don't abort the transaction when HEAD has been racily updated, but
    drop the log-only update instead.
  - Link to v3: https://lore.kernel.org/r/20250729-pks-reflog-append-v3-0-9614d310f073@pks.im

Changes in v5:
  - Revert back to the logic that aborts the transaction if we see a
    racy HEAD update. It's the pragmatic thing to do for an edge case
    that is very unlikely to ever happen.
  - Link to v4: https://lore.kernel.org/r/20250804-pks-reflog-append-v4-0-13213fef7200@pks.im

Changes in v6:
  - Convert options to use backticks in git-reflog(1) to appease the
    upcoming new manpage linter.
  - Link to v5: https://lore.kernel.org/r/20250805-pks-reflog-append-v5-0-050997db09d5@pks.im

Thanks!

Patrick

---
Patrick Steinhardt (9):
      Documentation/git-reflog: convert to use synopsis type
      builtin/reflog: improve grouping of subcommands
      refs: export `ref_transaction_update_reflog()`
      builtin/reflog: implement subcommand to write new entries
      ident: fix type of string length parameter
      refs: fix identity for migrated reflogs
      refs/files: detect race when generating reflog entry for HEAD
      refs: stop unsetting REF_HAVE_OLD for log-only updates
      refs: fix invalid old object IDs when migrating reflogs

 Documentation/git-reflog.adoc |  96 +++++++++++++++++---------------
 builtin/reflog.c              | 103 +++++++++++++++++++++++++++-------
 ident.c                       |   2 +-
 ident.h                       |   2 +-
 refs.c                        |  60 +++++++++++---------
 refs.h                        |  24 +++++++-
 refs/files-backend.c          |  65 +++++++++++++++++++---
 refs/refs-internal.h          |   3 +-
 refs/reftable-backend.c       |  26 ++++++---
 t/meson.build                 |   1 +
 t/t1421-reflog-write.sh       | 126 ++++++++++++++++++++++++++++++++++++++++++
 t/t1460-refs-migrate.sh       |  22 +++++---
 12 files changed, 413 insertions(+), 117 deletions(-)

Range-diff versus v5:

 1:  22b0ea5e69 <  -:  ---------- Documentation/git-reflog: convert to use synopsis type
 -:  ---------- >  1:  289dcbe595 Documentation/git-reflog: convert to use synopsis type
 2:  bf5d901269 !  2:  0363c102e6 builtin/reflog: improve grouping of subcommands
    @@ Documentation/git-reflog.adoc: Options for `show`
     +Options for `drop`
     +~~~~~~~~~~~~~~~~~~
     +
    -+--all::
    ++`--all`::
     +	Drop the reflogs of all references from all worktrees.
     +
    -+--single-worktree::
    ++`--single-worktree`::
     +	By default when `--all` is specified, reflogs from all working
     +	trees are dropped. This option limits the processing to reflogs
     +	from the current working tree only.
    @@ Documentation/git-reflog.adoc: which didn't protect objects referred to by reflo
     -Options for `drop`
     -~~~~~~~~~~~~~~~~~~
     -
    ----all::
    +-`--all`::
     -	Drop the reflogs of all references from all worktrees.
     -
    ----single-worktree::
    +-`--single-worktree`::
     -	By default when `--all` is specified, reflogs from all working
     -	trees are dropped. This option limits the processing to reflogs
     -	from the current working tree only.
 3:  709148c2a2 =  3:  ff885b29f4 refs: export `ref_transaction_update_reflog()`
 4:  3ad459bf6e =  4:  8ad1992cc9 builtin/reflog: implement subcommand to write new entries
 5:  207546401a =  5:  bb5dddf606 ident: fix type of string length parameter
 6:  0791e82645 =  6:  e15eba8029 refs: fix identity for migrated reflogs
 7:  e5c0280a36 =  7:  c83037d9a4 refs/files: detect race when generating reflog entry for HEAD
 8:  c31e03ebcb =  8:  8be3d5c126 refs: stop unsetting REF_HAVE_OLD for log-only updates
 9:  61acc7af7a =  9:  11a21b71f6 refs: fix invalid old object IDs when migrating reflogs

--- base-commit: d82adb61ba2fd11d8f2587fca1b6bd7925ce4044 change-id: 20250722-pks-reflog-append-634172d8ab2c