[PATCH v3 0/9] refs: fix migration of reflog entries

Hi,

after the announcement that "reftable" will become the default backend in Git 3.0 I've revived the efforts to implement this backend in libgit2. I'm happy to report that this implementation is almost done by now: out of 3000 tests only four are failing now.

For two of these tests I have been completely puzzled why those are failing, as everything really looked perfectly fine in libgit2. As it turned out, the bug wasn't in libgit2 though, but in Git. Namely, the way we migrate reflog entries between storage formats is broken in two ways:

  - The identity we write into the reflog entries is wrong.

  - The old commit ID of reflog entries is always set to all-zeroes.
    This is what caused the libgit2 tests to fail, as I used `git refs
    migrate` to convert test repositories to use reftables.

This patch series fixes both of these issues. Furthermore, it also adds a new `git reflog write` subcommand to write new reflog entries for a specific reference. This command was helpful to reproduce some test constellations in libgit2.

Changes in v2:
  - !!! The base of this topic has changed so that it sits on top of
    v2.50.1. This is done so that we can backport this change to older
    release tracks.
  - A couple of typo fixes and clarifications for commit messages.
  - Reorder sections in git-reflog(1) manpage according to the
    reordering we have in the synopsis.
  - Add a section for the new `write` command.
  - Improve test coverage for the `git reflog write` command.
  - Avoid `cat`ing a file into a Bash loop.
  - Remove a stale comment.
  - Make `ref_update_expects_existing_old_ref()` a bit more straight
    forward.
  - Link to v1: https://lore.kernel.org/r/20250722-pks-reflog-append-v1-0-183e5949de16@pks.im

Changes in v3:
  - `git reflog write` now requires fully-qualified refnames.
  - A new commit that plugs one part of the race around splitting of
    reflogs for HEAD in the "files" backend.
  - Link to v2: https://lore.kernel.org/r/20250725-pks-reflog-append-v2-0-e4e7cbe3f578@pks.im

Thanks!

Patrick

---
Patrick Steinhardt (9):
      Documentation/git-reflog: convert to use synopsis type
      builtin/reflog: improve grouping of subcommands
      refs: export `ref_transaction_update_reflog()`
      builtin/reflog: implement subcommand to write new entries
      ident: fix type of string length parameter
      refs: fix identity for migrated reflogs
      refs/files: detect race when generating reflog entry for HEAD
      refs: stop unsetting REF_HAVE_OLD for log-only updates
      refs: fix invalid old object IDs when migrating reflogs

 Documentation/git-reflog.adoc |  76 ++++++++++++++------------
 builtin/reflog.c              | 103 ++++++++++++++++++++++++++++-------
 ident.c                       |   2 +-
 ident.h                       |   2 +-
 refs.c                        |  60 +++++++++++---------
 refs.h                        |  24 +++++++-
 refs/files-backend.c          |  65 +++++++++++++++++++---
 refs/refs-internal.h          |   3 +-
 refs/reftable-backend.c       |  26 ++++++---
 t/meson.build                 |   1 +
 t/t1421-reflog-write.sh       | 124 ++++++++++++++++++++++++++++++++++++++++++
 t/t1460-refs-migrate.sh       |  22 +++++---
 12 files changed, 401 insertions(+), 107 deletions(-)

Range-diff versus v2:

 1:  65f4647df02 =  1:  027ac6d12f3 Documentation/git-reflog: convert to use synopsis type
 2:  e53a402a88d =  2:  1570cac0cb9 builtin/reflog: improve grouping of subcommands
 3:  4d060861f50 =  3:  af43c907fa0 refs: export `ref_transaction_update_reflog()`
 4:  ddd471f9891 !  4:  4322f98fcdd builtin/reflog: implement subcommand to write new entries
    @@ Documentation/git-reflog.adoc: The "exists" subcommand checks whether a ref has
      
     +The "write" subcommand writes a single entry to the reflog of a given
     +reference. This new entry is appended to the reflog and will thus become
    -+the most recent entry. Both the old and new object IDs must not be
    -+abbreviated and must point to existing objects. The reflog message gets
    -+normalized.
    ++the most recent entry. The reference name must be fully qualified. Both the old
    ++and new object IDs must not be abbreviated and must point to existing objects.
    ++The reflog message gets normalized.
     +
      The "delete" subcommand deletes single entries from the reflog, but
      not the reflog itself. Its argument must be an _exact_ entry (e.g. "`git
    @@ builtin/reflog.c: static int cmd_reflog_drop(int argc, const char **argv, const
     +		usage_with_options(reflog_write_usage, options);
     +
     +	ref = argv[0];
    -+	if (check_refname_format(ref, REFNAME_ALLOW_ONELEVEL))
    ++	if (!is_root_ref(ref) && check_refname_format(ref, 0))
     +		die(_("invalid reference name: %s"), ref);
     +
     +	ret = get_oid_hex_algop(argv[1], &old_oid, repo->hash_algo);
    @@ t/t1421-reflog-write.sh (new)
     +	)
     +'
     +
    ++test_expect_success 'unqualified refname is rejected' '
    ++	test_when_finished "rm -rf repo" &&
    ++	git init repo &&
    ++	(
    ++		cd repo &&
    ++		test_must_fail git reflog write unqualified $ZERO_OID $ZERO_OID first 2>err &&
    ++		test_grep "invalid reference name: " err
    ++	)
    ++'
    ++
     +test_expect_success 'nonexistent object IDs' '
     +	test_when_finished "rm -rf repo" &&
     +	git init repo &&
    @@ t/t1421-reflog-write.sh (new)
     +	)
     +'
     +
    ++test_expect_success 'can write to root ref' '
    ++	test_when_finished "rm -rf repo" &&
    ++	git init repo &&
    ++	(
    ++		cd repo &&
    ++		test_commit initial &&
    ++		COMMIT_OID=$(git rev-parse HEAD) &&
    ++
    ++		git reflog write ROOT_REF_HEAD $ZERO_OID $COMMIT_OID first &&
    ++		test_reflog_matches . ROOT_REF_HEAD <<-EOF
    ++		$ZERO_OID $COMMIT_OID $SIGNATURE	first
    ++		EOF
    ++	)
    ++'
    ++
     +test_done
 5:  67028ef4439 =  5:  66de5312e83 ident: fix type of string length parameter
 6:  a6bf88a4e89 =  6:  2b9fe08cf76 refs: fix identity for migrated reflogs
 -:  ----------- >  7:  7f87327e17c refs/files: detect race when generating reflog entry for HEAD
 7:  71b0f753dd3 =  8:  792a1d7ce61 refs: stop unsetting REF_HAVE_OLD for log-only updates
 8:  2d88a1e57b8 =  9:  121630b9d64 refs: fix invalid old object IDs when migrating reflogs

--- base-commit: d82adb61ba2fd11d8f2587fca1b6bd7925ce4044 change-id: 20250722-pks-reflog-append-634172d8ab2c