[PATCH v3 0/8] refs/reftable: add consistency checks

The reference subsystems allows for adding backend specific consistency checks. These checks are run as part of 'git refs verify'.

While the files backend has some consistency checks added, the reftable backend currently has none. This series first tightens the reftable backend to make it a little more strict and then also adds the required infrastructure and some simple consistency checks.

Since the reftable backend is treated as a library within the Git codebase, we don't want to spillover our internal fsck implementation into the library. At the same time, the fsck checks need to access internal structures of the reftable library which aren't exposed outside the library.

So we solve this by adding a 'reftable/fsck.[ch]' which implements and exposes a checker for the reftable library and returns specific errors as defined by the library. We then add glue code within 'refs/reftable-backend.c' to map these errors to errors which Git's fsck implementation would understand. This allows us to separate concerns.

We add the following consistency checks:

  1. Check for validating the reftable table name. This is treated as a
  warning since the reftable specification only suggests a table name
  but doesn't enforce it. Also there is a difference in the table name
  used in Git vs that in jGit.

  2. Check for checking additional files present in the reftable
  directory.

We tighten the reftable backend by raising a REFTABLE_FORMAT_ERROR error when:

1. The 'tables.list' file doesn't have a trailing newline.
  2. Tables added to a reftable stack are not sequential.

---
Changes in v3:
- I took a long hiatus from this topic, mostly due to other priorities.
  This has been rebased on top of '92c87bdc40 (The eighth batch,
  2025-09-12)' since there were conflicts.
- Junio suggested that two of the consistency checks (trailing newlines,
  sequential update indices for tables in stack) should actually be
  checked during runtime. I have made that change in this version.
- I've cleaned up the code and modularized the 'reftable/fsck.c' code.
- Invalid table name emits a warning, since the reftable spec doesn't
  enforce it but only makes a suggestion.
- Broken down the commits to make it easier to review.
- Link to v2: https://lore.kernel.org/r/20250902-228-reftable-introduce-consistency-checks-v2-0-4f96b3834779@gmail.com

Changes in v2:
- Ensured that 'struct reftable_fsck_info' is passed around as a
  pointer, this provides a smaller footprint (pointer size vs struct
  size).
- Run FSCK checks for other worktrees too, even if one of them fails.
- Separate messaging for table name vs table check and add additional
  test.
- Use the relative path in messages used.
- Small style and typo fixes.
- Link to v1: https://lore.kernel.org/r/20250819-228-reftable-introduce-consistency-checks-v1-0-8b8f6879fa9e@gmail.com

---
 Documentation/fsck-msgids.adoc   |   9 ++--
 Makefile                         |   3 +-
 fsck.h                           |  40 +++++++-------
 meson.build                      |   1 +
 refs.c                           |   4 ++
 refs/debug.c                     |   1 -
 refs/files-backend.c             |   3 --
 refs/reftable-backend.c          |  59 ++++++++++++++++++---
 reftable/basics.c                |  28 ++++++----
 reftable/basics.h                |   7 +--
 reftable/fsck.c                  | 112 +++++++++++++++++++++++++++++++++++++++
 reftable/reftable-fsck.h         |  42 +++++++++++++++
 reftable/stack.c                 |  15 ++++--
 t/meson.build                    |   1 +
 t/t0614-reftable-fsck.sh         |  55 +++++++++++++++++++
 t/unit-tests/u-reftable-basics.c |  23 ++++++--
 t/unit-tests/u-reftable-stack.c  |  28 ++++++++++
 17 files changed, 378 insertions(+), 53 deletions(-)

Karthik Nayak (8):
      refs: remove unused headers
      refs: move consistency check  msg to generic layer
      reftable: check for trailing newline in 'tables.list'
      reftable: ensure tables in a stack use sequential update indices
      Documentation/fsck-msgids: remove duplicate msg id
      fsck: order 'fsck_msg_type' alphabetically
      reftable: add code to facilitate consistency checks
      refs/reftable: add fsck check for checking the table name

Range-diff versus v2:

1: eea34c56f0 < -: ---------- fsck: order 'fsck_msg_type' alphabetically 2: dafcf618e9 < -: ---------- refs/reftable: add fsck check for checking the table name 3: 20294ade9b < -: ---------- refs/reftable: add fsck check for number of tables 4: 03c7979528 < -: ---------- refs/reftable: add fsck check for trailing newline 5: eb74502cd3 < -: ---------- refs/reftable: add fsck check for incorrect update index -: ---------- > 1: c9f39a04ca refs: remove unused headers -: ---------- > 2: e1baf61a8a refs: move consistency check msg to generic layer -: ---------- > 3: 88a2ae1171 reftable: check for trailing newline in 'tables.list' -: ---------- > 4: 2dd1750a9d reftable: ensure tables in a stack use sequential update indices -: ---------- > 5: a7f6c52385 Documentation/fsck-msgids: remove duplicate msg id -: ---------- > 6: 873c21c73f fsck: order 'fsck_msg_type' alphabetically -: ---------- > 7: cbaac94328 reftable: add code to facilitate consistency checks -: ---------- > 8: e7fcc15608 refs/reftable: add fsck check for checking the table name

base-commit: a483264b01b977f3e65a4419103c21e6af7412a2 change-id: 20250714-228-reftable-introduce-consistency-checks-379ded93c544

Thanks
- Karthik