[PATCH v5 00/18] reftable: stop using "git-compat-util.h"

Hi,

this patch series is the final step to fully decouple the reftable library from the rest of the Git codebase. The goal of this is to make the library reusable by other projects like libgit2 by simply copying over the source files, making Git the canonical upstream for reftable functionality.

This patch series stops using all kinds of helpers exposed by our "git-compat-util.h" header and open-codes them instead. In order to keep us from using these helpers by accident the final step is to pull out POSIX-related bits and pieces into a new "compat/posix.h" header, which the reftable library then uses instead of "git-compat-util.h".

The series is built on top of master at 5f8f7081f7 (The third batch, 2025-01-23) with ps/reftable-sign-compare at 33319b0976 (reftable: address trivial -Wsign-compare warnings, 2025-01-20) merged into it. There is a trivial merge conflict with ps/zlib-ng that can be solved like this:

    diff --cc reftable/system.h
    index e4a8944a70,d02eacea8f..0000000000
    --- a/reftable/system.h
    +++ b/reftable/system.h
    @@@ -11,15 -11,9 +11,15 @@@ https://developers.google.com/open-sour
      
      /* This header glues the reftable library to the rest of Git */
      
     -#include "git-compat-util.h"
     +#include "compat/posix.h"
    - #include <zlib.h>
    + #include "compat/zlib-compat.h"
      
     +/*
     + * Return a random 32 bit integer. This function is expected to return
     + * pre-seeded data.
     + */
     +uint32_t reftable_rand(void);
     +
      /*
       * An implementation-specific temporary file. By making this specific to the
       * implementation it becomes possible to tie temporary files into any kind of

Changes in v2:
  - The splitup of Windows headers has broken compilation because some
    of the headers couldn't be found anymore. I've fixed this more
    generally by converting includes in "compat/" to always be relative
    to the project source directory, dropping the platform-specific
    `-Icompat/` include.
  - Explain why we don't port over `EWOULDBLOCK` handling.
  - Fix commit message typos.
  - Link to v1: https://lore.kernel.org/r/20250127-pks-reftable-drop-git-compat-util-v1-0-6e280a564877@pks.im

Changes in v3:
  - Fix type of `total_read` variable used to track how many bytes we
    have read in `fd_read_lines()`.
  - Drop the patch use root-relative includes again. Let's rather
    discuss this outside of the scope of this series.
  - Link to v2: https://lore.kernel.org/r/20250128-pks-reftable-drop-git-compat-util-v2-0-c85c20336317@pks.im

Changes in v4:
  - Fix a couple of now-invalid relative includes that I missed. This
    fixes the build issue in "seen" with Meson.
  - Link to v3: https://lore.kernel.org/r/20250203-pks-reftable-drop-git-compat-util-v3-0-446c9ed4ee9e@pks.im

Changes in v5:
  - Rework the implementation of `REFTABLE_UNUSED` to use
    `__attribute__((__unused__))`.
  - Link to v4: https://lore.kernel.org/r/20250206-pks-reftable-drop-git-compat-util-v4-0-603d276d5f95@pks.im

Thanks!

Patrick

---
Patrick Steinhardt (18):
      reftable/stack: stop using `read_in_full()`
      reftable/stack: stop using `write_in_full()`
      reftable/blocksource: stop using `xmmap()`
      reftable/record: stop using `COPY_ARRAY()`
      reftable/record: stop using `BUG()` in `reftable_record_init()`
      reftable/record: don't `BUG()` in `reftable_record_cmp()`
      reftable: stop using `BUG()` in trivial cases
      reftable/basics: stop using `st_mult()` in array allocators
      reftable/basics: provide wrappers for big endian conversion
      reftable/reader: stop using `ARRAY_SIZE()` macro
      reftable/system: introduce `reftable_rand()`
      reftable/stack: stop using `sleep_millisec()`
      reftable/basics: stop using `SWAP()` macro
      reftable/basics: introduce `REFTABLE_UNUSED` annotation
      compat/mingw: split out POSIX-related bits
      git-compat-util.h: split out POSIX-emulating bits
      reftable: decouple from Git codebase by pulling in "compat/posix.h"
      Makefile: skip reftable library for Coccinelle

 Makefile                                |   2 +-
 compat/{mingw.c => mingw/compat-util.c} |  28 +-
 compat/mingw/compat-util.h              | 220 +++++++++++++
 compat/{mingw.h => mingw/posix.h}       | 216 +------------
 compat/msvc.c                           |   6 -
 compat/msvc/compat-util.c               |   6 +
 compat/msvc/compat-util.h               |   7 +
 compat/{msvc.h => msvc/posix.h}         |   8 +-
 compat/posix.h                          | 541 ++++++++++++++++++++++++++++++++
 config.mak.uname                        |   6 +-
 contrib/buildsystems/CMakeLists.txt     |   2 +-
 git-compat-util.h                       | 535 +------------------------------
 meson.build                             |   8 +-
 reftable/basics.c                       |  19 --
 reftable/basics.h                       | 123 +++++++-
 reftable/block.c                        |  16 +-
 reftable/blocksource.c                  |  17 +-
 reftable/iter.c                         |  13 +-
 reftable/merged.c                       |  27 +-
 reftable/pq.c                           |  40 ++-
 reftable/pq.h                           |   2 +-
 reftable/reader.c                       |  33 +-
 reftable/record.c                       |  84 ++---
 reftable/record.h                       |   6 +-
 reftable/stack.c                        |  52 ++-
 reftable/system.c                       |   7 +
 reftable/system.h                       |   9 +-
 reftable/writer.c                       |  27 +-
 t/unit-tests/t-reftable-basics.c        |  28 +-
 t/unit-tests/t-reftable-pq.c            |  22 +-
 t/unit-tests/t-reftable-record.c        |  42 ++-
 31 files changed, 1205 insertions(+), 947 deletions(-)

Range-diff versus v4:

 1:  7db7704c24 =  1:  74f576c975 reftable/stack: stop using `read_in_full()`
 2:  3ddd26284d =  2:  acd5854f56 reftable/stack: stop using `write_in_full()`
 3:  5f4a747da9 =  3:  25aab5acc1 reftable/blocksource: stop using `xmmap()`
 4:  64b87dd35f =  4:  5ea4f2d8ea reftable/record: stop using `COPY_ARRAY()`
 5:  49ab3789ac =  5:  2ca248c1cc reftable/record: stop using `BUG()` in `reftable_record_init()`
 6:  47aacb4064 =  6:  590b9097e3 reftable/record: don't `BUG()` in `reftable_record_cmp()`
 7:  4a777111bb =  7:  461cb30e1c reftable: stop using `BUG()` in trivial cases
 8:  743e70374d =  8:  72f672b731 reftable/basics: stop using `st_mult()` in array allocators
 9:  2a7cba7ec7 =  9:  c597bd1621 reftable/basics: provide wrappers for big endian conversion
10:  fd741883d7 = 10:  643b70b67e reftable/reader: stop using `ARRAY_SIZE()` macro
11:  2a4e188e96 = 11:  dc809eefbf reftable/system: introduce `reftable_rand()`
12:  57014b3de7 = 12:  bf59b53b60 reftable/stack: stop using `sleep_millisec()`
13:  7daddddd0d = 13:  68ffa9306f reftable/basics: stop using `SWAP()` macro
14:  e7556a7d53 <  -:  ---------- reftable/basics: stop using `UNUSED` annotation
 -:  ---------- > 14:  b444c64faf reftable/basics: introduce `REFTABLE_UNUSED` annotation
15:  4628c8a98e = 15:  2043277fb1 compat/mingw: split out POSIX-related bits
16:  055b865078 = 16:  f496f30aa2 git-compat-util.h: split out POSIX-emulating bits
17:  ae323e3b8d = 17:  3e4564d927 reftable: decouple from Git codebase by pulling in "compat/posix.h"
18:  6c3360b76a = 18:  63d85e0ace Makefile: skip reftable library for Coccinelle

--- base-commit: 8047765d092881ec4aef7dfc57772161eee7f0f5 change-id: 20241119-pks-reftable-drop-git-compat-util-470f2bfde562