[PATCH v6 0/9] Introduce Rust and announce that it will become mandatory

Hi,

this small patch series introduces Rust into the core of Git. This patch series is designed as a test balloon, similar to how we introduced test balloons for C99 features in the past. The goal is threefold:

  - Give us some time to experiment with Rust and introduce proper build
    infrastructure.

  - Give distributors time to ease into the new toolchain requirements.
    Introducing Rust is impossible for some platforms and hard for
    others.

  - Announce that Git 3.0 will make Rust a mandatory part of our build
    infrastructure.

The test balloon itself is quite uninteresting: I've chosen to convert the "varint.c" subsystem, mostly because it is trivial and does not have any dependencies. But it does allow us to verify that C to Rust interop works as expected, and to play around with tooling. All tests pass with the "varint.rs" implementation.

For now, the series only contains support for Meson. If we agree to go down this route I'll also introduce support for Rust into our Makefiles at a later point in time.

Furthermore missing is additional tooling:

  - At least one CI job to verify that Rust builds and works as
    expected.

  - Tooling and CI jobs to ensure that we have consistent formatting via
    `cargo format`.

And probably lots more. As said, the entire goal is for us to have an easy playground that we can experiment on and develop the infrastructure incrementally without yet having to commit to anything.

I'm mostly splitting out the topic of introducing Rust from the larger series that introduce it into xdiff so that we can focus more on the actual process of introducing Rust into Git and less on the potential features that we want to build on top of it.

Changes in v2:
  - Introduce support for building the Rust library via our Makefile.
  - Introduce a '-DWITH_RUST' define. This define is used to print
    whether or not Git is built with Rust via `git version
    --build-options`.
  - Adjust Meson to not depend on v1.9.0 and newer anymore.
  - Introduce a roadmap into our BreakingChanges document to explain how
    we'll iterate towards mandatory Rust support.
  - Rework the Fedora job to do a full compile-and-test run with Meson
    and breaking changes enabled.
  - Adapt our breaking-changes jobs to enable Rust support.
  - Link to v1: https://lore.kernel.org/r/20250904-b4-pks-rust-breaking-change-v1-0-3af1d25e0be9@pks.im

Changes in v3:
  - Reorder all uses of `WITH_RUST` after the include of "config.mak".
  - Add a test to verify overflow behaviour in Rust and explicitly use
    `add_wrapping()`.
  - Use explicit dependencies for the Rust library in our Makefile.
  - Fix Alma Linux CI job.
  - Stop tying maintenance of our LTS release to the availability of
    gcc-rs.
  - Add a fallback to Meson to use cargo directly.
  - I've fixed the Rust edition to 2018 for now. This is intentionally
    conservative so that we might be able to use Rust 1.49. For now, we
    don't have any reason to use a newer edition, either. So let's take
    the oldest version we can live with for now and then bump it as
    required.
  - Link to v2: https://lore.kernel.org/r/20250905-b4-pks-rust-breaking-change-v2-0-6939cbf4a0b8@pks.im

Changes in v4:
  - Convert "varint.c" to use explicit integer width so that we don't
    need to use C types in Rust.
  - Adapt Meson to unconditionally use Cargo.
  - Don't use the unstable `--out-dir` option in Cargo. Instead, we
    resort to a wrapper script in Meson.
  - Shorten the timeline a bit to drop the extra step that ties Rust
    support to `-Dbreaking_changes=true`. This accelerates the timeline
    until distros are made forcibly aware of the upcoming changes in
    Rust.
  - Link to v3: https://lore.kernel.org/r/20250908-b4-pks-rust-breaking-change-v3-0-1cd7189fed3b@pks.im

Changes in v5:
  - Fix indentation in the BreakingChanges document.
  - Fix a commit message typo.
  - Include "Cargo.lock" in the `make clean` target again.
  - Link to v4: https://lore.kernel.org/r/20250910-b4-pks-rust-breaking-change-v4-0-4a63fc69278d@pks.im

Changes in v6:
  - Give attribution to Ezekiel for kickstarting the Rust adoption
    again. I'm happy to change how I do the attribution.
  - Fix "varint.rs" to use `u64` instead of `usize`. Issues like these
    will eventually be catched by cbindgen.
  - Adapt the breaking changes document to mention that we already have
    Rust in our tree starting with Git 2.49.
  - Mention that we won't blindly make Rust mandatory, but consider the
    impact on downstream distributions.
  - Slightly reword how we'll handle LTS maintainership. This probably
    still is an ongoing discussion.
  - Link to v5: https://lore.kernel.org/r/20250915-b4-pks-rust-breaking-change-v5-0-dc3a32fbb216@pks.im

Thanks!

Patrick

---
Patrick Steinhardt (9):
      meson: add infrastructure to build internal Rust library
      Makefile: reorder sources after includes
      Makefile: introduce infrastructure to build internal Rust library
      help: report on whether or not Rust is enabled
      varint: use explicit width for integers
      varint: reimplement as test balloon for Rust
      BreakingChanges: announce Rust becoming mandatory
      ci: convert "pedantic" job into full build with breaking changes
      ci: enable Rust for breaking-changes jobs

 .github/workflows/main.yml         |   4 +-
 .gitignore                         |   2 +
 .gitlab-ci.yml                     |   4 +-
 Cargo.toml                         |   9 ++
 Documentation/BreakingChanges.adoc |  45 ++++++++
 Makefile                           | 214 ++++++++++++++++++++++---------------
 ci/install-dependencies.sh         |   8 +-
 ci/run-build-and-tests.sh          |  31 ++----
 dir.c                              |  18 ++--
 help.c                             |   6 ++
 meson.build                        |  15 ++-
 meson_options.txt                  |   2 +
 read-cache.c                       |   6 +-
 shared.mak                         |   1 +
 src/cargo-meson.sh                 |  32 ++++++
 src/lib.rs                         |   1 +
 src/meson.build                    |  41 +++++++
 src/varint.rs                      |  92 ++++++++++++++++
 varint.c                           |   6 +-
 varint.h                           |   4 +-
 20 files changed, 410 insertions(+), 131 deletions(-)

Range-diff versus v5:

 1:  22925bf016 !  1:  06872fe524 meson: add infrastructure to build internal Rust library
    @@ Commit message
         want to introduce features that were added in more recent editions of
         Rust though we should reevaluate that choice.
     
    +    Inspired-by: Ezekiel Newren <ezekielnewren@gmail.com>
         Signed-off-by: Patrick Steinhardt <ps@pks.im>
     
      ## Cargo.toml (new) ##
 2:  bcd30c4e0f =  2:  0f243f137e Makefile: reorder sources after includes
 3:  45663309c3 !  3:  cb078bdffc Makefile: introduce infrastructure to build internal Rust library
    @@ Commit message
         commit. Developers can enable the infrastructure by passing the new
         `WITH_RUST` build toggle.
     
    +    Inspired-by: Ezekiel Newren <ezekielnewren@gmail.com>
         Signed-off-by: Patrick Steinhardt <ps@pks.im>
     
      ## .gitignore ##
 4:  1eec68d6a3 =  4:  b043011938 help: report on whether or not Rust is enabled
 5:  95f705fc07 =  5:  b3b415b277 varint: use explicit width for integers
 6:  2d36e3bb54 !  6:  435e5c3ae5 varint: reimplement as test balloon for Rust
    @@ src/meson.build
      ## src/varint.rs (new) ##
     @@
     +#[no_mangle]
    -+pub unsafe extern "C" fn decode_varint(bufp: *mut *const u8) -> usize {
    ++pub unsafe extern "C" fn decode_varint(bufp: *mut *const u8) -> u64 {
     +    let mut buf = *bufp;
     +    let mut c = *buf;
    -+    let mut val = usize::from(c & 127);
    ++    let mut val = u64::from(c & 127);
     +
     +    buf = buf.add(1);
     +
    @@ src/varint.rs (new)
     +        c = *buf;
     +        buf = buf.add(1);
     +
    -+        val = (val << 7) + usize::from(c & 127);
    ++        val = (val << 7) + u64::from(c & 127);
     +    }
     +
     +    *bufp = buf;
    @@ src/varint.rs (new)
     +}
     +
     +#[no_mangle]
    -+pub unsafe extern "C" fn encode_varint(value: usize, buf: *mut u8) -> u8 {
    ++pub unsafe extern "C" fn encode_varint(value: u64, buf: *mut u8) -> u8 {
     +    let mut varint: [u8; 16] = [0; 16];
     +    let mut pos = varint.len() - 1;
     +
 7:  92590e9f87 !  7:  3f0cb3550a BreakingChanges: announce Rust becoming mandatory
    @@ Documentation/BreakingChanges.adoc: A prerequisite for this change is that the e
      JGit, libgit2 and Gitoxide need to support it.
      
     +* Git will require Rust as a mandatory part of the build process. While Git
    -+  already started to adopt Rust in Git 2.52, all parts written in Rust are
    ++  already started to adopt Rust in Git 2.49, all parts written in Rust are
     +  optional for the time being. This includes:
     ++
    ++  ** The Rust wrapper around libgit.a that is part of "contrib/" and which has
    ++     been introduced in Git 2.49.
     +  ** Subsystems that have an alternative implementation in Rust to test
     +     interoperability between our C and Rust codebase.
     +  ** Newly written features that are not mission critical for a fully functional
    @@ Documentation/BreakingChanges.adoc: A prerequisite for this change is that the e
     +project will hand over maintainership of the long-term release to distributors
     +in case they need to extend the life of that long-term release even further. In
     +that case, the backporting process will be handled by these distributors, but
    -+the backported patches will be reviewed on the mailing list and pulled in by the
    -+Git maintainer.
    ++the long-term release tags will be created in the canonical Git repository.
    +++
    ++We will evaluate the impact on downstream distributions before making Rust
    ++mandatory in Git 3.0. If we see that the impact on downstream distributions
    ++would be significant, we may decide to defer this breaking change to a
    ++subsequent minor release. This evaluation will also take into account our own
    ++learnings with how painful it is to keep Rust an optional component.
     +
      === Removals
      
 8:  3c7b2edeb4 =  8:  e5dc29bc1c ci: convert "pedantic" job into full build with breaking changes
 9:  c3803ab47b =  9:  9fcaa8c540 ci: enable Rust for breaking-changes jobs

--- base-commit: 2462961280690837670d997bde64bd4ebf8ae66d change-id: 20250904-b4-pks-rust-breaking-change-7167d9d3e37d