[RFC PATCH 0/6] [RFC] Introduce cgit-rs, a Rust wrapper around libgit.a

When we, the Git team at Google, first embarked on the libification journey, we didn’t have a specific consumer to build a library for but instead were interested in the various potential benefits of libification for many use cases such as VFSes and submodules. Without a specific consumer, it has been difficult to evaluate the scope of what is necessary or not even for the first library, git-std-lib. Attempting to solve problems such as error handling, symbol collisions, and internal/external interfaces, in addition to separating out a library turns out to be both too complex of a task to both develop and review all at once. While we strive to eventually build an ideal library, we have also realized in order to make meaningful and consistent progress, we have to solve these problems iteratively in smaller pieces. That is why over the last month, we have been working with the jj project [1] to understand their current usage of libgit2-rs [2] and gitoxide [3] and future library functionality they would be interested in. In doing so, we have built cgit-rs, a Rust wrapper around libgit.a that allows Rust code to call various basic Git functions.

[1] https://github.com/martinvonz/jj [2] https://github.com/rust-lang/git2-rs [3] https://github.com/Byron/gitoxide

This series provides a small Rust wrapper library around parts of libgit.a, and a proof-of-concept Rust executable that uses the library to interface with Git. Additionally, we have tested building JJ with our library and used it to replace some of the libgit2-rs uses.

This exercise has clarified a lot of things for us, and we believe that
developing this wrapper further provides benefits both for downstream
consumers and the Git project itself:
* cgit-rs provides wrappers for Rust consumers of libraries (eg. jj)
* cgit-rs suggests focus areas for libification
  * shows us what potential challenges we face with library consumers
* Git libification improves git interfaces
* Libification improves cgit-rs FFI.

We are putting error handling on hold for now since it is too complex and we intend other CLIs to be our first customers, in which case printing out errors is not the worst.

While the wrapper itself lives in contrib/, there are a couple of patches that touch git.git code. These patches are necessary for the wrapper, but not for git.git itself, which may seem unnecessary to merge. However, I would argue that other languages (not just limited to Rust) have issues calling functions that require a pointer to non-generic objects and essentially require a redefinition in their own language.

We're sending this series as RFC because there is remaining work we'd like to do, but we'd like to get early feedback on this approach, and particularly to ask for advice on a few topics:

* alternative methods of exposing only a subset of symbols in our
  library

* bikeshedding on the name (yes, really). There is an active, unrelated
  CGit project [4] that we only recently became aware of. We originally
  took the name "cgit" because at $DAYJOB we sometimes refer to git.git
  as "cgit" to distinguish it from jgit [5].

* gauging the level of interest in calling Git code from Rust

[4] https://git.zx2c4.com/cgit [5] https://www.eclipse.org/jgit

Remaining work includes:

* finding a better solution to the common-main split. We should probably
  have a separate initialization function including all of main() up to
  the call to cmd_main(), which can then be exposed in cgit-rs.

* adding unit and integration tests

* Makefile cleanup, particularly adding config.mak options that
  developers can set to run Rust builds and tests by default

* automating the process of exporting additional functions via cgit-rs
  (possibly with a wrapper script around bindgen [6])

[6] https://github.com/rust-lang/rust-bindgen

Finally, a quick discussion about symbol collisions: if functions are not prepended with “libgit_” or something similar, it leaves us open to collision issues in the future – so this probably would’ve happened with libification in general to begin with. Therefore it seems necessary to have to wrap all the symbols we are looking to expose. While this seem non-ideal, we couldn’t come up with a better method. Our next best alternative is to simply expose all symbols by default, but this leads to symbol collisions when library users link both cgit-rs and libgit2-rs.

Calvin Wan (2):
  contrib/cgit-rs: add repo initialization and config access
  contrib/cgit-rs: add a subset of configset wrappers

Josh Steadmon (4):
  common-main: split common_exit() into a new file
  repository: add initialize_repo wrapper without pointer
  contrib/cgit-rs: introduce Rust wrapper for libgit.a
  config: add git_configset_alloc

 .gitignore                             |  1 +
 Makefile                               | 14 ++++
 common-exit.c                          | 26 +++++++
 common-main.c                          | 24 -------
 config.c                               |  5 ++
 config.h                               |  5 ++
 contrib/cgit-rs/Cargo.lock             | 99 ++++++++++++++++++++++++++
 contrib/cgit-rs/Cargo.toml             | 17 +++++
 contrib/cgit-rs/README.md              | 15 ++++
 contrib/cgit-rs/build.rs               | 33 +++++++++
 contrib/cgit-rs/public_symbol_export.c | 72 +++++++++++++++++++
 contrib/cgit-rs/public_symbol_export.h | 26 +++++++
 contrib/cgit-rs/src/lib.rs             | 81 +++++++++++++++++++++
 contrib/cgit-rs/src/main.rs            | 44 ++++++++++++
 repository.c                           |  9 +++
 repository.h                           |  1 +
 16 files changed, 448 insertions(+), 24 deletions(-)
 create mode 100644 common-exit.c
 create mode 100644 contrib/cgit-rs/Cargo.lock
 create mode 100644 contrib/cgit-rs/Cargo.toml
 create mode 100644 contrib/cgit-rs/README.md
 create mode 100644 contrib/cgit-rs/build.rs
 create mode 100644 contrib/cgit-rs/public_symbol_export.c
 create mode 100644 contrib/cgit-rs/public_symbol_export.h
 create mode 100644 contrib/cgit-rs/src/lib.rs
 create mode 100644 contrib/cgit-rs/src/main.rs

base-commit: 557ae147e6cdc9db121269b058c757ac5092f9c9