{"thread":{"id":"63804","subject":"[PATCH 0/7] RFC: Accelerate xdiff and begin its rustification","startedAt":"2025-07-17T20:32:27Z","lastAt":"2025-11-17T13:37:12Z","messageCount":204,"participants":["Ezekiel Newren via GitGitGadget","brian m. carlson","Junio C Hamano","Taylor Blau","Elijah Newren","Christian Brabandt","Phillip Wood","Eli Schwartz","Ezekiel Newren","Haelwenn (lanodan) Monnier","Johannes Schindelin","Matthias Aßhauer","Patrick Steinhardt","Sam James","Collin Funk","Mike Hommey","Pierre-Emmanuel Patry","Ben Knoble","brian m. carlson via GitGitGadget","Johannes Schindelin via GitGitGadget","Ramsay Jones","Kristoffer Haugsbakk","rsbecker@nexbridge.com","D. Ben Knoble","Josh Steadmon","Jeff King","Yee Cheng Chin"],"isPatch":true,"patchVersion":1,"patchTotal":7},"messages":[{"id":"522232","messageId":"pull.1980.git.git.1752784344.gitgitgadget@gmail.com","threadId":"63804","inReplyTo":null,"subject":"[PATCH 0/7] RFC: Accelerate xdiff and begin its rustification","fromName":"Ezekiel Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2025-07-17T20:32:17Z","receivedAt":"2025-07-17T20:32:27Z","isPatch":true,"sender":{"key":"ezekielnewren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/18313324?v=4"},"body":"This series accelerates xdiff by 5-19%.\n\nIt also introduces Rust as a hard dependency.\n\n…and it doesn’t yet pass a couple of the github workflows; hints from\nWindows experts, and opinions on ambiguous primitives would be appreciated\n(see below).\n\nThis is just the beginning of many patches that I have to convert portions\nof, maybe eventually all of, xdiff to Rust. While working on that\nconversion, I found several ways to clarify the code, along with some\noptimizations.\n\nSo...\n\nThis obviously raises the question of whether we are ready to accept a hard\ndependency on Rust. Previous discussions on the mailing list and at Git\nMerge 2024 have not answered that question. If not now, will we be willing\nto accept such a hard dependency later? And what route do we want to take to\nget there?\n\nAbout the optimizations in this series:\n\n1. xdiff currently uses DJB2a for hashing (even though it is not explicitly named as such). This is an older hashing algorithm, and modern alternatives are superior. I chose xxhash because it’s faster, more collision resistant, and designed to be a standard. Other hash algorithms like aHash, MurMurHash, SipHash, and Fnv1a were considered, but my local testing made me feel like xxhash was the best choice for usage in xdiff.\n\n2. In support of switching to xxhash, parsing and hashing were split into separate steps. And it turns out that memchr() is faster for parsing than character-by-character iteration.\n\n\nAbout the workflow builds/tests that aren’t working with this series:\n\n1. Windows fails to build. I don’t know which rust toolchain is even correct for this or if multiple are needed. Example failed build: https://github.com/git/git/actions/runs/16353209191\n\n2. I386/ubuntu:focal will build, but fails the tests. The kernel reports the bitness as 64 despite the container being 32. I believe the issue is that C uses ambiguous primitives (which differ in size between platforms). The new code should use unambiguous primitives from Rust (u32, u64, etc.) rather than perpetuating ambiguous primitive types. Since the current xdiff API hardcodes the ambiguous types, though, those places will need to be migrated to unambiguous primitives. Much of the C code needs a slight refactor to be compatible with the Rust FFI and usually requires converting ambiguous to unambiguous types. What does this community think of this approach?\n\n\nMy brother (Elijah, cc’ed) has been guiding and reviewing my work here.\n\nEzekiel Newren (7):\n xdiff: introduce rust\n xdiff/xprepare: remove superfluous forward declarations\n xdiff: delete unnecessary fields from xrecord_t and xdfile_t\n xdiff: make fields of xrecord_t Rust friendly\n xdiff: separate parsing lines from hashing them\n xdiff: conditionally use Rust's implementation of xxhash\n github_workflows: install rust\n\n .github/workflows/main.yml | 1 +\n .gitignore | 1 +\n Makefile | 60 +++++++---\n build_rust.sh | 59 ++++++++++\n ci/install-dependencies.sh | 14 +--\n ci/install-rust.sh | 33 ++++++\n ci/lib.sh | 8 ++\n ci/make-test-artifacts.sh | 7 ++\n ci/run-build-and-tests.sh | 10 ++\n git-compat-util.h | 17 +++\n meson.build | 40 +++++--\n rust/Cargo.lock | 21 ++++\n rust/Cargo.toml | 6 +\n rust/interop/Cargo.toml | 14 +++\n rust/interop/src/lib.rs | 0\n rust/xdiff/Cargo.toml | 16 +++\n rust/xdiff/src/lib.rs | 7 ++\n xdiff/xdiffi.c | 8 +-\n xdiff/xemit.c | 2 +-\n xdiff/xmerge.c | 14 +--\n xdiff/xpatience.c | 2 +-\n xdiff/xprepare.c | 226 ++++++++++++++++++-------------------\n xdiff/xtypes.h | 9 +-\n xdiff/xutils.c | 4 +-\n 24 files changed, 414 insertions(+), 165 deletions(-)\n create mode 100755 build_rust.sh\n create mode 100644 ci/install-rust.sh\n create mode 100644 rust/Cargo.lock\n create mode 100644 rust/Cargo.toml\n create mode 100644 rust/interop/Cargo.toml\n create mode 100644 rust/interop/src/lib.rs\n create mode 100644 rust/xdiff/Cargo.toml\n create mode 100644 rust/xdiff/src/lib.rs\n\n\nbase-commit: 16bd9f20a403117f2e0d9bcda6c6e621d3763e77\nPublished-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-1980%2Fezekielnewren%2Fxdiff_rust_speedup-v1\nFetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-1980/ezekielnewren/xdiff_rust_speedup-v1\nPull-Request: https://github.com/git/git/pull/1980\n-- \ngitgitgadget\n"},{"id":"522233","messageId":"2a1f4be13dfbdee21811b7a4907f99042c791c2d.1752784344.git.gitgitgadget@gmail.com","threadId":"63804","inReplyTo":"pull.1980.git.git.1752784344.gitgitgadget@gmail.com","subject":"[PATCH 1/7] xdiff: introduce rust","fromName":"Ezekiel Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2025-07-17T20:32:18Z","receivedAt":"2025-07-17T20:32:29Z","isPatch":true,"sender":{"key":"ezekielnewren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/18313324?v=4"},"body":"From: Ezekiel Newren \n\nUpcoming patches will accelerate and simplify xdiff, while also\nporting parts of it to Rust. In preparation, add some stubs and setup\nthe Rust build. For now, it is easier to let cargo build rust and\nhave make or meson merely link against the static library that cargo\nbuilds. In line with ongoing libification efforts, use multiple\ncrates to allow more modularity on the Rust side. xdiff is the crate\nthat this series will focus on, but we also introduce the interop\ncrate for future patch series.\n\nIn order to facilitate interoperability between C and Rust, introduce\nC definitions for Rust primitive types in git-compat-util.h.\n\nSigned-off-by: Ezekiel Newren \n---\n Makefile | 20 +++++++++++++++++++-\n git-compat-util.h | 17 +++++++++++++++++\n meson.build | 32 ++++++++++++++++++++++++++++++++\n rust/Cargo.lock | 14 ++++++++++++++\n rust/Cargo.toml | 6 ++++++\n rust/interop/Cargo.toml | 14 ++++++++++++++\n rust/interop/src/lib.rs | 0\n rust/xdiff/Cargo.toml | 15 +++++++++++++++\n rust/xdiff/src/lib.rs | 0\n 9 files changed, 117 insertions(+), 1 deletion(-)\n create mode 100644 rust/Cargo.lock\n create mode 100644 rust/Cargo.toml\n create mode 100644 rust/interop/Cargo.toml\n create mode 100644 rust/interop/src/lib.rs\n create mode 100644 rust/xdiff/Cargo.toml\n create mode 100644 rust/xdiff/src/lib.rs\n\ndiff --git a/Makefile b/Makefile\nindex 70d1543b6b86..db39e6e1c28e 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -919,6 +919,11 @@ TEST_SHELL_PATH = $(SHELL_PATH)\n \n LIB_FILE = libgit.a\n XDIFF_LIB = xdiff/lib.a\n+ifeq ($(DEBUG), 1)\n+RUST_LIB = rust/target/debug/libxdiff.a\n+else\n+RUST_LIB = rust/target/release/libxdiff.a\n+endif\n REFTABLE_LIB = reftable/libreftable.a\n \n GENERATED_H += command-list.h\n@@ -1392,6 +1397,8 @@ UNIT_TEST_OBJS += $(UNIT_TEST_DIR)/lib-reftable.o\n GITLIBS = common-main.o $(LIB_FILE) $(XDIFF_LIB) $(REFTABLE_LIB) $(LIB_FILE)\n EXTLIBS =\n \n+GITLIBS += $(RUST_LIB)\n+\n GIT_USER_AGENT = git/$(GIT_VERSION)\n \n ifeq ($(wildcard sha1collisiondetection/lib/sha1.h),sha1collisiondetection/lib/sha1.h)\n@@ -2925,6 +2932,14 @@ $(LIB_FILE): $(LIB_OBJS)\n $(XDIFF_LIB): $(XDIFF_OBJS)\n \t$(QUIET_AR)$(RM) $@ && $(AR) $(ARFLAGS) $@ $^\n \n+.PHONY: $(RUST_LIB)\n+$(RUST_LIB):\n+ifeq ($(DEBUG), 1)\n+\tcd rust && RUSTFLAGS=\"-Aunused_imports -Adead_code\" cargo build --verbose\n+else\n+\tcd rust && RUSTFLAGS=\"-Aunused_imports -Adead_code\" cargo build --verbose --release\n+endif\n+\n $(REFTABLE_LIB): $(REFTABLE_OBJS)\n \t$(QUIET_AR)$(RM) $@ && $(AR) $(ARFLAGS) $@ $^\n \n@@ -3756,7 +3771,10 @@ cocciclean:\n \t$(RM) -r .build/contrib/coccinelle\n \t$(RM) contrib/coccinelle/*.cocci.patch\n \n-clean: profile-clean coverage-clean cocciclean\n+rustclean:\n+\tcd rust && cargo clean\n+\n+clean: profile-clean coverage-clean cocciclean rustclean\n \t$(RM) -r .build $(UNIT_TEST_BIN)\n \t$(RM) GIT-TEST-SUITES\n \t$(RM) po/git.pot po/git-core.pot\ndiff --git a/git-compat-util.h b/git-compat-util.h\nindex 4678e21c4cb8..82dc99764ac0 100644\n--- a/git-compat-util.h\n+++ b/git-compat-util.h\n@@ -196,6 +196,23 @@ static inline int is_xplatform_dir_sep(int c)\n #include \"compat/msvc.h\"\n #endif\n \n+/* rust types */\n+typedef uint8_t u8;\n+typedef uint16_t u16;\n+typedef uint32_t u32;\n+typedef uint64_t u64;\n+\n+typedef int8_t i8;\n+typedef int16_t i16;\n+typedef int32_t i32;\n+typedef int64_t i64;\n+\n+typedef float f32;\n+typedef double f64;\n+\n+typedef size_t usize;\n+typedef ptrdiff_t isize;\n+\n /* used on Mac OS X */\n #ifdef PRECOMPOSE_UNICODE\n #include \"compat/precompose_utf8.h\"\ndiff --git a/meson.build b/meson.build\nindex 596f5ac7110e..2d8da17f6515 100644\n--- a/meson.build\n+++ b/meson.build\n@@ -267,6 +267,36 @@ version_gen_environment.set('GIT_DATE', get_option('build_date'))\n version_gen_environment.set('GIT_USER_AGENT', get_option('user_agent'))\n version_gen_environment.set('GIT_VERSION', get_option('version'))\n \n+if get_option('optimization') in ['2', '3', 's', 'z']\n+ rust_target = 'release'\n+ rust_args = ['--release']\n+ rustflags = '-Aunused_imports -Adead_code'\n+else\n+ rust_target = 'debug'\n+ rust_args = []\n+ rustflags = '-Aunused_imports -Adead_code -C debuginfo=2 -C opt-level=1 -C force-frame-pointers=yes'\n+endif\n+\n+\n+rust_leaf = custom_target('rust_leaf',\n+ output: 'libxdiff.a',\n+ build_by_default: true,\n+ build_always_stale: true,\n+ command: ['cargo', 'build',\n+ '--manifest-path', meson.project_source_root() / 'rust/Cargo.toml'\n+ ] + rust_args,\n+ env: {\n+ 'RUSTFLAGS': rustflags,\n+ },\n+ install: false,\n+)\n+\n+rust_xdiff_dep = declare_dependency(\n+ link_args: ['-L' + meson.project_source_root() / 'rust/target' / rust_target, '-lxdiff'],\n+# include_directories: include_directories('xdiff/include'), # Adjust if you expose headers\n+)\n+\n+\n compiler = meson.get_compiler('c')\n \n libgit_sources = [\n@@ -1677,6 +1707,8 @@ version_def_h = custom_target(\n )\n libgit_sources += version_def_h\n \n+libgit_dependencies += rust_xdiff_dep\n+\n libgit = declare_dependency(\n link_with: static_library('git',\n sources: libgit_sources,\ndiff --git a/rust/Cargo.lock b/rust/Cargo.lock\nnew file mode 100644\nindex 000000000000..fb1eac690b39\n--- /dev/null\n+++ b/rust/Cargo.lock\n@@ -0,0 +1,14 @@\n+# This file is automatically @generated by Cargo.\n+# It is not intended for manual editing.\n+version = 4\n+\n+[[package]]\n+name = \"interop\"\n+version = \"0.1.0\"\n+\n+[[package]]\n+name = \"xdiff\"\n+version = \"0.1.0\"\n+dependencies = [\n+ \"interop\",\n+]\ndiff --git a/rust/Cargo.toml b/rust/Cargo.toml\nnew file mode 100644\nindex 000000000000..ed3d79d7f827\n--- /dev/null\n+++ b/rust/Cargo.toml\n@@ -0,0 +1,6 @@\n+[workspace]\n+members = [\n+ \"xdiff\",\n+ \"interop\",\n+]\n+resolver = \"2\"\ndiff --git a/rust/interop/Cargo.toml b/rust/interop/Cargo.toml\nnew file mode 100644\nindex 000000000000..045e3b01cfad\n--- /dev/null\n+++ b/rust/interop/Cargo.toml\n@@ -0,0 +1,14 @@\n+[package]\n+name = \"interop\"\n+version = \"0.1.0\"\n+edition = \"2021\"\n+\n+[lib]\n+name = \"interop\"\n+path = \"src/lib.rs\"\n+## staticlib to generate xdiff.a for use by gcc\n+## cdylib (optional) to generate xdiff.so for use by gcc\n+## rlib is required by the rust unit tests\n+crate-type = [\"staticlib\", \"rlib\"]\n+\n+[dependencies]\ndiff --git a/rust/interop/src/lib.rs b/rust/interop/src/lib.rs\nnew file mode 100644\nindex 000000000000..e69de29bb2d1\ndiff --git a/rust/xdiff/Cargo.toml b/rust/xdiff/Cargo.toml\nnew file mode 100644\nindex 000000000000..eb7966aada64\n--- /dev/null\n+++ b/rust/xdiff/Cargo.toml\n@@ -0,0 +1,15 @@\n+[package]\n+name = \"xdiff\"\n+version = \"0.1.0\"\n+edition = \"2021\"\n+\n+[lib]\n+name = \"xdiff\"\n+path = \"src/lib.rs\"\n+## staticlib to generate xdiff.a for use by gcc\n+## cdylib (optional) to generate xdiff.so for use by gcc\n+## rlib is required by the rust unit tests\n+crate-type = [\"staticlib\", \"rlib\"]\n+\n+[dependencies]\n+interop = { path = \"../interop\" }\ndiff --git a/rust/xdiff/src/lib.rs b/rust/xdiff/src/lib.rs\nnew file mode 100644\nindex 000000000000..e69de29bb2d1\n-- \ngitgitgadget\n\n"},{"id":"522234","messageId":"b0b744b9acf5299d323d56cbcc01411a228c1fc8.1752784344.git.gitgitgadget@gmail.com","threadId":"63804","inReplyTo":"pull.1980.git.git.1752784344.gitgitgadget@gmail.com","subject":"[PATCH 2/7] xdiff/xprepare: remove superfluous forward declarations","fromName":"Ezekiel Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2025-07-17T20:32:19Z","receivedAt":"2025-07-17T20:32:30Z","isPatch":true,"sender":{"key":"ezekielnewren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/18313324?v=4"},"body":"From: Ezekiel Newren \n\nMove xdl_prepare_env() later in the file to avoid the need\nfor forward declarations.\n\nSigned-off-by: Ezekiel Newren \n---\n xdiff/xprepare.c | 116 ++++++++++++++++++++---------------------------\n 1 file changed, 50 insertions(+), 66 deletions(-)\n\ndiff --git a/xdiff/xprepare.c b/xdiff/xprepare.c\nindex e1d4017b2dde..a45c5ee208c8 100644\n--- a/xdiff/xprepare.c\n+++ b/xdiff/xprepare.c\n@@ -53,21 +53,6 @@ typedef struct s_xdlclassifier {\n \n \n \n-static int xdl_init_classifier(xdlclassifier_t *cf, long size, long flags);\n-static void xdl_free_classifier(xdlclassifier_t *cf);\n-static int xdl_classify_record(unsigned int pass, xdlclassifier_t *cf, xrecord_t **rhash,\n-\t\t\t unsigned int hbits, xrecord_t *rec);\n-static int xdl_prepare_ctx(unsigned int pass, mmfile_t *mf, long narec, xpparam_t const *xpp,\n-\t\t\t xdlclassifier_t *cf, xdfile_t *xdf);\n-static void xdl_free_ctx(xdfile_t *xdf);\n-static int xdl_clean_mmatch(char const *dis, long i, long s, long e);\n-static int xdl_cleanup_records(xdlclassifier_t *cf, xdfile_t *xdf1, xdfile_t *xdf2);\n-static int xdl_trim_ends(xdfile_t *xdf1, xdfile_t *xdf2);\n-static int xdl_optimize_ctxs(xdlclassifier_t *cf, xdfile_t *xdf1, xdfile_t *xdf2);\n-\n-\n-\n-\n static int xdl_init_classifier(xdlclassifier_t *cf, long size, long flags) {\n \tcf->flags = flags;\n \n@@ -242,57 +227,6 @@ static void xdl_free_ctx(xdfile_t *xdf) {\n }\n \n \n-int xdl_prepare_env(mmfile_t *mf1, mmfile_t *mf2, xpparam_t const *xpp,\n-\t\t xdfenv_t *xe) {\n-\tlong enl1, enl2, sample;\n-\txdlclassifier_t cf;\n-\n-\tmemset(&cf, 0, sizeof(cf));\n-\n-\t/*\n-\t * For histogram diff, we can afford a smaller sample size and\n-\t * thus a poorer estimate of the number of lines, as the hash\n-\t * table (rhash) won't be filled up/grown. The number of lines\n-\t * (nrecs) will be updated correctly anyway by\n-\t * xdl_prepare_ctx().\n-\t */\n-\tsample = (XDF_DIFF_ALG(xpp->flags) == XDF_HISTOGRAM_DIFF\n-\t\t ? XDL_GUESS_NLINES2 : XDL_GUESS_NLINES1);\n-\n-\tenl1 = xdl_guess_lines(mf1, sample) + 1;\n-\tenl2 = xdl_guess_lines(mf2, sample) + 1;\n-\n-\tif (xdl_init_classifier(&cf, enl1 + enl2 + 1, xpp->flags) < 0)\n-\t\treturn -1;\n-\n-\tif (xdl_prepare_ctx(1, mf1, enl1, xpp, &cf, &xe->xdf1) < 0) {\n-\n-\t\txdl_free_classifier(&cf);\n-\t\treturn -1;\n-\t}\n-\tif (xdl_prepare_ctx(2, mf2, enl2, xpp, &cf, &xe->xdf2) < 0) {\n-\n-\t\txdl_free_ctx(&xe->xdf1);\n-\t\txdl_free_classifier(&cf);\n-\t\treturn -1;\n-\t}\n-\n-\tif ((XDF_DIFF_ALG(xpp->flags) != XDF_PATIENCE_DIFF) &&\n-\t (XDF_DIFF_ALG(xpp->flags) != XDF_HISTOGRAM_DIFF) &&\n-\t xdl_optimize_ctxs(&cf, &xe->xdf1, &xe->xdf2) < 0) {\n-\n-\t\txdl_free_ctx(&xe->xdf2);\n-\t\txdl_free_ctx(&xe->xdf1);\n-\t\txdl_free_classifier(&cf);\n-\t\treturn -1;\n-\t}\n-\n-\txdl_free_classifier(&cf);\n-\n-\treturn 0;\n-}\n-\n-\n void xdl_free_env(xdfenv_t *xe) {\n \n \txdl_free_ctx(&xe->xdf2);\n@@ -460,3 +394,53 @@ static int xdl_optimize_ctxs(xdlclassifier_t *cf, xdfile_t *xdf1, xdfile_t *xdf2\n \n \treturn 0;\n }\n+\n+int xdl_prepare_env(mmfile_t *mf1, mmfile_t *mf2, xpparam_t const *xpp,\n+\t\t xdfenv_t *xe) {\n+\tlong enl1, enl2, sample;\n+\txdlclassifier_t cf;\n+\n+\tmemset(&cf, 0, sizeof(cf));\n+\n+\t/*\n+\t * For histogram diff, we can afford a smaller sample size and\n+\t * thus a poorer estimate of the number of lines, as the hash\n+\t * table (rhash) won't be filled up/grown. The number of lines\n+\t * (nrecs) will be updated correctly anyway by\n+\t * xdl_prepare_ctx().\n+\t */\n+\tsample = (XDF_DIFF_ALG(xpp->flags) == XDF_HISTOGRAM_DIFF\n+\t\t ? XDL_GUESS_NLINES2 : XDL_GUESS_NLINES1);\n+\n+\tenl1 = xdl_guess_lines(mf1, sample) + 1;\n+\tenl2 = xdl_guess_lines(mf2, sample) + 1;\n+\n+\tif (xdl_init_classifier(&cf, enl1 + enl2 + 1, xpp->flags) < 0)\n+\t\treturn -1;\n+\n+\tif (xdl_prepare_ctx(1, mf1, enl1, xpp, &cf, &xe->xdf1) < 0) {\n+\n+\t\txdl_free_classifier(&cf);\n+\t\treturn -1;\n+\t}\n+\tif (xdl_prepare_ctx(2, mf2, enl2, xpp, &cf, &xe->xdf2) < 0) {\n+\n+\t\txdl_free_ctx(&xe->xdf1);\n+\t\txdl_free_classifier(&cf);\n+\t\treturn -1;\n+\t}\n+\n+\tif ((XDF_DIFF_ALG(xpp->flags) != XDF_PATIENCE_DIFF) &&\n+\t (XDF_DIFF_ALG(xpp->flags) != XDF_HISTOGRAM_DIFF) &&\n+\t xdl_optimize_ctxs(&cf, &xe->xdf1, &xe->xdf2) < 0) {\n+\n+\t\txdl_free_ctx(&xe->xdf2);\n+\t\txdl_free_ctx(&xe->xdf1);\n+\t\txdl_free_classifier(&cf);\n+\t\treturn -1;\n+\t }\n+\n+\txdl_free_classifier(&cf);\n+\n+\treturn 0;\n+}\n-- \ngitgitgadget\n\n"},{"id":"522235","messageId":"cc05150d6e142b6e0b2837b437903577cacb629a.1752784344.git.gitgitgadget@gmail.com","threadId":"63804","inReplyTo":"pull.1980.git.git.1752784344.gitgitgadget@gmail.com","subject":"[PATCH 3/7] xdiff: delete unnecessary fields from xrecord_t and xdfile_t","fromName":"Ezekiel Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2025-07-17T20:32:20Z","receivedAt":"2025-07-17T20:32:31Z","isPatch":true,"sender":{"key":"ezekielnewren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/18313324?v=4"},"body":"From: Ezekiel Newren \n\nxrecord_t.next, xdfile_t.hbits, xdfile_t.rhash are initialized,\nbut never used for anything by the code. Remove them.\n\nSigned-off-by: Ezekiel Newren \n---\n xdiff/xprepare.c | 24 +++---------------------\n xdiff/xtypes.h | 3 ---\n 2 files changed, 3 insertions(+), 24 deletions(-)\n\ndiff --git a/xdiff/xprepare.c b/xdiff/xprepare.c\nindex a45c5ee208c8..ad356281f939 100644\n--- a/xdiff/xprepare.c\n+++ b/xdiff/xprepare.c\n@@ -91,8 +91,7 @@ static void xdl_free_classifier(xdlclassifier_t *cf) {\n }\n \n \n-static int xdl_classify_record(unsigned int pass, xdlclassifier_t *cf, xrecord_t **rhash,\n-\t\t\t unsigned int hbits, xrecord_t *rec) {\n+static int xdl_classify_record(unsigned int pass, xdlclassifier_t *cf, xrecord_t *rec) {\n \tlong hi;\n \tchar const *line;\n \txdlclass_t *rcrec;\n@@ -126,23 +125,17 @@ static int xdl_classify_record(unsigned int pass, xdlclassifier_t *cf, xrecord_t\n \n \trec->ha = (unsigned long) rcrec->idx;\n \n-\thi = (long) XDL_HASHLONG(rec->ha, hbits);\n-\trec->next = rhash[hi];\n-\trhash[hi] = rec;\n-\n \treturn 0;\n }\n \n \n static int xdl_prepare_ctx(unsigned int pass, mmfile_t *mf, long narec, xpparam_t const *xpp,\n \t\t\t xdlclassifier_t *cf, xdfile_t *xdf) {\n-\tunsigned int hbits;\n-\tlong nrec, hsize, bsize;\n+\tlong nrec, bsize;\n \tunsigned long hav;\n \tchar const *blk, *cur, *top, *prev;\n \txrecord_t *crec;\n \txrecord_t **recs;\n-\txrecord_t **rhash;\n \tunsigned long *ha;\n \tchar *rchg;\n \tlong *rindex;\n@@ -150,7 +143,6 @@ static int xdl_prepare_ctx(unsigned int pass, mmfile_t *mf, long narec, xpparam_\n \tha = NULL;\n \trindex = NULL;\n \trchg = NULL;\n-\trhash = NULL;\n \trecs = NULL;\n \n \tif (xdl_cha_init(&xdf->rcha, sizeof(xrecord_t), narec / 4 + 1) < 0)\n@@ -158,11 +150,6 @@ static int xdl_prepare_ctx(unsigned int pass, mmfile_t *mf, long narec, xpparam_\n \tif (!XDL_ALLOC_ARRAY(recs, narec))\n \t\tgoto abort;\n \n-\thbits = xdl_hashbits((unsigned int) narec);\n-\thsize = 1 << hbits;\n-\tif (!XDL_CALLOC_ARRAY(rhash, hsize))\n-\t\tgoto abort;\n-\n \tnrec = 0;\n \tif ((cur = blk = xdl_mmfile_first(mf, &bsize))) {\n \t\tfor (top = blk + bsize; cur < top; ) {\n@@ -176,7 +163,7 @@ static int xdl_prepare_ctx(unsigned int pass, mmfile_t *mf, long narec, xpparam_\n \t\t\tcrec->size = (long) (cur - prev);\n \t\t\tcrec->ha = hav;\n \t\t\trecs[nrec++] = crec;\n-\t\t\tif (xdl_classify_record(pass, cf, rhash, hbits, crec) < 0)\n+\t\t\tif (xdl_classify_record(pass, cf, crec) < 0)\n \t\t\t\tgoto abort;\n \t\t}\n \t}\n@@ -194,8 +181,6 @@ static int xdl_prepare_ctx(unsigned int pass, mmfile_t *mf, long narec, xpparam_\n \n \txdf->nrec = nrec;\n \txdf->recs = recs;\n-\txdf->hbits = hbits;\n-\txdf->rhash = rhash;\n \txdf->rchg = rchg + 1;\n \txdf->rindex = rindex;\n \txdf->nreff = 0;\n@@ -209,7 +194,6 @@ abort:\n \txdl_free(ha);\n \txdl_free(rindex);\n \txdl_free(rchg);\n-\txdl_free(rhash);\n \txdl_free(recs);\n \txdl_cha_free(&xdf->rcha);\n \treturn -1;\n@@ -217,8 +201,6 @@ abort:\n \n \n static void xdl_free_ctx(xdfile_t *xdf) {\n-\n-\txdl_free(xdf->rhash);\n \txdl_free(xdf->rindex);\n \txdl_free(xdf->rchg - 1);\n \txdl_free(xdf->ha);\ndiff --git a/xdiff/xtypes.h b/xdiff/xtypes.h\nindex 8442bd436efe..8b8467360ecf 100644\n--- a/xdiff/xtypes.h\n+++ b/xdiff/xtypes.h\n@@ -39,7 +39,6 @@ typedef struct s_chastore {\n } chastore_t;\n \n typedef struct s_xrecord {\n-\tstruct s_xrecord *next;\n \tchar const *ptr;\n \tlong size;\n \tunsigned long ha;\n@@ -48,8 +47,6 @@ typedef struct s_xrecord {\n typedef struct s_xdfile {\n \tchastore_t rcha;\n \tlong nrec;\n-\tunsigned int hbits;\n-\txrecord_t **rhash;\n \tlong dstart, dend;\n \txrecord_t **recs;\n \tchar *rchg;\n-- \ngitgitgadget\n\n"},{"id":"522236","messageId":"6df9f50a8f4ca29b2c3ba1e39982b6d516146bb3.1752784344.git.gitgitgadget@gmail.com","threadId":"63804","inReplyTo":"pull.1980.git.git.1752784344.gitgitgadget@gmail.com","subject":"[PATCH 4/7] xdiff: make fields of xrecord_t Rust friendly","fromName":"Ezekiel Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2025-07-17T20:32:21Z","receivedAt":"2025-07-17T20:32:31Z","isPatch":true,"sender":{"key":"ezekielnewren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/18313324?v=4"},"body":"From: Ezekiel Newren \n\nA few commits ago, we added definitions for Rust primitive types,\nto facilitate interoperability between C and Rust. Switch a\nfew variables to use these types. Which, for now, will\nrequire adding some casts.\n\nSigned-off-by: Ezekiel Newren \n---\n xdiff/xdiffi.c | 8 ++++----\n xdiff/xemit.c | 2 +-\n xdiff/xmerge.c | 14 +++++++-------\n xdiff/xpatience.c | 2 +-\n xdiff/xprepare.c | 6 +++---\n xdiff/xtypes.h | 6 +++---\n xdiff/xutils.c | 4 ++--\n 7 files changed, 21 insertions(+), 21 deletions(-)\n\ndiff --git a/xdiff/xdiffi.c b/xdiff/xdiffi.c\nindex 5a96e36dfbea..3b364c61f671 100644\n--- a/xdiff/xdiffi.c\n+++ b/xdiff/xdiffi.c\n@@ -418,7 +418,7 @@ static int get_indent(xrecord_t *rec)\n \tlong i;\n \tint ret = 0;\n \n-\tfor (i = 0; i < rec->size; i++) {\n+\tfor (i = 0; i < (long) rec->size; i++) {\n \t\tchar c = rec->ptr[i];\n \n \t\tif (!XDL_ISSPACE(c))\n@@ -1005,11 +1005,11 @@ static void xdl_mark_ignorable_lines(xdchange_t *xscr, xdfenv_t *xe, long flags)\n \n \t\trec = &xe->xdf1.recs[xch->i1];\n \t\tfor (i = 0; i < xch->chg1 && ignore; i++)\n-\t\t\tignore = xdl_blankline(rec[i]->ptr, rec[i]->size, flags);\n+\t\t\tignore = xdl_blankline((const char*) rec[i]->ptr, rec[i]->size, flags);\n \n \t\trec = &xe->xdf2.recs[xch->i2];\n \t\tfor (i = 0; i < xch->chg2 && ignore; i++)\n-\t\t\tignore = xdl_blankline(rec[i]->ptr, rec[i]->size, flags);\n+\t\t\tignore = xdl_blankline((const char*)rec[i]->ptr, rec[i]->size, flags);\n \n \t\txch->ignore = ignore;\n \t}\n@@ -1020,7 +1020,7 @@ static int record_matches_regex(xrecord_t *rec, xpparam_t const *xpp) {\n \tsize_t i;\n \n \tfor (i = 0; i < xpp->ignore_regex_nr; i++)\n-\t\tif (!regexec_buf(xpp->ignore_regex[i], rec->ptr, rec->size, 1,\n+\t\tif (!regexec_buf(xpp->ignore_regex[i], (const char*) rec->ptr, rec->size, 1,\n \t\t\t\t ®match, 0))\n \t\t\treturn 1;\n \ndiff --git a/xdiff/xemit.c b/xdiff/xemit.c\nindex 1d40c9cb4076..bbf7b7f8c862 100644\n--- a/xdiff/xemit.c\n+++ b/xdiff/xemit.c\n@@ -24,7 +24,7 @@\n \n static long xdl_get_rec(xdfile_t *xdf, long ri, char const **rec) {\n \n-\t*rec = xdf->recs[ri]->ptr;\n+\t*rec = (char const*) xdf->recs[ri]->ptr;\n \n \treturn xdf->recs[ri]->size;\n }\ndiff --git a/xdiff/xmerge.c b/xdiff/xmerge.c\nindex af40c88a5b36..6fa6ea61a208 100644\n--- a/xdiff/xmerge.c\n+++ b/xdiff/xmerge.c\n@@ -101,8 +101,8 @@ static int xdl_merge_cmp_lines(xdfenv_t *xe1, int i1, xdfenv_t *xe2, int i2,\n \txrecord_t **rec2 = xe2->xdf2.recs + i2;\n \n \tfor (i = 0; i < line_count; i++) {\n-\t\tint result = xdl_recmatch(rec1[i]->ptr, rec1[i]->size,\n-\t\t\trec2[i]->ptr, rec2[i]->size, flags);\n+\t\tint result = xdl_recmatch((const char*) rec1[i]->ptr, rec1[i]->size,\n+\t\t\t(const char*) rec2[i]->ptr, rec2[i]->size, flags);\n \t\tif (!result)\n \t\t\treturn -1;\n \t}\n@@ -324,8 +324,8 @@ static int xdl_fill_merge_buffer(xdfenv_t *xe1, const char *name1,\n \n static int recmatch(xrecord_t *rec1, xrecord_t *rec2, unsigned long flags)\n {\n-\treturn xdl_recmatch(rec1->ptr, rec1->size,\n-\t\t\t rec2->ptr, rec2->size, flags);\n+\treturn xdl_recmatch((char const*) rec1->ptr, rec1->size,\n+\t\t\t (char const*) rec2->ptr, rec2->size, flags);\n }\n \n /*\n@@ -383,10 +383,10 @@ static int xdl_refine_conflicts(xdfenv_t *xe1, xdfenv_t *xe2, xdmerge_t *m,\n \t\t */\n \t\tt1.ptr = (char *)xe1->xdf2.recs[m->i1]->ptr;\n \t\tt1.size = xe1->xdf2.recs[m->i1 + m->chg1 - 1]->ptr\n-\t\t\t+ xe1->xdf2.recs[m->i1 + m->chg1 - 1]->size - t1.ptr;\n+\t\t\t+ xe1->xdf2.recs[m->i1 + m->chg1 - 1]->size - (u8 const*) t1.ptr;\n \t\tt2.ptr = (char *)xe2->xdf2.recs[m->i2]->ptr;\n \t\tt2.size = xe2->xdf2.recs[m->i2 + m->chg2 - 1]->ptr\n-\t\t\t+ xe2->xdf2.recs[m->i2 + m->chg2 - 1]->size - t2.ptr;\n+\t\t\t+ xe2->xdf2.recs[m->i2 + m->chg2 - 1]->size - (u8 const*) t2.ptr;\n \t\tif (xdl_do_diff(&t1, &t2, xpp, &xe) < 0)\n \t\t\treturn -1;\n \t\tif (xdl_change_compact(&xe.xdf1, &xe.xdf2, xpp->flags) < 0 ||\n@@ -440,7 +440,7 @@ static int line_contains_alnum(const char *ptr, long size)\n static int lines_contain_alnum(xdfenv_t *xe, int i, int chg)\n {\n \tfor (; chg; chg--, i++)\n-\t\tif (line_contains_alnum(xe->xdf2.recs[i]->ptr,\n+\t\tif (line_contains_alnum((char const*) xe->xdf2.recs[i]->ptr,\n \t\t\t\txe->xdf2.recs[i]->size))\n \t\t\treturn 1;\n \treturn 0;\ndiff --git a/xdiff/xpatience.c b/xdiff/xpatience.c\nindex 77dc411d1937..986a3a3f749a 100644\n--- a/xdiff/xpatience.c\n+++ b/xdiff/xpatience.c\n@@ -121,7 +121,7 @@ static void insert_record(xpparam_t const *xpp, int line, struct hashmap *map,\n \t\treturn;\n \tmap->entries[index].line1 = line;\n \tmap->entries[index].hash = record->ha;\n-\tmap->entries[index].anchor = is_anchor(xpp, map->env->xdf1.recs[line - 1]->ptr);\n+\tmap->entries[index].anchor = is_anchor(xpp, (const char*) map->env->xdf1.recs[line - 1]->ptr);\n \tif (!map->first)\n \t\tmap->first = map->entries + index;\n \tif (map->last) {\ndiff --git a/xdiff/xprepare.c b/xdiff/xprepare.c\nindex ad356281f939..747268e4fdf7 100644\n--- a/xdiff/xprepare.c\n+++ b/xdiff/xprepare.c\n@@ -96,12 +96,12 @@ static int xdl_classify_record(unsigned int pass, xdlclassifier_t *cf, xrecord_t\n \tchar const *line;\n \txdlclass_t *rcrec;\n \n-\tline = rec->ptr;\n+\tline = (char const*) rec->ptr;\n \thi = (long) XDL_HASHLONG(rec->ha, cf->hbits);\n \tfor (rcrec = cf->rchash[hi]; rcrec; rcrec = rcrec->next)\n \t\tif (rcrec->ha == rec->ha &&\n \t\t\t\txdl_recmatch(rcrec->line, rcrec->size,\n-\t\t\t\t\trec->ptr, rec->size, cf->flags))\n+\t\t\t\t\t(const char*) rec->ptr, rec->size, cf->flags))\n \t\t\tbreak;\n \n \tif (!rcrec) {\n@@ -159,7 +159,7 @@ static int xdl_prepare_ctx(unsigned int pass, mmfile_t *mf, long narec, xpparam_\n \t\t\t\tgoto abort;\n \t\t\tif (!(crec = xdl_cha_alloc(&xdf->rcha)))\n \t\t\t\tgoto abort;\n-\t\t\tcrec->ptr = prev;\n+\t\t\tcrec->ptr = (u8 const*) prev;\n \t\t\tcrec->size = (long) (cur - prev);\n \t\t\tcrec->ha = hav;\n \t\t\trecs[nrec++] = crec;\ndiff --git a/xdiff/xtypes.h b/xdiff/xtypes.h\nindex 8b8467360ecf..6e5f67ebf380 100644\n--- a/xdiff/xtypes.h\n+++ b/xdiff/xtypes.h\n@@ -39,9 +39,9 @@ typedef struct s_chastore {\n } chastore_t;\n \n typedef struct s_xrecord {\n-\tchar const *ptr;\n-\tlong size;\n-\tunsigned long ha;\n+\tu8 const* ptr;\n+\tusize size;\n+\tu64 ha;\n } xrecord_t;\n \n typedef struct s_xdfile {\ndiff --git a/xdiff/xutils.c b/xdiff/xutils.c\nindex 444a108f87c0..10e4f20b7c31 100644\n--- a/xdiff/xutils.c\n+++ b/xdiff/xutils.c\n@@ -418,10 +418,10 @@ int xdl_fall_back_diff(xdfenv_t *diff_env, xpparam_t const *xpp,\n \n \tsubfile1.ptr = (char *)diff_env->xdf1.recs[line1 - 1]->ptr;\n \tsubfile1.size = diff_env->xdf1.recs[line1 + count1 - 2]->ptr +\n-\t\tdiff_env->xdf1.recs[line1 + count1 - 2]->size - subfile1.ptr;\n+\t\tdiff_env->xdf1.recs[line1 + count1 - 2]->size - (u8 const*) subfile1.ptr;\n \tsubfile2.ptr = (char *)diff_env->xdf2.recs[line2 - 1]->ptr;\n \tsubfile2.size = diff_env->xdf2.recs[line2 + count2 - 2]->ptr +\n-\t\tdiff_env->xdf2.recs[line2 + count2 - 2]->size - subfile2.ptr;\n+\t\tdiff_env->xdf2.recs[line2 + count2 - 2]->size - (u8 const*) subfile2.ptr;\n \tif (xdl_do_diff(&subfile1, &subfile2, xpp, &env) < 0)\n \t\treturn -1;\n \n-- \ngitgitgadget\n\n"},{"id":"522237","messageId":"2db30cc739efadf8383bd9dc1b7825ce863e8f5a.1752784344.git.gitgitgadget@gmail.com","threadId":"63804","inReplyTo":"pull.1980.git.git.1752784344.gitgitgadget@gmail.com","subject":"[PATCH 5/7] xdiff: separate parsing lines from hashing them","fromName":"Ezekiel Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2025-07-17T20:32:22Z","receivedAt":"2025-07-17T20:32:32Z","isPatch":true,"sender":{"key":"ezekielnewren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/18313324?v=4"},"body":"From: Ezekiel Newren \n\nWe want to use xxhash for faster hashing. To facilitate that\nand to simplify the code. Separate the concerns of parsing\nand hashing into discrete steps. This makes swapping the hash\nfunction much easier. Since xdl_hash_record() both parses and\nhashses lines, this requires some slight code restructuring.\n\nSigned-off-by: Ezekiel Newren \n---\n xdiff/xprepare.c | 75 ++++++++++++++++++++++++++++--------------------\n 1 file changed, 44 insertions(+), 31 deletions(-)\n\ndiff --git a/xdiff/xprepare.c b/xdiff/xprepare.c\nindex 747268e4fdf7..c44005e9bbb8 100644\n--- a/xdiff/xprepare.c\n+++ b/xdiff/xprepare.c\n@@ -129,13 +129,39 @@ static int xdl_classify_record(unsigned int pass, xdlclassifier_t *cf, xrecord_t\n }\n \n \n+static void xdl_parse_lines(mmfile_t *mf, long narec, xdfile_t *xdf) {\n+\tu8 const* ptr = (u8 const*) mf->ptr;\n+\tusize len = (usize) mf->size;\n+\n+\txdf->recs = NULL;\n+\txdf->nrec = 0;\n+\tXDL_ALLOC_ARRAY(xdf->recs, narec);\n+\n+\twhile (len > 0) {\n+\t\txrecord_t *rec = NULL;\n+\t\tusize length;\n+\t\tu8 const* result = memchr(ptr, '\\n', len);\n+\t\tif (result) {\n+\t\t\tlength = result - ptr + 1;\n+\t\t} else {\n+\t\t\tlength = len;\n+\t\t}\n+\t\tif (XDL_ALLOC_GROW(xdf->recs, xdf->nrec + 1, narec))\n+\t\t\tdie(\"XDL_ALLOC_GROW failed\");\n+\t\trec = xdl_cha_alloc(&xdf->rcha);\n+\t\trec->ptr = ptr;\n+\t\trec->size = length;\n+\t\trec->ha = 0;\n+\t\txdf->recs[xdf->nrec++] = rec;\n+\t\tptr += length;\n+\t\tlen -= length;\n+\t}\n+\n+}\n+\n+\n static int xdl_prepare_ctx(unsigned int pass, mmfile_t *mf, long narec, xpparam_t const *xpp,\n \t\t\t xdlclassifier_t *cf, xdfile_t *xdf) {\n-\tlong nrec, bsize;\n-\tunsigned long hav;\n-\tchar const *blk, *cur, *top, *prev;\n-\txrecord_t *crec;\n-\txrecord_t **recs;\n \tunsigned long *ha;\n \tchar *rchg;\n \tlong *rindex;\n@@ -143,50 +169,37 @@ static int xdl_prepare_ctx(unsigned int pass, mmfile_t *mf, long narec, xpparam_\n \tha = NULL;\n \trindex = NULL;\n \trchg = NULL;\n-\trecs = NULL;\n \n \tif (xdl_cha_init(&xdf->rcha, sizeof(xrecord_t), narec / 4 + 1) < 0)\n \t\tgoto abort;\n-\tif (!XDL_ALLOC_ARRAY(recs, narec))\n-\t\tgoto abort;\n \n-\tnrec = 0;\n-\tif ((cur = blk = xdl_mmfile_first(mf, &bsize))) {\n-\t\tfor (top = blk + bsize; cur < top; ) {\n-\t\t\tprev = cur;\n-\t\t\thav = xdl_hash_record(&cur, top, xpp->flags);\n-\t\t\tif (XDL_ALLOC_GROW(recs, nrec + 1, narec))\n-\t\t\t\tgoto abort;\n-\t\t\tif (!(crec = xdl_cha_alloc(&xdf->rcha)))\n-\t\t\t\tgoto abort;\n-\t\t\tcrec->ptr = (u8 const*) prev;\n-\t\t\tcrec->size = (long) (cur - prev);\n-\t\t\tcrec->ha = hav;\n-\t\t\trecs[nrec++] = crec;\n-\t\t\tif (xdl_classify_record(pass, cf, crec) < 0)\n-\t\t\t\tgoto abort;\n-\t\t}\n+\txdl_parse_lines(mf, narec, xdf);\n+\n+\tfor (usize i = 0; i < (usize) xdf->nrec; i++) {\n+\t\txrecord_t *rec = xdf->recs[i];\n+\t\tchar const* dump = (char const*) rec->ptr;\n+\t\trec->ha = xdl_hash_record(&dump, (char const*) (rec->ptr + rec->size), xpp->flags);\n+\t\txdl_classify_record(pass, cf, rec);\n \t}\n \n-\tif (!XDL_CALLOC_ARRAY(rchg, nrec + 2))\n+\n+\tif (!XDL_CALLOC_ARRAY(rchg, xdf->nrec + 2))\n \t\tgoto abort;\n \n \tif ((XDF_DIFF_ALG(xpp->flags) != XDF_PATIENCE_DIFF) &&\n \t (XDF_DIFF_ALG(xpp->flags) != XDF_HISTOGRAM_DIFF)) {\n-\t\tif (!XDL_ALLOC_ARRAY(rindex, nrec + 1))\n+\t\tif (!XDL_ALLOC_ARRAY(rindex, xdf->nrec + 1))\n \t\t\tgoto abort;\n-\t\tif (!XDL_ALLOC_ARRAY(ha, nrec + 1))\n+\t\tif (!XDL_ALLOC_ARRAY(ha, xdf->nrec + 1))\n \t\t\tgoto abort;\n \t}\n \n-\txdf->nrec = nrec;\n-\txdf->recs = recs;\n \txdf->rchg = rchg + 1;\n \txdf->rindex = rindex;\n \txdf->nreff = 0;\n \txdf->ha = ha;\n \txdf->dstart = 0;\n-\txdf->dend = nrec - 1;\n+\txdf->dend = xdf->nrec - 1;\n \n \treturn 0;\n \n@@ -194,7 +207,7 @@ abort:\n \txdl_free(ha);\n \txdl_free(rindex);\n \txdl_free(rchg);\n-\txdl_free(recs);\n+\txdl_free(xdf->recs);\n \txdl_cha_free(&xdf->rcha);\n \treturn -1;\n }\n-- \ngitgitgadget\n\n"},{"id":"522238","messageId":"5a959c9bdad79cf972b95dcf4324135dd7c94dac.1752784344.git.gitgitgadget@gmail.com","threadId":"63804","inReplyTo":"pull.1980.git.git.1752784344.gitgitgadget@gmail.com","subject":"[PATCH 6/7] xdiff: conditionally use Rust's implementation of xxhash","fromName":"Ezekiel Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2025-07-17T20:32:23Z","receivedAt":"2025-07-17T20:32:34Z","isPatch":true,"sender":{"key":"ezekielnewren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/18313324?v=4"},"body":"From: Ezekiel Newren \n\nWhen no whitespace flags are present use xxhash, for faster\nhashing, otherwise use DJB2a (which is what xdiff has been\nusing all along).\n\nThe benchmark below compares my series with version v2.49.0\n(built in build_release/ and build_v2.49.0/ respectively),\nrunning log commands on linux kernel with 3 different machines.\n\n$ BASE=/path/to/git/root\n\n // laptop\n // CPU: 6-core Intel Core i7-8750H (-MT MCP-) speed/min/max: 726/800/4100 MHz\n $ hyperfine --warmup 3 -L exe $BASE/build_release/git,$BASE/build_v2.49.0/git '{exe} log --oneline --shortstat v6.8..v6.9 >/dev/null'\n Benchmark 1: /home/ezekiel/development/work/git/build_release/git log --oneline --shortstat v6.8..v6.9 >/dev/null\n Time (mean ± σ): 10.419 s ± 0.166 s [User: 10.097 s, System: 0.284 s]\n Range (min … max): 10.215 s … 10.680 s 10 runs\n\n Benchmark 2: /home/ezekiel/development/work/git/build_v2.49.0/git log --oneline --shortstat v6.8..v6.9 >/dev/null\n Time (mean ± σ): 10.980 s ± 0.137 s [User: 10.633 s, System: 0.308 s]\n Range (min … max): 10.791 s … 11.178 s 10 runs\n\n Summary\n /home/ezekiel/development/work/git/build_release/git log --oneline --shortstat v6.8..v6.9 >/dev/null ran\n 1.05 ± 0.02 times faster than /home/ezekiel/development/work/git/build_v2.49.0/git log --oneline --shortstat v6.8..v6.9 >/dev/null\n\n // desktop\n // CPU: 8-core Intel Core i7-9700 (-MCP-) speed/min/max: 800/800/4700 MHz\n $ hyperfine --warmup 3 -L exe $BASE/build_release/git,$BASE/build_v2.49.0/git '{exe} log --oneline --shortstat v6.8..v6.9 >/dev/null'\n Benchmark 1: /home/steamuser/dev/git/build_release/git log --oneline --shortstat v6.8..v6.9 >/dev/null\n Time (mean ± σ): 6.823 s ± 0.020 s [User: 6.624 s, System: 0.180 s]\n Range (min … max): 6.801 s … 6.858 s 10 runs\n\n Benchmark 2: /home/steamuser/dev/git/build_v2.49.0/git log --oneline --shortstat v6.8..v6.9 >/dev/null\n Time (mean ± σ): 8.151 s ± 0.024 s [User: 7.928 s, System: 0.198 s]\n Range (min … max): 8.105 s … 8.184 s 10 runs\n\n Summary\n /home/steamuser/dev/git/build_release/git log --oneline --shortstat v6.8..v6.9 >/dev/null ran\n 1.19 ± 0.01 times faster than /home/steamuser/dev/git/build_v2.49.0/git log --oneline --shortstat v6.8..v6.9 >/dev/null\n\n // router\n // CPU: dual core Intel Celeron 3965U (-MCP-) speed/min/max: 1300/400/2200 MHz\n $ hyperfine --warmup 3 -L exe $BASE/build_release/git,$BASE/build_v2.49.0/git '{exe} log --oneline --shortstat v6.8..v6.9 >/dev/null'\n Benchmark 1: /home/metal/dev/git/build_release/git log --oneline --shortstat v6.8..v6.9 >/dev/null\n Time (mean ± σ): 21.209 s ± 0.054 s [User: 20.341 s, System: 0.605 s]\n Range (min … max): 21.135 s … 21.309 s 10 runs\n\n Benchmark 2: /home/metal/dev/git/build_v2.49.0/git log --oneline --shortstat v6.8..v6.9 >/dev/null\n Time (mean ± σ): 23.683 s ± 0.060 s [User: 22.735 s, System: 0.672 s]\n Range (min … max): 23.566 s … 23.751 s 10 runs\n\n Summary\n /home/metal/dev/git/build_release/git log --oneline --shortstat v6.8..v6.9 >/dev/null ran\n 1.12 ± 0.00 times faster than /home/metal/dev/git/build_v2.49.0/git log --oneline --shortstat v6.8..v6.9 >/dev/null\n\nSigned-off-by: Ezekiel Newren \n---\n rust/Cargo.lock | 7 +++++++\n rust/xdiff/Cargo.toml | 1 +\n rust/xdiff/src/lib.rs | 7 +++++++\n xdiff/xprepare.c | 19 +++++++++++++++++--\n 4 files changed, 32 insertions(+), 2 deletions(-)\n\ndiff --git a/rust/Cargo.lock b/rust/Cargo.lock\nindex fb1eac690b39..5f84617b1049 100644\n--- a/rust/Cargo.lock\n+++ b/rust/Cargo.lock\n@@ -11,4 +11,11 @@ name = \"xdiff\"\n version = \"0.1.0\"\n dependencies = [\n \"interop\",\n+ \"xxhash-rust\",\n ]\n+\n+[[package]]\n+name = \"xxhash-rust\"\n+version = \"0.8.15\"\n+source = \"registry+https://github.com/rust-lang/crates.io-index\"\n+checksum = \"fdd20c5420375476fbd4394763288da7eb0cc0b8c11deed431a91562af7335d3\"\ndiff --git a/rust/xdiff/Cargo.toml b/rust/xdiff/Cargo.toml\nindex eb7966aada64..1516e829db18 100644\n--- a/rust/xdiff/Cargo.toml\n+++ b/rust/xdiff/Cargo.toml\n@@ -13,3 +13,4 @@ crate-type = [\"staticlib\", \"rlib\"]\n \n [dependencies]\n interop = { path = \"../interop\" }\n+xxhash-rust = { version = \"0.8.15\", features = [\"xxh3\"] }\ndiff --git a/rust/xdiff/src/lib.rs b/rust/xdiff/src/lib.rs\nindex e69de29bb2d1..96975975a1ba 100644\n--- a/rust/xdiff/src/lib.rs\n+++ b/rust/xdiff/src/lib.rs\n@@ -0,0 +1,7 @@\n+\n+\n+#[no_mangle]\n+unsafe extern \"C\" fn xxh3_64(ptr: *const u8, size: usize) -> u64 {\n+ let slice = std::slice::from_raw_parts(ptr, size);\n+ xxhash_rust::xxh3::xxh3_64(slice)\n+}\ndiff --git a/xdiff/xprepare.c b/xdiff/xprepare.c\nindex c44005e9bbb8..5a2e52f102cf 100644\n--- a/xdiff/xprepare.c\n+++ b/xdiff/xprepare.c\n@@ -160,6 +160,9 @@ static void xdl_parse_lines(mmfile_t *mf, long narec, xdfile_t *xdf) {\n }\n \n \n+extern u64 xxh3_64(u8 const* ptr, usize size);\n+\n+\n static int xdl_prepare_ctx(unsigned int pass, mmfile_t *mf, long narec, xpparam_t const *xpp,\n \t\t\t xdlclassifier_t *cf, xdfile_t *xdf) {\n \tunsigned long *ha;\n@@ -175,14 +178,26 @@ static int xdl_prepare_ctx(unsigned int pass, mmfile_t *mf, long narec, xpparam_\n \n \txdl_parse_lines(mf, narec, xdf);\n \n+\tif ((xpp->flags & XDF_WHITESPACE_FLAGS) == 0) {\n+\t\tfor (usize i = 0; i < (usize) xdf->nrec; i++) {\n+\t\t\txrecord_t *rec = xdf->recs[i];\n+\t\t\trec->ha = xxh3_64(rec->ptr, rec->size);\n+\t\t}\n+\t} else {\n+\t\tfor (usize i = 0; i < (usize) xdf->nrec; i++) {\n+\t\t\txrecord_t *rec = xdf->recs[i];\n+\t\t\tchar const* dump = (char const*) rec->ptr;\n+\t\t\trec->ha = xdl_hash_record(&dump, (char const*) (rec->ptr + rec->size), xpp->flags);\n+\t\t}\n+\t}\n+\n \tfor (usize i = 0; i < (usize) xdf->nrec; i++) {\n \t\txrecord_t *rec = xdf->recs[i];\n-\t\tchar const* dump = (char const*) rec->ptr;\n-\t\trec->ha = xdl_hash_record(&dump, (char const*) (rec->ptr + rec->size), xpp->flags);\n \t\txdl_classify_record(pass, cf, rec);\n \t}\n \n \n+\n \tif (!XDL_CALLOC_ARRAY(rchg, xdf->nrec + 2))\n \t\tgoto abort;\n \n-- \ngitgitgadget\n\n"},{"id":"522239","messageId":"0de0867ab44f316911bd34b9ceddbc8606e938f2.1752784344.git.gitgitgadget@gmail.com","threadId":"63804","inReplyTo":"pull.1980.git.git.1752784344.gitgitgadget@gmail.com","subject":"[PATCH 7/7] github_workflows: install rust","fromName":"Ezekiel Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2025-07-17T20:32:24Z","receivedAt":"2025-07-17T20:32:35Z","isPatch":true,"sender":{"key":"ezekielnewren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/18313324?v=4"},"body":"From: Ezekiel Newren \n\nSince we have introduced rust, it needs to be installed for the\ncontinuous integration build targets. Create an install script\n(build_rust.sh) that needs to be run as the same user that builds git.\nBecause of the limitations of meson, create build_rust.sh which makes\nit easy to centralize how rust is built between meson and make.\n\nThere are 2 interesting decisions worth calling out in this commit:\n\n* The 'output' field of custom_target() does not allow specifying a\n file nested inside the build directory. Thus create build_rust.sh to\n build rust with all of its parameters and then moves libxdiff.a to\n the root of the build directory.\n\n* Install curl, to facilitate the rustup install script.\n\nSigned-off-by: Ezekiel Newren \n---\n .github/workflows/main.yml | 1 +\n .gitignore | 1 +\n Makefile | 46 +++++++++++++++++++----------\n build_rust.sh | 59 ++++++++++++++++++++++++++++++++++++++\n ci/install-dependencies.sh | 14 ++++-----\n ci/install-rust.sh | 33 +++++++++++++++++++++\n ci/lib.sh | 8 ++++++\n ci/make-test-artifacts.sh | 7 +++++\n ci/run-build-and-tests.sh | 10 +++++++\n meson.build | 40 +++++++++++---------------\n 10 files changed, 173 insertions(+), 46 deletions(-)\n create mode 100755 build_rust.sh\n create mode 100644 ci/install-rust.sh\n\ndiff --git a/.github/workflows/main.yml b/.github/workflows/main.yml\nindex 7dbf9f7f123c..8aac18a6ba45 100644\n--- a/.github/workflows/main.yml\n+++ b/.github/workflows/main.yml\n@@ -4,6 +4,7 @@ on: [push, pull_request]\n \n env:\n DEVELOPER: 1\n+ RUST_VERSION: 1.87.0\n \n # If more than one workflow run is triggered for the very same commit hash\n # (which happens when multiple branches pointing to the same commit), only\ndiff --git a/.gitignore b/.gitignore\nindex 04c444404e4b..a1c0d212541e 100644\n--- a/.gitignore\n+++ b/.gitignore\n@@ -254,3 +254,4 @@ Release/\n /contrib/buildsystems/out\n /contrib/libgit-rs/target\n /contrib/libgit-sys/target\n+/rust/target\ndiff --git a/Makefile b/Makefile\nindex db39e6e1c28e..e659b6eefe82 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -919,11 +919,29 @@ TEST_SHELL_PATH = $(SHELL_PATH)\n \n LIB_FILE = libgit.a\n XDIFF_LIB = xdiff/lib.a\n+\n+EXTLIBS =\n+\n ifeq ($(DEBUG), 1)\n-RUST_LIB = rust/target/debug/libxdiff.a\n+ RUST_BUILD_MODE = debug\n else\n-RUST_LIB = rust/target/release/libxdiff.a\n+ RUST_BUILD_MODE = release\n+endif\n+\n+RUST_TARGET_DIR = rust/target/$(RUST_BUILD_MODE)\n+RUST_FLAGS_FOR_C = -L$(RUST_TARGET_DIR)\n+\n+.PHONY: compile_rust\n+compile_rust:\n+\t./build_rust.sh . $(RUST_BUILD_MODE) xdiff\n+\n+EXTLIBS += ./$(RUST_TARGET_DIR)/libxdiff.a\n+\n+UNAME_S := $(shell uname -s)\n+ifeq ($(UNAME_S),Linux)\n+ EXTLIBS += -ldl\n endif\n+\n REFTABLE_LIB = reftable/libreftable.a\n \n GENERATED_H += command-list.h\n@@ -1395,9 +1413,7 @@ UNIT_TEST_OBJS += $(UNIT_TEST_DIR)/lib-reftable.o\n \n # xdiff and reftable libs may in turn depend on what is in libgit.a\n GITLIBS = common-main.o $(LIB_FILE) $(XDIFF_LIB) $(REFTABLE_LIB) $(LIB_FILE)\n-EXTLIBS =\n \n-GITLIBS += $(RUST_LIB)\n \n GIT_USER_AGENT = git/$(GIT_VERSION)\n \n@@ -2548,7 +2564,7 @@ git.sp git.s git.o: EXTRA_CPPFLAGS = \\\n \t'-DGIT_MAN_PATH=\"$(mandir_relative_SQ)\"' \\\n \t'-DGIT_INFO_PATH=\"$(infodir_relative_SQ)\"'\n \n-git$X: git.o GIT-LDFLAGS $(BUILTIN_OBJS) $(GITLIBS)\n+git$X: git.o GIT-LDFLAGS $(BUILTIN_OBJS) $(GITLIBS) compile_rust\n \t$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) \\\n \t\t$(filter %.o,$^) $(LIBS)\n \n@@ -2898,17 +2914,17 @@ headless-git.o: compat/win32/headless.c GIT-CFLAGS\n headless-git$X: headless-git.o git.res GIT-LDFLAGS\n \t$(QUIET_LINK)$(CC) $(ALL_CFLAGS) $(ALL_LDFLAGS) -mwindows -o $@ $< git.res\n \n-git-%$X: %.o GIT-LDFLAGS $(GITLIBS)\n+git-%$X: %.o GIT-LDFLAGS $(GITLIBS) compile_rust\n \t$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) $(LIBS)\n \n-git-imap-send$X: imap-send.o $(IMAP_SEND_BUILDDEPS) GIT-LDFLAGS $(GITLIBS)\n+git-imap-send$X: imap-send.o $(IMAP_SEND_BUILDDEPS) GIT-LDFLAGS $(GITLIBS) compile_rust\n \t$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \\\n \t\t$(IMAP_SEND_LDFLAGS) $(LIBS)\n \n-git-http-fetch$X: http.o http-walker.o http-fetch.o GIT-LDFLAGS $(GITLIBS)\n+git-http-fetch$X: http.o http-walker.o http-fetch.o GIT-LDFLAGS $(GITLIBS) compile_rust\n \t$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \\\n \t\t$(CURL_LIBCURL) $(LIBS)\n-git-http-push$X: http.o http-push.o GIT-LDFLAGS $(GITLIBS)\n+git-http-push$X: http.o http-push.o GIT-LDFLAGS $(GITLIBS) compile_rust\n \t$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \\\n \t\t$(CURL_LIBCURL) $(EXPAT_LIBEXPAT) $(LIBS)\n \n@@ -2918,11 +2934,11 @@ $(REMOTE_CURL_ALIASES): $(REMOTE_CURL_PRIMARY)\n \tln -s $< $@ 2>/dev/null || \\\n \tcp $< $@\n \n-$(REMOTE_CURL_PRIMARY): remote-curl.o http.o http-walker.o GIT-LDFLAGS $(GITLIBS)\n+$(REMOTE_CURL_PRIMARY): remote-curl.o http.o http-walker.o GIT-LDFLAGS $(GITLIBS) compile_rust\n \t$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \\\n \t\t$(CURL_LIBCURL) $(EXPAT_LIBEXPAT) $(LIBS)\n \n-scalar$X: scalar.o GIT-LDFLAGS $(GITLIBS)\n+scalar$X: scalar.o GIT-LDFLAGS $(GITLIBS) compile_rust\n \t$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) \\\n \t\t$(filter %.o,$^) $(LIBS)\n \n@@ -3309,7 +3325,7 @@ perf: all\n \n t/helper/test-tool$X: $(patsubst %,t/helper/%,$(TEST_BUILTINS_OBJS)) $(UNIT_TEST_DIR)/test-lib.o\n \n-t/helper/test-%$X: t/helper/test-%.o GIT-LDFLAGS $(GITLIBS)\n+t/helper/test-%$X: t/helper/test-%.o GIT-LDFLAGS $(GITLIBS) compile_rust\n \t$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) $(filter %.a,$^) $(LIBS)\n \n check-sha1:: t/helper/test-tool$X\n@@ -3929,13 +3945,13 @@ FUZZ_CXXFLAGS ?= $(ALL_CFLAGS)\n .PHONY: fuzz-all\n fuzz-all: $(FUZZ_PROGRAMS)\n \n-$(FUZZ_PROGRAMS): %: %.o oss-fuzz/dummy-cmd-main.o $(GITLIBS) GIT-LDFLAGS\n+$(FUZZ_PROGRAMS): %: %.o oss-fuzz/dummy-cmd-main.o $(GITLIBS) GIT-LDFLAGS compile_rust\n \t$(QUIET_LINK)$(FUZZ_CXX) $(FUZZ_CXXFLAGS) -o $@ $(ALL_LDFLAGS) \\\n \t\t-Wl,--allow-multiple-definition \\\n \t\t$(filter %.o,$^) $(filter %.a,$^) $(LIBS) $(LIB_FUZZING_ENGINE)\n \n $(UNIT_TEST_PROGS): $(UNIT_TEST_BIN)/%$X: $(UNIT_TEST_DIR)/%.o $(UNIT_TEST_OBJS) \\\n-\t$(GITLIBS) GIT-LDFLAGS\n+\t$(GITLIBS) GIT-LDFLAGS compile_rust\n \t$(call mkdir_p_parent_template)\n \t$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) \\\n \t\t$(filter %.o,$^) $(filter %.a,$^) $(LIBS)\n@@ -3954,7 +3970,7 @@ $(UNIT_TEST_DIR)/clar.suite: $(UNIT_TEST_DIR)/clar-decls.h $(UNIT_TEST_DIR)/gene\n $(UNIT_TEST_DIR)/clar/clar.o: $(UNIT_TEST_DIR)/clar.suite\n $(CLAR_TEST_OBJS): $(UNIT_TEST_DIR)/clar-decls.h\n $(CLAR_TEST_OBJS): EXTRA_CPPFLAGS = -I$(UNIT_TEST_DIR)\n-$(CLAR_TEST_PROG): $(UNIT_TEST_DIR)/clar.suite $(CLAR_TEST_OBJS) $(GITLIBS) GIT-LDFLAGS\n+$(CLAR_TEST_PROG): $(UNIT_TEST_DIR)/clar.suite $(CLAR_TEST_OBJS) $(GITLIBS) GIT-LDFLAGS compile_rust\n \t$(call mkdir_p_parent_template)\n \t$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) $(LIBS)\n \ndiff --git a/build_rust.sh b/build_rust.sh\nnew file mode 100755\nindex 000000000000..4c12135cd205\n--- /dev/null\n+++ b/build_rust.sh\n@@ -0,0 +1,59 @@\n+#!/bin/sh\n+\n+if [ -z \"$CARGO_HOME\" ]; then\n+ export CARGO_HOME=$HOME/.cargo\n+ echo >&2 \"::warning:: CARGO_HOME is not set\"\n+fi\n+echo \"CARGO_HOME=$CARGO_HOME\"\n+\n+rustc -vV\n+cargo --version\n+\n+dir_git_root=${0%/*}\n+dir_build=$1\n+rust_target=$2\n+crate=$3\n+\n+dir_rust=$dir_git_root/rust\n+\n+if [ \"$dir_git_root\" = \"\" ]; then\n+ echo \"did not specify the directory for the root of git\"\n+ exit 1\n+fi\n+\n+if [ \"$dir_build\" = \"\" ]; then\n+ echo \"did not specify the build directory\"\n+ exit 1\n+fi\n+\n+if [ \"$rust_target\" = \"\" ]; then\n+ echo \"did not specify the rust_target\"\n+ exit 1\n+fi\n+\n+if [ \"$rust_target\" = \"release\" ]; then\n+ rust_args=\"--release\"\n+ export RUSTFLAGS='-Aunused_imports -Adead_code'\n+elif [ \"$rust_target\" = \"debug\" ]; then\n+ rust_args=\"\"\n+ export RUSTFLAGS='-Aunused_imports -Adead_code -C debuginfo=2 -C opt-level=1 -C force-frame-pointers=yes'\n+else\n+ echo \"illegal rust_target value $rust_target\"\n+ exit 1\n+fi\n+\n+cd $dir_rust && cargo clean && pwd && cargo build -p $crate $rust_args; cd ..\n+\n+libfile=\"lib${crate}.a\"\n+dst=$dir_build/$libfile\n+\n+if [ \"$dir_git_root\" != \"$dir_build\" ]; then\n+ src=$dir_rust/target/$rust_target/$libfile\n+ if [ ! -f $src ]; then\n+ echo >&2 \"::error:: cannot find path of static library\"\n+ exit 5\n+ fi\n+\n+ rm $dst 2>/dev/null\n+ mv $src $dst\n+fi\ndiff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh\nindex d061a4729339..7801075821ba 100755\n--- a/ci/install-dependencies.sh\n+++ b/ci/install-dependencies.sh\n@@ -24,14 +24,14 @@ fi\n \n case \"$distro\" in\n alpine-*)\n-\tapk add --update shadow sudo meson ninja-build gcc libc-dev curl-dev openssl-dev expat-dev gettext \\\n+\tapk add --update shadow sudo meson ninja-build gcc libc-dev curl curl-dev openssl-dev expat-dev gettext \\\n \t\tzlib-ng-dev pcre2-dev python3 musl-libintl perl-utils ncurses \\\n \t\tapache2 apache2-http2 apache2-proxy apache2-ssl apache2-webdav apr-util-dbd_sqlite3 \\\n \t\tbash cvs gnupg perl-cgi perl-dbd-sqlite perl-io-tty >/dev/null\n \t;;\n fedora-*|almalinux-*)\n \tdnf -yq update >/dev/null &&\n-\tdnf -yq install shadow-utils sudo make gcc findutils diffutils perl python3 gawk gettext zlib-devel expat-devel openssl-devel curl-devel pcre2-devel >/dev/null\n+\tdnf -yq install shadow-utils sudo make gcc findutils diffutils perl python3 gawk gettext zlib-devel expat-devel openssl-devel curl curl-devel pcre2-devel >/dev/null\n \t;;\n ubuntu-*|i386/ubuntu-*|debian-*)\n \t# Required so that apt doesn't wait for user input on certain packages.\n@@ -55,8 +55,8 @@ ubuntu-*|i386/ubuntu-*|debian-*)\n \tsudo apt-get -q update\n \tsudo apt-get -q -y install \\\n \t\t$LANGUAGES apache2 cvs cvsps git gnupg $SVN \\\n-\t\tmake libssl-dev libcurl4-openssl-dev libexpat-dev wget sudo default-jre \\\n-\t\ttcl tk gettext zlib1g-dev perl-modules liberror-perl libauthen-sasl-perl \\\n+\t\tmake libssl-dev curl libcurl4-openssl-dev libexpat-dev wget sudo default-jre \\\n+\t\ttcl tk gettext zlib1g zlib1g-dev perl-modules liberror-perl libauthen-sasl-perl \\\n \t\tlibemail-valid-perl libio-pty-perl libio-socket-ssl-perl libnet-smtp-ssl-perl libdbd-sqlite3-perl libcgi-pm-perl \\\n \t\tlibsecret-1-dev libpcre2-dev meson ninja-build pkg-config \\\n \t\t${CC_PACKAGE:-${CC:-gcc}} $PYTHON_PACKAGE\n@@ -121,13 +121,13 @@ ClangFormat)\n \t;;\n StaticAnalysis)\n \tsudo apt-get -q update\n-\tsudo apt-get -q -y install coccinelle libcurl4-openssl-dev libssl-dev \\\n+\tsudo apt-get -q -y install coccinelle curl libcurl4-openssl-dev libssl-dev \\\n \t\tlibexpat-dev gettext make\n \t;;\n sparse)\n \tsudo apt-get -q update -q\n-\tsudo apt-get -q -y install libssl-dev libcurl4-openssl-dev \\\n-\t\tlibexpat-dev gettext zlib1g-dev sparse\n+\tsudo apt-get -q -y install libssl-dev curl libcurl4-openssl-dev \\\n+\t\tlibexpat-dev gettext zlib1g zlib1g-dev sparse\n \t;;\n Documentation)\n \tsudo apt-get -q update\ndiff --git a/ci/install-rust.sh b/ci/install-rust.sh\nnew file mode 100644\nindex 000000000000..141ceddb17cf\n--- /dev/null\n+++ b/ci/install-rust.sh\n@@ -0,0 +1,33 @@\n+#!/bin/sh\n+\n+if [ \"$(id -u)\" -eq 0 ]; then\n+ echo >&2 \"::warning:: installing rust as root\"\n+fi\n+\n+if [ \"$CARGO_HOME\" = \"\" ]; then\n+ echo >&2 \"::warning:: CARGO_HOME is not set\"\n+ export CARGO_HOME=$HOME/.cargo\n+fi\n+\n+export RUSTUP_HOME=$CARGO_HOME\n+\n+if [ \"$RUST_VERSION\" = \"\" ]; then\n+ echo >&2 \"::error:: RUST_VERSION is not set\"\n+ exit 2\n+fi\n+\n+## install rustup\n+curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- --default-toolchain none -y\n+if [ ! -f $CARGO_HOME/env ]; then\n+ echo \"PATH=$CARGO_HOME/bin:\\$PATH\" > $CARGO_HOME/env\n+fi\n+## install a specific version of rust\n+if [ \"$BITNESS\" = \"32\" ]; then\n+ $CARGO_HOME/bin/rustup set default-host i686-unknown-linux-gnu || exit $?\n+ $CARGO_HOME/bin/rustup install $RUST_VERSION || exit $?\n+ $CARGO_HOME/bin/rustup default --force-non-host $RUST_VERSION || exit $?\n+else\n+ $CARGO_HOME/bin/rustup default $RUST_VERSION || exit $?\n+fi\n+\n+. $CARGO_HOME/env\ndiff --git a/ci/lib.sh b/ci/lib.sh\nindex f561884d4016..ad0e49a68dcb 100755\n--- a/ci/lib.sh\n+++ b/ci/lib.sh\n@@ -1,5 +1,13 @@\n # Library of functions shared by all CI scripts\n \n+\n+export BITNESS=\"64\"\n+if command -v getconf >/dev/null && [ \"$(getconf LONG_BIT 2>/dev/null)\" = \"32\" ]; then\n+ export BITNESS=\"32\"\n+fi\n+echo \"BITNESS=$BITNESS\"\n+\n+\n if test true = \"$GITHUB_ACTIONS\"\n then\n \tbegin_group () {\ndiff --git a/ci/make-test-artifacts.sh b/ci/make-test-artifacts.sh\nindex 74141af0cc74..56aa7efb1d53 100755\n--- a/ci/make-test-artifacts.sh\n+++ b/ci/make-test-artifacts.sh\n@@ -7,6 +7,13 @@ mkdir -p \"$1\" # in case ci/lib.sh decides to quit early\n \n . ${0%/*}/lib.sh\n \n+## install rust per user rather than system wide\n+. ${0%/*}/install-rust.sh\n+\n group Build make artifacts-tar ARTIFACTS_DIRECTORY=\"$1\"\n \n+if [ -d \"$CARGO_HOME\" ]; then\n+ rm -rf $CARGO_HOME\n+fi\n+\n check_unignored_build_artifacts\ndiff --git a/ci/run-build-and-tests.sh b/ci/run-build-and-tests.sh\nindex 01823fd0f140..dbab1cb2f936 100755\n--- a/ci/run-build-and-tests.sh\n+++ b/ci/run-build-and-tests.sh\n@@ -5,6 +5,12 @@\n \n . ${0%/*}/lib.sh\n \n+## install rust per user rather than system wide\n+. ${0%/*}/install-rust.sh\n+\n+rustc -vV\n+cargo --version || exit $?\n+\n run_tests=t\n \n case \"$jobname\" in\n@@ -72,5 +78,9 @@ case \"$jobname\" in\n \t;;\n esac\n \n+if [ -d \"$CARGO_HOME\" ]; then\n+ rm -rf $CARGO_HOME\n+fi\n+\n check_unignored_build_artifacts\n save_good_tree\ndiff --git a/meson.build b/meson.build\nindex 2d8da17f6515..047d7e5b6630 100644\n--- a/meson.build\n+++ b/meson.build\n@@ -277,26 +277,17 @@ else\n rustflags = '-Aunused_imports -Adead_code -C debuginfo=2 -C opt-level=1 -C force-frame-pointers=yes'\n endif\n \n-\n-rust_leaf = custom_target('rust_leaf',\n+rust_build_xdiff = custom_target('rust_build_xdiff',\n output: 'libxdiff.a',\n build_by_default: true,\n build_always_stale: true,\n- command: ['cargo', 'build',\n- '--manifest-path', meson.project_source_root() / 'rust/Cargo.toml'\n- ] + rust_args,\n- env: {\n- 'RUSTFLAGS': rustflags,\n- },\n+ command: [\n+ meson.project_source_root() / 'build_rust.sh',\n+ meson.current_build_dir(), rust_target, 'xdiff',\n+ ],\n install: false,\n )\n \n-rust_xdiff_dep = declare_dependency(\n- link_args: ['-L' + meson.project_source_root() / 'rust/target' / rust_target, '-lxdiff'],\n-# include_directories: include_directories('xdiff/include'), # Adjust if you expose headers\n-)\n-\n-\n compiler = meson.get_compiler('c')\n \n libgit_sources = [\n@@ -1707,17 +1698,18 @@ version_def_h = custom_target(\n )\n libgit_sources += version_def_h\n \n-libgit_dependencies += rust_xdiff_dep\n-\n libgit = declare_dependency(\n- link_with: static_library('git',\n- sources: libgit_sources,\n- c_args: libgit_c_args + [\n- '-DGIT_VERSION_H=\"' + version_def_h.full_path() + '\"',\n- ],\n- dependencies: libgit_dependencies,\n- include_directories: libgit_include_directories,\n- ),\n+ link_with: [\n+ static_library('git',\n+ sources: libgit_sources,\n+ c_args: libgit_c_args + [\n+ '-DGIT_VERSION_H=\"' + version_def_h.full_path() + '\"',\n+ ],\n+ dependencies: libgit_dependencies,\n+ include_directories: libgit_include_directories,\n+ ),\n+ rust_build_xdiff,\n+ ],\n compile_args: libgit_c_args,\n dependencies: libgit_dependencies,\n include_directories: libgit_include_directories,\n-- \ngitgitgadget\n"},{"id":"522241","messageId":"aHlp1joMwexLZAAb@fruit.crustytoothpaste.net","threadId":"63804","inReplyTo":"0de0867ab44f316911bd34b9ceddbc8606e938f2.1752784344.git.gitgitgadget@gmail.com","subject":"Re: [PATCH 7/7] github_workflows: install rust","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2025-07-17T21:23:34Z","receivedAt":"2025-07-17T21:23:41Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On 2025-07-17 at 20:32:24, Ezekiel Newren via GitGitGadget wrote:\n> diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml\n> index 7dbf9f7f123c..8aac18a6ba45 100644\n> --- a/.github/workflows/main.yml\n> +++ b/.github/workflows/main.yml\n> @@ -4,6 +4,7 @@ on: [push, pull_request]\n> \n> env:\n> DEVELOPER: 1\n> + RUST_VERSION: 1.87.0\n\nOur discussed plan is to support the version in Debian stable, plus a\nyear. So we'd be supporting 1.63.0 for a year after trixie's release.\n\nThe reason for that is that people build backports and security updates\nfor Git for stable releases of distros and they will use the distro\ntoolchain for doing so. Forcing distros to constantly build with the\nlatest toolchain is pretty hostile, especially since the lifespan of\nRust release is six weeks.\n\nIf the Rust project provides LTS releases in the future, then we can\nconsider adopting those.\n\n> +if [ \"$rust_target\" = \"release\" ]; then\n> + rust_args=\"--release\"\n> + export RUSTFLAGS='-Aunused_imports -Adead_code'\n> +elif [ \"$rust_target\" = \"debug\" ]; then\n> + rust_args=\"\"\n> + export RUSTFLAGS='-Aunused_imports -Adead_code -C debuginfo=2 -C opt-level=1 -C force-frame-pointers=yes'\n\nCan you say a little about why these options are needed and the defaults\nare inadequate? For instance, I build with the default options both in\nmy personal projects and at work and don't see a problem.\n\nI don't know if you plan to do this in a future series, but we'd also\nwant cargo's tests to be run as part of CI and we'd want a lint job that\nran clippy with both 1.63.0 and the latest stable version of Rust to\nmake sure things were tidy.\n-- \nbrian m. carlson (they/them)\nToronto, Ontario, CA\n"},{"id":"522242","messageId":"aHlrg7pbFqi2qNWH@fruit.crustytoothpaste.net","threadId":"63804","inReplyTo":"2a1f4be13dfbdee21811b7a4907f99042c791c2d.1752784344.git.gitgitgadget@gmail.com","subject":"Re: [PATCH 1/7] xdiff: introduce rust","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2025-07-17T21:30:43Z","receivedAt":"2025-07-17T21:30:45Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On 2025-07-17 at 20:32:18, Ezekiel Newren via GitGitGadget wrote:\n> diff --git a/rust/Cargo.lock b/rust/Cargo.lock\n> new file mode 100644\n> index 000000000000..fb1eac690b39\n> --- /dev/null\n> +++ b/rust/Cargo.lock\n> @@ -0,0 +1,14 @@\n> +# This file is automatically @generated by Cargo.\n> +# It is not intended for manual editing.\n> +version = 4\n> +\n> +[[package]]\n> +name = \"interop\"\n> +version = \"0.1.0\"\n> +\n> +[[package]]\n> +name = \"xdiff\"\n> +version = \"0.1.0\"\n> +dependencies = [\n> + \"interop\",\n> +]\n\nI would prefer that we not check in Cargo.lock in Git. Part of the\nreason is that it changes across versions and so building with a\ndifferent version of the toolchain can update the file.\n\nIn addition, as I mentioned downthread, because our intention is to\nsupport the Debian stable toolchain for a year after the new stable\nrelease, unless we are exceptionally careful about dependencies, we may\nend up with a case where distros need to use older dependencies patched\nfor security but other users may want to update the versions to newer\ndependencies with security fixes but that do not work on our pinned Rust\nversion. We can't possibly satisfy both sets of people if we pin\ndependencies in Cargo.lock, so we probably want to avoid checking it in\nand ignore it instead.\n-- \nbrian m. carlson (they/them)\nToronto, Ontario, CA\n"},{"id":"522243","messageId":"aHlwZPbiKnakMN75@fruit.crustytoothpaste.net","threadId":"63804","inReplyTo":"pull.1980.git.git.1752784344.gitgitgadget@gmail.com","subject":"Re: [PATCH 0/7] RFC: Accelerate xdiff and begin its rustification","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2025-07-17T21:51:32Z","receivedAt":"2025-07-17T21:51:34Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On 2025-07-17 at 20:32:17, Ezekiel Newren via GitGitGadget wrote:\n> This series accelerates xdiff by 5-19%.\n\nThat's great.\n\n> It also introduces Rust as a hard dependency.\n\nI think that's fine. We already discussed doing this at the last\nContributor Summit in Berlin and everyone was in favour. While we did\nnot have every contributor represented, I think that unanimity of the\ncontributors present is a compelling enough reason.\n\n> …and it doesn’t yet pass a couple of the github workflows; hints from\n> Windows experts, and opinions on ambiguous primitives would be appreciated\n> (see below).\n> \n> This is just the beginning of many patches that I have to convert portions\n> of, maybe eventually all of, xdiff to Rust. While working on that\n> conversion, I found several ways to clarify the code, along with some\n> optimizations.\n> \n> So...\n> \n> This obviously raises the question of whether we are ready to accept a hard\n> dependency on Rust. Previous discussions on the mailing list and at Git\n> Merge 2024 have not answered that question. If not now, will we be willing\n> to accept such a hard dependency later? And what route do we want to take to\n> get there?\n\nAgain, I think that's fine.\n\nI have a proposed policy at [0] (available from the `rust` branch on\nthat remote). Included in that policy is a link to [1], which I\nsummarize as \"the U.S. government is proposing to classify development\nin memory unsafe languages as a Product Security Bad Practice.\" The\nproposal is that a memory safety roadmap be introduced by the end of\n2025.\n\nNow, do let me be clear that I don't agree with everything that the U.S.\ngovernment says or does (far from it), but I do think this is a sensible\nproposal (or I wouldn't have cited it) and it will be showing up in a\nlot more security standards coming down the line, especially for those\nforges and companies which will be selling to governments around the\nworld. There's no time like the present to do this.\n\nI realize that that means that we will lose support for some platforms.\nI ultimately think that it's up to the porters and maintainers for a\nplatform to maintain appropriate toolchains on that platform and that,\nwhile we should be cognizant of the requirements for adding new\nplatforms or architectures, that shouldn't prevent the inclusion of\nimportant new tools like memory-safe languages.\n\nI would like to see a change to our platform policy and a policy on Rust\nbefore we merge this. I know your series adds support for 1.87, but\nbecause distros don't run the latest toolchain, we had discussed in the\npast targeting Debian stable's version plus an additional year after the\nnew release. This means we'll support a Rust version for three years,\nwhich is a reasonable amount of time for a toolchain and allows distros\nto easily backport security fixes.\n\nIf you would like, you are welcome to use my proposed policy as a basis\nfor this, or I can send that out as a separate document if you don't\nwant to write one or revise mine.\n\n[0] https://github.com/bk2204/git/commit/fbeb1180c7473635a964daed2da642c53487782d\n[1] https://www.cisa.gov/resources-tools/resources/product-security-bad-practices\n-- \nbrian m. carlson (they/them)\nToronto, Ontario, CA\n"},{"id":"522244","messageId":"xmqq1pqe5vpv.fsf@gitster.g","threadId":"63804","inReplyTo":"aHlrg7pbFqi2qNWH@fruit.crustytoothpaste.net","subject":"Re: [PATCH 1/7] xdiff: introduce rust","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-07-17T21:54:52Z","receivedAt":"2025-07-17T21:54:55Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"brian m. carlson\" writes:\n\n>> +# This file is automatically @generated by Cargo.\n>> +# It is not intended for manual editing.\n>> +version = 4\n>> +\n>> +[[package]]\n>> +name = \"interop\"\n>> +version = \"0.1.0\"\n>> +\n>> +[[package]]\n>> +name = \"xdiff\"\n>> +version = \"0.1.0\"\n>> +dependencies = [\n>> + \"interop\",\n>> +]\n>\n> I would prefer that we not check in Cargo.lock in Git. Part of the\n> reason is that it changes across versions and so building with a\n> different version of the toolchain can update the file.\n>\n> In addition, as I mentioned downthread, because our intention is to\n> support the Debian stable toolchain for a year after the new stable\n> release, unless we are exceptionally careful about dependencies, we may\n> end up with a case where distros need to use older dependencies patched\n> for security but other users may want to update the versions to newer\n> dependencies with security fixes but that do not work on our pinned Rust\n> version. We can't possibly satisfy both sets of people if we pin\n> dependencies in Cargo.lock, so we probably want to avoid checking it in\n> and ignore it instead.\n\nYup. \n\nThe comment in first few lines of the file says it very well ;-)\nThanks for flagging it.\n\n"},{"id":"522249","messageId":"aHl4U98BBvpA5eKF@nand.local","threadId":"63804","inReplyTo":"aHlwZPbiKnakMN75@fruit.crustytoothpaste.net","subject":"Re: [PATCH 0/7] RFC: Accelerate xdiff and begin its rustification","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2025-07-17T22:25:23Z","receivedAt":"2025-07-17T22:25:26Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"On Thu, Jul 17, 2025 at 09:51:32PM +0000, brian m. carlson wrote:\n> On 2025-07-17 at 20:32:17, Ezekiel Newren via GitGitGadget wrote:\n> > This series accelerates xdiff by 5-19%.\n>\n> That's great.\n>\n> > It also introduces Rust as a hard dependency.\n>\n> I think that's fine. We already discussed doing this at the last\n> Contributor Summit in Berlin and everyone was in favour. While we did\n> not have every contributor represented, I think that unanimity of the\n> contributors present is a compelling enough reason.\n\nI agree. I don't think that there is ever going to be a \"perfect\" time\nto introduce a hard dependency on Rust, and I don't think that should\nhold the project back from adopting it.\n\nI am far from a Rust expert, but I think that a more modern, memory-safe\nlanguage will attract newer contributors who may have a fresher\nperspective on the project, and I think that's a good thing.\n\nThe alternative, of course, is to continue to use C and not take any\ndependency on Rust. I think there is a middle-ground in there somewhere\nto be able to build with (e.g.) \"make\" or \"make RUST=1\", but I would\nreally like to see the project take a firmer stance here.\n\nI worry that having build support for both \"with Rust\" and \"C only\" will\ncreate a headache not just at the build system level, but also in the\ncode itself. Having a patchwork of features, optimizations, or bug fixes\nthat either are or aren't supported depending on whether Rust support\nwas specified at build-time seems like a worst-of-all-worlds outcome.\n\n> I realize that that means that we will lose support for some platforms.\n> I ultimately think that it's up to the porters and maintainers for a\n> platform to maintain appropriate toolchains on that platform and that,\n> while we should be cognizant of the requirements for adding new\n> platforms or architectures, that shouldn't prevent the inclusion of\n> important new tools like memory-safe languages.\n\nAgreed. Of course, I think we would all like Git to be able to build and\nrun on as many platforms as is reasonably possible. But we cannot\nsupport all platforms for all time. It is also not the Git project's\nresponsibility to ensure that every platform is Rust-friendly.\n\nHopefully the platforms that we currently support but won't after this\npatch series have niche enough workloads that they do not need the\nabsolute latest-and-greatest Git release at all times.\n\n> I would like to see a change to our platform policy and a policy on Rust\n> before we merge this. I know your series adds support for 1.87, but\n> because distros don't run the latest toolchain, we had discussed in the\n> past targeting Debian stable's version plus an additional year after the\n> new release. This means we'll support a Rust version for three years,\n> which is a reasonable amount of time for a toolchain and allows distros\n> to easily backport security fixes.\n>\n> If you would like, you are welcome to use my proposed policy as a basis\n> for this, or I can send that out as a separate document if you don't\n> want to write one or revise mine.\n\nYeah, I think that this is the most interesting part of the discussion\nhere. I am not knowledgeable enough about Rust's release cadence and\nplatform compatibility to have an opinion here. But I trust brian's\njudgement ;-).\n\nThanks,\nTaylor\n"},{"id":"522251","messageId":"aHl7W5bDOGsORLLO@nand.local","threadId":"63804","inReplyTo":"2a1f4be13dfbdee21811b7a4907f99042c791c2d.1752784344.git.gitgitgadget@gmail.com","subject":"Re: [PATCH 1/7] xdiff: introduce rust","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2025-07-17T22:38:19Z","receivedAt":"2025-07-17T22:38:23Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"On Thu, Jul 17, 2025 at 08:32:18PM +0000, Ezekiel Newren via GitGitGadget wrote:\n> From: Ezekiel Newren \n>\n> Upcoming patches will accelerate and simplify xdiff, while also\n> porting parts of it to Rust. In preparation, add some stubs and setup\n> the Rust build. For now, it is easier to let cargo build rust and\n> have make or meson merely link against the static library that cargo\n> builds. In line with ongoing libification efforts, use multiple\n> crates to allow more modularity on the Rust side. xdiff is the crate\n> that this series will focus on, but we also introduce the interop\n> crate for future patch series.\n>\n> In order to facilitate interoperability between C and Rust, introduce\n> C definitions for Rust primitive types in git-compat-util.h.\n\nExciting ;-).\n\n> diff --git a/Makefile b/Makefile\n> index 70d1543b6b86..db39e6e1c28e 100644\n> --- a/Makefile\n> +++ b/Makefile\n> @@ -919,6 +919,11 @@ TEST_SHELL_PATH = $(SHELL_PATH)\n>\n> LIB_FILE = libgit.a\n> XDIFF_LIB = xdiff/lib.a\n> +ifeq ($(DEBUG), 1)\n> +RUST_LIB = rust/target/debug/libxdiff.a\n> +else\n> +RUST_LIB = rust/target/release/libxdiff.a\n> +endif\n\nWe do have a DEBUG variable in our Makefile introduced via dce7d29551\n(msvc: support building Git using MS Visual C++, 2019-06-25), but I\ndon't think that it is very widely used. Perhaps that is because I don't\nbuild Git with MSVC, but I suspect that this is generally true.\n\nMuch more common is the DEVELOPER=1 setting, which adds more compiler\nwarnings and similar. I am not sure whether or not it would be\nappropriate to use DEVELOPER here to determine which libxdiff.a to use.\n\nIn any event, our convention would be to treat the defined-ness of DEBUG\nthe same way that this patch treats DEBUG=1, so I might suggest\nreplacing your \"ifeq\" with \"ifdef DEBUG\".\n\n> REFTABLE_LIB = reftable/libreftable.a\n>\n> GENERATED_H += command-list.h\n> @@ -1392,6 +1397,8 @@ UNIT_TEST_OBJS += $(UNIT_TEST_DIR)/lib-reftable.o\n> GITLIBS = common-main.o $(LIB_FILE) $(XDIFF_LIB) $(REFTABLE_LIB) $(LIB_FILE)\n> EXTLIBS =\n>\n> +GITLIBS += $(RUST_LIB)\n> +\n> GIT_USER_AGENT = git/$(GIT_VERSION)\n>\n> ifeq ($(wildcard sha1collisiondetection/lib/sha1.h),sha1collisiondetection/lib/sha1.h)\n> @@ -2925,6 +2932,14 @@ $(LIB_FILE): $(LIB_OBJS)\n> $(XDIFF_LIB): $(XDIFF_OBJS)\n> \t$(QUIET_AR)$(RM) $@ && $(AR) $(ARFLAGS) $@ $^\n>\n> +.PHONY: $(RUST_LIB)\n> +$(RUST_LIB):\n> +ifeq ($(DEBUG), 1)\n> +\tcd rust && RUSTFLAGS=\"-Aunused_imports -Adead_code\" cargo build --verbose\n\nA few thoughts here:\n\n - Does \"cargo\" support a flag similar to our -C? If so, I wonder if it\n might be worth writing \"cargo -C rust build ...\" instead of \"cd rust\n && ...\".\n\n - This conditional on DEBUG passes the \"--verbose\" option in both\n cases. Should we only pass the \"--verbose\" option when we have \"V=1\"?\n\n - Regardless of whether or not we condition passing \"--release\" (or\n not) on \"DEBUG\", this line should also be \"ifdef DEBUG\" similar to\n above.\n\n> +else\n> +\tcd rust && RUSTFLAGS=\"-Aunused_imports -Adead_code\" cargo build --verbose --release\n> +endif\n> +\n> $(REFTABLE_LIB): $(REFTABLE_OBJS)\n> \t$(QUIET_AR)$(RM) $@ && $(AR) $(ARFLAGS) $@ $^\n>\n> @@ -3756,7 +3771,10 @@ cocciclean:\n> \t$(RM) -r .build/contrib/coccinelle\n> \t$(RM) contrib/coccinelle/*.cocci.patch\n>\n> -clean: profile-clean coverage-clean cocciclean\n> +rustclean:\n\nI'm nitpicking, and we don't *really* have a convention here between\nseparating the clean target from \"clean\", as we have both\n\"profile-clean\" and \"cocciclean\". I prefer the former, and think that it\nwould be nice to use that convention, but this is pretty much textbook\nbike-shedding and not something that I really care about ;-).\n\n> +\tcd rust && cargo clean\n\nSame question here about whether or not this could be written as \"cargo\n-C clean\".\n\n> diff --git a/git-compat-util.h b/git-compat-util.h\n> index 4678e21c4cb8..82dc99764ac0 100644\n> --- a/git-compat-util.h\n> +++ b/git-compat-util.h\n> @@ -196,6 +196,23 @@ static inline int is_xplatform_dir_sep(int c)\n> #include \"compat/msvc.h\"\n> #endif\n>\n> +/* rust types */\n> +typedef uint8_t u8;\n> +typedef uint16_t u16;\n> +typedef uint32_t u32;\n> +typedef uint64_t u64;\n> +\n> +typedef int8_t i8;\n> +typedef int16_t i16;\n> +typedef int32_t i32;\n> +typedef int64_t i64;\n> +\n> +typedef float f32;\n> +typedef double f64;\n> +\n> +typedef size_t usize;\n> +typedef ptrdiff_t isize;\n> +\n\nMakes sense. Should we also have \"bool\" here (assuming that the series\ndeclaring the \"use bool\" experiment a success lands)? I guess maybe not\neither way, defines \"bool\" as the type name, identically to\nRust.\n\nThanks,\nTaylor\n"},{"id":"522252","messageId":"aHl7pRs9pgUTXKQk@nand.local","threadId":"63804","inReplyTo":"aHlrg7pbFqi2qNWH@fruit.crustytoothpaste.net","subject":"Re: [PATCH 1/7] xdiff: introduce rust","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2025-07-17T22:39:33Z","receivedAt":"2025-07-17T22:39:36Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"On Thu, Jul 17, 2025 at 09:30:43PM +0000, brian m. carlson wrote:\n> In addition, as I mentioned downthread, because our intention is to\n> support the Debian stable toolchain for a year after the new stable\n> release, unless we are exceptionally careful about dependencies, we may\n> end up with a case where distros need to use older dependencies patched\n> for security but other users may want to update the versions to newer\n> dependencies with security fixes but that do not work on our pinned Rust\n> version.\n\n...or Debian users who have an older version of the toolchain installed\nand got an unfriendly \"cannot parse 'version = 4'\" error when trying to\nbuild with this patch series applied locally ;-).\n\nThanks,\nTaylor\n"},{"id":"522253","messageId":"aHl8FebhIEd5+kah@nand.local","threadId":"63804","inReplyTo":"b0b744b9acf5299d323d56cbcc01411a228c1fc8.1752784344.git.gitgitgadget@gmail.com","subject":"Re: [PATCH 2/7] xdiff/xprepare: remove superfluous forward declarations","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2025-07-17T22:41:25Z","receivedAt":"2025-07-17T22:41:28Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"On Thu, Jul 17, 2025 at 08:32:19PM +0000, Ezekiel Newren via GitGitGadget wrote:\n> ---\n> xdiff/xprepare.c | 116 ++++++++++++++++++++---------------------------\n> 1 file changed, 50 insertions(+), 66 deletions(-)\n\nMakes sense. Reviewing with \"--color-moved\" makes it straightforward to\nsee that the contents of xdl_prepare_env() were not modified by this\npatch.\n\nThanks,\nTaylor\n"},{"id":"522256","messageId":"aHl9YLc823uWwgIp@nand.local","threadId":"63804","inReplyTo":"6df9f50a8f4ca29b2c3ba1e39982b6d516146bb3.1752784344.git.gitgitgadget@gmail.com","subject":"Re: [PATCH 4/7] xdiff: make fields of xrecord_t Rust friendly","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2025-07-17T22:46:56Z","receivedAt":"2025-07-17T22:46:58Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"On Thu, Jul 17, 2025 at 08:32:21PM +0000, Ezekiel Newren via GitGitGadget wrote:\n> From: Ezekiel Newren \n>\n> A few commits ago, we added definitions for Rust primitive types,\n> to facilitate interoperability between C and Rust. Switch a\n> few variables to use these types. Which, for now, will\n> require adding some casts.\n\nHmm, interesting. I am not super familiar with how people typically\nhandle interoperability between C and Rust, but having to change types\non the C side to make it work with Rust is a bit surprising to me.\n\nI would have expected that the Rust side would have declared its types\nusing libc::c_int, libc::size_t, and so on. I think I have a vague\npreference towards putting the burden of casting on the Rust side, but,\nagain, I am not super familiar with how transitions like these are\ntypically approached.\n\n> ---\n> xdiff/xdiffi.c | 8 ++++----\n> xdiff/xemit.c | 2 +-\n> xdiff/xmerge.c | 14 +++++++-------\n> xdiff/xpatience.c | 2 +-\n> xdiff/xprepare.c | 6 +++---\n> xdiff/xtypes.h | 6 +++---\n> xdiff/xutils.c | 4 ++--\n> 7 files changed, 21 insertions(+), 21 deletions(-)\n\nThe rest of the patch looks good to me, assuming that the burden of\ncasting is placed on the C side.\n\nThanks,\nTaylor\n"},{"id":"522257","messageId":"aHmAO0J7Ptr4OiCk@nand.local","threadId":"63804","inReplyTo":"2db30cc739efadf8383bd9dc1b7825ce863e8f5a.1752784344.git.gitgitgadget@gmail.com","subject":"Re: [PATCH 5/7] xdiff: separate parsing lines from hashing them","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2025-07-17T22:59:07Z","receivedAt":"2025-07-17T22:59:10Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"On Thu, Jul 17, 2025 at 08:32:22PM +0000, Ezekiel Newren via GitGitGadget wrote:\n> ---\n> xdiff/xprepare.c | 75 ++++++++++++++++++++++++++++--------------------\n> 1 file changed, 44 insertions(+), 31 deletions(-)\n\nNot being all that familiar with the xdiff code, this patch took me a\nlittle while longer to read and understand, but the transformation looks\ncorrect to me.\n\nThanks,\nTaylor\n"},{"id":"522258","messageId":"aHmDiqsBsDJJ6m8C@fruit.crustytoothpaste.net","threadId":"63804","inReplyTo":"aHl9YLc823uWwgIp@nand.local","subject":"Re: [PATCH 4/7] xdiff: make fields of xrecord_t Rust friendly","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2025-07-17T23:13:14Z","receivedAt":"2025-07-17T23:13:16Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On 2025-07-17 at 22:46:56, Taylor Blau wrote:\n> On Thu, Jul 17, 2025 at 08:32:21PM +0000, Ezekiel Newren via GitGitGadget wrote:\n> > From: Ezekiel Newren \n> >\n> > A few commits ago, we added definitions for Rust primitive types,\n> > to facilitate interoperability between C and Rust. Switch a\n> > few variables to use these types. Which, for now, will\n> > require adding some casts.\n> \n> Hmm, interesting. I am not super familiar with how people typically\n> handle interoperability between C and Rust, but having to change types\n> on the C side to make it work with Rust is a bit surprising to me.\n> \n> I would have expected that the Rust side would have declared its types\n> using libc::c_int, libc::size_t, and so on. I think I have a vague\n> preference towards putting the burden of casting on the Rust side, but,\n> again, I am not super familiar with how transitions like these are\n> typically approached.\n\nRust normally handles byte strings as slices or vectors of u8 (that is,\nC's uint8_t). C handles them as char, which may or may not be unsigned,\nas we all know, which leads to some \"entertaining\" problems from time to\ntime.\n\nAlso, in general, Rust doesn't offer generic system-specific types, such\nas `long`, except for C FFI. This is actually a strong benefit, since\nit means we're not inclined to write `unsigned long` and then wonder why\nthings are broken on Windows: instead, we write either `usize` (the\nequivalent of `size_t`) or `u64` (for things like file sizes). This is\nmuch more ingrained than it is in Go, which has a tendency to use `int`\n(Rust's `isize`) a lot and much less often specific types.\n\nIf we're going to move this code entirely into Rust, then it makes sense\nto cast temporarily, and I'm fine doing that in C, since it's C that has\nthe weird system-dependent behaviour (arbitrary decisions on the\nsignedness of char). That actually allows us to have more confidence in\nthe safety and maintainability of the Rust code since it is less system\ndependent and leave the suspect pieces in C. It may also, interestingly\nenough, also allow us to easily get rid of the weird 2 GB limit on diffs\ndue to the unpleasant dependency on `int` in the xdiff code, which I\nwould absolutely love to see.\n\nHowever, I'm not dead set against casting in Rust if that's what\neveryone else wants instead.\n-- \nbrian m. carlson (they/them)\nToronto, Ontario, CA\n"},{"id":"522259","messageId":"aHmHb99q/if6KTTw@nand.local","threadId":"63804","inReplyTo":"5a959c9bdad79cf972b95dcf4324135dd7c94dac.1752784344.git.gitgitgadget@gmail.com","subject":"Re: [PATCH 6/7] xdiff: conditionally use Rust's implementation of xxhash","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2025-07-17T23:29:51Z","receivedAt":"2025-07-17T23:29:59Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"On Thu, Jul 17, 2025 at 08:32:23PM +0000, Ezekiel Newren via GitGitGadget wrote:\n> // desktop\n> // CPU: 8-core Intel Core i7-9700 (-MCP-) speed/min/max: 800/800/4700 MHz\n> $ hyperfine --warmup 3 -L exe $BASE/build_release/git,$BASE/build_v2.49.0/git '{exe} log --oneline --shortstat v6.8..v6.9 >/dev/null'\n> Benchmark 1: /home/steamuser/dev/git/build_release/git log --oneline --shortstat v6.8..v6.9 >/dev/null\n> Time (mean ± σ): 6.823 s ± 0.020 s [User: 6.624 s, System: 0.180 s]\n> Range (min … max): 6.801 s … 6.858 s 10 runs\n>\n> Benchmark 2: /home/steamuser/dev/git/build_v2.49.0/git log --oneline --shortstat v6.8..v6.9 >/dev/null\n> Time (mean ± σ): 8.151 s ± 0.024 s [User: 7.928 s, System: 0.198 s]\n> Range (min … max): 8.105 s … 8.184 s 10 runs\n>\n> Summary\n> /home/steamuser/dev/git/build_release/git log --oneline --shortstat v6.8..v6.9 >/dev/null ran\n> 1.19 ± 0.01 times faster than /home/steamuser/dev/git/build_v2.49.0/git log --oneline --shortstat v6.8..v6.9 >/dev/null\n\nVery cool!\n\n> Signed-off-by: Ezekiel Newren \n> ---\n> rust/Cargo.lock | 7 +++++++\n> rust/xdiff/Cargo.toml | 1 +\n> rust/xdiff/src/lib.rs | 7 +++++++\n> xdiff/xprepare.c | 19 +++++++++++++++++--\n> 4 files changed, 32 insertions(+), 2 deletions(-)\n\nThis patch is delightfully simple. Thank you for carefully preparing the\nprevious five patches to make this one as tiny as it is.\n\n> @@ -175,14 +178,26 @@ static int xdl_prepare_ctx(unsigned int pass, mmfile_t *mf, long narec, xpparam_\n>\n> \txdl_parse_lines(mf, narec, xdf);\n>\n> +\tif ((xpp->flags & XDF_WHITESPACE_FLAGS) == 0) {\n\nIt may be worth adding a comment here to explain why we aren't using\nxdl_hash_record() when xpp->flags lacks XDF_WHITESPACE_FLAGS.\n\n(As a meta-note for reviewing this series, there are a handful of style\nnits that I haven't mentioned, e.g., if (... == 0) instead of if (!...).\nBut since the xdiff code doesn't match the project's style conventions,\nI have avoided mentioning it in my review.)\n\n> +\t\tfor (usize i = 0; i < (usize) xdf->nrec; i++) {\n> +\t\t\txrecord_t *rec = xdf->recs[i];\n> +\t\t\trec->ha = xxh3_64(rec->ptr, rec->size);\n> +\t\t}\n> +\t} else {\n> +\t\tfor (usize i = 0; i < (usize) xdf->nrec; i++) {\n> +\t\t\txrecord_t *rec = xdf->recs[i];\n> +\t\t\tchar const* dump = (char const*) rec->ptr;\n> +\t\t\trec->ha = xdl_hash_record(&dump, (char const*) (rec->ptr + rec->size), xpp->flags);\n> +\t\t}\n> +\t}\n> +\n> \tfor (usize i = 0; i < (usize) xdf->nrec; i++) {\n> \t\txrecord_t *rec = xdf->recs[i];\n> -\t\tchar const* dump = (char const*) rec->ptr;\n> -\t\trec->ha = xdl_hash_record(&dump, (char const*) (rec->ptr + rec->size), xpp->flags);\n> \t\txdl_classify_record(pass, cf, rec);\n\nI am curious why you are calling xdl_classify_record() here as a\npost-processing step rather than inline with the hash calculation above.\n\nThanks,\nTaylor\n"},{"id":"522260","messageId":"CABPp-BFqruHDF5He=iqrUe9sYJ3XdXBcthY8i_eKMXS9tAa0oA@mail.gmail.com","threadId":"63804","inReplyTo":"aHmDiqsBsDJJ6m8C@fruit.crustytoothpaste.net","subject":"Re: [PATCH 4/7] xdiff: make fields of xrecord_t Rust friendly","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2025-07-17T23:37:24Z","receivedAt":"2025-07-17T23:37:37Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"On Thu, Jul 17, 2025 at 4:13 PM brian m. carlson\n wrote:\n>\n> On 2025-07-17 at 22:46:56, Taylor Blau wrote:\n> > On Thu, Jul 17, 2025 at 08:32:21PM +0000, Ezekiel Newren via GitGitGadget wrote:\n> > > From: Ezekiel Newren \n> > >\n> > > A few commits ago, we added definitions for Rust primitive types,\n> > > to facilitate interoperability between C and Rust. Switch a\n> > > few variables to use these types. Which, for now, will\n> > > require adding some casts.\n> >\n> > Hmm, interesting. I am not super familiar with how people typically\n> > handle interoperability between C and Rust, but having to change types\n> > on the C side to make it work with Rust is a bit surprising to me.\n> >\n> > I would have expected that the Rust side would have declared its types\n> > using libc::c_int, libc::size_t, and so on. I think I have a vague\n> > preference towards putting the burden of casting on the Rust side, but,\n> > again, I am not super familiar with how transitions like these are\n> > typically approached.\n>\n> Rust normally handles byte strings as slices or vectors of u8 (that is,\n> C's uint8_t). C handles them as char, which may or may not be unsigned,\n> as we all know, which leads to some \"entertaining\" problems from time to\n> time.\n>\n> Also, in general, Rust doesn't offer generic system-specific types, such\n> as `long`, except for C FFI. This is actually a strong benefit, since\n> it means we're not inclined to write `unsigned long` and then wonder why\n> things are broken on Windows: instead, we write either `usize` (the\n> equivalent of `size_t`) or `u64` (for things like file sizes). This is\n> much more ingrained than it is in Go, which has a tendency to use `int`\n> (Rust's `isize`) a lot and much less often specific types.\n>\n> If we're going to move this code entirely into Rust, then it makes sense\n> to cast temporarily, and I'm fine doing that in C, since it's C that has\n> the weird system-dependent behaviour (arbitrary decisions on the\n> signedness of char). That actually allows us to have more confidence in\n> the safety and maintainability of the Rust code since it is less system\n> dependent and leave the suspect pieces in C. It may also, interestingly\n> enough, also allow us to easily get rid of the weird 2 GB limit on diffs\n> due to the unpleasant dependency on `int` in the xdiff code, which I\n> would absolutely love to see.\n>\n> However, I'm not dead set against casting in Rust if that's what\n> everyone else wants instead.\n\nIn general, I too would prefer to do the casting on the C side; after\nall, part of the reason for Rust is the language safety, which we\ncompromise if we force it into using ambiguously sized variables.\n\nHowever, I think it might be somewhat case-dependent...\n\nHere, we have C calling into APIs that will be defined and implemented\nin Rust. Further along the road of adopting Rust in more places, we\nmay have future cases where we have Rust calling into APIs defined and\nimplemented in C. I'm wondering if in such a world the rule of thumb\nthat makes the most sense would be to have a\ncaller-must-cast-as-necessary guideline, rather than specifying the\ncasting side by language.\n"},{"id":"522266","messageId":"aHmTpqVDkE4PqG3h@nand.local","threadId":"63804","inReplyTo":"aHmDiqsBsDJJ6m8C@fruit.crustytoothpaste.net","subject":"Re: [PATCH 4/7] xdiff: make fields of xrecord_t Rust friendly","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2025-07-18T00:21:58Z","receivedAt":"2025-07-18T00:22:00Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"On Thu, Jul 17, 2025 at 11:13:14PM +0000, brian m. carlson wrote:\n> On 2025-07-17 at 22:46:56, Taylor Blau wrote:\n> > On Thu, Jul 17, 2025 at 08:32:21PM +0000, Ezekiel Newren via GitGitGadget wrote:\n> > > From: Ezekiel Newren \n> > >\n> > > A few commits ago, we added definitions for Rust primitive types,\n> > > to facilitate interoperability between C and Rust. Switch a\n> > > few variables to use these types. Which, for now, will\n> > > require adding some casts.\n> >\n> > Hmm, interesting. I am not super familiar with how people typically\n> > handle interoperability between C and Rust, but having to change types\n> > on the C side to make it work with Rust is a bit surprising to me.\n> >\n> > I would have expected that the Rust side would have declared its types\n> > using libc::c_int, libc::size_t, and so on. I think I have a vague\n> > preference towards putting the burden of casting on the Rust side, but,\n> > again, I am not super familiar with how transitions like these are\n> > typically approached.\n>\n> Rust normally handles byte strings as slices or vectors of u8 (that is,\n> C's uint8_t). C handles them as char, which may or may not be unsigned,\n> as we all know, which leads to some \"entertaining\" problems from time to\n> time.\n\n;-)\n\n> Also, in general, Rust doesn't offer generic system-specific types, such\n> as `long`, except for C FFI. This is actually a strong benefit, since\n> it means we're not inclined to write `unsigned long` and then wonder why\n> things are broken on Windows: instead, we write either `usize` (the\n> equivalent of `size_t`) or `u64` (for things like file sizes). This is\n> much more ingrained than it is in Go, which has a tendency to use `int`\n> (Rust's `isize`) a lot and much less often specific types.\n>\n> If we're going to move this code entirely into Rust, then it makes sense\n> to cast temporarily, and I'm fine doing that in C, since it's C that has\n> the weird system-dependent behaviour (arbitrary decisions on the\n> signedness of char). That actually allows us to have more confidence in\n> the safety and maintainability of the Rust code since it is less system\n> dependent and leave the suspect pieces in C. It may also, interestingly\n> enough, also allow us to easily get rid of the weird 2 GB limit on diffs\n> due to the unpleasant dependency on `int` in the xdiff code, which I\n> would absolutely love to see.\n\nAhhh. Thanks for the patient explanation. That makes a lot of sense, and\ncasting on the Rust side seems like the right approach for this spot.\n\nThanks,\nTaylor\n"},{"id":"522267","messageId":"aHmUAbd8DlDbVO1i@nand.local","threadId":"63804","inReplyTo":"CABPp-BFqruHDF5He=iqrUe9sYJ3XdXBcthY8i_eKMXS9tAa0oA@mail.gmail.com","subject":"Re: [PATCH 4/7] xdiff: make fields of xrecord_t Rust friendly","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2025-07-18T00:23:29Z","receivedAt":"2025-07-18T00:23:31Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"On Thu, Jul 17, 2025 at 04:37:24PM -0700, Elijah Newren wrote:\n> > However, I'm not dead set against casting in Rust if that's what\n> > everyone else wants instead.\n>\n> In general, I too would prefer to do the casting on the C side; after\n> all, part of the reason for Rust is the language safety, which we\n> compromise if we force it into using ambiguously sized variables.\n>\n> However, I think it might be somewhat case-dependent...\n>\n> Here, we have C calling into APIs that will be defined and implemented\n> in Rust. Further along the road of adopting Rust in more places, we\n> may have future cases where we have Rust calling into APIs defined and\n> implemented in C. I'm wondering if in such a world the rule of thumb\n> that makes the most sense would be to have a\n> caller-must-cast-as-necessary guideline, rather than specifying the\n> casting side by language.\n\nYeah, I think having some sort of general guidance here spelled out in\nCodingGuidelines would be helpful. I don't think that it needs to\nprescribe a specific rule, but having some of what you and brian wrote\nabove captured more permanently would give contributors more information\nabout why they may want to place the casts on one side versus the other.\n\nThanks,\nTaylor\n"},{"id":"522268","messageId":"aHmVXDOiKzfKU8nb@fruit.crustytoothpaste.net","threadId":"63804","inReplyTo":"aHl4U98BBvpA5eKF@nand.local","subject":"Re: [PATCH 0/7] RFC: Accelerate xdiff and begin its rustification","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2025-07-18T00:29:16Z","receivedAt":"2025-07-18T00:29:18Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On 2025-07-17 at 22:25:23, Taylor Blau wrote:\n> I agree. I don't think that there is ever going to be a \"perfect\" time\n> to introduce a hard dependency on Rust, and I don't think that should\n> hold the project back from adopting it.\n> \n> I am far from a Rust expert, but I think that a more modern, memory-safe\n> language will attract newer contributors who may have a fresher\n> perspective on the project, and I think that's a good thing.\n\nYes, I think that's true. Rust is by far the most admired programming\nlanguage to work with, according to the 2024 Stack Overflow Developer\nSurvey. We will likely attract new contributors who find C intimidating\nor a bit of a hassle[0] but are excited about working on Rust,\nespecially in a project as compelling as Git[1].\n\n> The alternative, of course, is to continue to use C and not take any\n> dependency on Rust. I think there is a middle-ground in there somewhere\n> to be able to build with (e.g.) \"make\" or \"make RUST=1\", but I would\n> really like to see the project take a firmer stance here.\n> \n> I worry that having build support for both \"with Rust\" and \"C only\" will\n> create a headache not just at the build system level, but also in the\n> code itself. Having a patchwork of features, optimizations, or bug fixes\n> that either are or aren't supported depending on whether Rust support\n> was specified at build-time seems like a worst-of-all-worlds outcome.\n\nI definitely agree. I already find it terribly inconvenient when I end\nup when `git grep` doesn't support `-P` and I imagine that having lots\nof features that weren't available would be bothersome.\n\nI also think that using a combination of C and Rust will end up with us\nstill writing a lot of unsafe Rust code to interoperate with C. If we\nwant to reap the benefits in terms of memory and thread safety[2], we'll\nbe better off sticking with just Rust.\n\nI will also say that while it may be more challenging to compile Git at\nfirst on Windows, as we move more towards an all-Rust codebase, Git may\nend up being easier to maintain there as we depend more on the standard\nlibrary.\n\n> Agreed. Of course, I think we would all like Git to be able to build and\n> run on as many platforms as is reasonably possible. But we cannot\n> support all platforms for all time. It is also not the Git project's\n> responsibility to ensure that every platform is Rust-friendly.\n> \n> Hopefully the platforms that we currently support but won't after this\n> patch series have niche enough workloads that they do not need the\n> absolute latest-and-greatest Git release at all times.\n\nI will also point out that many OS and CPU architectures are actually\nsupported in Rust upstream. `rustc --print target-list` includes things\nlike the following:\n\n* m68k-unknown-linux-gnu (Amiga and other 68000 processors on Linux)\n* wasm32-unknown-unknown (Git in your browser?)\n* armv7a-nuttx-eabi (ARM processors running the embedded NuttX OS)\n* x86_64-pc-cygwin (Cygwin[3])\n* sparc64-unknown-openbsd (OpenBSD on UltraSPARC)\n\nAll Debian release architectures are supported, for instance, as well as\nseveral non-release architectures. The only Debian architectures that I\ndon't believe are supported are alpha, hppa, ia64, sh4, and x32 (which\nis an amd64 variant that can run amd64 code just fine).\n\n> Yeah, I think that this is the most interesting part of the discussion\n> here. I am not knowledgeable enough about Rust's release cadence and\n> platform compatibility to have an opinion here. But I trust brian's\n> judgement ;-).\n\nI'll see what Ezekiel thinks about this and I can send out a patch for\nreview if that's desired.\n\n[0] I've been writing C for three-quarters of my life and I still find\ndebugging segfaults and other memory problems to be annoying and\ntiresome, so I'm very interested in getting out of that business while\nworking on Git. While the limiting factor for my contributions to Git\nis often time, I would feel more excited about working on Git in Rust\nthan in C and I'm confident I'd write better quality code and more unit\ntests as well (which benefits us and our users as well).\n[1] I definitely think there's a cool factor to working on Git.\n[2] Having more thread-safe code might allow us to more easily add\nthreading to other parts of Git that would benefit from it and thus\nimprove performance in many cases.\n[3] This was missing for a long time, but it's finally here now, which\nis also good news for Git for Windows.\n-- \nbrian m. carlson (they/them)\nToronto, Ontario, CA\n"},{"id":"522276","messageId":"aHoSjbV2nMZkBn5l@256bit.org","threadId":"63804","inReplyTo":"pull.1980.git.git.1752784344.gitgitgadget@gmail.com","subject":"Re: [PATCH 0/7] RFC: Accelerate xdiff and begin its rustification","fromName":"Christian Brabandt","fromEmail":"cb@256bit.org","sentAt":"2025-07-18T09:23:25Z","receivedAt":"2025-07-18T09:49:23Z","isPatch":true,"sender":{"key":"cb@256bit.org","avatar":"https://gravatar.com/avatar/c72756321dd9fa10cada6d4b1f2c4e577a979373d76960608f816cb49c575b02?d=mp&s=160"},"body":"\nOn Do, 17 Jul 2025, Ezekiel Newren via GitGitGadget wrote:\n\n> This series accelerates xdiff by 5-19%.\n> \n> It also introduces Rust as a hard dependency.\n> \n> …and it doesn’t yet pass a couple of the github workflows; hints from\n> Windows experts, and opinions on ambiguous primitives would be appreciated\n> (see below).\n> \n> This is just the beginning of many patches that I have to convert portions\n> of, maybe eventually all of, xdiff to Rust. While working on that\n> conversion, I found several ways to clarify the code, along with some\n> optimizations.\n\nJust a quick heads-up: We (as in Vim/Neovim) have been using gits xdiff \nlibrary for use in Vim and Neovim.\n\nIs the plan to get rid of xdiffs C source completely and replace it by a \nRust implementation?\n\nThanks,\nChris\n-- \nEine gute Stellung ist besser als jede Arbeit.\n"},{"id":"522277","messageId":"f439958d-64ce-417f-8175-720f69387d48@gmail.com","threadId":"63804","inReplyTo":"pull.1980.git.git.1752784344.gitgitgadget@gmail.com","subject":"Re: [PATCH 0/7] RFC: Accelerate xdiff and begin its rustification","fromName":"Phillip Wood","fromEmail":"phillip.wood123@gmail.com","sentAt":"2025-07-18T13:34:34Z","receivedAt":"2025-07-18T13:34:37Z","isPatch":true,"sender":{"key":"phillip.wood@dunelm.org.uk","avatar":null},"body":"Hi Ezekiel\n\nThanks for working on this\n\nOn 17/07/2025 21:32, Ezekiel Newren via GitGitGadget wrote:\n> This series accelerates xdiff by 5-19%.\n> \n> It also introduces Rust as a hard dependency.\n> \n> …and it doesn’t yet pass a couple of the github workflows; hints from\n> Windows experts, and opinions on ambiguous primitives would be appreciated\n> (see below).\n> \n> This is just the beginning of many patches that I have to convert portions\n> of, maybe eventually all of, xdiff to Rust. While working on that\n> conversion, I found several ways to clarify the code, along with some\n> optimizations.\n> \n> So...\n> \n> This obviously raises the question of whether we are ready to accept a hard\n> dependency on Rust. Previous discussions on the mailing list and at Git\n> Merge 2024 have not answered that question. If not now, will we be willing\n> to accept such a hard dependency later? And what route do we want to take to\n> get there?\n\nAs far as git goes I think introducing a hard dependency on rust is \nfine. It is widely supported, the only issue I'm aware of is the lack of \nsupport on NonStop and I don't think it is reasonable for such a \nminority platform to hold the rest of the project to ransom. There is a \nquestion about the other users of the xdiff code though. libgit2 carries \na copy as do other projects like neovim. I've cc'd the libgit2 \nmaintainer and posted a link to this thread in neovim github [1]\n\nI've left a few comments on the patches\n\nThanks\n\nPhillip\n\n[1] https://github.com/neovim/neovim/discussions/34987\n\n> About the optimizations in this series:\n> \n> 1. xdiff currently uses DJB2a for hashing (even though it is not explicitly named as such). This is an older hashing algorithm, and modern alternatives are superior. I chose xxhash because it’s faster, more collision resistant, and designed to be a standard. Other hash algorithms like aHash, MurMurHash, SipHash, and Fnv1a were considered, but my local testing made me feel like xxhash was the best choice for usage in xdiff.\n> \n> 2. In support of switching to xxhash, parsing and hashing were split into separate steps. And it turns out that memchr() is faster for parsing than character-by-character iteration.\n> \n> \n> About the workflow builds/tests that aren’t working with this series:\n> \n> 1. Windows fails to build. I don’t know which rust toolchain is even correct for this or if multiple are needed. Example failed build: https://github.com/git/git/actions/runs/16353209191\n> \n> 2. I386/ubuntu:focal will build, but fails the tests. The kernel reports the bitness as 64 despite the container being 32. I believe the issue is that C uses ambiguous primitives (which differ in size between platforms). The new code should use unambiguous primitives from Rust (u32, u64, etc.) rather than perpetuating ambiguous primitive types. Since the current xdiff API hardcodes the ambiguous types, though, those places will need to be migrated to unambiguous primitives. Much of the C code needs a slight refactor to be compatible with the Rust FFI and usually requires converting ambiguous to unambiguous types. What does this community think of this approach?\n> \n> \n> My brother (Elijah, cc’ed) has been guiding and reviewing my work here.\n> \n> Ezekiel Newren (7):\n> xdiff: introduce rust\n> xdiff/xprepare: remove superfluous forward declarations\n> xdiff: delete unnecessary fields from xrecord_t and xdfile_t\n> xdiff: make fields of xrecord_t Rust friendly\n> xdiff: separate parsing lines from hashing them\n> xdiff: conditionally use Rust's implementation of xxhash\n> github_workflows: install rust\n> \n> .github/workflows/main.yml | 1 +\n> .gitignore | 1 +\n> Makefile | 60 +++++++---\n> build_rust.sh | 59 ++++++++++\n> ci/install-dependencies.sh | 14 +--\n> ci/install-rust.sh | 33 ++++++\n> ci/lib.sh | 8 ++\n> ci/make-test-artifacts.sh | 7 ++\n> ci/run-build-and-tests.sh | 10 ++\n> git-compat-util.h | 17 +++\n> meson.build | 40 +++++--\n> rust/Cargo.lock | 21 ++++\n> rust/Cargo.toml | 6 +\n> rust/interop/Cargo.toml | 14 +++\n> rust/interop/src/lib.rs | 0\n> rust/xdiff/Cargo.toml | 16 +++\n> rust/xdiff/src/lib.rs | 7 ++\n> xdiff/xdiffi.c | 8 +-\n> xdiff/xemit.c | 2 +-\n> xdiff/xmerge.c | 14 +--\n> xdiff/xpatience.c | 2 +-\n> xdiff/xprepare.c | 226 ++++++++++++++++++-------------------\n> xdiff/xtypes.h | 9 +-\n> xdiff/xutils.c | 4 +-\n> 24 files changed, 414 insertions(+), 165 deletions(-)\n> create mode 100755 build_rust.sh\n> create mode 100644 ci/install-rust.sh\n> create mode 100644 rust/Cargo.lock\n> create mode 100644 rust/Cargo.toml\n> create mode 100644 rust/interop/Cargo.toml\n> create mode 100644 rust/interop/src/lib.rs\n> create mode 100644 rust/xdiff/Cargo.toml\n> create mode 100644 rust/xdiff/src/lib.rs\n> \n> \n> base-commit: 16bd9f20a403117f2e0d9bcda6c6e621d3763e77\n> Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-1980%2Fezekielnewren%2Fxdiff_rust_speedup-v1\n> Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-1980/ezekielnewren/xdiff_rust_speedup-v1\n> Pull-Request: https://github.com/git/git/pull/1980\n\n"},{"id":"522278","messageId":"ec15f63f-1ddd-4b10-9c7d-c21d81ead953@gmail.com","threadId":"63804","inReplyTo":"2db30cc739efadf8383bd9dc1b7825ce863e8f5a.1752784344.git.gitgitgadget@gmail.com","subject":"Re: [PATCH 5/7] xdiff: separate parsing lines from hashing them","fromName":"Phillip Wood","fromEmail":"phillip.wood123@gmail.com","sentAt":"2025-07-18T13:34:50Z","receivedAt":"2025-07-18T13:34:53Z","isPatch":true,"sender":{"key":"phillip.wood@dunelm.org.uk","avatar":null},"body":"Hi Ezekiel\n\nOn 17/07/2025 21:32, Ezekiel Newren via GitGitGadget wrote:\n> From: Ezekiel Newren \n> \n> We want to use xxhash for faster hashing. To facilitate that\n> and to simplify the code. Separate the concerns of parsing\n> and hashing into discrete steps. This makes swapping the hash\n> function much easier. Since xdl_hash_record() both parses and\n> hashses lines, this requires some slight code restructuring.\n\nThat makes sense though unfortunately we seem to have lost some error \nhandling in the restructuring. How much does this extra pass over the \ninput data slow down the cases that don't end up using xxhash?\n\n> Signed-off-by: Ezekiel Newren \n> ---\n> xdiff/xprepare.c | 75 ++++++++++++++++++++++++++++--------------------\n> 1 file changed, 44 insertions(+), 31 deletions(-)\n> \n> diff --git a/xdiff/xprepare.c b/xdiff/xprepare.c\n> index 747268e4fdf7..c44005e9bbb8 100644\n> --- a/xdiff/xprepare.c\n> +++ b/xdiff/xprepare.c\n> @@ -129,13 +129,39 @@ static int xdl_classify_record(unsigned int pass, xdlclassifier_t *cf, xrecord_t\n> }\n> \n> \n> +static void xdl_parse_lines(mmfile_t *mf, long narec, xdfile_t *xdf) {\n> +\tu8 const* ptr = (u8 const*) mf->ptr;\n> +\tusize len = (usize) mf->size;\n> +\n> +\txdf->recs = NULL;\n> +\txdf->nrec = 0;\n> +\tXDL_ALLOC_ARRAY(xdf->recs, narec);\n\nThis should return error if the allocation fails like the original code. \nAlthough that does not make any difference for git a number of other \nprojects such as libgit2 carry a copy of our xdiff code and want to be \nable to handle allocation failures.\n\n> +\twhile (len > 0) {\n> +\t\txrecord_t *rec = NULL;\n> +\t\tusize length;\n> +\t\tu8 const* result = memchr(ptr, '\\n', len);\n> +\t\tif (result) {\n> +\t\t\tlength = result - ptr + 1;\n> +\t\t} else {\n> +\t\t\tlength = len;\n> +\t\t}\n> +\t\tif (XDL_ALLOC_GROW(xdf->recs, xdf->nrec + 1, narec))\n> +\t\t\tdie(\"XDL_ALLOC_GROW failed\");\n\nWe should return an error rather than dying here\n\n> +\t\trec = xdl_cha_alloc(&xdf->rcha);\n\nWe should return an error if the call fails like the original code\n\n> +\t\trec->ptr = ptr;\n> +\t\trec->size = length;\n> +\t\trec->ha = 0;\n> +\t\txdf->recs[xdf->nrec++] = rec;\n> +\t\tptr += length;\n> +\t\tlen -= length;\n> +\t}\n> +\n> +}\n> +\n> +\n> static int xdl_prepare_ctx(unsigned int pass, mmfile_t *mf, long narec, xpparam_t const *xpp,\n> \t\t\t xdlclassifier_t *cf, xdfile_t *xdf) {\n> -\tlong nrec, bsize;\n> -\tunsigned long hav;\n> -\tchar const *blk, *cur, *top, *prev;\n> -\txrecord_t *crec;\n> -\txrecord_t **recs;\n> \tunsigned long *ha;\n> \tchar *rchg;\n> \tlong *rindex;\n> @@ -143,50 +169,37 @@ static int xdl_prepare_ctx(unsigned int pass, mmfile_t *mf, long narec, xpparam_\n> \tha = NULL;\n> \trindex = NULL;\n> \trchg = NULL;\n> -\trecs = NULL;\n> \n> \tif (xdl_cha_init(&xdf->rcha, sizeof(xrecord_t), narec / 4 + 1) < 0)\n> \t\tgoto abort;\n> -\tif (!XDL_ALLOC_ARRAY(recs, narec))\n> -\t\tgoto abort;\n> \n> -\tnrec = 0;\n> -\tif ((cur = blk = xdl_mmfile_first(mf, &bsize))) {\n> -\t\tfor (top = blk + bsize; cur < top; ) {\n> -\t\t\tprev = cur;\n> -\t\t\thav = xdl_hash_record(&cur, top, xpp->flags);\n> -\t\t\tif (XDL_ALLOC_GROW(recs, nrec + 1, narec))\n> -\t\t\t\tgoto abort;\n> -\t\t\tif (!(crec = xdl_cha_alloc(&xdf->rcha)))\n> -\t\t\t\tgoto abort;\n> -\t\t\tcrec->ptr = (u8 const*) prev;\n> -\t\t\tcrec->size = (long) (cur - prev);\n> -\t\t\tcrec->ha = hav;\n> -\t\t\trecs[nrec++] = crec;\n> -\t\t\tif (xdl_classify_record(pass, cf, crec) < 0)\n> -\t\t\t\tgoto abort;\n> -\t\t}\n> +\txdl_parse_lines(mf, narec, xdf);\n> +\n> +\tfor (usize i = 0; i < (usize) xdf->nrec; i++) {\n> +\t\txrecord_t *rec = xdf->recs[i];\n> +\t\tchar const* dump = (char const*) rec->ptr;\n> +\t\trec->ha = xdl_hash_record(&dump, (char const*) (rec->ptr + rec->size), xpp->flags);\n\nI think we should update xdl_hash_record() to stop updating dump and use \nthe length from xdl_parse_lines(). Now that we parse the lines before \nhashing we should use that length in the hash function so we have a \nsingle definition of line length.\n\n> +\t\txdl_classify_record(pass, cf, rec);\n\nWe should return an error if this call fails like the original code\n\nThanks\n\nPhillip\n\n> \t}\n> \n> -\tif (!XDL_CALLOC_ARRAY(rchg, nrec + 2))\n> +\n> +\tif (!XDL_CALLOC_ARRAY(rchg, xdf->nrec + 2))\n> \t\tgoto abort;\n> \n> \tif ((XDF_DIFF_ALG(xpp->flags) != XDF_PATIENCE_DIFF) &&\n> \t (XDF_DIFF_ALG(xpp->flags) != XDF_HISTOGRAM_DIFF)) {\n> -\t\tif (!XDL_ALLOC_ARRAY(rindex, nrec + 1))\n> +\t\tif (!XDL_ALLOC_ARRAY(rindex, xdf->nrec + 1))\n> \t\t\tgoto abort;\n> -\t\tif (!XDL_ALLOC_ARRAY(ha, nrec + 1))\n> +\t\tif (!XDL_ALLOC_ARRAY(ha, xdf->nrec + 1))\n> \t\t\tgoto abort;\n> \t}\n> \n> -\txdf->nrec = nrec;\n> -\txdf->recs = recs;\n> \txdf->rchg = rchg + 1;\n> \txdf->rindex = rindex;\n> \txdf->nreff = 0;\n> \txdf->ha = ha;\n> \txdf->dstart = 0;\n> -\txdf->dend = nrec - 1;\n> +\txdf->dend = xdf->nrec - 1;\n> \n> \treturn 0;\n> \n> @@ -194,7 +207,7 @@ abort:\n> \txdl_free(ha);\n> \txdl_free(rindex);\n> \txdl_free(rchg);\n> -\txdl_free(recs);\n> +\txdl_free(xdf->recs);\n> \txdl_cha_free(&xdf->rcha);\n> \treturn -1;\n> }\n\n"},{"id":"522279","messageId":"91f6352f-abc4-4e99-938b-6a56aba2faed@gmail.com","threadId":"63804","inReplyTo":"6df9f50a8f4ca29b2c3ba1e39982b6d516146bb3.1752784344.git.gitgitgadget@gmail.com","subject":"Re: [PATCH 4/7] xdiff: make fields of xrecord_t Rust friendly","fromName":"Phillip Wood","fromEmail":"phillip.wood123@gmail.com","sentAt":"2025-07-18T13:35:05Z","receivedAt":"2025-07-18T13:35:08Z","isPatch":true,"sender":{"key":"phillip.wood@dunelm.org.uk","avatar":null},"body":"Hi Ezekiel\n\nOn 17/07/2025 21:32, Ezekiel Newren via GitGitGadget wrote:\n> From: Ezekiel Newren \n> \n> A few commits ago, we added definitions for Rust primitive types,\n> to facilitate interoperability between C and Rust. Switch a\n> few variables to use these types. Which, for now, will\n> require adding some casts.\n\nHow necessary is it to change char' to 'u8' so long as the rust and C \nsides both use a type that is the same size? Also what's the advantage \nof using these typedefs rather than the normal C types like unit8_t ?\n\n> diff --git a/xdiff/xdiffi.c b/xdiff/xdiffi.c\n> index 5a96e36dfbea..3b364c61f671 100644\n> --- a/xdiff/xdiffi.c\n> +++ b/xdiff/xdiffi.c\n> @@ -418,7 +418,7 @@ static int get_indent(xrecord_t *rec)\n> \tlong i;\n> \tint ret = 0;\n> \n> -\tfor (i = 0; i < rec->size; i++) {\n> +\tfor (i = 0; i < (long) rec->size; i++) {\n\ni is a loop counter and array index so we can lose this cast by \nchangeing i to size_t\n\nThanks\n\nPhillip\n\n> \t\tchar c = rec->ptr[i];\n> \n> \t\tif (!XDL_ISSPACE(c))\n> @@ -1005,11 +1005,11 @@ static void xdl_mark_ignorable_lines(xdchange_t *xscr, xdfenv_t *xe, long flags)\n> \n> \t\trec = &xe->xdf1.recs[xch->i1];\n> \t\tfor (i = 0; i < xch->chg1 && ignore; i++)\n> -\t\t\tignore = xdl_blankline(rec[i]->ptr, rec[i]->size, flags);\n> +\t\t\tignore = xdl_blankline((const char*) rec[i]->ptr, rec[i]->size, flags);\n> \n> \t\trec = &xe->xdf2.recs[xch->i2];\n> \t\tfor (i = 0; i < xch->chg2 && ignore; i++)\n> -\t\t\tignore = xdl_blankline(rec[i]->ptr, rec[i]->size, flags);\n> +\t\t\tignore = xdl_blankline((const char*)rec[i]->ptr, rec[i]->size, flags);\n> \n> \t\txch->ignore = ignore;\n> \t}\n> @@ -1020,7 +1020,7 @@ static int record_matches_regex(xrecord_t *rec, xpparam_t const *xpp) {\n> \tsize_t i;\n> \n> \tfor (i = 0; i < xpp->ignore_regex_nr; i++)\n> -\t\tif (!regexec_buf(xpp->ignore_regex[i], rec->ptr, rec->size, 1,\n> +\t\tif (!regexec_buf(xpp->ignore_regex[i], (const char*) rec->ptr, rec->size, 1,\n> \t\t\t\t ®match, 0))\n> \t\t\treturn 1;\n> \n> diff --git a/xdiff/xemit.c b/xdiff/xemit.c\n> index 1d40c9cb4076..bbf7b7f8c862 100644\n> --- a/xdiff/xemit.c\n> +++ b/xdiff/xemit.c\n> @@ -24,7 +24,7 @@\n> \n> static long xdl_get_rec(xdfile_t *xdf, long ri, char const **rec) {\n> \n> -\t*rec = xdf->recs[ri]->ptr;\n> +\t*rec = (char const*) xdf->recs[ri]->ptr;\n> \n> \treturn xdf->recs[ri]->size;\n> }\n> diff --git a/xdiff/xmerge.c b/xdiff/xmerge.c\n> index af40c88a5b36..6fa6ea61a208 100644\n> --- a/xdiff/xmerge.c\n> +++ b/xdiff/xmerge.c\n> @@ -101,8 +101,8 @@ static int xdl_merge_cmp_lines(xdfenv_t *xe1, int i1, xdfenv_t *xe2, int i2,\n> \txrecord_t **rec2 = xe2->xdf2.recs + i2;\n> \n> \tfor (i = 0; i < line_count; i++) {\n> -\t\tint result = xdl_recmatch(rec1[i]->ptr, rec1[i]->size,\n> -\t\t\trec2[i]->ptr, rec2[i]->size, flags);\n> +\t\tint result = xdl_recmatch((const char*) rec1[i]->ptr, rec1[i]->size,\n> +\t\t\t(const char*) rec2[i]->ptr, rec2[i]->size, flags);\n> \t\tif (!result)\n> \t\t\treturn -1;\n> \t}\n> @@ -324,8 +324,8 @@ static int xdl_fill_merge_buffer(xdfenv_t *xe1, const char *name1,\n> \n> static int recmatch(xrecord_t *rec1, xrecord_t *rec2, unsigned long flags)\n> {\n> -\treturn xdl_recmatch(rec1->ptr, rec1->size,\n> -\t\t\t rec2->ptr, rec2->size, flags);\n> +\treturn xdl_recmatch((char const*) rec1->ptr, rec1->size,\n> +\t\t\t (char const*) rec2->ptr, rec2->size, flags);\n> }\n> \n> /*\n> @@ -383,10 +383,10 @@ static int xdl_refine_conflicts(xdfenv_t *xe1, xdfenv_t *xe2, xdmerge_t *m,\n> \t\t */\n> \t\tt1.ptr = (char *)xe1->xdf2.recs[m->i1]->ptr;\n> \t\tt1.size = xe1->xdf2.recs[m->i1 + m->chg1 - 1]->ptr\n> -\t\t\t+ xe1->xdf2.recs[m->i1 + m->chg1 - 1]->size - t1.ptr;\n> +\t\t\t+ xe1->xdf2.recs[m->i1 + m->chg1 - 1]->size - (u8 const*) t1.ptr;\n> \t\tt2.ptr = (char *)xe2->xdf2.recs[m->i2]->ptr;\n> \t\tt2.size = xe2->xdf2.recs[m->i2 + m->chg2 - 1]->ptr\n> -\t\t\t+ xe2->xdf2.recs[m->i2 + m->chg2 - 1]->size - t2.ptr;\n> +\t\t\t+ xe2->xdf2.recs[m->i2 + m->chg2 - 1]->size - (u8 const*) t2.ptr;\n> \t\tif (xdl_do_diff(&t1, &t2, xpp, &xe) < 0)\n> \t\t\treturn -1;\n> \t\tif (xdl_change_compact(&xe.xdf1, &xe.xdf2, xpp->flags) < 0 ||\n> @@ -440,7 +440,7 @@ static int line_contains_alnum(const char *ptr, long size)\n> static int lines_contain_alnum(xdfenv_t *xe, int i, int chg)\n> {\n> \tfor (; chg; chg--, i++)\n> -\t\tif (line_contains_alnum(xe->xdf2.recs[i]->ptr,\n> +\t\tif (line_contains_alnum((char const*) xe->xdf2.recs[i]->ptr,\n> \t\t\t\txe->xdf2.recs[i]->size))\n> \t\t\treturn 1;\n> \treturn 0;\n> diff --git a/xdiff/xpatience.c b/xdiff/xpatience.c\n> index 77dc411d1937..986a3a3f749a 100644\n> --- a/xdiff/xpatience.c\n> +++ b/xdiff/xpatience.c\n> @@ -121,7 +121,7 @@ static void insert_record(xpparam_t const *xpp, int line, struct hashmap *map,\n> \t\treturn;\n> \tmap->entries[index].line1 = line;\n> \tmap->entries[index].hash = record->ha;\n> -\tmap->entries[index].anchor = is_anchor(xpp, map->env->xdf1.recs[line - 1]->ptr);\n> +\tmap->entries[index].anchor = is_anchor(xpp, (const char*) map->env->xdf1.recs[line - 1]->ptr);\n> \tif (!map->first)\n> \t\tmap->first = map->entries + index;\n> \tif (map->last) {\n> diff --git a/xdiff/xprepare.c b/xdiff/xprepare.c\n> index ad356281f939..747268e4fdf7 100644\n> --- a/xdiff/xprepare.c\n> +++ b/xdiff/xprepare.c\n> @@ -96,12 +96,12 @@ static int xdl_classify_record(unsigned int pass, xdlclassifier_t *cf, xrecord_t\n> \tchar const *line;\n> \txdlclass_t *rcrec;\n> \n> -\tline = rec->ptr;\n> +\tline = (char const*) rec->ptr;\n> \thi = (long) XDL_HASHLONG(rec->ha, cf->hbits);\n> \tfor (rcrec = cf->rchash[hi]; rcrec; rcrec = rcrec->next)\n> \t\tif (rcrec->ha == rec->ha &&\n> \t\t\t\txdl_recmatch(rcrec->line, rcrec->size,\n> -\t\t\t\t\trec->ptr, rec->size, cf->flags))\n> +\t\t\t\t\t(const char*) rec->ptr, rec->size, cf->flags))\n> \t\t\tbreak;\n> \n> \tif (!rcrec) {\n> @@ -159,7 +159,7 @@ static int xdl_prepare_ctx(unsigned int pass, mmfile_t *mf, long narec, xpparam_\n> \t\t\t\tgoto abort;\n> \t\t\tif (!(crec = xdl_cha_alloc(&xdf->rcha)))\n> \t\t\t\tgoto abort;\n> -\t\t\tcrec->ptr = prev;\n> +\t\t\tcrec->ptr = (u8 const*) prev;\n> \t\t\tcrec->size = (long) (cur - prev);\n> \t\t\tcrec->ha = hav;\n> \t\t\trecs[nrec++] = crec;\n> diff --git a/xdiff/xtypes.h b/xdiff/xtypes.h\n> index 8b8467360ecf..6e5f67ebf380 100644\n> --- a/xdiff/xtypes.h\n> +++ b/xdiff/xtypes.h\n> @@ -39,9 +39,9 @@ typedef struct s_chastore {\n> } chastore_t;\n> \n> typedef struct s_xrecord {\n> -\tchar const *ptr;\n> -\tlong size;\n> -\tunsigned long ha;\n> +\tu8 const* ptr;\n> +\tusize size;\n> +\tu64 ha;\n> } xrecord_t;\n> \n> typedef struct s_xdfile {\n> diff --git a/xdiff/xutils.c b/xdiff/xutils.c\n> index 444a108f87c0..10e4f20b7c31 100644\n> --- a/xdiff/xutils.c\n> +++ b/xdiff/xutils.c\n> @@ -418,10 +418,10 @@ int xdl_fall_back_diff(xdfenv_t *diff_env, xpparam_t const *xpp,\n> \n> \tsubfile1.ptr = (char *)diff_env->xdf1.recs[line1 - 1]->ptr;\n> \tsubfile1.size = diff_env->xdf1.recs[line1 + count1 - 2]->ptr +\n> -\t\tdiff_env->xdf1.recs[line1 + count1 - 2]->size - subfile1.ptr;\n> +\t\tdiff_env->xdf1.recs[line1 + count1 - 2]->size - (u8 const*) subfile1.ptr;\n> \tsubfile2.ptr = (char *)diff_env->xdf2.recs[line2 - 1]->ptr;\n> \tsubfile2.size = diff_env->xdf2.recs[line2 + count2 - 2]->ptr +\n> -\t\tdiff_env->xdf2.recs[line2 + count2 - 2]->size - subfile2.ptr;\n> +\t\tdiff_env->xdf2.recs[line2 + count2 - 2]->size - (u8 const*) subfile2.ptr;\n> \tif (xdl_do_diff(&subfile1, &subfile2, xpp, &env) < 0)\n> \t\treturn -1;\n> \n\n"},{"id":"522281","messageId":"xmqqjz454l96.fsf@gitster.g","threadId":"63804","inReplyTo":"pull.1980.git.git.1752784344.gitgitgadget@gmail.com","subject":"Re: [PATCH 0/7] RFC: Accelerate xdiff and begin its rustification","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-07-18T14:38:29Z","receivedAt":"2025-07-18T14:38:32Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Ezekiel Newren via GitGitGadget\" writes:\n\n> This series accelerates xdiff by 5-19%.\n\n;-)\n\nDo we know how much of that can be attributed to the hash algorithm\ndifference, and how much for languages?\n\nThe earlier parts of the series to trim unused code and refactor\nlook to me that they are good changes regardless of whether we\nintroduce a different hash algorithm, and/or we use an\nimplementation of that different hash algorithm written in Rust.\nIOW, even if neither of these two happens, I would think that the\nearlier parts are independently good pieces.\n\nThanks for starting this effort. And thanks Elijah for helping.\n\nAnd in case nobody has said this yet, welcome to the Git development\ncommunity.\n"},{"id":"522283","messageId":"xmqqcy9x4g8n.fsf@gitster.g","threadId":"63804","inReplyTo":"aHoSjbV2nMZkBn5l@256bit.org","subject":"Re: [PATCH 0/7] RFC: Accelerate xdiff and begin its rustification","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-07-18T16:26:48Z","receivedAt":"2025-07-18T16:26:51Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Christian Brabandt writes:\n\n> On Do, 17 Jul 2025, Ezekiel Newren via GitGitGadget wrote:\n>\n>> This series accelerates xdiff by 5-19%.\n>> \n>> It also introduces Rust as a hard dependency.\n>> \n>> …and it doesn’t yet pass a couple of the github workflows; hints from\n>> Windows experts, and opinions on ambiguous primitives would be appreciated\n>> (see below).\n>> \n>> This is just the beginning of many patches that I have to convert portions\n>> of, maybe eventually all of, xdiff to Rust. While working on that\n>> conversion, I found several ways to clarify the code, along with some\n>> optimizations.\n>\n> Just a quick heads-up: We (as in Vim/Neovim) have been using gits xdiff \n> library for use in Vim and Neovim.\n>\n> Is the plan to get rid of xdiffs C source completely and replace it by a \n> Rust implementation?\n\nAs far as I know, there is no such plan that is widely agreed upon\n(yet).\n\nThe discussion starter thread you are looking at only introduces a\nnew code path that uses a different line hash function written in\nRust when whitespace munging search is not enabled, and everything\nelse still is written in C, but since it is just a discussion\nstarter so far.\n\nI would personally have liked the effort to start with xmerge code,\nnot xdiff machinery, for various reasons, but that may be just me\n;-)\n"},{"id":"522286","messageId":"xmqqzfd12ujv.fsf@gitster.g","threadId":"63804","inReplyTo":"5a959c9bdad79cf972b95dcf4324135dd7c94dac.1752784344.git.gitgitgadget@gmail.com","subject":"Re: [PATCH 6/7] xdiff: conditionally use Rust's implementation of xxhash","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-07-18T19:00:36Z","receivedAt":"2025-07-18T19:00:39Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Ezekiel Newren via GitGitGadget\" writes:\n\n> +extern u64 xxh3_64(u8 const* ptr, usize size);\n> +\n> +\n> static int xdl_prepare_ctx(unsigned int pass, mmfile_t *mf, long narec, xpparam_t const *xpp,\n> \t\t\t xdlclassifier_t *cf, xdfile_t *xdf) {\n> \tunsigned long *ha;\n> @@ -175,14 +178,26 @@ static int xdl_prepare_ctx(unsigned int pass, mmfile_t *mf, long narec, xpparam_\n> \n> \txdl_parse_lines(mf, narec, xdf);\n> \n> +\tif ((xpp->flags & XDF_WHITESPACE_FLAGS) == 0) {\n> +\t\tfor (usize i = 0; i < (usize) xdf->nrec; i++) {\n> +\t\t\txrecord_t *rec = xdf->recs[i];\n> +\t\t\trec->ha = xxh3_64(rec->ptr, rec->size);\n> +\t\t}\n> +\t} else {\n> +\t\tfor (usize i = 0; i < (usize) xdf->nrec; i++) {\n> +\t\t\txrecord_t *rec = xdf->recs[i];\n> +\t\t\tchar const* dump = (char const*) rec->ptr;\n> +\t\t\trec->ha = xdl_hash_record(&dump, (char const*) (rec->ptr + rec->size), xpp->flags);\n> +\t\t}\n> +\t}\n\nAs a technology demonstration and proof of concept patch, this is\nvery nice, but to be upstreamed for real, we'd want a variant of\nxxhash that can work with the contents with whitespace squashed to\nbe usable with various whitespace ignoring modes of operation. When\nthat happens, and when the result turns out to be more performant,\nwe can lose the xdl_hash_record() and require only the xxhash, which\nwould be great.\n\nAnd that variant of xxhash that understands whitespace squashing can\nof course be written in Rust as a part of this effort when the\nseries loses its RFC status. At the same time, those who want to\nuse our xdiff code in third-party software (like libgit2 and vim)\nmay want to reimplement it in C in their copy.\n\nThanks.\n\n"},{"id":"522289","messageId":"79c1b3ab-af2e-4c93-b033-349221d82ad9@gentoo.org","threadId":"63804","inReplyTo":"f439958d-64ce-417f-8175-720f69387d48@gmail.com","subject":"Re: [PATCH 0/7] RFC: Accelerate xdiff and begin its rustification","fromName":"Eli Schwartz","fromEmail":"eschwartz@gentoo.org","sentAt":"2025-07-18T21:25:01Z","receivedAt":"2025-07-18T21:25:09Z","isPatch":true,"sender":{"key":"eschwartz@gentoo.org","avatar":"https://avatars.githubusercontent.com/u/6551424?v=4"},"body":"On 7/18/25 9:34 AM, Phillip Wood wrote:\n> Hi Ezekiel\n> \n> Thanks for working on this\n> \n> On 17/07/2025 21:32, Ezekiel Newren via GitGitGadget wrote:\n>\n>> So...\n>>\n>> This obviously raises the question of whether we are ready to accept a\n>> hard\n>> dependency on Rust. Previous discussions on the mailing list and at Git\n>> Merge 2024 have not answered that question. If not now, will we be\n>> willing\n>> to accept such a hard dependency later? And what route do we want to\n>> take to\n>> get there?\n> \n> As far as git goes I think introducing a hard dependency on rust is\n> fine. It is widely supported, the only issue I'm aware of is the lack of\n> support on NonStop and I don't think it is reasonable for such a\n> minority platform to hold the rest of the project to ransom. There is a\n> question about the other users of the xdiff code though. libgit2 carries\n> a copy as do other projects like neovim. I've cc'd the libgit2\n> maintainer and posted a link to this thread in neovim github [1]\n\n\nA hard dependency on rust for Gentoo amd64 would potentially require\nbuilding https://github.com/thepowersgang/mrustc followed by building 13\nand counting versions of rustc in order to get to the latest version.\nWhat is the minimum supported version in this series, by the way?\n\nbin packages for rust do exist but not everyone wants to use non-distro\nprovided binaries, sometimes for auditability reasons.\n\n\nFor Gentoo HPPA, Alpha, m68k it will simply mean the removal (or end of\nlife and staying forever on 2.50, perhaps) of Git. There is no rust\ncompiler there.\n\nEven s390 support for rust is limited to a precompiled version not\neveryone is willing to use.\n\nGCC-rs will probably fix this general issue.\n\n-- \nEli Schwartz\n"},{"id":"522290","messageId":"CAH=ZcbCVVOMEFmWp1JEDNRWGE2+F3zQ5jT48JhD_2ycR2kOv3Q@mail.gmail.com","threadId":"63804","inReplyTo":"xmqqjz454l96.fsf@gitster.g","subject":"Re: [PATCH 0/7] RFC: Accelerate xdiff and begin its rustification","fromName":"Ezekiel Newren","fromEmail":"ezekielnewren@gmail.com","sentAt":"2025-07-18T21:56:22Z","receivedAt":"2025-07-18T21:56:35Z","isPatch":true,"sender":{"key":"ezekielnewren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/18313324?v=4"},"body":"On Fri, Jul 18, 2025 at 8:38 AM Junio C Hamano wrote:\n>\n> \"Ezekiel Newren via GitGitGadget\" writes:\n>\n> > This series accelerates xdiff by 5-19%.\n>\n> ;-)\n>\n> Do we know how much of that can be attributed to the hash algorithm\n> difference, and how much for languages?\n\nThis is difficult to answer because xdl_hash_record() hashes the\nstring as it determines its length. Xxhash uses simd instructions, so\nall data must be contiguous and processed as blocks rather than byte\nby byte. The components cannot be directly compared due to the nature\nof processing differences.\n\n> The earlier parts of the series to trim unused code and refactor\n> look to me that they are good changes regardless of whether we\n> introduce a different hash algorithm, and/or we use an\n> implementation of that different hash algorithm written in Rust.\n> IOW, even if neither of these two happens, I would think that the\n> earlier parts are independently good pieces.\n>\n> Thanks for starting this effort. And thanks Elijah for helping.\n>\n> And in case nobody has said this yet, welcome to the Git development\n> community.\n\nThanks to you and everyone else for your review comments. I'm going to\nneed time to investigate and respond.\n"},{"id":"522293","messageId":"CAH=ZcbAeMv7oO-X_o_WOvgS6_igO3XAUjmgLoEfDN0CqqTMH_g@mail.gmail.com","threadId":"63804","inReplyTo":"aHlp1joMwexLZAAb@fruit.crustytoothpaste.net","subject":"Re: [PATCH 7/7] github_workflows: install rust","fromName":"Ezekiel Newren","fromEmail":"ezekielnewren@gmail.com","sentAt":"2025-07-18T23:01:30Z","receivedAt":"2025-07-18T23:01:44Z","isPatch":true,"sender":{"key":"ezekielnewren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/18313324?v=4"},"body":"On Thu, Jul 17, 2025 at 3:23 PM brian m. carlson\n wrote:\n>\n> On 2025-07-17 at 20:32:24, Ezekiel Newren via GitGitGadget wrote:\n> > diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml\n> > index 7dbf9f7f123c..8aac18a6ba45 100644\n> > --- a/.github/workflows/main.yml\n> > +++ b/.github/workflows/main.yml\n> > @@ -4,6 +4,7 @@ on: [push, pull_request]\n> >\n> > env:\n> > DEVELOPER: 1\n> > + RUST_VERSION: 1.87.0\n>\n> Our discussed plan is to support the version in Debian stable, plus a\n> year. So we'd be supporting 1.63.0 for a year after trixie's release.\n>\n> The reason for that is that people build backports and security updates\n> for Git for stable releases of distros and they will use the distro\n> toolchain for doing so. Forcing distros to constantly build with the\n> latest toolchain is pretty hostile, especially since the lifespan of\n> Rust release is six weeks.\n>\n> If the Rust project provides LTS releases in the future, then we can\n> consider adopting those.\n\nThe RUST_VERSION variable in .github/workflows/main.yaml had to have a\nspecific version. 1.87.0 was selected since that's what I was using\nlocally. Elijah made me aware that an older version of rust might be\ndesired, but didn't know which one. I'll switch to 1.63.0 or whatever\nthe community decides.\n\n> > +if [ \"$rust_target\" = \"release\" ]; then\n> > + rust_args=\"--release\"\n> > + export RUSTFLAGS='-Aunused_imports -Adead_code'\n> > +elif [ \"$rust_target\" = \"debug\" ]; then\n> > + rust_args=\"\"\n> > + export RUSTFLAGS='-Aunused_imports -Adead_code -C debuginfo=2 -C opt-level=1 -C force-frame-pointers=yes'\n>\n> Can you say a little about why these options are needed and the defaults\n> are inadequate? For instance, I build with the default options both in\n> my personal projects and at work and don't see a problem.\n\nWhat I found is that if I have a Rust function\n\n#[no_mangle]\npub fn call_from_c(arg: u64) {}\n\nwhich is only meant to be called from C and isn’t called from\nelsewhere in Rust, then cargo will misidentify this function as dead\ncode. This was the reason for adding ‘-Adead_code’.\n\nThe reason for adding ‘-Aunused_imports’ is somewhat IDE related; if I\npaste code somewhere, RustRover will sometimes automatically add the\nnecessary imports. However, if I delete a chunk of code, it’ll\nhighlight the imports that are no longer used if I scroll to the top\nof the file, but it won’t automatically remove them. Since they\naren’t automatically removed, it’s easier to build with\n‘-Aunused_imports’.\n\nThe remaining arguments, ‘-C debuginfo=2 -C opt-level=1 -C\nforce-frame-pointers=yes’ is to make /usr/bin/perf output more\namenable to analysis.\n\n> I don't know if you plan to do this in a future series, but we'd also\n> want cargo's tests to be run as part of CI and we'd want a lint job that\n> ran clippy with both 1.63.0 and the latest stable version of Rust to\n> make sure things were tidy.\n\nYeah I'd like that too; we can add that in a future patch series.\n\n> --\n> brian m. carlson (they/them)\n> Toronto, Ontario, CA\n"},{"id":"522294","messageId":"CAH=ZcbBebM6CememqOUFY2YPOXpk_mC=zE0OnLOKDqcJQTdMuA@mail.gmail.com","threadId":"63804","inReplyTo":"aHlrg7pbFqi2qNWH@fruit.crustytoothpaste.net","subject":"Re: [PATCH 1/7] xdiff: introduce rust","fromName":"Ezekiel Newren","fromEmail":"ezekielnewren@gmail.com","sentAt":"2025-07-18T23:15:19Z","receivedAt":"2025-07-18T23:15:32Z","isPatch":true,"sender":{"key":"ezekielnewren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/18313324?v=4"},"body":"On Thu, Jul 17, 2025 at 3:30 PM brian m. carlson\n wrote:\n>\n> On 2025-07-17 at 20:32:18, Ezekiel Newren via GitGitGadget wrote:\n> > diff --git a/rust/Cargo.lock b/rust/Cargo.lock\n> > new file mode 100644\n> > index 000000000000..fb1eac690b39\n> > --- /dev/null\n> > +++ b/rust/Cargo.lock\n> > @@ -0,0 +1,14 @@\n> > +# This file is automatically @generated by Cargo.\n> > +# It is not intended for manual editing.\n> > +version = 4\n> > +\n> > +[[package]]\n> > +name = \"interop\"\n> > +version = \"0.1.0\"\n> > +\n> > +[[package]]\n> > +name = \"xdiff\"\n> > +version = \"0.1.0\"\n> > +dependencies = [\n> > + \"interop\",\n> > +]\n>\n> I would prefer that we not check in Cargo.lock in Git. Part of the\n> reason is that it changes across versions and so building with a\n> different version of the toolchain can update the file.\n\nThis goes against what I think is best practices. Don’t we need\nCargo.lock to audit and debug platform specific issues, and to ensure\nreproducibility? Without Cargo.lock, we might get different results\none minute to the next if one of our dependencies releases a new\nversion. Checking in Cargo.lock aligns with Cargo’s documented best\npractices (https://doc.rust-lang.org/cargo/faq.html#why-have-cargolock-in-version-control).\n\n\n> In addition, as I mentioned downthread, because our intention is to\n> support the Debian stable toolchain for a year after the new stable\n> release, unless we are exceptionally careful about dependencies, we may\n> end up with a case where distros need to use older dependencies patched\n> for security but other users may want to update the versions to newer\n> dependencies with security fixes but that do not work on our pinned Rust\n> version. We can't possibly satisfy both sets of people if we pin\n> dependencies in Cargo.lock, so we probably want to avoid checking it in\n> and ignore it instead.\n\nI understand your concern and I agree that this could become a\nproblem. I’m totally flexible on which rust version should be used,\nbut without Cargo.lock checked in we lose the ability to audit why a\nbuild failed. I think that this will be a pain point, but numbing that\npain means we can’t solve intermittent problems due to dependencies in\nthe future.\n\n> --\n> brian m. carlson (they/them)\n> Toronto, Ontario, CA\n"},{"id":"522302","messageId":"CABPp-BHH1gYrv66k1dmeE7+W2jPfcOCxEH4orKOFpze-kBcEQA@mail.gmail.com","threadId":"63804","inReplyTo":"xmqqcy9x4g8n.fsf@gitster.g","subject":"Re: [PATCH 0/7] RFC: Accelerate xdiff and begin its rustification","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2025-07-19T00:32:00Z","receivedAt":"2025-07-19T00:32:12Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"On Fri, Jul 18, 2025 at 9:26 AM Junio C Hamano wrote:\n>\n> Christian Brabandt writes:\n>\n[...]\n> > Just a quick heads-up: We (as in Vim/Neovim) have been using gits xdiff\n> > library for use in Vim and Neovim.\n> >\n> > Is the plan to get rid of xdiffs C source completely and replace it by a\n> > Rust implementation?\n>\n> As far as I know, there is no such plan that is widely agreed upon\n> (yet).\n\nYeah, Ezekiel just barely notified the community of his efforts\nyesterday with this patch series. :-)\n\n> The discussion starter thread you are looking at only introduces a\n> new code path that uses a different line hash function written in\n> Rust when whitespace munging search is not enabled, and everything\n> else still is written in C, but since it is just a discussion\n> starter so far.\n>\n> I would personally have liked the effort to start with xmerge code,\n> not xdiff machinery, for various reasons, but that may be just me\n> ;-)\n\nWe have both xdiff/xmerge.[ch] and xdiff/{xdiff.h,xdiffi.[ch]}. When\nyou say \"xdiff\", I suspect that you're referring to the files within\nthe directory rather than to the whole directory, yes?\n\nI actually pointed Ezekiel at xhistogram to start (and thought he\nmight only do that file), then he backed up to xprepare, and then he\ncontinued from there on to other bits of xdiff/, including xmerge.\nDifferent parts are at different stages of conversion and testing.\nHe's not just transliterating but also trying to both clean up the\ncode and look for performance improvements (and it's sometimes hard to\ndo both; he's hit a few maintainability vs. performance tradeoffs and\nthose will likely result in some questions on the list at some point).\nIt's been a long slog, especially given how arcane xdiff sometimes\nfeels. Anyway, along the way, he recognized the DJB2a hash --\nsomething I certainly wouldn't have recognized or even thought to\ninvestigate. It led him to this optimization, which I thought was a\nreally good find, and it seemed like it'd make for a good initial\nseries to send to the list to get a feel for what people thought about\npossibly Rustifying xdiff/.\n"},{"id":"522303","messageId":"aHrrZyrDw_CYmFQF@cloudsdale.the-delta.net.eu.org","threadId":"63804","inReplyTo":"79c1b3ab-af2e-4c93-b033-349221d82ad9@gentoo.org","subject":"Re: [PATCH 0/7] RFC: Accelerate xdiff and begin its rustification","fromName":"Haelwenn (lanodan) Monnier","fromEmail":"contact@hacktivis.me","sentAt":"2025-07-19T00:48:39Z","receivedAt":"2025-07-19T00:55:21Z","isPatch":true,"sender":{"key":"contact@hacktivis.me","avatar":null},"body":"[2025-07-18 17:25:01-0400] Eli Schwartz:\n>On 7/18/25 9:34 AM, Phillip Wood wrote:\n>> Hi Ezekiel\n>>\n>> Thanks for working on this\n>>\n>> On 17/07/2025 21:32, Ezekiel Newren via GitGitGadget wrote:\n>>\n>>> So...\n>>>\n>>> This obviously raises the question of whether we are ready to accept a\n>>> hard\n>>> dependency on Rust. Previous discussions on the mailing list and at Git\n>>> Merge 2024 have not answered that question. If not now, will we be\n>>> willing\n>>> to accept such a hard dependency later? And what route do we want to\n>>> take to\n>>> get there?\n>>\n>> As far as git goes I think introducing a hard dependency on rust is\n>> fine. It is widely supported, the only issue I'm aware of is the lack of\n>> support on NonStop and I don't think it is reasonable for such a\n>> minority platform to hold the rest of the project to ransom. There is a\n>> question about the other users of the xdiff code though. libgit2 carries\n>> a copy as do other projects like neovim. I've cc'd the libgit2\n>> maintainer and posted a link to this thread in neovim github [1]\n>\n>\n>A hard dependency on rust for Gentoo amd64 would potentially require\n>building https://github.com/thepowersgang/mrustc followed by building 13\n>and counting versions of rustc in order to get to the latest version.\n>What is the minimum supported version in this series, by the way?\n>\n>bin packages for rust do exist but not everyone wants to use non-distro\n>provided binaries, sometimes for auditability reasons.\n>\n>\n>For Gentoo HPPA, Alpha, m68k it will simply mean the removal (or end of\n>life and staying forever on 2.50, perhaps) of Git. There is no rust\n>compiler there.\n>\n>Even s390 support for rust is limited to a precompiled version not\n>everyone is willing to use.\n\nAlso in other distro concerns, if it trickles down to libgit2,\nextra care should be taken to avoid creating circular dependencies\ndue to cargo depending on libgit2 (via git2 crate).\n\nFor example with making sure it can reasonably be built via meson's\nRust support rather than through cargo.\n\n>\n>GCC-rs will probably fix this general issue.\n>\n>-- \n>Eli Schwartz\n"},{"id":"522322","messageId":"ac871bc4-df93-31f4-55f2-d6fc538a422d@gmx.de","threadId":"63804","inReplyTo":"pull.1980.git.git.1752784344.gitgitgadget@gmail.com","subject":"Re: [PATCH 0/7] RFC: Accelerate xdiff and begin its rustification","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2025-07-19T21:53:24Z","receivedAt":"2025-07-19T21:53:28Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Ezekiel, \npleasure to make your acquaintance!\n\nOn Thu, 17 Jul 2025, Ezekiel Newren via GitGitGadget wrote:\n\n> 1. Windows fails to build. I don’t know which rust toolchain is even\n> correct for this or if multiple are needed. Example failed build:\n> https://github.com/git/git/actions/runs/16353209191\n\nThere are a couple of problems, not just one. Here are the patches that I\nwould like to ask you to take custody of (for your convenience, I have\npushed them to https://github.com/dscho/git as the `xdiff_rust_speedup`\nbranch). Please find them below. They _just_ fix the build, but the tests\nwith win+Meson still fail (and as \"win+Meson test\" jobs keep the logs of\nthe failed tests a well-guarded secret, due to time constraints I have to\nstop looking into this for now).\n\nThank you for working on this,\nJohannes\n\n-- snipsnap --\nFrom 72c50ee3f9df5ccfe48bf6f44b2c6bba05a680bf Mon Sep 17 00:00:00 2001\nFrom: Johannes Schindelin \nDate: Sat, 19 Jul 2025 21:24:07 +0200\nSubject: [PATCH 1/3] Do support Windows again after requiring Rust\n\nBy default, Rust wants to build MS Visual C-compatible libraries on\nWindows, because that is _the_ native C compiler.\n\nGit is historically lacking in its MSVC support, and the official Git\nfor Windows versions are built using GCC instead. As a consequence, a\n(subset of a) GCC toolchain is installed as part of the `windows-build`\njob of every CI build.\n\nNaturally, this requires adjustments in how Rust is called, most\nimportantly it requires installing support for a GCC-compatible build\ntarget.\n\nLet's make the necessary adjustment both in the CI-specific code that\ninstalls Rust as well as in the Windows-specific configuration in\n`config.mak.uname`.\n\nSigned-off-by: Johannes Schindelin \n---\n ci/install-rust.sh | 3 +++\n config.mak.uname | 9 +++++++++\n 2 files changed, 12 insertions(+)\n\ndiff --git a/ci/install-rust.sh b/ci/install-rust.sh\nindex 141ceddb17cfe..c22baa629ceb7 100644\n--- a/ci/install-rust.sh\n+++ b/ci/install-rust.sh\n@@ -28,6 +28,9 @@ if [ \"$BITNESS\" = \"32\" ]; then\n $CARGO_HOME/bin/rustup default --force-non-host $RUST_VERSION || exit $?\n else\n $CARGO_HOME/bin/rustup default $RUST_VERSION || exit $?\n+ if [ \"$CI_OS_NAME\" = \"windows\" ]; then\n+ $CARGO_HOME/bin/rustup target add x86_64-pc-windows-gnu || exit $?\n+ fi\n fi\n \n . $CARGO_HOME/env\ndiff --git a/config.mak.uname b/config.mak.uname\nindex 3e26bb074a4b5..fbe7cebf40edd 100644\n--- a/config.mak.uname\n+++ b/config.mak.uname\n@@ -727,19 +727,28 @@ ifeq ($(uname_S),MINGW)\n \t\tprefix = /mingw32\n \t\tHOST_CPU = i686\n \t\tBASIC_LDFLAGS += -Wl,--pic-executable,-e,_mainCRTStartup\n+\t\tCARGO_BUILD_TARGET = i686-pc-windows-gnu\n endif\n ifeq (MINGW64,$(MSYSTEM))\n \t\tprefix = /mingw64\n \t\tHOST_CPU = x86_64\n \t\tBASIC_LDFLAGS += -Wl,--pic-executable,-e,mainCRTStartup\n+\t\tCARGO_BUILD_TARGET = x86_64-pc-windows-gnu\n else ifeq (CLANGARM64,$(MSYSTEM))\n \t\tprefix = /clangarm64\n \t\tHOST_CPU = aarch64\n \t\tBASIC_LDFLAGS += -Wl,--pic-executable,-e,mainCRTStartup\n+\t\tCARGO_BUILD_TARGET = aarch64-pc-windows-gnu\n else\n \t\tCOMPAT_CFLAGS += -D_USE_32BIT_TIME_T\n \t\tBASIC_LDFLAGS += -Wl,--large-address-aware\n endif\n+\n+\texport CARGO_BUILD_TARGET\n+\tRUST_TARGET_DIR = rust/target/$(CARGO_BUILD_TARGET)/$(RUST_BUILD_MODE)\n+\t# Unfortunately now needed because of Rust\n+\tEXTLIBS += -luserenv\n+\n \tCC = gcc\n \tCOMPAT_CFLAGS += -D__USE_MINGW_ANSI_STDIO=0 -DDETECT_MSYS_TTY \\\n \t\t-fstack-protector-strong\n-- \n2.50.1.windows.1\n\n\nFrom ef6e4394ae26d8f28cb0d9e456810ce0818e623b Mon Sep 17 00:00:00 2001\nFrom: Johannes Schindelin \nDate: Sat, 19 Jul 2025 23:08:11 +0200\nSubject: [PATCH 2/3] win+Meson: allow for xdiff to be compiled with MSVC\n\nThe `build_rust.sh` script is quite opinionated about the naming scheme\nof the C compiler: It assumes that the xdiff library file will be named\n`libxdiff.a`.\n\nHowever, MS Visual C generates `xdiff.lib` files instead; This naming\nscheme has been in use in a very, very long time.\n\nLet's allow for that.\n\nSigned-off-by: Johannes Schindelin \n---\n build_rust.sh | 7 ++++++-\n meson.build | 12 +++++++++---\n 2 files changed, 15 insertions(+), 4 deletions(-)\n\ndiff --git a/build_rust.sh b/build_rust.sh\nindex 4c12135cd2050..694d48d857a58 100755\n--- a/build_rust.sh\n+++ b/build_rust.sh\n@@ -44,7 +44,12 @@ fi\n \n cd $dir_rust && cargo clean && pwd && cargo build -p $crate $rust_args; cd ..\n \n-libfile=\"lib${crate}.a\"\n+if grep x86_64-pc-windows-msvc rust/target/.rustc_info.json\n+then\n+ libfile=\"${crate}.lib\"\n+else\n+ libfile=\"lib${crate}.a\"\n+fi\n dst=$dir_build/$libfile\n \n if [ \"$dir_git_root\" != \"$dir_build\" ]; then\ndiff --git a/meson.build b/meson.build\nindex 047d7e5b66306..5e89a5dd0e00f 100644\n--- a/meson.build\n+++ b/meson.build\n@@ -277,8 +277,16 @@ else\n rustflags = '-Aunused_imports -Adead_code -C debuginfo=2 -C opt-level=1 -C force-frame-pointers=yes'\n endif\n \n+compiler = meson.get_compiler('c')\n+\n+if compiler.get_id() == 'msvc'\n+ xdiff_lib_filename = 'xdiff.lib'\n+else\n+ xdiff_lib_filename = 'libxdiff.a'\n+endif\n+\n rust_build_xdiff = custom_target('rust_build_xdiff',\n- output: 'libxdiff.a',\n+ output: xdiff_lib_filename,\n build_by_default: true,\n build_always_stale: true,\n command: [\n@@ -288,8 +296,6 @@ rust_build_xdiff = custom_target('rust_build_xdiff',\n install: false,\n )\n \n-compiler = meson.get_compiler('c')\n-\n libgit_sources = [\n 'abspath.c',\n 'add-interactive.c',\n-- \n2.50.1.windows.1\n\n\nFrom 9c3b017cfa069211027fbb1f6d3b97c8e7edda81 Mon Sep 17 00:00:00 2001\nFrom: Johannes Schindelin \nDate: Sat, 19 Jul 2025 23:22:57 +0200\nSubject: [PATCH 3/3] win+Meson: do allow linking with the Rust-built xdiff\n\nWhen linking against the Rust-built `xdiff`, there is now a new required\ndependency: Without _also_ linking to the system library `userenv`, the\ncompile would fail with this error message:\n\n xdiff.lib(std-c85e9beb7923f636.std.df32d1bc89881d89-cgu.0.rcgu.o) :\n error LNK2019: unresolved external symbol __imp_GetUserProfileDirectoryW\n referenced in function _ZN3std3env8home_dir17hfd1c3b6676cd78f6E\n\nTherefore, just like we do in case of Makefile-based builds on Windows,\nwe now also link to that library when building with Meson.\n\nSigned-off-by: Johannes Schindelin \n---\n meson.build | 1 +\n 1 file changed, 1 insertion(+)\n\ndiff --git a/meson.build b/meson.build\nindex 5e89a5dd0e00f..af015f04763fd 100644\n--- a/meson.build\n+++ b/meson.build\n@@ -1260,6 +1260,7 @@ elif host_machine.system() == 'windows'\n ]\n \n libgit_dependencies += compiler.find_library('ntdll')\n+ libgit_dependencies += compiler.find_library('userenv')\n libgit_include_directories += 'compat/win32'\n if compiler.get_id() == 'msvc'\n libgit_include_directories += 'compat/vcbuild/include'\n-- \n2.50.1.windows.1\n"},{"id":"522323","messageId":"5596e569-6632-c2b1-37af-a978de5408cd@gmx.de","threadId":"63804","inReplyTo":"5a959c9bdad79cf972b95dcf4324135dd7c94dac.1752784344.git.gitgitgadget@gmail.com","subject":"Re: [PATCH 6/7] xdiff: conditionally use Rust's implementation of xxhash","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2025-07-19T21:53:35Z","receivedAt":"2025-07-19T21:53:38Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Ezekiel,\n\nOn Thu, 17 Jul 2025, Ezekiel Newren via GitGitGadget wrote:\n\n> diff --git a/rust/xdiff/src/lib.rs b/rust/xdiff/src/lib.rs\n> index e69de29bb2d1..96975975a1ba 100644\n> --- a/rust/xdiff/src/lib.rs\n> +++ b/rust/xdiff/src/lib.rs\n> @@ -0,0 +1,7 @@\n> +\n> +\n> +#[no_mangle]\n> +unsafe extern \"C\" fn xxh3_64(ptr: *const u8, size: usize) -> u64 {\n> + let slice = std::slice::from_raw_parts(ptr, size);\n> + xxhash_rust::xxh3::xxh3_64(slice)\n> +}\n\nI know that this is a pretty small file, but I do notice that it does not\nhave a license header.\n\nThis reminds me of the unfortunate oversight to be careful about making\n(and keeping) libgit.a's source files compatible with libgit2's license to\nnurture a fruitful exchange between those two projects.\n\nWith Rust, we still have a really good chance to learn from history and\navoid that mistake: Gitoxide is a very exciting project with clear overlap\nin its mission to implement Git functionality in Rust. Gitoxide is\ndual-licensed under the Apache License v2 and the MIT license (see\nhttps://github.com/GitoxideLabs/gitoxide?tab=readme-ov-file#license).\n\nWould you mind adding a license header to that file that explicitly allows\nthe contents of the file to be used in Gitoxide, to get the Rust effort\nstarted on a good foot?\n\nThank you,\nJohannes\n"},{"id":"522324","messageId":"835beb3f-cc31-16cc-ea2f-55d70f700cfe@gmx.de","threadId":"63804","inReplyTo":"0de0867ab44f316911bd34b9ceddbc8606e938f2.1752784344.git.gitgitgadget@gmail.com","subject":"Re: [PATCH 7/7] github_workflows: install rust","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2025-07-19T21:54:00Z","receivedAt":"2025-07-19T21:54:04Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Ezekiel,\n\nOn Thu, 17 Jul 2025, Ezekiel Newren via GitGitGadget wrote:\n\n> +if [ \"$dir_git_root\" != \"$dir_build\" ]; then\n> + src=$dir_rust/target/$rust_target/$libfile\n> + if [ ! -f $src ]; then\n> + echo >&2 \"::error:: cannot find path of static library\"\n> + exit 5\n> + fi\n\nAs I found out the hard way, this error message could be more helpful if\nit specified a couple of those variables that play into the failure (or\nall of them).\n\nWould you mind changing the error message accordingly?\n\nThank you,\nJohannes\n"},{"id":"522334","messageId":"295a911b-b461-e66d-5f9a-501b522f0e12@gmx.de","threadId":"63804","inReplyTo":"6df9f50a8f4ca29b2c3ba1e39982b6d516146bb3.1752784344.git.gitgitgadget@gmail.com","subject":"Re: [PATCH 4/7] xdiff: make fields of xrecord_t Rust friendly","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2025-07-20T01:39:44Z","receivedAt":"2025-07-20T01:39:48Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Ezekiel,\n\nOn Thu, 17 Jul 2025, Ezekiel Newren via GitGitGadget wrote:\n\n> diff --git a/xdiff/xtypes.h b/xdiff/xtypes.h\n> index 8b8467360ecf..6e5f67ebf380 100644\n> --- a/xdiff/xtypes.h\n> +++ b/xdiff/xtypes.h\n> @@ -39,9 +39,9 @@ typedef struct s_chastore {\n> } chastore_t;\n> \n> typedef struct s_xrecord {\n> -\tchar const *ptr;\n> -\tlong size;\n> -\tunsigned long ha;\n> +\tu8 const* ptr;\n> +\tusize size;\n> +\tu64 ha;\n> } xrecord_t;\n> \n> typedef struct s_xdfile {\n\nYou cannot do this on its own, you'll also have to do the following (which\nincidentally fixes the linux32 failures as well as the win test and\nwin+Meson test failures, see\nhttps://github.com/dscho/git/actions/runs/16394351471):\n\n-- snipsnap --\nFrom 8693c83858a7c9308e54fb470cd7e82bcf67c758 Mon Sep 17 00:00:00 2001\nFrom: Johannes Schindelin \nDate: Sun, 20 Jul 2025 02:34:35 +0200\nSubject: [PATCH] fixup! xdiff: make fields of xrecord_t Rust friendly\n\nTo make `xdl_classify_record()` work, the `ha` attributes of `xrecord_t`\nand of `s_xdlclass` _must_ have the same range. Otherwise the function\nwon't be able to recognize previously-classified records correctly when\nthe `ha` recorded in the `xrecord_t` is so wide that it won't fit into\nthe `s_xdlclass`' attribute and therefore they won't match when they\nneed to match.\n\nSigned-off-by: Johannes Schindelin \n---\n xdiff/xprepare.c | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)\n\ndiff --git a/xdiff/xprepare.c b/xdiff/xprepare.c\nindex 5a2e52f102cf7..c0463bacd94b0 100644\n--- a/xdiff/xprepare.c\n+++ b/xdiff/xprepare.c\n@@ -32,7 +32,7 @@\n \n typedef struct s_xdlclass {\n \tstruct s_xdlclass *next;\n-\tunsigned long ha;\n+\tu64 ha;\n \tchar const *line;\n \tlong size;\n \tlong idx;\n-- \n2.50.1.windows.1\n\n"},{"id":"522336","messageId":"DB9P250MB06925F4833A534245FAA0A2BA552A@DB9P250MB0692.EURP250.PROD.OUTLOOK.COM","threadId":"63804","inReplyTo":"ac871bc4-df93-31f4-55f2-d6fc538a422d@gmx.de","subject":"Re: [PATCH 0/7] RFC: Accelerate xdiff and begin its rustification","fromName":"Matthias Aßhauer","fromEmail":"mha1993@live.de","sentAt":"2025-07-20T08:45:32Z","receivedAt":"2025-07-20T08:45:42Z","isPatch":true,"sender":{"key":"mha1993@live.de","avatar":"https://avatars.githubusercontent.com/u/6178234?v=4"},"body":"\n\nOn Sat, 19 Jul 2025, Johannes Schindelin wrote:\n\n> Hi Ezekiel,\n> pleasure to make your acquaintance!\n>\n> On Thu, 17 Jul 2025, Ezekiel Newren via GitGitGadget wrote:\n>\n>> 1. Windows fails to build. I don’t know which rust toolchain is even\n>> correct for this or if multiple are needed. Example failed build:\n>> https://github.com/git/git/actions/runs/16353209191\n>\n> There are a couple of problems, not just one. Here are the patches that I\n> would like to ask you to take custody of (for your convenience, I have\n> pushed them to https://github.com/dscho/git as the `xdiff_rust_speedup`\n> branch). Please find them below. They _just_ fix the build, but the tests\n> with win+Meson still fail (and as \"win+Meson test\" jobs keep the logs of\n> the failed tests a well-guarded secret, due to time constraints I have to\n> stop looking into this for now).\n>\n> Thank you for working on this,\n> Johannes\n>\n> -- snipsnap --\n> From 72c50ee3f9df5ccfe48bf6f44b2c6bba05a680bf Mon Sep 17 00:00:00 2001\n> From: Johannes Schindelin \n> Date: Sat, 19 Jul 2025 21:24:07 +0200\n> Subject: [PATCH 1/3] Do support Windows again after requiring Rust\n>\n> By default, Rust wants to build MS Visual C-compatible libraries on\n> Windows, because that is _the_ native C compiler.\n>\n> Git is historically lacking in its MSVC support, and the official Git\n> for Windows versions are built using GCC instead. As a consequence, a\n> (subset of a) GCC toolchain is installed as part of the `windows-build`\n> job of every CI build.\n>\n> Naturally, this requires adjustments in how Rust is called, most\n> importantly it requires installing support for a GCC-compatible build\n> target.\n>\n> Let's make the necessary adjustment both in the CI-specific code that\n> installs Rust as well as in the Windows-specific configuration in\n> `config.mak.uname`.\n>\n> Signed-off-by: Johannes Schindelin \n> ---\n> ci/install-rust.sh | 3 +++\n> config.mak.uname | 9 +++++++++\n> 2 files changed, 12 insertions(+)\n>\n> diff --git a/ci/install-rust.sh b/ci/install-rust.sh\n> index 141ceddb17cfe..c22baa629ceb7 100644\n> --- a/ci/install-rust.sh\n> +++ b/ci/install-rust.sh\n> @@ -28,6 +28,9 @@ if [ \"$BITNESS\" = \"32\" ]; then\n> $CARGO_HOME/bin/rustup default --force-non-host $RUST_VERSION || exit $?\n> else\n> $CARGO_HOME/bin/rustup default $RUST_VERSION || exit $?\n> + if [ \"$CI_OS_NAME\" = \"windows\" ]; then\n> + $CARGO_HOME/bin/rustup target add x86_64-pc-windows-gnu || exit $?\n> + fi\n> fi\n>\n> . $CARGO_HOME/env\n> diff --git a/config.mak.uname b/config.mak.uname\n> index 3e26bb074a4b5..fbe7cebf40edd 100644\n> --- a/config.mak.uname\n> +++ b/config.mak.uname\n> @@ -727,19 +727,28 @@ ifeq ($(uname_S),MINGW)\n> \t\tprefix = /mingw32\n> \t\tHOST_CPU = i686\n> \t\tBASIC_LDFLAGS += -Wl,--pic-executable,-e,_mainCRTStartup\n> +\t\tCARGO_BUILD_TARGET = i686-pc-windows-gnu\n\nWhile i686-pc-windows-gnu is fine for CI, it would mean we'd have to bump \nour supported Windows version up to Windows 10. If we want to keep \nsupporting Windows 8.1, we'll need i686-win7-windows-gnu, at least on rust \n1.78 and newer.[1][2] We'd probably build Windows versions on rust 1.88 \ncurrently.[3]\n\n[1] https://blog.rust-lang.org/2024/02/26/Windows-7/\n[2] https://doc.rust-lang.org/rustc/platform-support/win7-windows-gnu.html\n[3] https://packages.msys2.org/base/mingw-w64-rust\n\n> endif\n> ifeq (MINGW64,$(MSYSTEM))\n> \t\tprefix = /mingw64\n> \t\tHOST_CPU = x86_64\n> \t\tBASIC_LDFLAGS += -Wl,--pic-executable,-e,mainCRTStartup\n> +\t\tCARGO_BUILD_TARGET = x86_64-pc-windows-gnu\n\nFor x86_64 we'llprobably also want x86_64-win7-windows-gnu if we want to \nkeep Windows 8.1 support.\n\n> else ifeq (CLANGARM64,$(MSYSTEM))\n> \t\tprefix = /clangarm64\n> \t\tHOST_CPU = aarch64\n> \t\tBASIC_LDFLAGS += -Wl,--pic-executable,-e,mainCRTStartup\n> +\t\tCARGO_BUILD_TARGET = aarch64-pc-windows-gnu\n\nAre we sure this target currently exists? It's at least undocumented.[4] I \nthink we might want aarch64-pc-windows-gnullvm for CLANGARM64, either \nway.[5]\n\n[4] https://doc.rust-lang.org/rustc/platform-support/windows-gnu.html\n[5] https://doc.rust-lang.org/rustc/platform-support/windows-gnullvm.html\n\nBest regards\n\nMatthias\n\n> else\n> \t\tCOMPAT_CFLAGS += -D_USE_32BIT_TIME_T\n> \t\tBASIC_LDFLAGS += -Wl,--large-address-aware\n> endif\n> +\n> +\texport CARGO_BUILD_TARGET\n> +\tRUST_TARGET_DIR = rust/target/$(CARGO_BUILD_TARGET)/$(RUST_BUILD_MODE)\n> +\t# Unfortunately now needed because of Rust\n> +\tEXTLIBS += -luserenv\n> +\n> \tCC = gcc\n> \tCOMPAT_CFLAGS += -D__USE_MINGW_ANSI_STDIO=0 -DDETECT_MSYS_TTY \\\n> \t\t-fstack-protector-strong\n> -- \n> 2.50.1.windows.1\n>\n>\n> From ef6e4394ae26d8f28cb0d9e456810ce0818e623b Mon Sep 17 00:00:00 2001\n> From: Johannes Schindelin \n> Date: Sat, 19 Jul 2025 23:08:11 +0200\n> Subject: [PATCH 2/3] win+Meson: allow for xdiff to be compiled with MSVC\n>\n> The `build_rust.sh` script is quite opinionated about the naming scheme\n> of the C compiler: It assumes that the xdiff library file will be named\n> `libxdiff.a`.\n>\n> However, MS Visual C generates `xdiff.lib` files instead; This naming\n> scheme has been in use in a very, very long time.\n>\n> Let's allow for that.\n>\n> Signed-off-by: Johannes Schindelin \n> ---\n> build_rust.sh | 7 ++++++-\n> meson.build | 12 +++++++++---\n> 2 files changed, 15 insertions(+), 4 deletions(-)\n>\n> diff --git a/build_rust.sh b/build_rust.sh\n> index 4c12135cd2050..694d48d857a58 100755\n> --- a/build_rust.sh\n> +++ b/build_rust.sh\n> @@ -44,7 +44,12 @@ fi\n>\n> cd $dir_rust && cargo clean && pwd && cargo build -p $crate $rust_args; cd ..\n>\n> -libfile=\"lib${crate}.a\"\n> +if grep x86_64-pc-windows-msvc rust/target/.rustc_info.json\n> +then\n> + libfile=\"${crate}.lib\"\n> +else\n> + libfile=\"lib${crate}.a\"\n> +fi\n> dst=$dir_build/$libfile\n>\n> if [ \"$dir_git_root\" != \"$dir_build\" ]; then\n> diff --git a/meson.build b/meson.build\n> index 047d7e5b66306..5e89a5dd0e00f 100644\n> --- a/meson.build\n> +++ b/meson.build\n> @@ -277,8 +277,16 @@ else\n> rustflags = '-Aunused_imports -Adead_code -C debuginfo=2 -C opt-level=1 -C force-frame-pointers=yes'\n> endif\n>\n> +compiler = meson.get_compiler('c')\n> +\n> +if compiler.get_id() == 'msvc'\n> + xdiff_lib_filename = 'xdiff.lib'\n> +else\n> + xdiff_lib_filename = 'libxdiff.a'\n> +endif\n> +\n> rust_build_xdiff = custom_target('rust_build_xdiff',\n> - output: 'libxdiff.a',\n> + output: xdiff_lib_filename,\n> build_by_default: true,\n> build_always_stale: true,\n> command: [\n> @@ -288,8 +296,6 @@ rust_build_xdiff = custom_target('rust_build_xdiff',\n> install: false,\n> )\n>\n> -compiler = meson.get_compiler('c')\n> -\n> libgit_sources = [\n> 'abspath.c',\n> 'add-interactive.c',\n> -- \n> 2.50.1.windows.1\n>\n>\n> From 9c3b017cfa069211027fbb1f6d3b97c8e7edda81 Mon Sep 17 00:00:00 2001\n> From: Johannes Schindelin \n> Date: Sat, 19 Jul 2025 23:22:57 +0200\n> Subject: [PATCH 3/3] win+Meson: do allow linking with the Rust-built xdiff\n>\n> When linking against the Rust-built `xdiff`, there is now a new required\n> dependency: Without _also_ linking to the system library `userenv`, the\n> compile would fail with this error message:\n>\n> xdiff.lib(std-c85e9beb7923f636.std.df32d1bc89881d89-cgu.0.rcgu.o) :\n> error LNK2019: unresolved external symbol __imp_GetUserProfileDirectoryW\n> referenced in function _ZN3std3env8home_dir17hfd1c3b6676cd78f6E\n>\n> Therefore, just like we do in case of Makefile-based builds on Windows,\n> we now also link to that library when building with Meson.\n>\n> Signed-off-by: Johannes Schindelin \n> ---\n> meson.build | 1 +\n> 1 file changed, 1 insertion(+)\n>\n> diff --git a/meson.build b/meson.build\n> index 5e89a5dd0e00f..af015f04763fd 100644\n> --- a/meson.build\n> +++ b/meson.build\n> @@ -1260,6 +1260,7 @@ elif host_machine.system() == 'windows'\n> ]\n>\n> libgit_dependencies += compiler.find_library('ntdll')\n> + libgit_dependencies += compiler.find_library('userenv')\n> libgit_include_directories += 'compat/win32'\n> if compiler.get_id() == 'msvc'\n> libgit_include_directories += 'compat/vcbuild/include'\n> -- \n> 2.50.1.windows.1\n>"},{"id":"522338","messageId":"dd3a7ab0-947b-4592-a086-8c7028f02ffd@gmail.com","threadId":"63804","inReplyTo":"5596e569-6632-c2b1-37af-a978de5408cd@gmx.de","subject":"Re: [PATCH 6/7] xdiff: conditionally use Rust's implementation of xxhash","fromName":"Phillip Wood","fromEmail":"phillip.wood123@gmail.com","sentAt":"2025-07-20T10:14:58Z","receivedAt":"2025-07-20T10:15:21Z","isPatch":true,"sender":{"key":"phillip.wood@dunelm.org.uk","avatar":null},"body":"Hi Johannes\n\nOn 19/07/2025 22:53, Johannes Schindelin wrote:\n> Hi Ezekiel,\n> \n> On Thu, 17 Jul 2025, Ezekiel Newren via GitGitGadget wrote:\n> \n>> diff --git a/rust/xdiff/src/lib.rs b/rust/xdiff/src/lib.rs\n>> index e69de29bb2d1..96975975a1ba 100644\n>> --- a/rust/xdiff/src/lib.rs\n>> +++ b/rust/xdiff/src/lib.rs\n>> @@ -0,0 +1,7 @@\n>> +\n>> +\n>> +#[no_mangle]\n>> +unsafe extern \"C\" fn xxh3_64(ptr: *const u8, size: usize) -> u64 {\n>> + let slice = std::slice::from_raw_parts(ptr, size);\n>> + xxhash_rust::xxh3::xxh3_64(slice)\n>> +}\n> \n> I know that this is a pretty small file, but I do notice that it does not\n> have a license header.\n> \n> This reminds me of the unfortunate oversight to be careful about making\n> (and keeping) libgit.a's source files compatible with libgit2's license to\n> nurture a fruitful exchange between those two projects.\n\nI'm not sure I follow your reasoning here. libgit2 was started after git \nand chose to use an incompatible license. I wasn't around at the time \nbut isn't there a list of git contributors who are happy to re-license \ntheir contributions with the linking exception used by libgit2?\n\n> With Rust, we still have a really good chance to learn from history and\n> avoid that mistake: Gitoxide is a very exciting project with clear overlap\n> in its mission to implement Git functionality in Rust. Gitoxide is\n> dual-licensed under the Apache License v2 and the MIT license (see\n> https://github.com/GitoxideLabs/gitoxide?tab=readme-ov-file#license).\n> \n> Would you mind adding a license header to that file that explicitly allows\n> the contents of the file to be used in Gitoxide, to get the Rust effort\n> started on a good foot?\n\nI wary of that for two reasons. Firstly over time it is de-facto \nre-licensing git as the amount of rust code grows and the amount of C \ncode shrinks which deserves a wider discussion. Secondly it makes it \nharder to convert our C code which is licensed under GPL2 (or in the \ncase of xdiff LGPL) to rust if the rust code uses a different license.\n\nIf someone wants to start a discussion about re-licensing git (and is \nprepared to do all of the associated admin in the event that it happens) \nthen by all means do so but I don't think it we want to slip such a \nchange into this series.\n\nThanks\n\nPhillip\n\n"},{"id":"522344","messageId":"45ea5d1d-05dd-4f7a-bee5-ea3936d23d0a@gmail.com","threadId":"63804","inReplyTo":"xmqqjz454l96.fsf@gitster.g","subject":"Re: [PATCH 0/7] RFC: Accelerate xdiff and begin its rustification","fromName":"Phillip Wood","fromEmail":"phillip.wood123@gmail.com","sentAt":"2025-07-21T10:14:38Z","receivedAt":"2025-07-21T10:15:03Z","isPatch":true,"sender":{"key":"phillip.wood@dunelm.org.uk","avatar":null},"body":"On 18/07/2025 15:38, Junio C Hamano wrote:\n> \"Ezekiel Newren via GitGitGadget\" writes:\n> \n>> This series accelerates xdiff by 5-19%.\n> \n> ;-)\n> \n> Do we know how much of that can be attributed to the hash algorithm\n> difference, and how much for languages?\n\nThat's an interesting question. The two patches below [1] switch\nxdiff to use xxhash from libxxhash. On my computer the rust and\nC implementations both speed up \"git log --oneline --shortstat\"\nby 15%. Just over half of that seems to come from hoisting the\ncheck for whitespace flags in xdl_hash_record() out of the loop\nin xdl_prepare_ctx() and the rest comes from the change in hash\nfunction. As I understand it the hash is implemented using SIMD\ncompiler intrinsics and the rust implementation is basically a\ncopy of the C code in libxxhash. I wonder how well xxhash performs\ncompared to our existing hash on platforms without an optimized\nimplementation.\n\nThanks\n\nPhillip\n\n[1] These patches are available in the xdiff-hashing-experiments\n branch at https://github.com/phillipwood/git\n\n---- 8< ----\n From 06e7abdcfb9fc3f143ef84644966d6fce128d8ae Mon Sep 17 00:00:00 2001\nFrom: Phillip Wood \nDate: Sat, 19 Jul 2025 10:58:48 +0100\nSubject: [PATCH 1/2] xdiff: refactor xdl_hash_record()\nMIME-Version: 1.0\nContent-Type: text/plain; charset=UTF-8\nContent-Transfer-Encoding: 8bit\n\nInline the check for whitespace flags so that the compiler can hoist\nit out of the loop in xdl_prepare_ctx(). This improves the performance\nby 8%.\n\n$ hyperfine --warmup=1 -L rev HEAD,HEAD^ --setup='git checkout {rev} -- :/ && make git' ': {rev}; GIT_CONFIG_GLOBAL=/dev/null ./git log --oneline --shortstat v2.0.0..v2.5.0'\nBenchmark 1: : HEAD; GIT_CONFIG_GLOBAL=/dev/null ./git log --oneline --shortstat v2.0.0..v2.5.0\n Time (mean ┬▒ ¤â): 1.670 s ┬▒ 0.044 s [User: 1.473 s, System: 0.196 s]\n Range (min ÔǪ max): 1.619 s ÔǪ 1.754 s 10 runs\n\nBenchmark 2: : HEAD^; GIT_CONFIG_GLOBAL=/dev/null ./git log --oneline --shortstat v2.0.0..v2.5.0\n Time (mean ┬▒ ¤â): 1.801 s ┬▒ 0.021 s [User: 1.605 s, System: 0.192 s]\n Range (min ÔǪ max): 1.766 s ÔǪ 1.831 s 10 runs\n\nSummary\n ': HEAD^; GIT_CONFIG_GLOBAL=/dev/null ./git log --oneline --shortstat v2.0.0..v2.5.0' ran\n 1.08 ┬▒ 0.03 times faster than ': HEAD^^; GIT_CONFIG_GLOBAL=/dev/null ./git log --oneline --shortstat v2.0.0..v2.5.0'\n\nSigned-off-by: Phillip Wood \n---\n xdiff/xutils.c | 7 ++-----\n xdiff/xutils.h | 10 +++++++++-\n 2 files changed, 11 insertions(+), 6 deletions(-)\n\ndiff --git a/xdiff/xutils.c b/xdiff/xutils.c\nindex 444a108f87..e070ed649f 100644\n--- a/xdiff/xutils.c\n+++ b/xdiff/xutils.c\n@@ -249,7 +249,7 @@ int xdl_recmatch(const char *l1, long s1, const char *l2, long s2, long flags)\n \treturn 1;\n }\n \n-static unsigned long xdl_hash_record_with_whitespace(char const **data,\n+unsigned long xdl_hash_record_with_whitespace(char const **data,\n \t\tchar const *top, long flags) {\n \tunsigned long ha = 5381;\n \tchar const *ptr = *data;\n@@ -294,13 +294,10 @@ static unsigned long xdl_hash_record_with_whitespace(char const **data,\n \treturn ha;\n }\n \n-unsigned long xdl_hash_record(char const **data, char const *top, long flags) {\n+unsigned long xdl_hash_record_verbatim(char const **data, char const *top) {\n \tunsigned long ha = 5381;\n \tchar const *ptr = *data;\n \n-\tif (flags & XDF_WHITESPACE_FLAGS)\n-\t\treturn xdl_hash_record_with_whitespace(data, top, flags);\n-\n \tfor (; ptr < top && *ptr != '\\n'; ptr++) {\n \t\tha += (ha << 5);\n \t\tha ^= (unsigned long) *ptr;\ndiff --git a/xdiff/xutils.h b/xdiff/xutils.h\nindex fd0bba94e8..13f6831047 100644\n--- a/xdiff/xutils.h\n+++ b/xdiff/xutils.h\n@@ -34,7 +34,15 @@ void *xdl_cha_alloc(chastore_t *cha);\n long xdl_guess_lines(mmfile_t *mf, long sample);\n int xdl_blankline(const char *line, long size, long flags);\n int xdl_recmatch(const char *l1, long s1, const char *l2, long s2, long flags);\n-unsigned long xdl_hash_record(char const **data, char const *top, long flags);\n+unsigned long xdl_hash_record_verbatim(char const **data, char const *top);\n+unsigned long xdl_hash_record_with_whitespace(char const **data, char const *top, long flags);\n+static inline unsigned long xdl_hash_record(char const **data, char const *top, long flags)\n+{\n+\tif (flags & XDF_WHITESPACE_FLAGS)\n+\t\treturn xdl_hash_record_with_whitespace(data, top, flags);\n+\telse\n+\t\treturn xdl_hash_record_verbatim(data, top);\n+}\n unsigned int xdl_hashbits(unsigned int size);\n int xdl_num_out(char *out, long val);\n int xdl_emit_hunk_hdr(long s1, long c1, long s2, long c2,\n-- \n2.49.0.897.gfad3eb7d21\n\n\n From 16f3b26624dc17002f3e507cd1e260deadfe1de8 Mon Sep 17 00:00:00 2001\nFrom: Phillip Wood \nDate: Sat, 19 Jul 2025 14:52:48 +0100\nSubject: [PATCH 2/2] xdiff: use xxhash\nMIME-Version: 1.0\nContent-Type: text/plain; charset=UTF-8\nContent-Transfer-Encoding: 8bit\n\nUsing XXH3_64bits() from libxxhash to hash the input lines improves\nthe performance by about 6% and equals the performance of using\nxxhash-rust.\n\n$ hyperfine --warmup=1 -L rev en/xdiff-rust/v1,HEAD,HEAD^,HEAD^^ --setup='git checkout {rev} -- :/ && make git' ': {rev}; GIT_CONFIG_GLOBAL=/dev/null ./git log --oneline --shortstat v2.0.0..v2.5.0'\nBenchmark 1: : en/xdiff-rust/v1; GIT_CONFIG_GLOBAL=/dev/null ./git log --oneline --shortstat v2.0.0..v2.5.0\n Time (mean ┬▒ ¤â): 1.575 s ┬▒ 0.032 s [User: 1.406 s, System: 0.168 s]\n Range (min ÔǪ max): 1.541 s ÔǪ 1.651 s 10 runs\n\nBenchmark 2: : HEAD; GIT_CONFIG_GLOBAL=/dev/null ./git log --oneline --shortstat v2.0.0..v2.5.0\n Time (mean ┬▒ ¤â): 1.569 s ┬▒ 0.018 s [User: 1.382 s, System: 0.185 s]\n Range (min ÔǪ max): 1.546 s ÔǪ 1.596 s 10 runs\n\nBenchmark 3: : HEAD^; GIT_CONFIG_GLOBAL=/dev/null ./git log --oneline --shortstat v2.0.0..v2.5.0\n Time (mean ┬▒ ¤â): 1.661 s ┬▒ 0.026 s [User: 1.475 s, System: 0.186 s]\n Range (min ÔǪ max): 1.630 s ÔǪ 1.696 s 10 runs\n\nBenchmark 4: : HEAD^^; GIT_CONFIG_GLOBAL=/dev/null ./git log --oneline --shortstat v2.0.0..v2.5.0\n Time (mean ┬▒ ¤â): 1.800 s ┬▒ 0.023 s [User: 1.611 s, System: 0.187 s]\n Range (min ÔǪ max): 1.772 s ÔǪ 1.837 s 10 runs\n\nSummary\n ': HEAD; GIT_CONFIG_GLOBAL=/dev/null ./git log --oneline --shortstat v2.0.0..v2.5.0' ran\n 1.00 ┬▒ 0.02 times faster than ': en/xdiff-rust/v1; GIT_CONFIG_GLOBAL=/dev/null ./git log --oneline --shortstat v2.0.0..v2.5.0'\n 1.06 ┬▒ 0.02 times faster than ': HEAD^; GIT_CONFIG_GLOBAL=/dev/null ./git log --oneline --shortstat v2.0.0..v2.5.0'\n 1.15 ┬▒ 0.02 times faster than ': HEAD^^; GIT_CONFIG_GLOBAL=/dev/null ./git log --oneline --shortstat v2.0.0..v2.5.0'\n\nSigned-off-by: Phillip Wood \n---\n Makefile | 1 +\n xdiff/xutils.c | 14 ++++++--------\n 2 files changed, 7 insertions(+), 8 deletions(-)\n\ndiff --git a/Makefile b/Makefile\nindex 5f7dd79dfa..6de7ccdf3b 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -1390,6 +1390,7 @@ UNIT_TEST_OBJS += $(UNIT_TEST_DIR)/lib-reftable.o\n # xdiff and reftable libs may in turn depend on what is in libgit.a\n GITLIBS = common-main.o $(LIB_FILE) $(XDIFF_LIB) $(REFTABLE_LIB) $(LIB_FILE)\n EXTLIBS =\n+EXTLIBS += -lxxhash\n \n GIT_USER_AGENT = git/$(GIT_VERSION)\n \ndiff --git a/xdiff/xutils.c b/xdiff/xutils.c\nindex e070ed649f..43fce4b5b1 100644\n--- a/xdiff/xutils.c\n+++ b/xdiff/xutils.c\n@@ -21,7 +21,7 @@\n */\n \n #include \"xinclude.h\"\n-\n+#include \n \n long xdl_bogosqrt(long n) {\n \tlong i;\n@@ -295,14 +295,12 @@ unsigned long xdl_hash_record_with_whitespace(char const **data,\n }\n \n unsigned long xdl_hash_record_verbatim(char const **data, char const *top) {\n-\tunsigned long ha = 5381;\n-\tchar const *ptr = *data;\n+\tlong ha;\n+\tchar const *eol = memchr(*data, '\\n', top - *data);\n+\tsize_t len = (eol ? eol : top) - *data;\n \n-\tfor (; ptr < top && *ptr != '\\n'; ptr++) {\n-\t\tha += (ha << 5);\n-\t\tha ^= (unsigned long) *ptr;\n-\t}\n-\t*data = ptr < top ? ptr + 1: ptr;\n+\tha = XXH3_64bits(*data, len);\n+\t*data += len + !!eol;\n \n \treturn ha;\n }\n-- \n2.49.0.897.gfad3eb7d21\n\n\n"},{"id":"522365","messageId":"xmqq4iv5z94f.fsf@gitster.g","threadId":"63804","inReplyTo":"45ea5d1d-05dd-4f7a-bee5-ea3936d23d0a@gmail.com","subject":"Re: [PATCH 0/7] RFC: Accelerate xdiff and begin its rustification","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-07-21T18:33:52Z","receivedAt":"2025-07-21T18:33:55Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Phillip Wood writes:\n\n> ... by 15%. Just over half of that seems to come from hoisting the\n> check for whitespace flags in xdl_hash_record() out of the loop\n> in xdl_prepare_ctx() and the rest comes from the change in hash\n> function.\n\nThe first half of that alone is interesting enough ;-).\n\n> As I understand it the hash is implemented using SIMD\n> compiler intrinsics and the rust implementation is basically a\n> copy of the C code in libxxhash. I wonder how well xxhash performs\n> compared to our existing hash on platforms without an optimized\n> implementation.\n\nYeah, that indeed is worth investigating.\n\nThanks.\n\n"},{"id":"522436","messageId":"aH-CN0RYFmpm7fMt@pks.im","threadId":"63804","inReplyTo":"aHmVXDOiKzfKU8nb@fruit.crustytoothpaste.net","subject":"Re: [PATCH 0/7] RFC: Accelerate xdiff and begin its rustification","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-07-22T12:21:11Z","receivedAt":"2025-07-22T12:21:23Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Fri, Jul 18, 2025 at 12:29:16AM +0000, brian m. carlson wrote:\n> On 2025-07-17 at 22:25:23, Taylor Blau wrote:\n> > I agree. I don't think that there is ever going to be a \"perfect\" time\n> > to introduce a hard dependency on Rust, and I don't think that should\n> > hold the project back from adopting it.\n> > \n> > I am far from a Rust expert, but I think that a more modern, memory-safe\n> > language will attract newer contributors who may have a fresher\n> > perspective on the project, and I think that's a good thing.\n> \n> Yes, I think that's true. Rust is by far the most admired programming\n> language to work with, according to the 2024 Stack Overflow Developer\n> Survey. We will likely attract new contributors who find C intimidating\n> or a bit of a hassle[0] but are excited about working on Rust,\n> especially in a project as compelling as Git[1].\n\nI am also aligned with allowing Rust into Git. I think the ecosystem has\nkind of settled on Rust as the next system-level programming language,\nand it does have good interop with C.\n\nI think with the ongoing efforts to reduce our reliance on global state\nwe should eventually be able to encapsulate more and more of our\nsubsystems. And once they are neatly encapsulated we would be able to\nswap out their respective implementation and plug in a Rust replacement.\n\nGood candidates are for example the reftable library, as I've already\nproposed in the past.\n\n> > The alternative, of course, is to continue to use C and not take any\n> > dependency on Rust. I think there is a middle-ground in there somewhere\n> > to be able to build with (e.g.) \"make\" or \"make RUST=1\", but I would\n> > really like to see the project take a firmer stance here.\n> > \n> > I worry that having build support for both \"with Rust\" and \"C only\" will\n> > create a headache not just at the build system level, but also in the\n> > code itself. Having a patchwork of features, optimizations, or bug fixes\n> > that either are or aren't supported depending on whether Rust support\n> > was specified at build-time seems like a worst-of-all-worlds outcome.\n> \n> I definitely agree. I already find it terribly inconvenient when I end\n> up when `git grep` doesn't support `-P` and I imagine that having lots\n> of features that weren't available would be bothersome.\n> \n> I also think that using a combination of C and Rust will end up with us\n> still writing a lot of unsafe Rust code to interoperate with C. If we\n> want to reap the benefits in terms of memory and thread safety[2], we'll\n> be better off sticking with just Rust.\n> \n> I will also say that while it may be more challenging to compile Git at\n> first on Windows, as we move more towards an all-Rust codebase, Git may\n> end up being easier to maintain there as we depend more on the standard\n> library.\n\nFully agreed. I've said so at the last contributors summit, but I think\nit would become awfully unmaintainable if we retain two implementations\nof every subsystem that we convert to Rust. If we decide to use Rust I\nwould strongly advocate for going all-in.\n\nPatrick\n"},{"id":"522437","messageId":"aH-CROwg57Juo9mH@pks.im","threadId":"63804","inReplyTo":"aHrrZyrDw_CYmFQF@cloudsdale.the-delta.net.eu.org","subject":"Re: [PATCH 0/7] RFC: Accelerate xdiff and begin its rustification","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-07-22T12:21:24Z","receivedAt":"2025-07-22T12:21:30Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Sat, Jul 19, 2025 at 02:48:39AM +0200, Haelwenn (lanodan) Monnier wrote:\n> [2025-07-18 17:25:01-0400] Eli Schwartz:\n> > On 7/18/25 9:34 AM, Phillip Wood wrote:\n> > > Hi Ezekiel\n> > > \n> > > Thanks for working on this\n> > > \n> > > On 17/07/2025 21:32, Ezekiel Newren via GitGitGadget wrote:\n> > > \n> > > > So...\n> > > > \n> > > > This obviously raises the question of whether we are ready to accept a\n> > > > hard\n> > > > dependency on Rust. Previous discussions on the mailing list and at Git\n> > > > Merge 2024 have not answered that question. If not now, will we be\n> > > > willing\n> > > > to accept such a hard dependency later? And what route do we want to\n> > > > take to\n> > > > get there?\n> > > \n> > > As far as git goes I think introducing a hard dependency on rust is\n> > > fine. It is widely supported, the only issue I'm aware of is the lack of\n> > > support on NonStop and I don't think it is reasonable for such a\n> > > minority platform to hold the rest of the project to ransom. There is a\n> > > question about the other users of the xdiff code though. libgit2 carries\n> > > a copy as do other projects like neovim. I've cc'd the libgit2\n> > > maintainer and posted a link to this thread in neovim github [1]\n> > \n> > \n> > A hard dependency on rust for Gentoo amd64 would potentially require\n> > building https://github.com/thepowersgang/mrustc followed by building 13\n> > and counting versions of rustc in order to get to the latest version.\n> > What is the minimum supported version in this series, by the way?\n> > \n> > bin packages for rust do exist but not everyone wants to use non-distro\n> > provided binaries, sometimes for auditability reasons.\n> > \n> > \n> > For Gentoo HPPA, Alpha, m68k it will simply mean the removal (or end of\n> > life and staying forever on 2.50, perhaps) of Git. There is no rust\n> > compiler there.\n> > \n> > Even s390 support for rust is limited to a precompiled version not\n> > everyone is willing to use.\n> \n> Also in other distro concerns, if it trickles down to libgit2,\n> extra care should be taken to avoid creating circular dependencies\n> due to cargo depending on libgit2 (via git2 crate).\n> \n> For example with making sure it can reasonably be built via meson's\n> Rust support rather than through cargo.\n\nI think it's unlikely that this eventually trickles down into libgit2.\nThe bundled versions of xdiff have already diverged for a long time, and\nunfortunately libgit2 is mostly in maintenance mode nowadays. So I guess\nthat this change here just means that things will diverge even further\nin the future, which is probably okay-ish. After all, the whole xdiff\nlibrary didn't really evolve in a fast pace over the last years.\n\nThat being said, there is an xdiff fork located at [1] that libgit2\nmaintains nowadays. So if the Rust dependency ever became a problem for\nany of the downstream users I think we could simply redirect them to\nthat fork and make it the canonical upstream for C-only xdiff.\n\nPatrick\n\n[1]: https://github.com/libgit2/xdiff\n"},{"id":"522443","messageId":"aH-fDEX7gdpALJ6w@pks.im","threadId":"63804","inReplyTo":"79c1b3ab-af2e-4c93-b033-349221d82ad9@gentoo.org","subject":"Re: [PATCH 0/7] RFC: Accelerate xdiff and begin its rustification","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-07-22T14:24:12Z","receivedAt":"2025-07-22T14:24:19Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Fri, Jul 18, 2025 at 05:25:01PM -0400, Eli Schwartz wrote:\n> On 7/18/25 9:34 AM, Phillip Wood wrote:\n> > Hi Ezekiel\n> > \n> > Thanks for working on this\n> > \n> > On 17/07/2025 21:32, Ezekiel Newren via GitGitGadget wrote:\n> >\n> >> So...\n> >>\n> >> This obviously raises the question of whether we are ready to accept a\n> >> hard\n> >> dependency on Rust. Previous discussions on the mailing list and at Git\n> >> Merge 2024 have not answered that question. If not now, will we be\n> >> willing\n> >> to accept such a hard dependency later? And what route do we want to\n> >> take to\n> >> get there?\n> > \n> > As far as git goes I think introducing a hard dependency on rust is\n> > fine. It is widely supported, the only issue I'm aware of is the lack of\n> > support on NonStop and I don't think it is reasonable for such a\n> > minority platform to hold the rest of the project to ransom. There is a\n> > question about the other users of the xdiff code though. libgit2 carries\n> > a copy as do other projects like neovim. I've cc'd the libgit2\n> > maintainer and posted a link to this thread in neovim github [1]\n> \n> \n> A hard dependency on rust for Gentoo amd64 would potentially require\n> building https://github.com/thepowersgang/mrustc followed by building 13\n> and counting versions of rustc in order to get to the latest version.\n> What is the minimum supported version in this series, by the way?\n> \n> bin packages for rust do exist but not everyone wants to use non-distro\n> provided binaries, sometimes for auditability reasons.\n> \n> \n> For Gentoo HPPA, Alpha, m68k it will simply mean the removal (or end of\n> life and staying forever on 2.50, perhaps) of Git. There is no rust\n> compiler there.\n> \n> Even s390 support for rust is limited to a precompiled version not\n> everyone is willing to use.\n> \n> GCC-rs will probably fix this general issue.\n\nHm. It would be nice to assemble a list of common or semi-common\ndistributions that do not have proper support for Rust for all or at\nleast some platforms. Should we maybe consider reaching out to other\ndistros (e.g. Debian, Fedora, BSDs) before we commit to any change that\nhas an outsized impact on the larger ecosystem?\n\nI would really love to start adopting Rust, and if it's only going to be\narchitectures that are extremely niche I'm probably fine with that. But\nif there are many small systems that are impacted by such a change we\nmight have to reconsider.\n\nMeh :/\n\nPatrick\n"},{"id":"522446","messageId":"4bb0e99a-0c0d-49e4-82f8-02443a7f6783@gentoo.org","threadId":"63804","inReplyTo":"aH-fDEX7gdpALJ6w@pks.im","subject":"Re: [PATCH 0/7] RFC: Accelerate xdiff and begin its rustification","fromName":"Eli Schwartz","fromEmail":"eschwartz@gentoo.org","sentAt":"2025-07-22T15:14:21Z","receivedAt":"2025-07-22T15:14:30Z","isPatch":true,"sender":{"key":"eschwartz@gentoo.org","avatar":"https://avatars.githubusercontent.com/u/6551424?v=4"},"body":"On 7/22/25 10:24 AM, Patrick Steinhardt wrote:\n> On Fri, Jul 18, 2025 at 05:25:01PM -0400, Eli Schwartz wrote:\n\n>> For Gentoo HPPA, Alpha, m68k it will simply mean the removal (or end of\n>> life and staying forever on 2.50, perhaps) of Git. There is no rust\n>> compiler there.\n>>\n>> Even s390 support for rust is limited to a precompiled version not\n>> everyone is willing to use.\n>>\n>> GCC-rs will probably fix this general issue.\n> \n> Hm. It would be nice to assemble a list of common or semi-common\n> distributions that do not have proper support for Rust for all or at\n> least some platforms. Should we maybe consider reaching out to other\n> distros (e.g. Debian, Fedora, BSDs) before we commit to any change that\n> has an outsized impact on the larger ecosystem?\n> \n> I would really love to start adopting Rust, and if it's only going to be\n> architectures that are extremely niche I'm probably fine with that. But\n> if there are many small systems that are impacted by such a change we\n> might have to reconsider.\n> \n> Meh :/\n\n\nTo elaborate a bit w.r.t. Gentooo.\n\nGentoo Prefix-on-macOS and Prefix-on-Solaris don't support rust either.\nI think at least macOS is reasonably popular. Obviously Rust supports\nmacOS, and the Prefix maintainer would like it to work but hasn't been\nable to -- no idea why. Arguably you can tell these users \"install a\nbetter OS so you can use git\".\n\nmusl has lots of issues with rust, and is disabled for Gentoo musl\neditions on arm (not arm64), ppc, i686, m68k, mips. Arguably you can\ntell these users \"musl sucks, why are you using it, use glibc like a\nsensible person\".\n\ni486 is entirely disabled due to mandatory sse2. Hopefully those users\nare rare even compared to i686 users. ;)\n\ns390 only works on s390x\n\nsparc 64ul works, but 32ul does not.\n\nriscv rv64gc works, rv32imac does not.\n\nA general trend here is 32-bit issues.\n\n\nFor alpha/hppa, no references at all -- not even tier 3 support -- on\nhttps://doc.rust-lang.org/beta/rustc/platform-support.html, and Gentoo\ndoesn't support LLVM there either. ;) In general, porting rustc to a new\narch means *first* porting LLVM, and then after that, *also* porting\nrustc, so who's going to try the latter before the former? ;)\n\nHence the interest in GCC-rs, which already has a backend supporting all\nthis for C/C++/Fortran plus interest by users of these arches in a\nportable rust compiler.\n\nIf rust is added and doesn't have a fallback C impl, all this becomes a\nrelevant topic for consideration. (I don't have strong opinions on\noptional rust.)\n\n\n...\n\n\nSee\n\n$ git clone https://github.com/gentoo/gentoo && cd gentoo\n$ git grep --name-only features/wd40 profiles/| grep -v 17.0\n\n\n(wd40 is the inheritance tree for disabling all features in any package\nthat rely on a rust compiler. See README at\nhttps://github.com/gentoo/gentoo/tree/master/profiles/features/wd40 for\ndetails)\n\n-- \nEli Schwartz\n"},{"id":"522461","messageId":"xmqqa54wte23.fsf@gitster.g","threadId":"63804","inReplyTo":"aH-CN0RYFmpm7fMt@pks.im","subject":"Re: [PATCH 0/7] RFC: Accelerate xdiff and begin its rustification","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-07-22T15:56:04Z","receivedAt":"2025-07-22T15:56:07Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Patrick Steinhardt writes:\n\n> Fully agreed. I've said so at the last contributors summit, but I think\n> it would become awfully unmaintainable if we retain two implementations\n> of every subsystem that we convert to Rust. If we decide to use Rust I\n> would strongly advocate for going all-in.\n\nTrue. \n\nWe do not have subsystems with clear boundaries yet, and introducing\nRust in such a state would not allow us to pick some parts (e.g.\nmerge backends, etc.) and do them optionally in Rust, while keeping\nand/or adding others in C.\n"},{"id":"522462","messageId":"874iv4gqxv.fsf@gentoo.org","threadId":"63804","inReplyTo":"aH-fDEX7gdpALJ6w@pks.im","subject":"Re: [PATCH 0/7] RFC: Accelerate xdiff and begin its rustification","fromName":"Sam James","fromEmail":"sam@gentoo.org","sentAt":"2025-07-22T15:56:12Z","receivedAt":"2025-07-22T15:56:17Z","isPatch":true,"sender":{"key":"sam@gentoo.org","avatar":"https://avatars.githubusercontent.com/u/11667869?v=4"},"body":"There's a few issues from our perspective:\n\n* Old platforms which don't have LLVM can't yet have Rust either, as\n rustc is based on LLVM.\n\n These need gccrs to be unblocked. I can understand not caring too much\n about these, though it is unfortunate, because I think if git hadn't\n supported many platforms to begin with, I doubt it'd have the adoption\n it does today.\n\n (There is another effort which seeks to take rustc and bolt on\n libgccjit as a replacement backend, but that isn't feasible for use\n yet either.)\n\n* New platforms where rustc or various Rust crates don't support it\n and we have to go around patching them.\n\n The crate model makes this much harder. Not having git available when\n doing such porting if doing it natively is going to suck. It also\n means even more software needs Rust ported first.\n \n* Platforms which aren't ancient, just not \"the default\", which tend not\n to work well with Rust.\n\n For example, rustc assumes that all musl configurations will be\n statically linked, which isn't the case. Working around this is a\n hassle.\n\n* rustc doesn't have LTS releases or the like.\n\n The only supported release is the latest one. Upgrading to the latest\n release often means we have to deal with new portability problems\n but we can't not upgrade because:\n a) some software will start to require bleeding-edge Rust immediately,\n and\n b) it means we're missing out on bug fixes (miscompilations are\n serious)\n\n* Crate creep\n\n Rust projects tend to end up having a huge list of crates that they\n pull-in which makes us worried about something nasty creeping in, but\n there's also popular crates with serious portability problems like the\n 'ring' crate for TLS.\n\ngit is a fundamental piece of system software and making it harder to\nbuild it or use it is a real worry for us.\n"},{"id":"522465","messageId":"87seiofc0x.fsf@gentoo.org","threadId":"63804","inReplyTo":"aHl4U98BBvpA5eKF@nand.local","subject":"Re: [PATCH 0/7] RFC: Accelerate xdiff and begin its rustification","fromName":"Sam James","fromEmail":"sam@gentoo.org","sentAt":"2025-07-22T16:03:42Z","receivedAt":"2025-07-22T16:03:46Z","isPatch":true,"sender":{"key":"sam@gentoo.org","avatar":"https://avatars.githubusercontent.com/u/11667869?v=4"},"body":"> I am far from a Rust expert, but I think that a more modern, memory-safe\n> language will attract newer contributors who may have a fresher\n> perspective on the project, and I think that's a good thing.\n\nAren't they likely to contribute to gitoxide? There, they get a clean\nslate without having to deal with the least-fun part (bidings).\n\n> It is also not the Git project's responsibility to ensure that every\n> platform is Rust-friendly.\n\nThat's true, of course. And nobody is entitled to indefinie updates, but\non the other hand, there's still some implicit contract with users. I\nreally don't think git would have the adoption it does today if it had\nadopted a Rust-like language in the same state Rust is now from the\nstart.\n\n(In exactly the same way, git doesn't gratuitously break compatibility\nevery release either. Can it? Yes, and git can change the platforms it\nruns on, but it's something to be taken seriously.)\n\n> Hopefully the platforms that we currently support but won't after this\n> patch series have niche enough workloads that they do not need the\n> absolute latest-and-greatest Git release at all times.\n\nI mention this in my other email, but it's not just about ancient\nplatforms. It's also about new ones, or ones where Rust supports them\npoorly despite them being relevant.\n\n> Yeah, I think that this is the most interesting part of the discussion\n> here. I am not knowledgeable enough about Rust's release cadence and\n> platform compatibility to have an opinion here. But I trust brian's\n> judgement ;-).\n\nIt gets a new release every 6 weeks and no other releases are supported.\n"},{"id":"522499","messageId":"CABPp-BEf2O12jx-wN5ig941SyoL=X2OJkQY26bac=8+v+jx8ZQ@mail.gmail.com","threadId":"63804","inReplyTo":"87seiofc0x.fsf@gentoo.org","subject":"Re: [PATCH 0/7] RFC: Accelerate xdiff and begin its rustification","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2025-07-22T21:37:11Z","receivedAt":"2025-07-22T21:37:24Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"Hi,\n\nOn Tue, Jul 22, 2025 at 9:03 AM Sam James wrote:\n\nFirst of all, thanks to all the Gentoo folks for chiming in and\nproviding specifics about platforms and their state.\n\n> > I am far from a Rust expert, but I think that a more modern, memory-safe\n> > language will attract newer contributors who may have a fresher\n> > perspective on the project, and I think that's a good thing.\n>\n> Aren't they likely to contribute to gitoxide? There, they get a clean\n> slate without having to deal with the least-fun part (bidings).\n\nI'm sure some are. But clearly there are others where the draw is\nimproving git itself because of its installed base; in fact, we need\nlook no further than this exact series we are commenting on to find\nproof of that -- one such new contributor submitted patches to use\nRust in git, and found a significant speedup while doing so.\n\nFurther, there's considerable interest from existing git developers to\nuse Rust in git as well; last year at the Git contributor summit,\nusage of Rust in git was not only one of the topics of discussion, it\nwas the top voted topic (meaning, the topic that the greatest number\nof git contributors wanted to discuss).\n\n> > It is also not the Git project's responsibility to ensure that every\n> > platform is Rust-friendly.\n>\n> That's true, of course. And nobody is entitled to indefinie updates, but\n> on the other hand, there's still some implicit contract with users. I\n> really don't think git would have the adoption it does today if it had\n> adopted a Rust-like language in the same state Rust is now from the\n> start.\n>\n> (In exactly the same way, git doesn't gratuitously break compatibility\n> every release either. Can it? Yes, and git can change the platforms it\n> runs on, but it's something to be taken seriously.)\n\nThis feels kind of close to a false dichotomy between breaking\ncompatibility every release and indefinite update entitlements. There\nis certainly some middle ground: discussing reducing the breadth of\nplatform support in order to gain other benefits, then gathering\nfeedback, making a plan, and announcing the upcoming change, etc.\n\nAnd we're already pretty deep into it. Concerns about losing out on\nsome platforms have repeatedly slowed us down from adopting Rust years\nago. Yet, the desire for Rust adoption keeps coming up anyway; see\nthe threads starting at\n\n * https://lore.kernel.org/git/ZZ77NQkSuiRxRDwt@nand.local/\n * https://lore.kernel.org/git/Zu2D%2Fb1ZJbTlC1ml@nand.local/\n * https://lore.kernel.org/git/20241128-pks-meson-v10-22-79a3fb0cb3a6@pks.im/\n(search for \"Rust\")\n * https://lore.kernel.org/git/cover.1723242556.git.steadmon@google.com/\n\nThe discussion has also been picked up and reported outside the Git\nmailing list, e.g. https://lwn.net/Articles/998115/.\n\nAnd so, in addition to the optional contrib/libgit-rs and\ncontrib/libgit-sys Rust components that have already been merged into\ngit, and a new build system added in part to make it easier to adopt\nRust, we now have the first patch series that proposes a hard\ndependency on Rust.\n\nFurther, I'd like to comment a bit on the support of our users from\nanother angle. We're also responsible for security for our users, and\nfeel Rust would help (see e.g.\nhttps://litchipi.github.io/infosec/2023/01/24/git-code-audit-viewed-as-rust-programmer.html\nand https://github.com/bk2204/git/commit/fbeb1180c7473635a964daed2da642c53487782d).\nWe're responsible for performance of Git for our users, and feel Rust\nwould help (see the email that started this thread,\nhttps://lore.kernel.org/git/CABPp-BFOmwV-xBtjvtenb6RFz9wx2VWVpTeho0k=D8wsCCVwqQ@mail.gmail.com/,\nand brian's notes about [CPU multi-]threading elsewhere in this email\nthread we are in). And there are other benefits from using Rust that\nwe believe would benefit our users. Thus, it's not just a question of\nresponsibility to our users, because such a responsibility pulls us in\ndifferent directions regarding usage of Rust. So we need to figure\nout how to weigh the needs of our different users. For many of us,\nand forgive the geeky comparison, we'll probably weigh those needs\nwith something more akin to an L2 norm (most good for the most users)\nrather than an L-infinity norm (maximal difference in usability for a\nsingle user), which probably isn't to your liking.\n\nAnyway, there's been lots of discussion already. We can certainly\nstill discuss more about exactly how to announce, when to adopt Rust,\nwhether we'll support an existing C-only version of git for a longer\nperiod of time than normal, and even whether to continue to delay\nadopting Rust for a little longer. But my personal guess is that\nattempting to stop adoption of Rust is unlikely to win at this point.\n\n> > Hopefully the platforms that we currently support but won't after this\n> > patch series have niche enough workloads that they do not need the\n> > absolute latest-and-greatest Git release at all times.\n>\n> I mention this in my other email, but it's not just about ancient\n> platforms. It's also about new ones, or ones where Rust supports them\n> poorly despite them being relevant.\n\nThis feels like you're trying to push the decision for a given\nplatform to be a dichotomy between latest-and-greatest-Git or\nno-version-of-Git-at-all, despite the fact that Taylor suggested an\nalternative and you even quoted him. Can you comment on that\nalternative? Why would using the last C-only version of Git[1] until\ngccrs bridges the gap be a problem for these platforms?\n\nThanks,\nElijah\n\n[1] Well, C-only other than optional Rust components like\ncontrib/libgit-rs and contrib/libgit-sys that have already been\nreleased.\n"},{"id":"522502","messageId":"87o6tbevql.fsf@gentoo.org","threadId":"63804","inReplyTo":"CABPp-BEf2O12jx-wN5ig941SyoL=X2OJkQY26bac=8+v+jx8ZQ@mail.gmail.com","subject":"Re: [PATCH 0/7] RFC: Accelerate xdiff and begin its rustification","fromName":"Sam James","fromEmail":"sam@gentoo.org","sentAt":"2025-07-22T21:55:30Z","receivedAt":"2025-07-22T21:55:34Z","isPatch":true,"sender":{"key":"sam@gentoo.org","avatar":"https://avatars.githubusercontent.com/u/11667869?v=4"},"body":"Elijah Newren writes:\n\n> Hi,\n>\n> On Tue, Jul 22, 2025 at 9:03 AM Sam James wrote:\n>\n> First of all, thanks to all the Gentoo folks for chiming in and\n> providing specifics about platforms and their state.\n\nThanks. I've been trying to be very specific about what the issues\nare. I don't deny Rust is the future of many projects, but it still has\nrough parts, and I'd like to ensure they're discussed. It is not my\nintention to just scream whenever someone considers adopting Rust.\n\n>\n>> > I am far from a Rust expert, but I think that a more modern, memory-safe\n>> > language will attract newer contributors who may have a fresher\n>> > perspective on the project, and I think that's a good thing.\n>>\n>> Aren't they likely to contribute to gitoxide? There, they get a clean\n>> slate without having to deal with the least-fun part (bidings).\n>\n> I'm sure some are. But clearly there are others where the draw is\n> improving git itself because of its installed base; in fact, we need\n> look no further than this exact series we are commenting on to find\n> proof of that -- one such new contributor submitted patches to use\n> Rust in git, and found a significant speedup while doing so.\n\nPart of my opinion there is coloured by how generally working on a\npolylang codebase often has pain when dealing with bindings and the\nedges, so I figure that anyone most-keen on Rust would surely want to\navoid that ;)\n\n>\n> Further, there's considerable interest from existing git developers to\n> use Rust in git as well; last year at the Git contributor summit,\n> usage of Rust in git was not only one of the topics of discussion, it\n> was the top voted topic (meaning, the topic that the greatest number\n> of git contributors wanted to discuss).\n>\n>> > It is also not the Git project's responsibility to ensure that every\n>> > platform is Rust-friendly.\n>>\n>> That's true, of course. And nobody is entitled to indefinie updates, but\n>> on the other hand, there's still some implicit contract with users. I\n>> really don't think git would have the adoption it does today if it had\n>> adopted a Rust-like language in the same state Rust is now from the\n>> start.\n>>\n>> (In exactly the same way, git doesn't gratuitously break compatibility\n>> every release either. Can it? Yes, and git can change the platforms it\n>> runs on, but it's something to be taken seriously.)\n>\n> This feels kind of close to a false dichotomy between breaking\n> compatibility every release and indefinite update entitlements. There\n> is certainly some middle ground: discussing reducing the breadth of\n> platform support in order to gain other benefits, then gathering\n> feedback, making a plan, and announcing the upcoming change, etc.\n>\n\nOf course. I'm just making the point that it is indeed a compatibility\nchange, and perhaps that perspective is useful.\n\n> And we're already pretty deep into it. Concerns about losing out on\n> some platforms have repeatedly slowed us down from adopting Rust years\n> ago. Yet, the desire for Rust adoption keeps coming up anyway; see\n> the threads starting at\n>\n> * https://lore.kernel.org/git/ZZ77NQkSuiRxRDwt@nand.local/\n> * https://lore.kernel.org/git/Zu2D%2Fb1ZJbTlC1ml@nand.local/\n> * https://lore.kernel.org/git/20241128-pks-meson-v10-22-79a3fb0cb3a6@pks.im/\n> (search for \"Rust\")\n> * https://lore.kernel.org/git/cover.1723242556.git.steadmon@google.com/\n>\n> The discussion has also been picked up and reported outside the Git\n> mailing list, e.g. https://lwn.net/Articles/998115/.\n>\n> And so, in addition to the optional contrib/libgit-rs and\n> contrib/libgit-sys Rust components that have already been merged into\n> git, and a new build system added in part to make it easier to adopt\n> Rust, we now have the first patch series that proposes a hard\n> dependency on Rust.\n\nYes, that's why it's of concern. I have no issue with the optional parts.\n\n>\n> Further, I'd like to comment a bit on the support of our users from\n> another angle. We're also responsible for security for our users\n\nSupply-chain issues become more of a problem with Rust if we end up\nmaking heavy use of crates. A policy moderating their use is something\nwe should talk about.\n\n> and\n> feel Rust would help (see e.g.\n> https://litchipi.github.io/infosec/2023/01/24/git-code-audit-viewed-as-rust-programmer.html\n> and https://github.com/bk2204/git/commit/fbeb1180c7473635a964daed2da642c53487782d).\n> We're responsible for performance of Git for our users, and feel Rust\n> would help (see the email that started this thread,\n> https://lore.kernel.org/git/CABPp-BFOmwV-xBtjvtenb6RFz9wx2VWVpTeho0k=D8wsCCVwqQ@mail.gmail.com/,\n> and brian's notes about [CPU multi-]threading elsewhere in this email\n> thread we are in). And there are other benefits from using Rust that\n> we believe would benefit our users. Thus, it's not just a question of\n> responsibility to our users, because such a responsibility pulls us in\n> different directions regarding usage of Rust. So we need to figure\n> out how to weigh the needs of our different users. For many of us,\n> and forgive the geeky comparison, we'll probably weigh those needs\n> with something more akin to an L2 norm (most good for the most users)\n> rather than an L-infinity norm (maximal difference in usability for a\n> single user), which probably isn't to your liking.\n>\n\n:)\n\n> Anyway, there's been lots of discussion already. We can certainly\n> still discuss more about exactly how to announce, when to adopt Rust,\n> whether we'll support an existing C-only version of git for a longer\n> period of time than normal, and even whether to continue to delay\n> adopting Rust for a little longer. But my personal guess is that\n> attempting to stop adoption of Rust is unlikely to win at this point.\n\nI wouldn't characterise my position as attempting to flat-out stop\nadoption of Rust (see beginning of this email).\n\n>\n>> > Hopefully the platforms that we currently support but won't after this\n>> > patch series have niche enough workloads that they do not need the\n>> > absolute latest-and-greatest Git release at all times.\n>>\n>> I mention this in my other email, but it's not just about ancient\n>> platforms. It's also about new ones, or ones where Rust supports them\n>> poorly despite them being relevant.\n>\n> This feels like you're trying to push the decision for a given\n> platform to be a dichotomy between latest-and-greatest-Git or\n> no-version-of-Git-at-all, despite the fact that Taylor suggested an\n> alternative and you even quoted him. Can you comment on that\n> alternative? Why would using the last C-only version of Git[1] until\n> gccrs bridges the gap be a problem for these platforms?\n\nWhat I was saying there was: it matters for platforms where they may not\nhave a git at all (because they're new, and we have a bit of a\nbootstrapping problem), not just old ones where they're stuck on an old git.\n\nPart of what I had in mind here is that sticking on old versions even\ntemporarily isn't necessarily a great option, see the recent issues w/\nbackports done (https://lore.kernel.org/git/xmqqldov4rpt.fsf@gitster.g/,\nand https://lore.kernel.org/git/20250708210529.1214574-1-tmz@pobox.com/).\n\n---\n\nAs a final note: I am genuinely not trying to be a member of the peanut\ngallery wishing to prevent git's progress if the project wants to adopt\nRust, just there's some real practical obstacles for us right now.\n\nI hope it didn't come across that way, but \"dichotomy\" appearing a few\ntimes in your reply made me fear it did.\n\n>\n> Thanks,\n> Elijah\n\nthanks,\nsam\n\n>\n> [1] Well, C-only other than optional Rust components like\n> contrib/libgit-rs and contrib/libgit-sys that have already been\n> released.\n"},{"id":"522504","messageId":"871pq7yj2x.fsf@gmail.com","threadId":"63804","inReplyTo":"87o6tbevql.fsf@gentoo.org","subject":"Re: [PATCH 0/7] RFC: Accelerate xdiff and begin its rustification","fromName":"Collin Funk","fromEmail":"collin.funk1@gmail.com","sentAt":"2025-07-22T22:08:38Z","receivedAt":"2025-07-22T22:08:40Z","isPatch":true,"sender":{"key":"collin.funk1@gmail.com","avatar":"https://avatars.githubusercontent.com/u/65689063?v=4"},"body":"Sam James writes:\n\n>> Further, I'd like to comment a bit on the support of our users from\n>> another angle. We're also responsible for security for our users\n>\n> Supply-chain issues become more of a problem with Rust if we end up\n> making heavy use of crates. A policy moderating their use is something\n> we should talk about.\n\n+1. I find it a bit worrying when I see 500+ dependencies (mostly\ntransitive) being downloaded when running 'cargo build'.\n\nNot saying we should go to the extreme of Not Invented Here syndrome\n[1], since easy use of packages via 'cargo' is a major reason why people\nenjoy Rust. But we should consider whether they provide enough value to\nbe included.\n\nCollin\n\n[1] https://en.wikipedia.org/wiki/Not_invented_here\n"},{"id":"522506","messageId":"20250722220233.7rdako7zjwyjqlmv@glandium.org","threadId":"63804","inReplyTo":"aHlrg7pbFqi2qNWH@fruit.crustytoothpaste.net","subject":"Re: [PATCH 1/7] xdiff: introduce rust","fromName":"Mike Hommey","fromEmail":"mh@glandium.org","sentAt":"2025-07-22T22:02:33Z","receivedAt":"2025-07-22T22:18:45Z","isPatch":true,"sender":{"key":"mh@glandium.org","avatar":"https://avatars.githubusercontent.com/u/1038527?v=4"},"body":"On Thu, Jul 17, 2025 at 09:30:43PM +0000, brian m. carlson wrote:\n> On 2025-07-17 at 20:32:18, Ezekiel Newren via GitGitGadget wrote:\n> > diff --git a/rust/Cargo.lock b/rust/Cargo.lock\n> > new file mode 100644\n> > index 000000000000..fb1eac690b39\n> > --- /dev/null\n> > +++ b/rust/Cargo.lock\n> > @@ -0,0 +1,14 @@\n> > +# This file is automatically @generated by Cargo.\n> > +# It is not intended for manual editing.\n> > +version = 4\n> > +\n> > +[[package]]\n> > +name = \"interop\"\n> > +version = \"0.1.0\"\n> > +\n> > +[[package]]\n> > +name = \"xdiff\"\n> > +version = \"0.1.0\"\n> > +dependencies = [\n> > + \"interop\",\n> > +]\n> \n> I would prefer that we not check in Cargo.lock in Git. Part of the\n> reason is that it changes across versions and so building with a\n> different version of the toolchain can update the file.\n\nThat actually doesn't happen unless the file needs to be updated for\nsome reason, like Cargo.toml having new dependencies or `cargo update`\nbeing run.\n\nMike\n"},{"id":"522513","messageId":"aIAkSiPeDy5MLe4N@fruit.crustytoothpaste.net","threadId":"63804","inReplyTo":"20250722220233.7rdako7zjwyjqlmv@glandium.org","subject":"Re: [PATCH 1/7] xdiff: introduce rust","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2025-07-22T23:52:42Z","receivedAt":"2025-07-22T23:52:44Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On 2025-07-22 at 22:02:33, Mike Hommey wrote:\n> On Thu, Jul 17, 2025 at 09:30:43PM +0000, brian m. carlson wrote:\n> > I would prefer that we not check in Cargo.lock in Git. Part of the\n> > reason is that it changes across versions and so building with a\n> > different version of the toolchain can update the file.\n> \n> That actually doesn't happen unless the file needs to be updated for\n> some reason, like Cargo.toml having new dependencies or `cargo update`\n> being run.\n\nI've actually seen several cases in my local Rust development where\nCargo wants to update the file despite it not being necessary and\n`--locked` simply refusing to work without good cause. Perhaps those\ncases have been fixed, but it has happened in older versions.\n-- \nbrian m. carlson (they/them)\nToronto, Ontario, CA\n"},{"id":"522518","messageId":"aIBlxnoOqwHhGzMd@pks.im","threadId":"63804","inReplyTo":"874iv4gqxv.fsf@gentoo.org","subject":"Re: [PATCH 0/7] RFC: Accelerate xdiff and begin its rustification","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-07-23T04:32:06Z","receivedAt":"2025-07-23T04:32:19Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Tue, Jul 22, 2025 at 04:56:12PM +0100, Sam James wrote:\n> There's a few issues from our perspective:\n> \n> * Old platforms which don't have LLVM can't yet have Rust either, as\n> rustc is based on LLVM.\n> \n> These need gccrs to be unblocked. I can understand not caring too much\n> about these, though it is unfortunate, because I think if git hadn't\n> supported many platforms to begin with, I doubt it'd have the adoption\n> it does today.\n> \n> (There is another effort which seeks to take rustc and bolt on\n> libgccjit as a replacement backend, but that isn't feasible for use\n> yet either.)\n\nIt would be great to know about the general timelines of these\nalternative implementations. If e.g. gccrs were to achieve compatibility\nwith one of the editions of Rust next year it would be a good enough\nreason to defer the rustification from my point of view so that we don't\nbreak the ecosystem and have wider platform support. If the answer is\n\"They'll land in 10 years\" then I don't know...\n\nI sifted through their project sites and found various status reports,\nand they do seem to be making steady progress. But as far as I see\ncritical language features are still missing as of now.\n\n[snip]\n> * rustc doesn't have LTS releases or the like.\n> \n> The only supported release is the latest one. Upgrading to the latest\n> release often means we have to deal with new portability problems\n> but we can't not upgrade because:\n> a) some software will start to require bleeding-edge Rust immediately,\n> and\n> b) it means we're missing out on bug fixes (miscompilations are\n> serious)\n\nI'm not a big fan of this in the Rust ecosystem indeed. It feels like\nevery second project requires nightly features or at least a version of\nthe compiler that was released in the last couple months. This may work\nfor a language like Go, which is more targeted towards deploying server\napplications. But for a system-level language like Rust I think it's\nrather a sign of it being immature.\n\nIn any case, the burden would fall on us to ensure that we carefully\nconsider which version of Rust to target. And as it was said elsewhere\nin the thread, we would need to make sure that things build on old\nversions of Debian. Which may be easier said than done if we also rely\non lots of crates which may update to newer Rust versions at any point\nin time.\n\n> * Crate creep\n> \n> Rust projects tend to end up having a huge list of crates that they\n> pull-in which makes us worried about something nasty creeping in, but\n> there's also popular crates with serious portability problems like the\n> 'ring' crate for TLS.\n\nTrue. I think if we were to adopt Rust we ought to be as conservative as\nwe are now with picking up new dependencies. I don't want to have a big\nopen door for supply chain attacks. And neither do I want to be forced\ninto the situation where we cannot update a crate because they decided\nto drop support for older Rust versions.\n\nPatrick\n"},{"id":"522609","messageId":"aIFauT8M0wRfaZV8@fruit.crustytoothpaste.net","threadId":"63804","inReplyTo":"CAH=ZcbBebM6CememqOUFY2YPOXpk_mC=zE0OnLOKDqcJQTdMuA@mail.gmail.com","subject":"Re: [PATCH 1/7] xdiff: introduce rust","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2025-07-23T21:57:13Z","receivedAt":"2025-07-23T21:57:24Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On 2025-07-18 at 23:15:19, Ezekiel Newren wrote:\n> This goes against what I think is best practices. Don’t we need\n> Cargo.lock to audit and debug platform specific issues, and to ensure\n> reproducibility? Without Cargo.lock, we might get different results\n> one minute to the next if one of our dependencies releases a new\n> version. Checking in Cargo.lock aligns with Cargo’s documented best\n> practices (https://doc.rust-lang.org/cargo/faq.html#why-have-cargolock-in-version-control).\n\nI appreciate that, but best practices also don't limit software to a\nsix-week lifespan. Rust the language is a great tool, but we also have\na special case here in that we need to support software that upstream\ndoes not and that we care about OS distros, which upstream does not.\n\nNote that when someone builds locally, a Cargo.lock will be created and\nthey will get reproducible builds from that point on. It is only on\nfirst build that they will get whatever's the latest.\n\n> I understand your concern and I agree that this could become a\n> problem. I’m totally flexible on which rust version should be used,\n> but without Cargo.lock checked in we lose the ability to audit why a\n> build failed. I think that this will be a pain point, but numbing that\n> pain means we can’t solve intermittent problems due to dependencies in\n> the future.\n\nI was one of the maintainers for Git LFS for several years. We\nroutinely had people come to us and say, \"This dependency you're using\nhas a portion that you're not using, which has a CVE. I demand you\nupdate it and do a new release immediately because our security scanner\nis going off and our company policy is that there be no exceptions.\"\nThis happens literally all the time and I absolutely in no case want to\nsee those people on this list or the security list.\n\nSo the options as I see them are (a) we don't check in Cargo.lock, (b)\nwe convince the Rust project and the ecosystem to provide LTS releases\nwith security fixes, or (c) we only accept dependencies that have our\nsame lifetime policy (which are very few and far between). I know this\nmakes builds unreproducible (although not under the Reproducible Builds\nproject's definitions), but we really don't have many alternatives.\n-- \nbrian m. carlson (they/them)\nToronto, Ontario, CA\n"},{"id":"522613","messageId":"xmqq4iv2lf1j.fsf@gitster.g","threadId":"63804","inReplyTo":"aIFauT8M0wRfaZV8@fruit.crustytoothpaste.net","subject":"Re: [PATCH 1/7] xdiff: introduce rust","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-07-23T22:26:32Z","receivedAt":"2025-07-23T22:26:34Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"brian m. carlson\" writes:\n\n> I was one of the maintainers for Git LFS for several years. We\n> routinely had people come to us and say, \"This dependency you're using\n> has a portion that you're not using, which has a CVE. I demand you\n> update it and do a new release immediately because our security scanner\n> is going off and our company policy is that there be no exceptions.\"\n> This happens literally all the time and I absolutely in no case want to\n> see those people on this list or the security list.\n\nAhh, the kind we love not to have.\n\n> So the options as I see them are (a) we don't check in Cargo.lock, (b)\n> we convince the Rust project and the ecosystem to provide LTS releases\n> with security fixes, or (c) we only accept dependencies that have our\n> same lifetime policy (which are very few and far between). I know this\n> makes builds unreproducible (although not under the Reproducible Builds\n> project's definitions), but we really don't have many alternatives.\n\nThanks for a well reasoned argument.\n\nHopefully as Rust matures more, some of these issues (starting with\n\"6 weeks and it is too old to bother\") would resolve themselves, but\nuntil then we'd need to be careful.\n\n"},{"id":"522646","messageId":"7bf054a1-0196-4ad8-aaa4-a432cd2c93a5@embecosm.com","threadId":"63804","inReplyTo":"aIBlxnoOqwHhGzMd@pks.im","subject":"Re: [PATCH 0/7] RFC: Accelerate xdiff and begin its rustification","fromName":"Pierre-Emmanuel Patry","fromEmail":"pierre-emmanuel.patry@embecosm.com","sentAt":"2025-07-24T09:01:22Z","receivedAt":"2025-07-24T09:01:25Z","isPatch":true,"sender":{"key":"pierre-emmanuel.patry@embecosm.com","avatar":null},"body":"\nOn Tue, Jul 23, 2025 at 06:32:06 +0200, Patrick Steinhardt wrote:\n > It would be great to know about the general timelines of these\n > alternative implementations.\n\nWe still think we'll be able to compile libcore before the end of the \nsummer, we've made great progress and few items are left. But keep in \nmind we're targeting an older version of rust (1.49) and libcore is \nsmaller than the standard library. We still have a lot of testing to do \nand we expect many bugs.\n\nThe next targeted version will probably be rust 1.78 as we want to keep \nup with rust for linux. This shouldn't be too long as most of the \nfeatures are coming from either standard library modifications or \nnightly features we already had to support for 1.49.\n\nWe expect to be able to compile some 1.49 code correctly next year at \nbest. I would like to bring to your attention rustc_codegen_gcc which \nadds a gcc backend to the rustc frontend, although not a full gcc \ncompiler it could help supporting some architectures that are currently \nnot supported by llvm.\n\nPierre-Emmanuel\n"},{"id":"522650","messageId":"aIIETOdK4Nrsy5Jb@pks.im","threadId":"63804","inReplyTo":"7bf054a1-0196-4ad8-aaa4-a432cd2c93a5@embecosm.com","subject":"Re: [PATCH 0/7] RFC: Accelerate xdiff and begin its rustification","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-07-24T10:00:44Z","receivedAt":"2025-07-24T10:00:52Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Thu, Jul 24, 2025 at 11:01:22AM +0200, Pierre-Emmanuel Patry wrote:\n> \n> On Tue, Jul 23, 2025 at 06:32:06 +0200, Patrick Steinhardt wrote:\n> > It would be great to know about the general timelines of these\n> > alternative implementations.\n> \n> We still think we'll be able to compile libcore before the end of the\n> summer, we've made great progress and few items are left. But keep in mind\n> we're targeting an older version of rust (1.49) and libcore is smaller than\n> the standard library. We still have a lot of testing to do and we expect\n> many bugs.\n\nUnderstood. Given that we don't plan to roll with the latest version of\nRust anyway I think it could be a viable tradeoff for us to also\nconsider gccrs when we determine the minimum required Rust version.\n\n> The next targeted version will probably be rust 1.78 as we want to keep up\n> with rust for linux. This shouldn't be too long as most of the features are\n> coming from either standard library modifications or nightly features we\n> already had to support for 1.49.\n> \n> We expect to be able to compile some 1.49 code correctly next year at best.\n\nAnd I expect that 1.78 will be another significant effort that won't\nland before the year after?\n\n> I would like to bring to your attention rustc_codegen_gcc which adds a gcc\n> backend to the rustc frontend, although not a full gcc compiler it could\n> help supporting some architectures that are currently not supported by llvm.\n\nFor my own understanding: is this something that the Git project would\nhave to support or something that the distributor needs to set up?\n\nPatrick\n"},{"id":"522778","messageId":"3F4EE706-9666-40B0-9638-AC433D7048ED@gmail.com","threadId":"63804","inReplyTo":"CAH=ZcbAeMv7oO-X_o_WOvgS6_igO3XAUjmgLoEfDN0CqqTMH_g@mail.gmail.com","subject":"Re: [PATCH 7/7] github_workflows: install rust","fromName":"Ben Knoble","fromEmail":"ben.knoble@gmail.com","sentAt":"2025-07-25T23:56:47Z","receivedAt":"2025-07-25T23:56:59Z","isPatch":true,"sender":{"key":"ben.knoble@gmail.com","avatar":"https://avatars.githubusercontent.com/u/22802209?v=4"},"body":"\n> Le 18 juil. 2025 à 19:04, Ezekiel Newren a écrit :\n> \n> On Thu, Jul 17, 2025 at 3:23 PM brian m. carlson\n> wrote:\n>> \n>>> On 2025-07-17 at 20:32:24, Ezekiel Newren via GitGitGadget wrote:\n>>> diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml\n>>> index 7dbf9f7f123c..8aac18a6ba45 100644\n>>> --- a/.github/workflows/main.yml\n>>> +++ b/.github/workflows/main.yml\n>>> @@ -4,6 +4,7 @@ on: [push, pull_request]\n>>> \n>>> env:\n>>> DEVELOPER: 1\n>>> + RUST_VERSION: 1.87.0\n>> \n>> Our discussed plan is to support the version in Debian stable, plus a\n>> year. So we'd be supporting 1.63.0 for a year after trixie's release.\n>> \n>> The reason for that is that people build backports and security updates\n>> for Git for stable releases of distros and they will use the distro\n>> toolchain for doing so. Forcing distros to constantly build with the\n>> latest toolchain is pretty hostile, especially since the lifespan of\n>> Rust release is six weeks.\n>> \n>> If the Rust project provides LTS releases in the future, then we can\n>> consider adopting those.\n> \n> The RUST_VERSION variable in .github/workflows/main.yaml had to have a\n> specific version. 1.87.0 was selected since that's what I was using\n> locally. Elijah made me aware that an older version of rust might be\n> desired, but didn't know which one. I'll switch to 1.63.0 or whatever\n> the community decides.\n> \n>>> +if [ \"$rust_target\" = \"release\" ]; then\n>>> + rust_args=\"--release\"\n>>> + export RUSTFLAGS='-Aunused_imports -Adead_code'\n>>> +elif [ \"$rust_target\" = \"debug\" ]; then\n>>> + rust_args=\"\"\n>>> + export RUSTFLAGS='-Aunused_imports -Adead_code -C debuginfo=2 -C opt-level=1 -C force-frame-pointers=yes'\n>> \n>> Can you say a little about why these options are needed and the defaults\n>> are inadequate? For instance, I build with the default options both in\n>> my personal projects and at work and don't see a problem.\n> \n> What I found is that if I have a Rust function\n> \n> #[no_mangle]\n> pub fn call_from_c(arg: u64) {}\n> \n> which is only meant to be called from C and isn’t called from\n> elsewhere in Rust, then cargo will misidentify this function as dead\n> code. This was the reason for adding ‘-Adead_code’.\n\nAre functions that exist for C FFI callers supposed to be marked unsafe, and if so: does that prevent the dead code analyzer from removing them w/o the allow flag?\n\nOr, alternatively, can we #[] annotate them as allowed? It might be noisy, but it also lets the checkers flag actually dead code?\n\n> \n> The reason for adding ‘-Aunused_imports’ is somewhat IDE related; if I\n> paste code somewhere, RustRover will sometimes automatically add the\n> necessary imports. However, if I delete a chunk of code, it’ll\n> highlight the imports that are no longer used if I scroll to the top\n> of the file, but it won’t automatically remove them. Since they\n> aren’t automatically removed, it’s easier to build with\n> ‘-Aunused_imports’.\n\nSimilarly, I would think this is a problem with the IDE rather than something we want to end up with in the implementation.\n\nI wonder if one of the cargo fix type things can remove them for you, too, so that it’s easier to just drop this allow. "},{"id":"522845","messageId":"d452bb67-569a-4772-a943-950e3edb4e16@embecosm.com","threadId":"63804","inReplyTo":"aIIETOdK4Nrsy5Jb@pks.im","subject":"Re: [PATCH 0/7] RFC: Accelerate xdiff and begin its rustification","fromName":"Pierre-Emmanuel Patry","fromEmail":"pierre-emmanuel.patry@embecosm.com","sentAt":"2025-07-28T09:06:52Z","receivedAt":"2025-07-28T09:06:56Z","isPatch":true,"sender":{"key":"pierre-emmanuel.patry@embecosm.com","avatar":null},"body":"\n> And I expect that 1.78 will be another significant effort that won't\n> land before the year after?\n\nYes, even though it will be easier once the foundations are laid off, I \nwouldn't expect 1.78 before at least a year after that.\n\n> For my own understanding: is this something that the Git project would\n> have to support or something that the distributor needs to set up?\n\nI would say it is something the distributor needs to set up. From what I \nremember rustc requires a flag with the path to the rustc_codegen_gcc \nbackend. This means that has long as the distributor has the alternative \nbackend and a way to inject a flag to rustc through an environment \nvariable it should be mostly fine for them.\n\nPierre-Emmanuel\n"},{"id":"522876","messageId":"CAH=ZcbBNg0Ku0VKvF0HUyksrcZdbT=8Xmk6_kQV0178ROATf8Q@mail.gmail.com","threadId":"63804","inReplyTo":"aIFauT8M0wRfaZV8@fruit.crustytoothpaste.net","subject":"Re: [PATCH 1/7] xdiff: introduce rust","fromName":"Ezekiel Newren","fromEmail":"ezekielnewren@gmail.com","sentAt":"2025-07-28T19:11:34Z","receivedAt":"2025-07-28T19:11:49Z","isPatch":true,"sender":{"key":"ezekielnewren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/18313324?v=4"},"body":"On Wed, Jul 23, 2025 at 3:57 PM brian m. carlson\n wrote:\n>\n> On 2025-07-18 at 23:15:19, Ezekiel Newren wrote:\n> > This goes against what I think is best practices. Don’t we need\n> > Cargo.lock to audit and debug platform specific issues, and to ensure\n> > reproducibility? Without Cargo.lock, we might get different results\n> > one minute to the next if one of our dependencies releases a new\n> > version. Checking in Cargo.lock aligns with Cargo’s documented best\n> > practices (https://doc.rust-lang.org/cargo/faq.html#why-have-cargolock-in-version-control).\n>\n> I appreciate that, but best practices also don't limit software to a\n> six-week lifespan. Rust the language is a great tool, but we also have\n> a special case here in that we need to support software that upstream\n> does not and that we care about OS distros, which upstream does not.\n>\n> Note that when someone builds locally, a Cargo.lock will be created and\n> they will get reproducible builds from that point on. It is only on\n> first build that they will get whatever's the latest.\n>\n> > I understand your concern and I agree that this could become a\n> > problem. I’m totally flexible on which rust version should be used,\n> > but without Cargo.lock checked in we lose the ability to audit why a\n> > build failed. I think that this will be a pain point, but numbing that\n> > pain means we can’t solve intermittent problems due to dependencies in\n> > the future.\n>\n> I was one of the maintainers for Git LFS for several years. We\n> routinely had people come to us and say, \"This dependency you're using\n> has a portion that you're not using, which has a CVE. I demand you\n> update it and do a new release immediately because our security scanner\n> is going off and our company policy is that there be no exceptions.\"\n> This happens literally all the time and I absolutely in no case want to\n> see those people on this list or the security list.\n>\n> So the options as I see them are (a) we don't check in Cargo.lock, (b)\n> we convince the Rust project and the ecosystem to provide LTS releases\n> with security fixes, or (c) we only accept dependencies that have our\n> same lifetime policy (which are very few and far between). I know this\n> makes builds unreproducible (although not under the Reproducible Builds\n> project's definitions), but we really don't have many alternatives.\n> --\n> brian m. carlson (they/them)\n> Toronto, Ontario, CA\n\nI like having the Cargo.lock file to figure out why a build worked on\none system, but not another. After talking with Elijah I've decided\nthat a good solution would be to add Cargo.lock to .gitignore and\nchange the github workflows to ensure that Cargo.lock is preserved for\nall builds. We should also add a comment to Cargo.toml stating that\nany build or test issues should include the Cargo.lock that was\ngenerated when asking for help. What does the community think of this\nsolution?\n"},{"id":"522880","messageId":"CAH=ZcbCnEpBokM9rxmmkeM9GT948n7+RipXODHLfPssuwJuVCw@mail.gmail.com","threadId":"63804","inReplyTo":"91f6352f-abc4-4e99-938b-6a56aba2faed@gmail.com","subject":"Re: [PATCH 4/7] xdiff: make fields of xrecord_t Rust friendly","fromName":"Ezekiel Newren","fromEmail":"ezekielnewren@gmail.com","sentAt":"2025-07-28T19:34:32Z","receivedAt":"2025-07-28T19:34:45Z","isPatch":true,"sender":{"key":"ezekielnewren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/18313324?v=4"},"body":"On Fri, Jul 18, 2025 at 7:35 AM Phillip Wood wrote:\n>\n> Hi Ezekiel\n>\n> On 17/07/2025 21:32, Ezekiel Newren via GitGitGadget wrote:\n> > From: Ezekiel Newren \n> >\n> > A few commits ago, we added definitions for Rust primitive types,\n> > to facilitate interoperability between C and Rust. Switch a\n> > few variables to use these types. Which, for now, will\n> > require adding some casts.\n>\n> How necessary is it to change char' to 'u8' so long as the rust and C\n> sides both use a type that is the same size? Also what's the advantage\n> of using these typedefs rather than the normal C types like unit8_t ?\n\nRust defines char as 32 bits. C treats char as signed 8 bits. What git\nreally means by char* is treat everything like a byte string, and u8\nis how raw bytes are handled in Rust.\n\n> > diff --git a/xdiff/xdiffi.c b/xdiff/xdiffi.c\n> > index 5a96e36dfbea..3b364c61f671 100644\n> > --- a/xdiff/xdiffi.c\n> > +++ b/xdiff/xdiffi.c\n> > @@ -418,7 +418,7 @@ static int get_indent(xrecord_t *rec)\n> > long i;\n> > int ret = 0;\n> >\n> > - for (i = 0; i < rec->size; i++) {\n> > + for (i = 0; i < (long) rec->size; i++) {\n>\n> i is a loop counter and array index so we can lose this cast by\n> changeing i to size_t\n\nOk, but I'm going to change the type of i to usize and stuff it inside\nthe loop i.e. for (usize i = 0; ...\n\n> Thanks\n>\n> Phillip\n"},{"id":"522882","messageId":"a765cde9-0fad-414a-996f-2ec162d1e4f3@gmail.com","threadId":"63804","inReplyTo":"CAH=ZcbCnEpBokM9rxmmkeM9GT948n7+RipXODHLfPssuwJuVCw@mail.gmail.com","subject":"Re: [PATCH 4/7] xdiff: make fields of xrecord_t Rust friendly","fromName":"Phillip Wood","fromEmail":"phillip.wood123@gmail.com","sentAt":"2025-07-28T19:52:48Z","receivedAt":"2025-07-28T19:52:53Z","isPatch":true,"sender":{"key":"phillip.wood@dunelm.org.uk","avatar":null},"body":"On 28/07/2025 20:34, Ezekiel Newren wrote:\n> On Fri, Jul 18, 2025 at 7:35 AM Phillip Wood wrote:\n>> On 17/07/2025 21:32, Ezekiel Newren via GitGitGadget wrote:\n>>> From: Ezekiel Newren \n>>>\n>>> A few commits ago, we added definitions for Rust primitive types,\n>>> to facilitate interoperability between C and Rust. Switch a\n>>> few variables to use these types. Which, for now, will\n>>> require adding some casts.\n>>\n>> How necessary is it to change char' to 'u8' so long as the rust and C\n>> sides both use a type that is the same size? Also what's the advantage\n>> of using these typedefs rather than the normal C types like unit8_t ?\n> \n> Rust defines char as 32 bits. C treats char as signed 8 bits. What git\n> really means by char* is treat everything like a byte string, and u8\n> is how raw bytes are handled in Rust.\n\nRight - we need to use u8 on the rust side but I'm trying to understand \nwhy we need to change the type on the C side and why do we need typedefs \nlike usize and u32 on the C side when we already have size_t and uint32_t?\n\nThanks\n\nPhillip\n\n>>> diff --git a/xdiff/xdiffi.c b/xdiff/xdiffi.c\n>>> index 5a96e36dfbea..3b364c61f671 100644\n>>> --- a/xdiff/xdiffi.c\n>>> +++ b/xdiff/xdiffi.c\n>>> @@ -418,7 +418,7 @@ static int get_indent(xrecord_t *rec)\n>>> long i;\n>>> int ret = 0;\n>>>\n>>> - for (i = 0; i < rec->size; i++) {\n>>> + for (i = 0; i < (long) rec->size; i++) {\n>>\n>> i is a loop counter and array index so we can lose this cast by\n>> changeing i to size_t\n> \n> Ok, but I'm going to change the type of i to usize and stuff it inside\n> the loop i.e. for (usize i = 0; ...\n> \n>> Thanks\n>>\n>> Phillip\n\n"},{"id":"522884","messageId":"87ecu0nl0q.fsf@gmail.com","threadId":"63804","inReplyTo":"CAH=ZcbCnEpBokM9rxmmkeM9GT948n7+RipXODHLfPssuwJuVCw@mail.gmail.com","subject":"Re: [PATCH 4/7] xdiff: make fields of xrecord_t Rust friendly","fromName":"Collin Funk","fromEmail":"collin.funk1@gmail.com","sentAt":"2025-07-28T20:00:21Z","receivedAt":"2025-07-28T20:00:23Z","isPatch":true,"sender":{"key":"collin.funk1@gmail.com","avatar":"https://avatars.githubusercontent.com/u/65689063?v=4"},"body":"Ezekiel Newren writes:\n\n> Rust defines char as 32 bits. C treats char as signed 8 bits. What git\n> really means by char* is treat everything like a byte string, and u8\n> is how raw bytes are handled in Rust.\n\nMinor correction, but the C standard leaves the signedness of 'char' up\nto the implementation. Portable code must be written to assume a plain\n'char' can be signed or unsigned.\n\nUsing the test program below:\n\n #include \n #define TYPE_SIGNED(t) (! ((t) 0 < (t) -1))\n int\n main (void)\n {\n printf (\"%d\\n\", TYPE_SIGNED (char));\n return 0;\n }\n\nOn GNU/Linux x86_64:\n\n $ ./a.out \n 1\n\nOn GNU/Linux aarch64:\n\n $ ./a.out \n 0\n\nCollin\n"},{"id":"522885","messageId":"CAH=ZcbALsQqTrvNJ4ZKmVWc6PHtTA+8k8p6_D=x=BfMXxnayfA@mail.gmail.com","threadId":"63804","inReplyTo":"a765cde9-0fad-414a-996f-2ec162d1e4f3@gmail.com","subject":"Re: [PATCH 4/7] xdiff: make fields of xrecord_t Rust friendly","fromName":"Ezekiel Newren","fromEmail":"ezekielnewren@gmail.com","sentAt":"2025-07-28T20:14:15Z","receivedAt":"2025-07-28T20:14:29Z","isPatch":true,"sender":{"key":"ezekielnewren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/18313324?v=4"},"body":"On Mon, Jul 28, 2025 at 1:52 PM Phillip Wood wrote:\n>\n> On 28/07/2025 20:34, Ezekiel Newren wrote:\n> > On Fri, Jul 18, 2025 at 7:35 AM Phillip Wood wrote:\n> >> On 17/07/2025 21:32, Ezekiel Newren via GitGitGadget wrote:\n> >>> From: Ezekiel Newren \n> >>>\n> >>> A few commits ago, we added definitions for Rust primitive types,\n> >>> to facilitate interoperability between C and Rust. Switch a\n> >>> few variables to use these types. Which, for now, will\n> >>> require adding some casts.\n> >>\n> >> How necessary is it to change char' to 'u8' so long as the rust and C\n> >> sides both use a type that is the same size? Also what's the advantage\n> >> of using these typedefs rather than the normal C types like unit8_t ?\n> >\n> > Rust defines char as 32 bits. C treats char as signed 8 bits. What git\n> > really means by char* is treat everything like a byte string, and u8\n> > is how raw bytes are handled in Rust.\n>\n> Right - we need to use u8 on the rust side but I'm trying to understand\n> why we need to change the type on the C side and why do we need typedefs\n> like usize and u32 on the C side when we already have size_t and uint32_t?\n\nAh, I misunderstood the scope of your question. I could not fit an\nexample of why this design pattern made sense into this patch series,\nso I'll explain with an example here:\n\nIf C defines a struct like below then it's obvious how to translate\nthat into rust for ffi purposes. It also makes it clear that this C\nstruct is expressly for the purpose of C <-> Rust interoperability.\nstruct some_struct {\n u8* ptr;\n usize length;\n u64 counter;\n};\n\nThis is how that C struct needs to be defined in Rust so that it can\ninteroperate with C, and making C use the Rust types reduces the\nchance of copy paste, and primitive type definition mismatch errors.\n#[repr(C)]\npub struct some_struct {\n ptr: *mut u8,\n length: usize,\n counter: u64,\n};\n\nThe Rust function would look like:\n#[no_mangle]\nunsafe extern \"C\" fn do_something(data: *mut some_struct) {...}\n\nAnd C would have a forward declaration like:\nextern void do_something(struct some_struct *data);\n\nvoid some_c_function() {\n struct some_struct x;\n do_something(&x);\n}\n\n> Thanks\n>\n> Phillip\n"},{"id":"522898","messageId":"xmqqwm7st4ts.fsf@gitster.g","threadId":"63804","inReplyTo":"a765cde9-0fad-414a-996f-2ec162d1e4f3@gmail.com","subject":"Re: [PATCH 4/7] xdiff: make fields of xrecord_t Rust friendly","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-07-28T20:53:35Z","receivedAt":"2025-07-28T20:53:37Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Phillip Wood writes:\n\n> On 28/07/2025 20:34, Ezekiel Newren wrote:\n>> On Fri, Jul 18, 2025 at 7:35 AM Phillip Wood wrote:\n>>> On 17/07/2025 21:32, Ezekiel Newren via GitGitGadget wrote:\n>>>> From: Ezekiel Newren \n>>>>\n>>>> A few commits ago, we added definitions for Rust primitive types,\n>>>> to facilitate interoperability between C and Rust. Switch a\n>>>> few variables to use these types. Which, for now, will\n>>>> require adding some casts.\n>>>\n>>> How necessary is it to change char' to 'u8' so long as the rust and C\n>>> sides both use a type that is the same size? Also what's the advantage\n>>> of using these typedefs rather than the normal C types like unit8_t ?\n>> Rust defines char as 32 bits. C treats char as signed 8 bits. What\n>> git\n>> really means by char* is treat everything like a byte string, and u8\n>> is how raw bytes are handled in Rust.\n>\n> Right - we need to use u8 on the rust side but I'm trying to\n> understand why we need to change the type on the C side and why do we\n> need typedefs like usize and u32 on the C side when we already have\n> size_t and uint32_t?\n\nOr uint8_t? Ah, eh, that is \"unsigned char\" so it would be\nredundant, I guess?\n"},{"id":"523094","messageId":"ad453eee-23cd-42fe-97bd-1ff0fc2f3edf@gmail.com","threadId":"63804","inReplyTo":"CAH=ZcbALsQqTrvNJ4ZKmVWc6PHtTA+8k8p6_D=x=BfMXxnayfA@mail.gmail.com","subject":"Re: [PATCH 4/7] xdiff: make fields of xrecord_t Rust friendly","fromName":"Phillip Wood","fromEmail":"phillip.wood123@gmail.com","sentAt":"2025-07-31T14:20:44Z","receivedAt":"2025-07-31T14:20:53Z","isPatch":true,"sender":{"key":"phillip.wood@dunelm.org.uk","avatar":null},"body":"On 28/07/2025 21:14, Ezekiel Newren wrote:\n> On Mon, Jul 28, 2025 at 1:52 PM Phillip Wood wrote:\n> \n> Ah, I misunderstood the scope of your question. I could not fit an\n> example of why this design pattern made sense into this patch series,\n> so I'll explain with an example here:\n> \n> If C defines a struct like below then it's obvious how to translate\n> that into rust for ffi purposes. It also makes it clear that this C\n> struct is expressly for the purpose of C <-> Rust interoperability.\n> struct some_struct {\n> u8* ptr;\n> usize length;\n> u64 counter;\n> };\n> \n> This is how that C struct needs to be defined in Rust so that it can\n> interoperate with C, and making C use the Rust types reduces the\n> chance of copy paste, and primitive type definition mismatch errors.\n> #[repr(C)]\n> pub struct some_struct {\n> ptr: *mut u8,\n> length: usize,\n> counter: u64,\n> };\n\nHow is the pointer, length pair used in rust? Normally one would use a \nslice so do we have to construct a slice every time we want to use the \ndata in this struct, or do we copy the data in this struct into to a an \nidiomatic struct with a slice member? If we end up copying there doesn't \nseem much point in changing all the types in the C struct as we can \ndefine a rust struct using *c_char, c_long etc. to interface with the C \ncode and covert them to an appropriate rust type when we copy the data \nto the idiomatic version that is then used by the rust of the rust code. \nI can see the value of the typedefs for documenting C<->rust interop if \nthe same struct is used by both but if we end up copying data on the \nrust side I'm not so sure.\n\nThanks\n\nPhillip\n\n"},{"id":"523126","messageId":"CAH=ZcbBa=1iUTcxaBOvG_kcuWsF_nJQiWGkL+BUzsNYLpzFG5w@mail.gmail.com","threadId":"63804","inReplyTo":"ad453eee-23cd-42fe-97bd-1ff0fc2f3edf@gmail.com","subject":"Re: [PATCH 4/7] xdiff: make fields of xrecord_t Rust friendly","fromName":"Ezekiel Newren","fromEmail":"ezekielnewren@gmail.com","sentAt":"2025-07-31T20:58:08Z","receivedAt":"2025-07-31T20:58:21Z","isPatch":true,"sender":{"key":"ezekielnewren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/18313324?v=4"},"body":"On Thu, Jul 31, 2025 at 8:20 AM Phillip Wood wrote:\n>\n> On 28/07/2025 21:14, Ezekiel Newren wrote:\n> > On Mon, Jul 28, 2025 at 1:52 PM Phillip Wood wrote:\n> >\n> > Ah, I misunderstood the scope of your question. I could not fit an\n> > example of why this design pattern made sense into this patch series,\n> > so I'll explain with an example here:\n> >\n> > If C defines a struct like below then it's obvious how to translate\n> > that into rust for ffi purposes. It also makes it clear that this C\n> > struct is expressly for the purpose of C <-> Rust interoperability.\n> > struct some_struct {\n> > u8* ptr;\n> > usize length;\n> > u64 counter;\n> > };\n> >\n> > This is how that C struct needs to be defined in Rust so that it can\n> > interoperate with C, and making C use the Rust types reduces the\n> > chance of copy paste, and primitive type definition mismatch errors.\n> > #[repr(C)]\n> > pub struct some_struct {\n> > ptr: *mut u8,\n> > length: usize,\n> > counter: u64,\n> > };\n>\n> How is the pointer, length pair used in rust? Normally one would use a\n> slice so do we have to construct a slice every time we want to use the\n> data in this struct, or do we copy the data in this struct into to a an\n> idiomatic struct with a slice member? If we end up copying there doesn't\n> seem much point in changing all the types in the C struct as we can\n> define a rust struct using *c_char, c_long etc. to interface with the C\n> code and covert them to an appropriate rust type when we copy the data\n> to the idiomatic version that is then used by the rust of the rust code.\n> I can see the value of the typedefs for documenting C<->rust interop if\n> the same struct is used by both but if we end up copying data on the\n> rust side I'm not so sure.\n>\n> Thanks\n>\n> Phillip\n\nPassing pointer + length from c to Rust does not incur a memory copy\noverhead. Take a look at rust/xdiff/src/lib.rs wich has the following\nrust function defined:\n\n#[no_mangle]\nunsafe extern \"C\" fn xxh3_64(ptr: *const u8, size: usize) -> u64 {\n let slice = std::slice::from_raw_parts(ptr, size);\n xxhash_rust::xxh3::xxh3_64(slice)\n}\n\nCreating a slice tells the compiler what assumptions it can make about\nthat memory. On the C side in xdiff/xprepare.c:\n\nextern u64 xxh3_64(u8 const* ptr, usize size);\n\nand then it's called like this in that same file:\n\nrec->ha = xxh3_64(rec->ptr, rec->size);\n\nI really wanted to show my ivec type that made passing an\ninteroperable vector type between C and Rust easy and fast, but this\npatch series is already getting very long.\n"},{"id":"523127","messageId":"CAH=ZcbA-OWxbLJoqf1EtDetnXwAieXQjBr5Jmf+G4GiQsTv-hA@mail.gmail.com","threadId":"63804","inReplyTo":"xmqqzfd12ujv.fsf@gitster.g","subject":"Re: [PATCH 6/7] xdiff: conditionally use Rust's implementation of xxhash","fromName":"Ezekiel Newren","fromEmail":"ezekielnewren@gmail.com","sentAt":"2025-07-31T21:13:33Z","receivedAt":"2025-07-31T21:13:46Z","isPatch":true,"sender":{"key":"ezekielnewren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/18313324?v=4"},"body":"On Fri, Jul 18, 2025 at 1:00 PM Junio C Hamano wrote:\n>\n> \"Ezekiel Newren via GitGitGadget\" writes:\n>\n> > +extern u64 xxh3_64(u8 const* ptr, usize size);\n> > +\n> > +\n> > static int xdl_prepare_ctx(unsigned int pass, mmfile_t *mf, long narec, xpparam_t const *xpp,\n> > xdlclassifier_t *cf, xdfile_t *xdf) {\n> > unsigned long *ha;\n> > @@ -175,14 +178,26 @@ static int xdl_prepare_ctx(unsigned int pass, mmfile_t *mf, long narec, xpparam_\n> >\n> > xdl_parse_lines(mf, narec, xdf);\n> >\n> > + if ((xpp->flags & XDF_WHITESPACE_FLAGS) == 0) {\n> > + for (usize i = 0; i < (usize) xdf->nrec; i++) {\n> > + xrecord_t *rec = xdf->recs[i];\n> > + rec->ha = xxh3_64(rec->ptr, rec->size);\n> > + }\n> > + } else {\n> > + for (usize i = 0; i < (usize) xdf->nrec; i++) {\n> > + xrecord_t *rec = xdf->recs[i];\n> > + char const* dump = (char const*) rec->ptr;\n> > + rec->ha = xdl_hash_record(&dump, (char const*) (rec->ptr + rec->size), xpp->flags);\n> > + }\n> > + }\n>\n> As a technology demonstration and proof of concept patch, this is\n> very nice, but to be upstreamed for real, we'd want a variant of\n> xxhash that can work with the contents with whitespace squashed to\n> be usable with various whitespace ignoring modes of operation. When\n> that happens, and when the result turns out to be more performant,\n> we can lose the xdl_hash_record() and require only the xxhash, which\n> would be great.\n>\n> And that variant of xxhash that understands whitespace squashing can\n> of course be written in Rust as a part of this effort when the\n> series loses its RFC status. At the same time, those who want to\n> use our xdiff code in third-party software (like libgit2 and vim)\n> may want to reimplement it in C in their copy.\n>\n> Thanks.\n\nWhat is the git precedent for replacement code that is easier to read\nand maintain while also being more secure, but is slower? I think\nhashing with whitespace handling in Rust might fall in that category.\n\nAs far as I can tell the Rust code for dealing with whitespace is\ngoing to be slower than the C code because xdiff used a hash algorithm\n(DJB2a) that can operate 1 byte at a time and combined hashing with\ndetermining the length. Xxhash requires that the length be known\nbeforehand and the memory to be contiguous or to hash it in chunks.\nHashing 1 byte at a time with Xxhash is VERY slow since it's just\ncopying to an internal buffer until a full block is ready.\n\nOn a broader note. How do I show the mailing list the changes that\nI've made to this branch/patch series? I'm not sure what the proper\nprocedure is or even how to do it. What commands would I run, or web\nbrowser steps would I take to show my newest commits?\n"},{"id":"523131","messageId":"aIvwQtLCSNHo7D_3@fruit.crustytoothpaste.net","threadId":"63804","inReplyTo":"CAH=ZcbBNg0Ku0VKvF0HUyksrcZdbT=8Xmk6_kQV0178ROATf8Q@mail.gmail.com","subject":"Re: [PATCH 1/7] xdiff: introduce rust","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2025-07-31T22:37:54Z","receivedAt":"2025-07-31T22:38:04Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On 2025-07-28 at 19:11:34, Ezekiel Newren wrote:\n> I like having the Cargo.lock file to figure out why a build worked on\n> one system, but not another. After talking with Elijah I've decided\n> that a good solution would be to add Cargo.lock to .gitignore and\n> change the github workflows to ensure that Cargo.lock is preserved for\n> all builds. We should also add a comment to Cargo.toml stating that\n> any build or test issues should include the Cargo.lock that was\n> generated when asking for help. What does the community think of this\n> solution?\n\nThat sounds like a good solution.\n-- \nbrian m. carlson (they/them)\nToronto, Ontario, CA\n"},{"id":"523183","messageId":"a0fea0dc-44ed-4b8b-a28e-762f3a964ccd@gmail.com","threadId":"63804","inReplyTo":"CAH=ZcbBa=1iUTcxaBOvG_kcuWsF_nJQiWGkL+BUzsNYLpzFG5w@mail.gmail.com","subject":"Re: [PATCH 4/7] xdiff: make fields of xrecord_t Rust friendly","fromName":"Phillip Wood","fromEmail":"phillip.wood123@gmail.com","sentAt":"2025-08-01T09:14:55Z","receivedAt":"2025-08-01T09:14:59Z","isPatch":true,"sender":{"key":"phillip.wood@dunelm.org.uk","avatar":null},"body":"Hi Ezekiel\n\nOn 31/07/2025 21:58, Ezekiel Newren wrote:\n> On Thu, Jul 31, 2025 at 8:20 AM Phillip Wood wrote:\n>>\n>> On 28/07/2025 21:14, Ezekiel Newren wrote:\n>>> On Mon, Jul 28, 2025 at 1:52 PM Phillip Wood wrote:\n>>>\n>>> Ah, I misunderstood the scope of your question. I could not fit an\n>>> example of why this design pattern made sense into this patch series,\n>>> so I'll explain with an example here:\n>>>\n>>> If C defines a struct like below then it's obvious how to translate\n>>> that into rust for ffi purposes. It also makes it clear that this C\n>>> struct is expressly for the purpose of C <-> Rust interoperability.\n>>> struct some_struct {\n>>> u8* ptr;\n>>> usize length;\n>>> u64 counter;\n>>> };\n>>>\n>>> This is how that C struct needs to be defined in Rust so that it can\n>>> interoperate with C, and making C use the Rust types reduces the\n>>> chance of copy paste, and primitive type definition mismatch errors.\n>>> #[repr(C)]\n>>> pub struct some_struct {\n>>> ptr: *mut u8,\n>>> length: usize,\n>>> counter: u64,\n>>> };\n>>\n>> How is the pointer, length pair used in rust? Normally one would use a\n>> slice so do we have to construct a slice every time we want to use the\n>> data in this struct, or do we copy the data in this struct into to a an\n>> idiomatic struct with a slice member? If we end up copying there doesn't\n>> seem much point in changing all the types in the C struct as we can\n>> define a rust struct using *c_char, c_long etc. to interface with the C\n>> code and covert them to an appropriate rust type when we copy the data\n>> to the idiomatic version that is then used by the rust of the rust code.\n>> I can see the value of the typedefs for documenting C<->rust interop if\n>> the same struct is used by both but if we end up copying data on the\n>> rust side I'm not so sure.\n>>\n>> Thanks\n>>\n>> Phillip\n> \n> Passing pointer + length from c to Rust does not incur a memory copy\n> overhead. Take a look at rust/xdiff/src/lib.rs wich has the following\n> rust function defined:\n> \n> #[no_mangle]\n> unsafe extern \"C\" fn xxh3_64(ptr: *const u8, size: usize) -> u64 {\n> let slice = std::slice::from_raw_parts(ptr, size);\n> xxhash_rust::xxh3::xxh3_64(slice)\n> }\nI'm afraid I don't find this simple unsafe function example very \nilluminating. I'm trying to understand how we are going to use a struct \ncontaining a pointer, length pair in code that are more complex than \nthis. For example if we implement an entire diff algorithm in rust are \nwe going to call std::slice::from_raw_parts() every time we want to \naccess a string passed from C? If we're doing that I assume we'd impl a \nsafe method on the struct that wraps std::slice::from_raw_parts(). If \nthat's the case the method can easily access a field that has type \n*c_char and we don't have to sprinkle casts throughout our C code.\n\nFor example (ignoring lifetimes)\n#repr[\"C\"]\npub struct SomeStruct {\n ptr *std::ffi::c_char,\n usize len,\n // more members\n}\n\nimpl SomeStruct {\n get_line(&self) -> &[u8] {\n unsafe {\n std::slice::from_raw_parts(self.ptr as *u8, self.len);\n }\n }\n}\n\nOn the other hand if at the interface between rust and C, we create a \nslice that we can pass to the rest of the rust code then we also don't \nneed to change the C type as there is a single place in the rust code \nwhere we convert from c_char when we create the slice.\n\nThe casts on the C side are pretty invasive. At least casting from char \nto u8 is not going to break anything. The long -> usize and long -> u64 \nchanges and their associated casts are going to need some careful review \nbut in the long run I think the C code also benefits to using those types\n\n> Creating a slice tells the compiler what assumptions it can make about\n> that memory. On the C side in xdiff/xprepare.c:\n> \n> extern u64 xxh3_64(u8 const* ptr, usize size);\n> \n> and then it's called like this in that same file:\n> \n> rec->ha = xxh3_64(rec->ptr, rec->size);\n> \n> I really wanted to show my ivec type that made passing an\n> interoperable vector type between C and Rust easy and fast, but this\n> patch series is already getting very long.\nThat sounds interesting\n\nThanks\n\nPhillip\n\n"},{"id":"523257","messageId":"DB9P250MB06923B3E881C33DE457DDA63A521A@DB9P250MB0692.EURP250.PROD.OUTLOOK.COM","threadId":"63804","inReplyTo":"CAH=ZcbA-OWxbLJoqf1EtDetnXwAieXQjBr5Jmf+G4GiQsTv-hA@mail.gmail.com","subject":"Re: [PATCH 6/7] xdiff: conditionally use Rust's implementation of xxhash","fromName":"Matthias Aßhauer","fromEmail":"mha1993@live.de","sentAt":"2025-08-02T07:53:25Z","receivedAt":"2025-08-02T07:53:34Z","isPatch":true,"sender":{"key":"mha1993@live.de","avatar":"https://avatars.githubusercontent.com/u/6178234?v=4"},"body":"\n\nOn Thu, 31 Jul 2025, Ezekiel Newren wrote:\n\n> On Fri, Jul 18, 2025 at 1:00 PM Junio C Hamano wrote:\n>>\n>> \"Ezekiel Newren via GitGitGadget\" writes:\n>>\n>>> +extern u64 xxh3_64(u8 const* ptr, usize size);\n>>> +\n>>> +\n>>> static int xdl_prepare_ctx(unsigned int pass, mmfile_t *mf, long narec, xpparam_t const *xpp,\n>>> xdlclassifier_t *cf, xdfile_t *xdf) {\n>>> unsigned long *ha;\n>>> @@ -175,14 +178,26 @@ static int xdl_prepare_ctx(unsigned int pass, mmfile_t *mf, long narec, xpparam_\n>>>\n>>> xdl_parse_lines(mf, narec, xdf);\n>>>\n>>> + if ((xpp->flags & XDF_WHITESPACE_FLAGS) == 0) {\n>>> + for (usize i = 0; i < (usize) xdf->nrec; i++) {\n>>> + xrecord_t *rec = xdf->recs[i];\n>>> + rec->ha = xxh3_64(rec->ptr, rec->size);\n>>> + }\n>>> + } else {\n>>> + for (usize i = 0; i < (usize) xdf->nrec; i++) {\n>>> + xrecord_t *rec = xdf->recs[i];\n>>> + char const* dump = (char const*) rec->ptr;\n>>> + rec->ha = xdl_hash_record(&dump, (char const*) (rec->ptr + rec->size), xpp->flags);\n>>> + }\n>>> + }\n>>\n>> As a technology demonstration and proof of concept patch, this is\n>> very nice, but to be upstreamed for real, we'd want a variant of\n>> xxhash that can work with the contents with whitespace squashed to\n>> be usable with various whitespace ignoring modes of operation. When\n>> that happens, and when the result turns out to be more performant,\n>> we can lose the xdl_hash_record() and require only the xxhash, which\n>> would be great.\n>>\n>> And that variant of xxhash that understands whitespace squashing can\n>> of course be written in Rust as a part of this effort when the\n>> series loses its RFC status. At the same time, those who want to\n>> use our xdiff code in third-party software (like libgit2 and vim)\n>> may want to reimplement it in C in their copy.\n>>\n>> Thanks.\n>\n> What is the git precedent for replacement code that is easier to read\n> and maintain while also being more secure, but is slower? I think\n> hashing with whitespace handling in Rust might fall in that category.\n>\n> As far as I can tell the Rust code for dealing with whitespace is\n> going to be slower than the C code because xdiff used a hash algorithm\n> (DJB2a) that can operate 1 byte at a time and combined hashing with\n> determining the length. Xxhash requires that the length be known\n> beforehand and the memory to be contiguous or to hash it in chunks.\n> Hashing 1 byte at a time with Xxhash is VERY slow since it's just\n> copying to an internal buffer until a full block is ready.\n>\n> On a broader note. How do I show the mailing list the changes that\n> I've made to this branch/patch series? I'm not sure what the proper\n> procedure is or even how to do it. What commands would I run, or web\n> browser steps would I take to show my newest commits?\n>\n\nSince you've used GitGitGadget for the original submission of this patch \nseries, the easiest way is to force push your updated commits to your PR \nbranch (xdiff_rust_speedup) and comment \"/submit\" on the PR again.\n\nAlternatively you can send a version 2 of your patch series using git \nformat-patch and git send-email, but that is a few more manual steps. \nMyFirstContribution.adoc has a detailed section about this process, \nincluding a part about a v2. [1]\n\n[1] \nhttps://github.com/git/git/blob/master/Documentation/MyFirstContribution.adoc#sending-patches-with-git-send-email\n\nBest regards\n\nMatthias"},{"id":"524204","messageId":"pull.1980.v2.git.git.1755220973.gitgitgadget@gmail.com","threadId":"63804","inReplyTo":"pull.1980.git.git.1752784344.gitgitgadget@gmail.com","subject":"[PATCH v2 00/17] RFC: Accelerate xdiff and begin its rustification","fromName":"Ezekiel Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2025-08-15T01:22:35Z","receivedAt":"2025-08-15T01:22:57Z","isPatch":true,"sender":{"key":"ezekielnewren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/18313324?v=4"},"body":"Changes in this second round of this RFC:\n\n * Now builds and passes tests on all platforms (example run:\n https://github.com/ezekielnewren/git/actions/runs/16974821401). Special\n thanks to Johannes Schindelin for patches to things for Windows and\n linux32.\n * Includes brian’s rust-support documentation as the new 1st patch\n * Removed the Cargo.lock file from version control, but now CI will upload\n these files as build artifacts so we can audit dependencies and notice\n if/when new dependencies cause issues.\n * Added handling of whitespace flags. These are slower; see below\n\nParticular points I’m interested in feedback on:\n\n * Code style: Should we adopt a Rust code style of some sort? Perhaps have\n the code always be formatted by rustfmt in its default configuration?\n * Rust version: We are not using the same Rust version on all platforms in\n CI; 32-bit builds and Windows builds require a newer Rust version to\n successfully build.\n * Performance with whitepsace flags: I originally intended to leave out the\n whitespace handling because I knew it was slower, and I think it’d be\n difficult to fix that until more of xdiff is converted to Rust, but since\n Junio requested it, I have an implementation here. I made sure that both\n –ignore-cr-at-eol (the default on Windows) and no whitespace flags remain\n fast (or are faster), but other whitespace flag combinations are\n currently significantly slower. Are folks okay with merging this, since\n it’ll only affect those that specify some special flag, should we perhaps\n only convert the code path with no whitespace flags for now, or something\n else?\n * Types/Aliases/Data passing: The discussion between Phillip on I on\n types/translation; this longer series has examples with e.g.\n xdl_line_hash() and line_hash() which might give us more to talk about,\n though I think we can’t fully address that discussion until we have an\n example which I’m planning with a later series with an IVec type.\n * There was lots of feedback on v1, and I might have missed some; let me\n know if there’s something I need to still look at.\n\n==Original cover letter==\n\nThis series accelerates xdiff by 5-19%.\n\nIt also introduces Rust as a hard dependency.\n\n…and it doesn’t yet pass a couple of the github workflows; hints from\nWindows experts, and opinions on ambiguous primitives would be appreciated\n(see below).\n\nThis is just the beginning of many patches that I have to convert portions\nof, maybe eventually all of, xdiff to Rust. While working on that\nconversion, I found several ways to clarify the code, along with some\noptimizations.\n\nSo...\n\nThis obviously raises the question of whether we are ready to accept a hard\ndependency on Rust. Previous discussions on the mailing list and at Git\nMerge 2024 have not answered that question. If not now, will we be willing\nto accept such a hard dependency later? And what route do we want to take to\nget there?\n\nAbout the optimizations in this series:\n\n1. xdiff currently uses DJB2a for hashing (even though it is not explicitly named as such). This is an older hashing algorithm, and modern alternatives are superior. I chose xxhash because it’s faster, more collision resistant, and designed to be a standard. Other hash algorithms like aHash, MurMurHash, SipHash, and Fnv1a were considered, but my local testing made me feel like xxhash was the best choice for usage in xdiff.\n\n2. In support of switching to xxhash, parsing and hashing were split into separate steps. And it turns out that memchr() is faster for parsing than character-by-character iteration.\n\n\nAbout the workflow builds/tests that aren’t working with this series:\n\n1. Windows fails to build. I don’t know which rust toolchain is even correct for this or if multiple are needed. Example failed build: https://github.com/git/git/actions/runs/16353209191\n\n2. I386/ubuntu:focal will build, but fails the tests. The kernel reports the bitness as 64 despite the container being 32. I believe the issue is that C uses ambiguous primitives (which differ in size between platforms). The new code should use unambiguous primitives from Rust (u32, u64, etc.) rather than perpetuating ambiguous primitive types. Since the current xdiff API hardcodes the ambiguous types, though, those places will need to be migrated to unambiguous primitives. Much of the C code needs a slight refactor to be compatible with the Rust FFI and usually requires converting ambiguous to unambiguous types. What does this community think of this approach?\n\n\nMy brother (Elijah, cc’ed) has been guiding and reviewing my work here.\n\nEzekiel Newren (13):\n xdiff: introduce rust\n xdiff/xprepare: remove superfluous forward declarations\n xdiff: delete unnecessary fields from xrecord_t and xdfile_t\n xdiff: make fields of xrecord_t Rust friendly\n xdiff: separate parsing lines from hashing them\n xdiff: conditionally use Rust's implementation of xxhash\n github workflows: install rust\n github workflows: define rust versions and targets in the same place\n github workflows: upload Cargo.lock\n xdiff: implement a white space iterator in Rust\n xdiff: create line_hash() and line_equal()\n xdiff: optimize case where --ignore-cr-at-eol is the only whitespace\n flag\n xdiff: use rust's version of whitespace processing\n\nJohannes Schindelin (3):\n Do support Windows again after requiring Rust\n win+Meson: allow for xdiff to be compiled with MSVC\n win+Meson: do allow linking with the Rust-built xdiff\n\nbrian m. carlson (1):\n doc: add a policy for using Rust\n\n .github/workflows/main.yml | 61 +++\n .gitignore | 3 +\n Documentation/Makefile | 1 +\n Documentation/technical/platform-support.adoc | 2 +\n Documentation/technical/rust-support.adoc | 119 ++++++\n Makefile | 60 ++-\n build_rust.sh | 59 +++\n ci/install-dependencies.sh | 14 +-\n ci/install-rust.sh | 37 ++\n ci/lib.sh | 1 +\n ci/make-test-artifacts.sh | 7 +\n ci/run-build-and-tests.sh | 12 +\n config.mak.uname | 9 +\n git-compat-util.h | 17 +\n meson.build | 48 ++-\n rust/Cargo.toml | 6 +\n rust/interop/Cargo.toml | 14 +\n rust/interop/src/lib.rs | 0\n rust/xdiff/Cargo.toml | 16 +\n rust/xdiff/src/lib.rs | 30 ++\n rust/xdiff/src/xutils.rs | 354 ++++++++++++++++++\n xdiff-interface.c | 4 +-\n xdiff/xdiffi.c | 8 +-\n xdiff/xemit.c | 2 +-\n xdiff/xmerge.c | 10 +-\n xdiff/xpatience.c | 2 +-\n xdiff/xprepare.c | 219 +++++------\n xdiff/xtypes.h | 9 +-\n xdiff/xutils.c | 162 +-------\n xdiff/xutils.h | 4 +-\n 30 files changed, 961 insertions(+), 329 deletions(-)\n create mode 100644 Documentation/technical/rust-support.adoc\n create mode 100755 build_rust.sh\n create mode 100755 ci/install-rust.sh\n create mode 100644 rust/Cargo.toml\n create mode 100644 rust/interop/Cargo.toml\n create mode 100644 rust/interop/src/lib.rs\n create mode 100644 rust/xdiff/Cargo.toml\n create mode 100644 rust/xdiff/src/lib.rs\n create mode 100644 rust/xdiff/src/xutils.rs\n\n\nbase-commit: 16bd9f20a403117f2e0d9bcda6c6e621d3763e77\nPublished-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-1980%2Fezekielnewren%2Fxdiff_rust_speedup-v2\nFetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-1980/ezekielnewren/xdiff_rust_speedup-v2\nPull-Request: https://github.com/git/git/pull/1980\n\nRange-diff vs v1:\n\n -: ----------- > 1: 75dfb40ead3 doc: add a policy for using Rust\n 1: 2a1f4be13df ! 2: 7709e5eddba xdiff: introduce rust\n @@ Commit message\n \n Signed-off-by: Ezekiel Newren \n \n + ## .gitignore ##\n +@@ .gitignore: Release/\n + /contrib/buildsystems/out\n + /contrib/libgit-rs/target\n + /contrib/libgit-sys/target\n ++/.idea/\n ++/rust/target/\n ++/rust/Cargo.lock\n +\n ## Makefile ##\n @@ Makefile: TEST_SHELL_PATH = $(SHELL_PATH)\n \n @@ meson.build: version_def_h = custom_target(\n link_with: static_library('git',\n sources: libgit_sources,\n \n - ## rust/Cargo.lock (new) ##\n -@@\n -+# This file is automatically @generated by Cargo.\n -+# It is not intended for manual editing.\n -+version = 4\n -+\n -+[[package]]\n -+name = \"interop\"\n -+version = \"0.1.0\"\n -+\n -+[[package]]\n -+name = \"xdiff\"\n -+version = \"0.1.0\"\n -+dependencies = [\n -+ \"interop\",\n -+]\n -\n ## rust/Cargo.toml (new) ##\n @@\n +[workspace]\n 2: b0b744b9acf = 3: 56c96d35554 xdiff/xprepare: remove superfluous forward declarations\n 3: cc05150d6e1 = 4: ebec3689dce xdiff: delete unnecessary fields from xrecord_t and xdfile_t\n 4: 6df9f50a8f4 ! 5: 769d1a5b9d2 xdiff: make fields of xrecord_t Rust friendly\n @@ Commit message\n few variables to use these types. Which, for now, will\n require adding some casts.\n \n + Also change xdlclass_t::ha to be u64 to match xrecord_t::ha, as\n + pointed out by Johannes.\n +\n + Helped-by: Johannes Schindelin \n Signed-off-by: Ezekiel Newren \n \n ## xdiff/xdiffi.c ##\n @@ xdiff/xpatience.c: static void insert_record(xpparam_t const *xpp, int line, str\n \tif (map->last) {\n \n ## xdiff/xprepare.c ##\n +@@\n + \n + typedef struct s_xdlclass {\n + \tstruct s_xdlclass *next;\n +-\tunsigned long ha;\n ++\tu64 ha;\n + \tchar const *line;\n + \tlong size;\n + \tlong idx;\n @@ xdiff/xprepare.c: static int xdl_classify_record(unsigned int pass, xdlclassifier_t *cf, xrecord_t\n \tchar const *line;\n \txdlclass_t *rcrec;\n 5: 2db30cc739e = 6: 87623495994 xdiff: separate parsing lines from hashing them\n 6: 5a959c9bdad ! 7: d74fd4ef67a xdiff: conditionally use Rust's implementation of xxhash\n @@ Commit message\n \n Signed-off-by: Ezekiel Newren \n \n - ## rust/Cargo.lock ##\n -@@ rust/Cargo.lock: name = \"xdiff\"\n - version = \"0.1.0\"\n - dependencies = [\n - \"interop\",\n -+ \"xxhash-rust\",\n - ]\n -+\n -+[[package]]\n -+name = \"xxhash-rust\"\n -+version = \"0.8.15\"\n -+source = \"registry+https://github.com/rust-lang/crates.io-index\"\n -+checksum = \"fdd20c5420375476fbd4394763288da7eb0cc0b8c11deed431a91562af7335d3\"\n -\n ## rust/xdiff/Cargo.toml ##\n @@ rust/xdiff/Cargo.toml: crate-type = [\"staticlib\", \"rlib\"]\n \n 7: 0de0867ab44 ! 8: 7dc241e6682 github_workflows: install rust\n @@ Metadata\n Author: Ezekiel Newren \n \n ## Commit message ##\n - github_workflows: install rust\n + github workflows: install rust\n \n Since we have introduced rust, it needs to be installed for the\n continuous integration build targets. Create an install script\n @@ .github/workflows/main.yml: on: [push, pull_request]\n # If more than one workflow run is triggered for the very same commit hash\n # (which happens when multiple branches pointing to the same commit), only\n \n - ## .gitignore ##\n -@@ .gitignore: Release/\n - /contrib/buildsystems/out\n - /contrib/libgit-rs/target\n - /contrib/libgit-sys/target\n -+/rust/target\n -\n ## Makefile ##\n @@ Makefile: TEST_SHELL_PATH = $(SHELL_PATH)\n \n -: ----------- > 9: 96041a10d54 Do support Windows again after requiring Rust\n -: ----------- > 10: 1194de3f39c win+Meson: allow for xdiff to be compiled with MSVC\n -: ----------- > 11: 382067a09e3 win+Meson: do allow linking with the Rust-built xdiff\n -: ----------- > 12: fffdb326710 github workflows: define rust versions and targets in the same place\n -: ----------- > 13: 44784f0d672 github workflows: upload Cargo.lock\n -: ----------- > 14: f20efdff7aa xdiff: implement a white space iterator in Rust\n -: ----------- > 15: c8d41173274 xdiff: create line_hash() and line_equal()\n -: ----------- > 16: f7829c55871 xdiff: optimize case where --ignore-cr-at-eol is the only whitespace flag\n -: ----------- > 17: 395609aff4b xdiff: use rust's version of whitespace processing\n\n-- \ngitgitgadget\n"},{"id":"524207","messageId":"75dfb40ead370e80dda423998f8220ac19c2ff46.1755220973.git.gitgitgadget@gmail.com","threadId":"63804","inReplyTo":"pull.1980.v2.git.git.1755220973.gitgitgadget@gmail.com","subject":"[PATCH v2 01/17] doc: add a policy for using Rust","fromName":"brian m. carlson via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2025-08-15T01:22:36Z","receivedAt":"2025-08-15T01:22:57Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"From: \"brian m. carlson\" \n\nGit has historically been written primarily in C, with some shell and\nPerl. However, C is not memory safe, which makes it more likely that\nsecurity vulnerabilities or other bugs will be introduced, and it is\nalso more verbose and less ergonomic than other, more modern languages.\n\nOne of the most common modern compiled languages which is easily\ninteroperable with C is Rust. It is popular (the most admired language\non the 2024 Stack Overflow Developer Survey), efficient, portable, and\nrobust.\n\nIntroduce a document laying out the incremental introduction of Rust to\nGit and provide a detailed rationale for doing so, including the points\nabove. Propose a design for this approach that addresses the needs of\ndownstreams and distributors, as well as contributors.\n\nSince we don't want to carry both a C and Rust version of code and want\nto be able to add new features only in Rust, mention that Rust is a\nrequired part of our platform support policy.\n\nIt should be noted that a recent discussion at the Berlin Git Merge\nContributor Summit found widespread support for the addition of Rust to\nGit. While of course not all contributors were represented, the\nproposal appeared to have the support of a majority of active\ncontributors.\n\nSigned-off-by: brian m. carlson \nSigned-off-by: Ezekiel Newren \n---\n Documentation/Makefile | 1 +\n Documentation/technical/platform-support.adoc | 2 +\n Documentation/technical/rust-support.adoc | 119 ++++++++++++++++++\n 3 files changed, 122 insertions(+)\n create mode 100644 Documentation/technical/rust-support.adoc\n\ndiff --git a/Documentation/Makefile b/Documentation/Makefile\nindex b109d25e9c80..066b761c01b9 100644\n--- a/Documentation/Makefile\n+++ b/Documentation/Makefile\n@@ -127,6 +127,7 @@ TECH_DOCS += technical/parallel-checkout\n TECH_DOCS += technical/partial-clone\n TECH_DOCS += technical/platform-support\n TECH_DOCS += technical/racy-git\n+TECH_DOCS += technical/rust-support\n TECH_DOCS += technical/reftable\n TECH_DOCS += technical/scalar\n TECH_DOCS += technical/send-pack-pipeline\ndiff --git a/Documentation/technical/platform-support.adoc b/Documentation/technical/platform-support.adoc\nindex 0a2fb28d6277..42b04b186105 100644\n--- a/Documentation/technical/platform-support.adoc\n+++ b/Documentation/technical/platform-support.adoc\n@@ -33,6 +33,8 @@ meet the following minimum requirements:\n \n * Has active security support (taking security releases of dependencies, etc)\n \n+* Supports Rust and the toolchain version specified in link:rust-support.txt[].\n+\n These requirements are a starting point, and not sufficient on their own for the\n Git community to be enthusiastic about supporting your platform. Maintainers of\n platforms which do meet these requirements can follow the steps below to make it\ndiff --git a/Documentation/technical/rust-support.adoc b/Documentation/technical/rust-support.adoc\nnew file mode 100644\nindex 000000000000..a63327ebc575\n--- /dev/null\n+++ b/Documentation/technical/rust-support.adoc\n@@ -0,0 +1,119 @@\n+Usage of Rust in Git\n+====================\n+\n+Objective\n+---------\n+Introduce Rust into Git incrementally to improve security and maintainability.\n+\n+Background\n+----------\n+Git has historically been written primarily in C, with some portions in shell,\n+Perl, or other languages. At the time it was originally written, this was\n+important for portability and was a logical choice for software development.\n+\n+:0: link:https://security.googleblog.com/2024/09/eliminating-memory-safety-vulnerabilities-Android.html\n+:1: link:https://www.cisa.gov/resources-tools/resources/product-security-bad-practices\n+\n+However, as time has progressed, we've seen an increased concern with memory\n+safety vulnerabilities and the development of newer languages, such as Rust,\n+that substantially limit or eliminate this class of vulnerabilities.\n+Development in a variety of projects has found that memory safety\n+vulnerabilities constitute about 70% of vulnerabilities of software in\n+languages that are not memory safe. For instance, {0}[one survey of Android]\n+found that memory safety vulnerabilities decreased from 76% to 24% over six\n+years due to an increase in memory safe code. Similarly, the U.S. government\n+is {1}[proposing to classify development in memory unsafe languages as a\n+Product Security Bad Practice\"].\n+\n+These risks are even more substantial when we consider the fact that Git is a\n+network-facing service. Many organizations run Git servers internally or use a\n+cloud-based forge, and the risk of accidental exposure or compromise of user\n+data is substantial. It's important to ensure that Git, whether it's used\n+locally or remotely, is robustly secure.\n+\n+In addition, C is a difficult language to write well and concisely. While it\n+is of course possible to do anything with C, it lacks built-in support for\n+niceties found in modern languages, such as hash tables, generics, typed\n+errors, and automatic destruction, and most modern language offer shorter, more\n+ergonomic syntax for expressing code. This is valuable functionality that can\n+allow Git to be developed more rapidly, more easily, by more developers of a\n+variety of levels, and with more confidence in the correctness of the code.\n+\n+For these reasons, adding Rust to Git is a sensible and prudent move that will\n+allow us to improve the quality of the code and potentially attract new developers.\n+\n+Goals\n+-----\n+1. Git continues to build, run, and pass tests on a wide variety of operating\n+ systems and architectures.\n+2. Transition from C to Rust is incremental; that is, code can be ported as it\n+ is convenient and Git does not need to transition all at once.\n+3. Git continues to support older operating systems in conformance with the\n+ platform support policy.\n+\n+Non-Goals\n+---------\n+1. Support for every possible operating system and architecture. Git already\n+ has a platform support policy which defines what is supported and we already\n+ exclude some operating systems for various reasons (e.g., lacking enough POSIX\n+ tools to pass the test suite).\n+2. Implementing C-only versions of Rust code or compiling a C-only Git. This\n+ would be difficult to maintain and would not offer the ergonomic benefits we\n+ desire.\n+\n+Design\n+------\n+Git will adopt Rust incrementally. This transition will start with the\n+creation of a static library that can be linked into the existing Git binaries.\n+At some point, we may wish to expose a dynamic library and compile the Git\n+binaries themselves using Rust. Using an incremental approach allows us to\n+determine as we go along how to structure our code in the best way for the\n+project and avoids the need to make hard, potentially disruptive, transitions\n+caused by porting a binary wholesale from one language to another that might\n+introduce bugs.\n+\n+We will use the `bindgen` and `cbindgen` crates for handling C-compatible\n+bindings and the `rustix` crate for POSIX-compatible interfaces. The `libc`\n+crate, which is used by `rustix`, does not expose safe interfaces and does not\n+handle differences between platforms, such as differing 64-bit `stat` call\n+names, and so is less desirable as a target than `rustix`. We may still choose\n+to use it in some cases if `rustix` does not offer suitable interfaces.\n+\n+Rust upstream releases every six weeks and only supports the latest stable\n+release. While it is nice that upstream is active, we would like our software\n+releases to have a lifespan exceeding six weeks. To allow compiling our code\n+on a variety of systems, we will support the version of Rust in Debian stable,\n+plus, for a year after a new Debian stable is released, the version in Debian\n+oldstable.\n+\n+This provides an approximately three-year lifespan of support for a Rust\n+release and allows us to support a variety of operating systems and\n+architectures, including those for which Rust upstream does not build binaries.\n+Debian stable is the benchmark distribution used by many Rust projects when\n+determining supported Rust versions, and it is an extremely portable and\n+popular free software operating system that is available to the public at no\n+charge, which makes it a sensible choice for us as well.\n+\n+We may change this policy if the Rust project issues long-term support releases\n+or the Rust community and distributors agree on releases to target as if they\n+were long-term support releases.\n+\n+This version support policy necessitates that we be very careful about the\n+dependencies we include, since many Rust projects support only the latest\n+stable version. However, we typically have been careful about dependencies in\n+the first place, so this should not be a major departure from existing policy,\n+although it may be a change for some existing Rust developers.\n+\n+We will avoid including the `Cargo.lock` file in the repository and instead\n+specify minimum dependency versions in the `Cargo.toml` file. We want to allow\n+people to use newer versions of dependencies if necessary to support newer\n+platforms without needing to force upgrades of dependencies on all users, and\n+it provides additional flexibility for distribution maintainers.\n+\n+We do not plan to support beta or nightly versions of the Rust compiler. These\n+versions may change rapidly and especially parts of the toolchain such as\n+Clippy, the lint tool, can have false positives or add additional warnings with\n+too great of a frequency to be supportable by the project. However, we do plan\n+to support alternate compilers, such as the rust_codegen_gcc backend and gccrs\n+when they are stable and support our desired release versions. This will\n+provide greater support for more operating systems and architectures.\n-- \ngitgitgadget\n\n"},{"id":"524205","messageId":"7709e5eddba671db0e772724c0c71516d18f2cb2.1755220973.git.gitgitgadget@gmail.com","threadId":"63804","inReplyTo":"pull.1980.v2.git.git.1755220973.gitgitgadget@gmail.com","subject":"[PATCH v2 02/17] xdiff: introduce rust","fromName":"Ezekiel Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2025-08-15T01:22:37Z","receivedAt":"2025-08-15T01:22:58Z","isPatch":true,"sender":{"key":"ezekielnewren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/18313324?v=4"},"body":"From: Ezekiel Newren \n\nUpcoming patches will accelerate and simplify xdiff, while also\nporting parts of it to Rust. In preparation, add some stubs and setup\nthe Rust build. For now, it is easier to let cargo build rust and\nhave make or meson merely link against the static library that cargo\nbuilds. In line with ongoing libification efforts, use multiple\ncrates to allow more modularity on the Rust side. xdiff is the crate\nthat this series will focus on, but we also introduce the interop\ncrate for future patch series.\n\nIn order to facilitate interoperability between C and Rust, introduce\nC definitions for Rust primitive types in git-compat-util.h.\n\nSigned-off-by: Ezekiel Newren \n---\n .gitignore | 3 +++\n Makefile | 20 +++++++++++++++++++-\n git-compat-util.h | 17 +++++++++++++++++\n meson.build | 32 ++++++++++++++++++++++++++++++++\n rust/Cargo.toml | 6 ++++++\n rust/interop/Cargo.toml | 14 ++++++++++++++\n rust/interop/src/lib.rs | 0\n rust/xdiff/Cargo.toml | 15 +++++++++++++++\n rust/xdiff/src/lib.rs | 0\n 9 files changed, 106 insertions(+), 1 deletion(-)\n create mode 100644 rust/Cargo.toml\n create mode 100644 rust/interop/Cargo.toml\n create mode 100644 rust/interop/src/lib.rs\n create mode 100644 rust/xdiff/Cargo.toml\n create mode 100644 rust/xdiff/src/lib.rs\n\ndiff --git a/.gitignore b/.gitignore\nindex 04c444404e4b..ff81e3580c4e 100644\n--- a/.gitignore\n+++ b/.gitignore\n@@ -254,3 +254,6 @@ Release/\n /contrib/buildsystems/out\n /contrib/libgit-rs/target\n /contrib/libgit-sys/target\n+/.idea/\n+/rust/target/\n+/rust/Cargo.lock\ndiff --git a/Makefile b/Makefile\nindex 70d1543b6b86..db39e6e1c28e 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -919,6 +919,11 @@ TEST_SHELL_PATH = $(SHELL_PATH)\n \n LIB_FILE = libgit.a\n XDIFF_LIB = xdiff/lib.a\n+ifeq ($(DEBUG), 1)\n+RUST_LIB = rust/target/debug/libxdiff.a\n+else\n+RUST_LIB = rust/target/release/libxdiff.a\n+endif\n REFTABLE_LIB = reftable/libreftable.a\n \n GENERATED_H += command-list.h\n@@ -1392,6 +1397,8 @@ UNIT_TEST_OBJS += $(UNIT_TEST_DIR)/lib-reftable.o\n GITLIBS = common-main.o $(LIB_FILE) $(XDIFF_LIB) $(REFTABLE_LIB) $(LIB_FILE)\n EXTLIBS =\n \n+GITLIBS += $(RUST_LIB)\n+\n GIT_USER_AGENT = git/$(GIT_VERSION)\n \n ifeq ($(wildcard sha1collisiondetection/lib/sha1.h),sha1collisiondetection/lib/sha1.h)\n@@ -2925,6 +2932,14 @@ $(LIB_FILE): $(LIB_OBJS)\n $(XDIFF_LIB): $(XDIFF_OBJS)\n \t$(QUIET_AR)$(RM) $@ && $(AR) $(ARFLAGS) $@ $^\n \n+.PHONY: $(RUST_LIB)\n+$(RUST_LIB):\n+ifeq ($(DEBUG), 1)\n+\tcd rust && RUSTFLAGS=\"-Aunused_imports -Adead_code\" cargo build --verbose\n+else\n+\tcd rust && RUSTFLAGS=\"-Aunused_imports -Adead_code\" cargo build --verbose --release\n+endif\n+\n $(REFTABLE_LIB): $(REFTABLE_OBJS)\n \t$(QUIET_AR)$(RM) $@ && $(AR) $(ARFLAGS) $@ $^\n \n@@ -3756,7 +3771,10 @@ cocciclean:\n \t$(RM) -r .build/contrib/coccinelle\n \t$(RM) contrib/coccinelle/*.cocci.patch\n \n-clean: profile-clean coverage-clean cocciclean\n+rustclean:\n+\tcd rust && cargo clean\n+\n+clean: profile-clean coverage-clean cocciclean rustclean\n \t$(RM) -r .build $(UNIT_TEST_BIN)\n \t$(RM) GIT-TEST-SUITES\n \t$(RM) po/git.pot po/git-core.pot\ndiff --git a/git-compat-util.h b/git-compat-util.h\nindex 4678e21c4cb8..82dc99764ac0 100644\n--- a/git-compat-util.h\n+++ b/git-compat-util.h\n@@ -196,6 +196,23 @@ static inline int is_xplatform_dir_sep(int c)\n #include \"compat/msvc.h\"\n #endif\n \n+/* rust types */\n+typedef uint8_t u8;\n+typedef uint16_t u16;\n+typedef uint32_t u32;\n+typedef uint64_t u64;\n+\n+typedef int8_t i8;\n+typedef int16_t i16;\n+typedef int32_t i32;\n+typedef int64_t i64;\n+\n+typedef float f32;\n+typedef double f64;\n+\n+typedef size_t usize;\n+typedef ptrdiff_t isize;\n+\n /* used on Mac OS X */\n #ifdef PRECOMPOSE_UNICODE\n #include \"compat/precompose_utf8.h\"\ndiff --git a/meson.build b/meson.build\nindex 596f5ac7110e..2d8da17f6515 100644\n--- a/meson.build\n+++ b/meson.build\n@@ -267,6 +267,36 @@ version_gen_environment.set('GIT_DATE', get_option('build_date'))\n version_gen_environment.set('GIT_USER_AGENT', get_option('user_agent'))\n version_gen_environment.set('GIT_VERSION', get_option('version'))\n \n+if get_option('optimization') in ['2', '3', 's', 'z']\n+ rust_target = 'release'\n+ rust_args = ['--release']\n+ rustflags = '-Aunused_imports -Adead_code'\n+else\n+ rust_target = 'debug'\n+ rust_args = []\n+ rustflags = '-Aunused_imports -Adead_code -C debuginfo=2 -C opt-level=1 -C force-frame-pointers=yes'\n+endif\n+\n+\n+rust_leaf = custom_target('rust_leaf',\n+ output: 'libxdiff.a',\n+ build_by_default: true,\n+ build_always_stale: true,\n+ command: ['cargo', 'build',\n+ '--manifest-path', meson.project_source_root() / 'rust/Cargo.toml'\n+ ] + rust_args,\n+ env: {\n+ 'RUSTFLAGS': rustflags,\n+ },\n+ install: false,\n+)\n+\n+rust_xdiff_dep = declare_dependency(\n+ link_args: ['-L' + meson.project_source_root() / 'rust/target' / rust_target, '-lxdiff'],\n+# include_directories: include_directories('xdiff/include'), # Adjust if you expose headers\n+)\n+\n+\n compiler = meson.get_compiler('c')\n \n libgit_sources = [\n@@ -1677,6 +1707,8 @@ version_def_h = custom_target(\n )\n libgit_sources += version_def_h\n \n+libgit_dependencies += rust_xdiff_dep\n+\n libgit = declare_dependency(\n link_with: static_library('git',\n sources: libgit_sources,\ndiff --git a/rust/Cargo.toml b/rust/Cargo.toml\nnew file mode 100644\nindex 000000000000..ed3d79d7f827\n--- /dev/null\n+++ b/rust/Cargo.toml\n@@ -0,0 +1,6 @@\n+[workspace]\n+members = [\n+ \"xdiff\",\n+ \"interop\",\n+]\n+resolver = \"2\"\ndiff --git a/rust/interop/Cargo.toml b/rust/interop/Cargo.toml\nnew file mode 100644\nindex 000000000000..045e3b01cfad\n--- /dev/null\n+++ b/rust/interop/Cargo.toml\n@@ -0,0 +1,14 @@\n+[package]\n+name = \"interop\"\n+version = \"0.1.0\"\n+edition = \"2021\"\n+\n+[lib]\n+name = \"interop\"\n+path = \"src/lib.rs\"\n+## staticlib to generate xdiff.a for use by gcc\n+## cdylib (optional) to generate xdiff.so for use by gcc\n+## rlib is required by the rust unit tests\n+crate-type = [\"staticlib\", \"rlib\"]\n+\n+[dependencies]\ndiff --git a/rust/interop/src/lib.rs b/rust/interop/src/lib.rs\nnew file mode 100644\nindex 000000000000..e69de29bb2d1\ndiff --git a/rust/xdiff/Cargo.toml b/rust/xdiff/Cargo.toml\nnew file mode 100644\nindex 000000000000..eb7966aada64\n--- /dev/null\n+++ b/rust/xdiff/Cargo.toml\n@@ -0,0 +1,15 @@\n+[package]\n+name = \"xdiff\"\n+version = \"0.1.0\"\n+edition = \"2021\"\n+\n+[lib]\n+name = \"xdiff\"\n+path = \"src/lib.rs\"\n+## staticlib to generate xdiff.a for use by gcc\n+## cdylib (optional) to generate xdiff.so for use by gcc\n+## rlib is required by the rust unit tests\n+crate-type = [\"staticlib\", \"rlib\"]\n+\n+[dependencies]\n+interop = { path = \"../interop\" }\ndiff --git a/rust/xdiff/src/lib.rs b/rust/xdiff/src/lib.rs\nnew file mode 100644\nindex 000000000000..e69de29bb2d1\n-- \ngitgitgadget\n\n"},{"id":"524206","messageId":"56c96d355544609b26be69b6d2ae04b5bdf3d7a3.1755220973.git.gitgitgadget@gmail.com","threadId":"63804","inReplyTo":"pull.1980.v2.git.git.1755220973.gitgitgadget@gmail.com","subject":"[PATCH v2 03/17] xdiff/xprepare: remove superfluous forward declarations","fromName":"Ezekiel Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2025-08-15T01:22:38Z","receivedAt":"2025-08-15T01:22:59Z","isPatch":true,"sender":{"key":"ezekielnewren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/18313324?v=4"},"body":"From: Ezekiel Newren \n\nMove xdl_prepare_env() later in the file to avoid the need\nfor forward declarations.\n\nSigned-off-by: Ezekiel Newren \n---\n xdiff/xprepare.c | 116 ++++++++++++++++++++---------------------------\n 1 file changed, 50 insertions(+), 66 deletions(-)\n\ndiff --git a/xdiff/xprepare.c b/xdiff/xprepare.c\nindex e1d4017b2dde..a45c5ee208c8 100644\n--- a/xdiff/xprepare.c\n+++ b/xdiff/xprepare.c\n@@ -53,21 +53,6 @@ typedef struct s_xdlclassifier {\n \n \n \n-static int xdl_init_classifier(xdlclassifier_t *cf, long size, long flags);\n-static void xdl_free_classifier(xdlclassifier_t *cf);\n-static int xdl_classify_record(unsigned int pass, xdlclassifier_t *cf, xrecord_t **rhash,\n-\t\t\t unsigned int hbits, xrecord_t *rec);\n-static int xdl_prepare_ctx(unsigned int pass, mmfile_t *mf, long narec, xpparam_t const *xpp,\n-\t\t\t xdlclassifier_t *cf, xdfile_t *xdf);\n-static void xdl_free_ctx(xdfile_t *xdf);\n-static int xdl_clean_mmatch(char const *dis, long i, long s, long e);\n-static int xdl_cleanup_records(xdlclassifier_t *cf, xdfile_t *xdf1, xdfile_t *xdf2);\n-static int xdl_trim_ends(xdfile_t *xdf1, xdfile_t *xdf2);\n-static int xdl_optimize_ctxs(xdlclassifier_t *cf, xdfile_t *xdf1, xdfile_t *xdf2);\n-\n-\n-\n-\n static int xdl_init_classifier(xdlclassifier_t *cf, long size, long flags) {\n \tcf->flags = flags;\n \n@@ -242,57 +227,6 @@ static void xdl_free_ctx(xdfile_t *xdf) {\n }\n \n \n-int xdl_prepare_env(mmfile_t *mf1, mmfile_t *mf2, xpparam_t const *xpp,\n-\t\t xdfenv_t *xe) {\n-\tlong enl1, enl2, sample;\n-\txdlclassifier_t cf;\n-\n-\tmemset(&cf, 0, sizeof(cf));\n-\n-\t/*\n-\t * For histogram diff, we can afford a smaller sample size and\n-\t * thus a poorer estimate of the number of lines, as the hash\n-\t * table (rhash) won't be filled up/grown. The number of lines\n-\t * (nrecs) will be updated correctly anyway by\n-\t * xdl_prepare_ctx().\n-\t */\n-\tsample = (XDF_DIFF_ALG(xpp->flags) == XDF_HISTOGRAM_DIFF\n-\t\t ? XDL_GUESS_NLINES2 : XDL_GUESS_NLINES1);\n-\n-\tenl1 = xdl_guess_lines(mf1, sample) + 1;\n-\tenl2 = xdl_guess_lines(mf2, sample) + 1;\n-\n-\tif (xdl_init_classifier(&cf, enl1 + enl2 + 1, xpp->flags) < 0)\n-\t\treturn -1;\n-\n-\tif (xdl_prepare_ctx(1, mf1, enl1, xpp, &cf, &xe->xdf1) < 0) {\n-\n-\t\txdl_free_classifier(&cf);\n-\t\treturn -1;\n-\t}\n-\tif (xdl_prepare_ctx(2, mf2, enl2, xpp, &cf, &xe->xdf2) < 0) {\n-\n-\t\txdl_free_ctx(&xe->xdf1);\n-\t\txdl_free_classifier(&cf);\n-\t\treturn -1;\n-\t}\n-\n-\tif ((XDF_DIFF_ALG(xpp->flags) != XDF_PATIENCE_DIFF) &&\n-\t (XDF_DIFF_ALG(xpp->flags) != XDF_HISTOGRAM_DIFF) &&\n-\t xdl_optimize_ctxs(&cf, &xe->xdf1, &xe->xdf2) < 0) {\n-\n-\t\txdl_free_ctx(&xe->xdf2);\n-\t\txdl_free_ctx(&xe->xdf1);\n-\t\txdl_free_classifier(&cf);\n-\t\treturn -1;\n-\t}\n-\n-\txdl_free_classifier(&cf);\n-\n-\treturn 0;\n-}\n-\n-\n void xdl_free_env(xdfenv_t *xe) {\n \n \txdl_free_ctx(&xe->xdf2);\n@@ -460,3 +394,53 @@ static int xdl_optimize_ctxs(xdlclassifier_t *cf, xdfile_t *xdf1, xdfile_t *xdf2\n \n \treturn 0;\n }\n+\n+int xdl_prepare_env(mmfile_t *mf1, mmfile_t *mf2, xpparam_t const *xpp,\n+\t\t xdfenv_t *xe) {\n+\tlong enl1, enl2, sample;\n+\txdlclassifier_t cf;\n+\n+\tmemset(&cf, 0, sizeof(cf));\n+\n+\t/*\n+\t * For histogram diff, we can afford a smaller sample size and\n+\t * thus a poorer estimate of the number of lines, as the hash\n+\t * table (rhash) won't be filled up/grown. The number of lines\n+\t * (nrecs) will be updated correctly anyway by\n+\t * xdl_prepare_ctx().\n+\t */\n+\tsample = (XDF_DIFF_ALG(xpp->flags) == XDF_HISTOGRAM_DIFF\n+\t\t ? XDL_GUESS_NLINES2 : XDL_GUESS_NLINES1);\n+\n+\tenl1 = xdl_guess_lines(mf1, sample) + 1;\n+\tenl2 = xdl_guess_lines(mf2, sample) + 1;\n+\n+\tif (xdl_init_classifier(&cf, enl1 + enl2 + 1, xpp->flags) < 0)\n+\t\treturn -1;\n+\n+\tif (xdl_prepare_ctx(1, mf1, enl1, xpp, &cf, &xe->xdf1) < 0) {\n+\n+\t\txdl_free_classifier(&cf);\n+\t\treturn -1;\n+\t}\n+\tif (xdl_prepare_ctx(2, mf2, enl2, xpp, &cf, &xe->xdf2) < 0) {\n+\n+\t\txdl_free_ctx(&xe->xdf1);\n+\t\txdl_free_classifier(&cf);\n+\t\treturn -1;\n+\t}\n+\n+\tif ((XDF_DIFF_ALG(xpp->flags) != XDF_PATIENCE_DIFF) &&\n+\t (XDF_DIFF_ALG(xpp->flags) != XDF_HISTOGRAM_DIFF) &&\n+\t xdl_optimize_ctxs(&cf, &xe->xdf1, &xe->xdf2) < 0) {\n+\n+\t\txdl_free_ctx(&xe->xdf2);\n+\t\txdl_free_ctx(&xe->xdf1);\n+\t\txdl_free_classifier(&cf);\n+\t\treturn -1;\n+\t }\n+\n+\txdl_free_classifier(&cf);\n+\n+\treturn 0;\n+}\n-- \ngitgitgadget\n\n"},{"id":"524208","messageId":"ebec3689dcea838cb57b11465ea4340b8b84d842.1755220973.git.gitgitgadget@gmail.com","threadId":"63804","inReplyTo":"pull.1980.v2.git.git.1755220973.gitgitgadget@gmail.com","subject":"[PATCH v2 04/17] xdiff: delete unnecessary fields from xrecord_t and xdfile_t","fromName":"Ezekiel Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2025-08-15T01:22:39Z","receivedAt":"2025-08-15T01:23:00Z","isPatch":true,"sender":{"key":"ezekielnewren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/18313324?v=4"},"body":"From: Ezekiel Newren \n\nxrecord_t.next, xdfile_t.hbits, xdfile_t.rhash are initialized,\nbut never used for anything by the code. Remove them.\n\nSigned-off-by: Ezekiel Newren \n---\n xdiff/xprepare.c | 24 +++---------------------\n xdiff/xtypes.h | 3 ---\n 2 files changed, 3 insertions(+), 24 deletions(-)\n\ndiff --git a/xdiff/xprepare.c b/xdiff/xprepare.c\nindex a45c5ee208c8..ad356281f939 100644\n--- a/xdiff/xprepare.c\n+++ b/xdiff/xprepare.c\n@@ -91,8 +91,7 @@ static void xdl_free_classifier(xdlclassifier_t *cf) {\n }\n \n \n-static int xdl_classify_record(unsigned int pass, xdlclassifier_t *cf, xrecord_t **rhash,\n-\t\t\t unsigned int hbits, xrecord_t *rec) {\n+static int xdl_classify_record(unsigned int pass, xdlclassifier_t *cf, xrecord_t *rec) {\n \tlong hi;\n \tchar const *line;\n \txdlclass_t *rcrec;\n@@ -126,23 +125,17 @@ static int xdl_classify_record(unsigned int pass, xdlclassifier_t *cf, xrecord_t\n \n \trec->ha = (unsigned long) rcrec->idx;\n \n-\thi = (long) XDL_HASHLONG(rec->ha, hbits);\n-\trec->next = rhash[hi];\n-\trhash[hi] = rec;\n-\n \treturn 0;\n }\n \n \n static int xdl_prepare_ctx(unsigned int pass, mmfile_t *mf, long narec, xpparam_t const *xpp,\n \t\t\t xdlclassifier_t *cf, xdfile_t *xdf) {\n-\tunsigned int hbits;\n-\tlong nrec, hsize, bsize;\n+\tlong nrec, bsize;\n \tunsigned long hav;\n \tchar const *blk, *cur, *top, *prev;\n \txrecord_t *crec;\n \txrecord_t **recs;\n-\txrecord_t **rhash;\n \tunsigned long *ha;\n \tchar *rchg;\n \tlong *rindex;\n@@ -150,7 +143,6 @@ static int xdl_prepare_ctx(unsigned int pass, mmfile_t *mf, long narec, xpparam_\n \tha = NULL;\n \trindex = NULL;\n \trchg = NULL;\n-\trhash = NULL;\n \trecs = NULL;\n \n \tif (xdl_cha_init(&xdf->rcha, sizeof(xrecord_t), narec / 4 + 1) < 0)\n@@ -158,11 +150,6 @@ static int xdl_prepare_ctx(unsigned int pass, mmfile_t *mf, long narec, xpparam_\n \tif (!XDL_ALLOC_ARRAY(recs, narec))\n \t\tgoto abort;\n \n-\thbits = xdl_hashbits((unsigned int) narec);\n-\thsize = 1 << hbits;\n-\tif (!XDL_CALLOC_ARRAY(rhash, hsize))\n-\t\tgoto abort;\n-\n \tnrec = 0;\n \tif ((cur = blk = xdl_mmfile_first(mf, &bsize))) {\n \t\tfor (top = blk + bsize; cur < top; ) {\n@@ -176,7 +163,7 @@ static int xdl_prepare_ctx(unsigned int pass, mmfile_t *mf, long narec, xpparam_\n \t\t\tcrec->size = (long) (cur - prev);\n \t\t\tcrec->ha = hav;\n \t\t\trecs[nrec++] = crec;\n-\t\t\tif (xdl_classify_record(pass, cf, rhash, hbits, crec) < 0)\n+\t\t\tif (xdl_classify_record(pass, cf, crec) < 0)\n \t\t\t\tgoto abort;\n \t\t}\n \t}\n@@ -194,8 +181,6 @@ static int xdl_prepare_ctx(unsigned int pass, mmfile_t *mf, long narec, xpparam_\n \n \txdf->nrec = nrec;\n \txdf->recs = recs;\n-\txdf->hbits = hbits;\n-\txdf->rhash = rhash;\n \txdf->rchg = rchg + 1;\n \txdf->rindex = rindex;\n \txdf->nreff = 0;\n@@ -209,7 +194,6 @@ abort:\n \txdl_free(ha);\n \txdl_free(rindex);\n \txdl_free(rchg);\n-\txdl_free(rhash);\n \txdl_free(recs);\n \txdl_cha_free(&xdf->rcha);\n \treturn -1;\n@@ -217,8 +201,6 @@ abort:\n \n \n static void xdl_free_ctx(xdfile_t *xdf) {\n-\n-\txdl_free(xdf->rhash);\n \txdl_free(xdf->rindex);\n \txdl_free(xdf->rchg - 1);\n \txdl_free(xdf->ha);\ndiff --git a/xdiff/xtypes.h b/xdiff/xtypes.h\nindex 8442bd436efe..8b8467360ecf 100644\n--- a/xdiff/xtypes.h\n+++ b/xdiff/xtypes.h\n@@ -39,7 +39,6 @@ typedef struct s_chastore {\n } chastore_t;\n \n typedef struct s_xrecord {\n-\tstruct s_xrecord *next;\n \tchar const *ptr;\n \tlong size;\n \tunsigned long ha;\n@@ -48,8 +47,6 @@ typedef struct s_xrecord {\n typedef struct s_xdfile {\n \tchastore_t rcha;\n \tlong nrec;\n-\tunsigned int hbits;\n-\txrecord_t **rhash;\n \tlong dstart, dend;\n \txrecord_t **recs;\n \tchar *rchg;\n-- \ngitgitgadget\n\n"},{"id":"524209","messageId":"769d1a5b9d2f56d9b8266124507a28e1b78b2352.1755220973.git.gitgitgadget@gmail.com","threadId":"63804","inReplyTo":"pull.1980.v2.git.git.1755220973.gitgitgadget@gmail.com","subject":"[PATCH v2 05/17] xdiff: make fields of xrecord_t Rust friendly","fromName":"Ezekiel Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2025-08-15T01:22:40Z","receivedAt":"2025-08-15T01:23:01Z","isPatch":true,"sender":{"key":"ezekielnewren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/18313324?v=4"},"body":"From: Ezekiel Newren \n\nA few commits ago, we added definitions for Rust primitive types,\nto facilitate interoperability between C and Rust. Switch a\nfew variables to use these types. Which, for now, will\nrequire adding some casts.\n\nAlso change xdlclass_t::ha to be u64 to match xrecord_t::ha, as\npointed out by Johannes.\n\nHelped-by: Johannes Schindelin \nSigned-off-by: Ezekiel Newren \n---\n xdiff/xdiffi.c | 8 ++++----\n xdiff/xemit.c | 2 +-\n xdiff/xmerge.c | 14 +++++++-------\n xdiff/xpatience.c | 2 +-\n xdiff/xprepare.c | 8 ++++----\n xdiff/xtypes.h | 6 +++---\n xdiff/xutils.c | 4 ++--\n 7 files changed, 22 insertions(+), 22 deletions(-)\n\ndiff --git a/xdiff/xdiffi.c b/xdiff/xdiffi.c\nindex 5a96e36dfbea..3b364c61f671 100644\n--- a/xdiff/xdiffi.c\n+++ b/xdiff/xdiffi.c\n@@ -418,7 +418,7 @@ static int get_indent(xrecord_t *rec)\n \tlong i;\n \tint ret = 0;\n \n-\tfor (i = 0; i < rec->size; i++) {\n+\tfor (i = 0; i < (long) rec->size; i++) {\n \t\tchar c = rec->ptr[i];\n \n \t\tif (!XDL_ISSPACE(c))\n@@ -1005,11 +1005,11 @@ static void xdl_mark_ignorable_lines(xdchange_t *xscr, xdfenv_t *xe, long flags)\n \n \t\trec = &xe->xdf1.recs[xch->i1];\n \t\tfor (i = 0; i < xch->chg1 && ignore; i++)\n-\t\t\tignore = xdl_blankline(rec[i]->ptr, rec[i]->size, flags);\n+\t\t\tignore = xdl_blankline((const char*) rec[i]->ptr, rec[i]->size, flags);\n \n \t\trec = &xe->xdf2.recs[xch->i2];\n \t\tfor (i = 0; i < xch->chg2 && ignore; i++)\n-\t\t\tignore = xdl_blankline(rec[i]->ptr, rec[i]->size, flags);\n+\t\t\tignore = xdl_blankline((const char*)rec[i]->ptr, rec[i]->size, flags);\n \n \t\txch->ignore = ignore;\n \t}\n@@ -1020,7 +1020,7 @@ static int record_matches_regex(xrecord_t *rec, xpparam_t const *xpp) {\n \tsize_t i;\n \n \tfor (i = 0; i < xpp->ignore_regex_nr; i++)\n-\t\tif (!regexec_buf(xpp->ignore_regex[i], rec->ptr, rec->size, 1,\n+\t\tif (!regexec_buf(xpp->ignore_regex[i], (const char*) rec->ptr, rec->size, 1,\n \t\t\t\t ®match, 0))\n \t\t\treturn 1;\n \ndiff --git a/xdiff/xemit.c b/xdiff/xemit.c\nindex 1d40c9cb4076..bbf7b7f8c862 100644\n--- a/xdiff/xemit.c\n+++ b/xdiff/xemit.c\n@@ -24,7 +24,7 @@\n \n static long xdl_get_rec(xdfile_t *xdf, long ri, char const **rec) {\n \n-\t*rec = xdf->recs[ri]->ptr;\n+\t*rec = (char const*) xdf->recs[ri]->ptr;\n \n \treturn xdf->recs[ri]->size;\n }\ndiff --git a/xdiff/xmerge.c b/xdiff/xmerge.c\nindex af40c88a5b36..6fa6ea61a208 100644\n--- a/xdiff/xmerge.c\n+++ b/xdiff/xmerge.c\n@@ -101,8 +101,8 @@ static int xdl_merge_cmp_lines(xdfenv_t *xe1, int i1, xdfenv_t *xe2, int i2,\n \txrecord_t **rec2 = xe2->xdf2.recs + i2;\n \n \tfor (i = 0; i < line_count; i++) {\n-\t\tint result = xdl_recmatch(rec1[i]->ptr, rec1[i]->size,\n-\t\t\trec2[i]->ptr, rec2[i]->size, flags);\n+\t\tint result = xdl_recmatch((const char*) rec1[i]->ptr, rec1[i]->size,\n+\t\t\t(const char*) rec2[i]->ptr, rec2[i]->size, flags);\n \t\tif (!result)\n \t\t\treturn -1;\n \t}\n@@ -324,8 +324,8 @@ static int xdl_fill_merge_buffer(xdfenv_t *xe1, const char *name1,\n \n static int recmatch(xrecord_t *rec1, xrecord_t *rec2, unsigned long flags)\n {\n-\treturn xdl_recmatch(rec1->ptr, rec1->size,\n-\t\t\t rec2->ptr, rec2->size, flags);\n+\treturn xdl_recmatch((char const*) rec1->ptr, rec1->size,\n+\t\t\t (char const*) rec2->ptr, rec2->size, flags);\n }\n \n /*\n@@ -383,10 +383,10 @@ static int xdl_refine_conflicts(xdfenv_t *xe1, xdfenv_t *xe2, xdmerge_t *m,\n \t\t */\n \t\tt1.ptr = (char *)xe1->xdf2.recs[m->i1]->ptr;\n \t\tt1.size = xe1->xdf2.recs[m->i1 + m->chg1 - 1]->ptr\n-\t\t\t+ xe1->xdf2.recs[m->i1 + m->chg1 - 1]->size - t1.ptr;\n+\t\t\t+ xe1->xdf2.recs[m->i1 + m->chg1 - 1]->size - (u8 const*) t1.ptr;\n \t\tt2.ptr = (char *)xe2->xdf2.recs[m->i2]->ptr;\n \t\tt2.size = xe2->xdf2.recs[m->i2 + m->chg2 - 1]->ptr\n-\t\t\t+ xe2->xdf2.recs[m->i2 + m->chg2 - 1]->size - t2.ptr;\n+\t\t\t+ xe2->xdf2.recs[m->i2 + m->chg2 - 1]->size - (u8 const*) t2.ptr;\n \t\tif (xdl_do_diff(&t1, &t2, xpp, &xe) < 0)\n \t\t\treturn -1;\n \t\tif (xdl_change_compact(&xe.xdf1, &xe.xdf2, xpp->flags) < 0 ||\n@@ -440,7 +440,7 @@ static int line_contains_alnum(const char *ptr, long size)\n static int lines_contain_alnum(xdfenv_t *xe, int i, int chg)\n {\n \tfor (; chg; chg--, i++)\n-\t\tif (line_contains_alnum(xe->xdf2.recs[i]->ptr,\n+\t\tif (line_contains_alnum((char const*) xe->xdf2.recs[i]->ptr,\n \t\t\t\txe->xdf2.recs[i]->size))\n \t\t\treturn 1;\n \treturn 0;\ndiff --git a/xdiff/xpatience.c b/xdiff/xpatience.c\nindex 77dc411d1937..986a3a3f749a 100644\n--- a/xdiff/xpatience.c\n+++ b/xdiff/xpatience.c\n@@ -121,7 +121,7 @@ static void insert_record(xpparam_t const *xpp, int line, struct hashmap *map,\n \t\treturn;\n \tmap->entries[index].line1 = line;\n \tmap->entries[index].hash = record->ha;\n-\tmap->entries[index].anchor = is_anchor(xpp, map->env->xdf1.recs[line - 1]->ptr);\n+\tmap->entries[index].anchor = is_anchor(xpp, (const char*) map->env->xdf1.recs[line - 1]->ptr);\n \tif (!map->first)\n \t\tmap->first = map->entries + index;\n \tif (map->last) {\ndiff --git a/xdiff/xprepare.c b/xdiff/xprepare.c\nindex ad356281f939..00cdf7d8a038 100644\n--- a/xdiff/xprepare.c\n+++ b/xdiff/xprepare.c\n@@ -32,7 +32,7 @@\n \n typedef struct s_xdlclass {\n \tstruct s_xdlclass *next;\n-\tunsigned long ha;\n+\tu64 ha;\n \tchar const *line;\n \tlong size;\n \tlong idx;\n@@ -96,12 +96,12 @@ static int xdl_classify_record(unsigned int pass, xdlclassifier_t *cf, xrecord_t\n \tchar const *line;\n \txdlclass_t *rcrec;\n \n-\tline = rec->ptr;\n+\tline = (char const*) rec->ptr;\n \thi = (long) XDL_HASHLONG(rec->ha, cf->hbits);\n \tfor (rcrec = cf->rchash[hi]; rcrec; rcrec = rcrec->next)\n \t\tif (rcrec->ha == rec->ha &&\n \t\t\t\txdl_recmatch(rcrec->line, rcrec->size,\n-\t\t\t\t\trec->ptr, rec->size, cf->flags))\n+\t\t\t\t\t(const char*) rec->ptr, rec->size, cf->flags))\n \t\t\tbreak;\n \n \tif (!rcrec) {\n@@ -159,7 +159,7 @@ static int xdl_prepare_ctx(unsigned int pass, mmfile_t *mf, long narec, xpparam_\n \t\t\t\tgoto abort;\n \t\t\tif (!(crec = xdl_cha_alloc(&xdf->rcha)))\n \t\t\t\tgoto abort;\n-\t\t\tcrec->ptr = prev;\n+\t\t\tcrec->ptr = (u8 const*) prev;\n \t\t\tcrec->size = (long) (cur - prev);\n \t\t\tcrec->ha = hav;\n \t\t\trecs[nrec++] = crec;\ndiff --git a/xdiff/xtypes.h b/xdiff/xtypes.h\nindex 8b8467360ecf..6e5f67ebf380 100644\n--- a/xdiff/xtypes.h\n+++ b/xdiff/xtypes.h\n@@ -39,9 +39,9 @@ typedef struct s_chastore {\n } chastore_t;\n \n typedef struct s_xrecord {\n-\tchar const *ptr;\n-\tlong size;\n-\tunsigned long ha;\n+\tu8 const* ptr;\n+\tusize size;\n+\tu64 ha;\n } xrecord_t;\n \n typedef struct s_xdfile {\ndiff --git a/xdiff/xutils.c b/xdiff/xutils.c\nindex 444a108f87c0..10e4f20b7c31 100644\n--- a/xdiff/xutils.c\n+++ b/xdiff/xutils.c\n@@ -418,10 +418,10 @@ int xdl_fall_back_diff(xdfenv_t *diff_env, xpparam_t const *xpp,\n \n \tsubfile1.ptr = (char *)diff_env->xdf1.recs[line1 - 1]->ptr;\n \tsubfile1.size = diff_env->xdf1.recs[line1 + count1 - 2]->ptr +\n-\t\tdiff_env->xdf1.recs[line1 + count1 - 2]->size - subfile1.ptr;\n+\t\tdiff_env->xdf1.recs[line1 + count1 - 2]->size - (u8 const*) subfile1.ptr;\n \tsubfile2.ptr = (char *)diff_env->xdf2.recs[line2 - 1]->ptr;\n \tsubfile2.size = diff_env->xdf2.recs[line2 + count2 - 2]->ptr +\n-\t\tdiff_env->xdf2.recs[line2 + count2 - 2]->size - subfile2.ptr;\n+\t\tdiff_env->xdf2.recs[line2 + count2 - 2]->size - (u8 const*) subfile2.ptr;\n \tif (xdl_do_diff(&subfile1, &subfile2, xpp, &env) < 0)\n \t\treturn -1;\n \n-- \ngitgitgadget\n\n"},{"id":"524210","messageId":"8762349599411de6ce85347c74f2fcaf129124cd.1755220973.git.gitgitgadget@gmail.com","threadId":"63804","inReplyTo":"pull.1980.v2.git.git.1755220973.gitgitgadget@gmail.com","subject":"[PATCH v2 06/17] xdiff: separate parsing lines from hashing them","fromName":"Ezekiel Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2025-08-15T01:22:41Z","receivedAt":"2025-08-15T01:23:01Z","isPatch":true,"sender":{"key":"ezekielnewren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/18313324?v=4"},"body":"From: Ezekiel Newren \n\nWe want to use xxhash for faster hashing. To facilitate that\nand to simplify the code. Separate the concerns of parsing\nand hashing into discrete steps. This makes swapping the hash\nfunction much easier. Since xdl_hash_record() both parses and\nhashses lines, this requires some slight code restructuring.\n\nSigned-off-by: Ezekiel Newren \n---\n xdiff/xprepare.c | 75 ++++++++++++++++++++++++++++--------------------\n 1 file changed, 44 insertions(+), 31 deletions(-)\n\ndiff --git a/xdiff/xprepare.c b/xdiff/xprepare.c\nindex 00cdf7d8a038..031c1752cc1a 100644\n--- a/xdiff/xprepare.c\n+++ b/xdiff/xprepare.c\n@@ -129,13 +129,39 @@ static int xdl_classify_record(unsigned int pass, xdlclassifier_t *cf, xrecord_t\n }\n \n \n+static void xdl_parse_lines(mmfile_t *mf, long narec, xdfile_t *xdf) {\n+\tu8 const* ptr = (u8 const*) mf->ptr;\n+\tusize len = (usize) mf->size;\n+\n+\txdf->recs = NULL;\n+\txdf->nrec = 0;\n+\tXDL_ALLOC_ARRAY(xdf->recs, narec);\n+\n+\twhile (len > 0) {\n+\t\txrecord_t *rec = NULL;\n+\t\tusize length;\n+\t\tu8 const* result = memchr(ptr, '\\n', len);\n+\t\tif (result) {\n+\t\t\tlength = result - ptr + 1;\n+\t\t} else {\n+\t\t\tlength = len;\n+\t\t}\n+\t\tif (XDL_ALLOC_GROW(xdf->recs, xdf->nrec + 1, narec))\n+\t\t\tdie(\"XDL_ALLOC_GROW failed\");\n+\t\trec = xdl_cha_alloc(&xdf->rcha);\n+\t\trec->ptr = ptr;\n+\t\trec->size = length;\n+\t\trec->ha = 0;\n+\t\txdf->recs[xdf->nrec++] = rec;\n+\t\tptr += length;\n+\t\tlen -= length;\n+\t}\n+\n+}\n+\n+\n static int xdl_prepare_ctx(unsigned int pass, mmfile_t *mf, long narec, xpparam_t const *xpp,\n \t\t\t xdlclassifier_t *cf, xdfile_t *xdf) {\n-\tlong nrec, bsize;\n-\tunsigned long hav;\n-\tchar const *blk, *cur, *top, *prev;\n-\txrecord_t *crec;\n-\txrecord_t **recs;\n \tunsigned long *ha;\n \tchar *rchg;\n \tlong *rindex;\n@@ -143,50 +169,37 @@ static int xdl_prepare_ctx(unsigned int pass, mmfile_t *mf, long narec, xpparam_\n \tha = NULL;\n \trindex = NULL;\n \trchg = NULL;\n-\trecs = NULL;\n \n \tif (xdl_cha_init(&xdf->rcha, sizeof(xrecord_t), narec / 4 + 1) < 0)\n \t\tgoto abort;\n-\tif (!XDL_ALLOC_ARRAY(recs, narec))\n-\t\tgoto abort;\n \n-\tnrec = 0;\n-\tif ((cur = blk = xdl_mmfile_first(mf, &bsize))) {\n-\t\tfor (top = blk + bsize; cur < top; ) {\n-\t\t\tprev = cur;\n-\t\t\thav = xdl_hash_record(&cur, top, xpp->flags);\n-\t\t\tif (XDL_ALLOC_GROW(recs, nrec + 1, narec))\n-\t\t\t\tgoto abort;\n-\t\t\tif (!(crec = xdl_cha_alloc(&xdf->rcha)))\n-\t\t\t\tgoto abort;\n-\t\t\tcrec->ptr = (u8 const*) prev;\n-\t\t\tcrec->size = (long) (cur - prev);\n-\t\t\tcrec->ha = hav;\n-\t\t\trecs[nrec++] = crec;\n-\t\t\tif (xdl_classify_record(pass, cf, crec) < 0)\n-\t\t\t\tgoto abort;\n-\t\t}\n+\txdl_parse_lines(mf, narec, xdf);\n+\n+\tfor (usize i = 0; i < (usize) xdf->nrec; i++) {\n+\t\txrecord_t *rec = xdf->recs[i];\n+\t\tchar const* dump = (char const*) rec->ptr;\n+\t\trec->ha = xdl_hash_record(&dump, (char const*) (rec->ptr + rec->size), xpp->flags);\n+\t\txdl_classify_record(pass, cf, rec);\n \t}\n \n-\tif (!XDL_CALLOC_ARRAY(rchg, nrec + 2))\n+\n+\tif (!XDL_CALLOC_ARRAY(rchg, xdf->nrec + 2))\n \t\tgoto abort;\n \n \tif ((XDF_DIFF_ALG(xpp->flags) != XDF_PATIENCE_DIFF) &&\n \t (XDF_DIFF_ALG(xpp->flags) != XDF_HISTOGRAM_DIFF)) {\n-\t\tif (!XDL_ALLOC_ARRAY(rindex, nrec + 1))\n+\t\tif (!XDL_ALLOC_ARRAY(rindex, xdf->nrec + 1))\n \t\t\tgoto abort;\n-\t\tif (!XDL_ALLOC_ARRAY(ha, nrec + 1))\n+\t\tif (!XDL_ALLOC_ARRAY(ha, xdf->nrec + 1))\n \t\t\tgoto abort;\n \t}\n \n-\txdf->nrec = nrec;\n-\txdf->recs = recs;\n \txdf->rchg = rchg + 1;\n \txdf->rindex = rindex;\n \txdf->nreff = 0;\n \txdf->ha = ha;\n \txdf->dstart = 0;\n-\txdf->dend = nrec - 1;\n+\txdf->dend = xdf->nrec - 1;\n \n \treturn 0;\n \n@@ -194,7 +207,7 @@ abort:\n \txdl_free(ha);\n \txdl_free(rindex);\n \txdl_free(rchg);\n-\txdl_free(recs);\n+\txdl_free(xdf->recs);\n \txdl_cha_free(&xdf->rcha);\n \treturn -1;\n }\n-- \ngitgitgadget\n\n"},{"id":"524211","messageId":"d74fd4ef67a35fe453cbddab4641dfbef161c9c0.1755220973.git.gitgitgadget@gmail.com","threadId":"63804","inReplyTo":"pull.1980.v2.git.git.1755220973.gitgitgadget@gmail.com","subject":"[PATCH v2 07/17] xdiff: conditionally use Rust's implementation of xxhash","fromName":"Ezekiel Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2025-08-15T01:22:42Z","receivedAt":"2025-08-15T01:23:02Z","isPatch":true,"sender":{"key":"ezekielnewren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/18313324?v=4"},"body":"From: Ezekiel Newren \n\nWhen no whitespace flags are present use xxhash, for faster\nhashing, otherwise use DJB2a (which is what xdiff has been\nusing all along).\n\nThe benchmark below compares my series with version v2.49.0\n(built in build_release/ and build_v2.49.0/ respectively),\nrunning log commands on linux kernel with 3 different machines.\n\n$ BASE=/path/to/git/root\n\n // laptop\n // CPU: 6-core Intel Core i7-8750H (-MT MCP-) speed/min/max: 726/800/4100 MHz\n $ hyperfine --warmup 3 -L exe $BASE/build_release/git,$BASE/build_v2.49.0/git '{exe} log --oneline --shortstat v6.8..v6.9 >/dev/null'\n Benchmark 1: /home/ezekiel/development/work/git/build_release/git log --oneline --shortstat v6.8..v6.9 >/dev/null\n Time (mean ± σ): 10.419 s ± 0.166 s [User: 10.097 s, System: 0.284 s]\n Range (min … max): 10.215 s … 10.680 s 10 runs\n\n Benchmark 2: /home/ezekiel/development/work/git/build_v2.49.0/git log --oneline --shortstat v6.8..v6.9 >/dev/null\n Time (mean ± σ): 10.980 s ± 0.137 s [User: 10.633 s, System: 0.308 s]\n Range (min … max): 10.791 s … 11.178 s 10 runs\n\n Summary\n /home/ezekiel/development/work/git/build_release/git log --oneline --shortstat v6.8..v6.9 >/dev/null ran\n 1.05 ± 0.02 times faster than /home/ezekiel/development/work/git/build_v2.49.0/git log --oneline --shortstat v6.8..v6.9 >/dev/null\n\n // desktop\n // CPU: 8-core Intel Core i7-9700 (-MCP-) speed/min/max: 800/800/4700 MHz\n $ hyperfine --warmup 3 -L exe $BASE/build_release/git,$BASE/build_v2.49.0/git '{exe} log --oneline --shortstat v6.8..v6.9 >/dev/null'\n Benchmark 1: /home/steamuser/dev/git/build_release/git log --oneline --shortstat v6.8..v6.9 >/dev/null\n Time (mean ± σ): 6.823 s ± 0.020 s [User: 6.624 s, System: 0.180 s]\n Range (min … max): 6.801 s … 6.858 s 10 runs\n\n Benchmark 2: /home/steamuser/dev/git/build_v2.49.0/git log --oneline --shortstat v6.8..v6.9 >/dev/null\n Time (mean ± σ): 8.151 s ± 0.024 s [User: 7.928 s, System: 0.198 s]\n Range (min … max): 8.105 s … 8.184 s 10 runs\n\n Summary\n /home/steamuser/dev/git/build_release/git log --oneline --shortstat v6.8..v6.9 >/dev/null ran\n 1.19 ± 0.01 times faster than /home/steamuser/dev/git/build_v2.49.0/git log --oneline --shortstat v6.8..v6.9 >/dev/null\n\n // router\n // CPU: dual core Intel Celeron 3965U (-MCP-) speed/min/max: 1300/400/2200 MHz\n $ hyperfine --warmup 3 -L exe $BASE/build_release/git,$BASE/build_v2.49.0/git '{exe} log --oneline --shortstat v6.8..v6.9 >/dev/null'\n Benchmark 1: /home/metal/dev/git/build_release/git log --oneline --shortstat v6.8..v6.9 >/dev/null\n Time (mean ± σ): 21.209 s ± 0.054 s [User: 20.341 s, System: 0.605 s]\n Range (min … max): 21.135 s … 21.309 s 10 runs\n\n Benchmark 2: /home/metal/dev/git/build_v2.49.0/git log --oneline --shortstat v6.8..v6.9 >/dev/null\n Time (mean ± σ): 23.683 s ± 0.060 s [User: 22.735 s, System: 0.672 s]\n Range (min … max): 23.566 s … 23.751 s 10 runs\n\n Summary\n /home/metal/dev/git/build_release/git log --oneline --shortstat v6.8..v6.9 >/dev/null ran\n 1.12 ± 0.00 times faster than /home/metal/dev/git/build_v2.49.0/git log --oneline --shortstat v6.8..v6.9 >/dev/null\n\nSigned-off-by: Ezekiel Newren \n---\n rust/xdiff/Cargo.toml | 1 +\n rust/xdiff/src/lib.rs | 7 +++++++\n xdiff/xprepare.c | 19 +++++++++++++++++--\n 3 files changed, 25 insertions(+), 2 deletions(-)\n\ndiff --git a/rust/xdiff/Cargo.toml b/rust/xdiff/Cargo.toml\nindex eb7966aada64..1516e829db18 100644\n--- a/rust/xdiff/Cargo.toml\n+++ b/rust/xdiff/Cargo.toml\n@@ -13,3 +13,4 @@ crate-type = [\"staticlib\", \"rlib\"]\n \n [dependencies]\n interop = { path = \"../interop\" }\n+xxhash-rust = { version = \"0.8.15\", features = [\"xxh3\"] }\ndiff --git a/rust/xdiff/src/lib.rs b/rust/xdiff/src/lib.rs\nindex e69de29bb2d1..96975975a1ba 100644\n--- a/rust/xdiff/src/lib.rs\n+++ b/rust/xdiff/src/lib.rs\n@@ -0,0 +1,7 @@\n+\n+\n+#[no_mangle]\n+unsafe extern \"C\" fn xxh3_64(ptr: *const u8, size: usize) -> u64 {\n+ let slice = std::slice::from_raw_parts(ptr, size);\n+ xxhash_rust::xxh3::xxh3_64(slice)\n+}\ndiff --git a/xdiff/xprepare.c b/xdiff/xprepare.c\nindex 031c1752cc1a..c0463bacd94b 100644\n--- a/xdiff/xprepare.c\n+++ b/xdiff/xprepare.c\n@@ -160,6 +160,9 @@ static void xdl_parse_lines(mmfile_t *mf, long narec, xdfile_t *xdf) {\n }\n \n \n+extern u64 xxh3_64(u8 const* ptr, usize size);\n+\n+\n static int xdl_prepare_ctx(unsigned int pass, mmfile_t *mf, long narec, xpparam_t const *xpp,\n \t\t\t xdlclassifier_t *cf, xdfile_t *xdf) {\n \tunsigned long *ha;\n@@ -175,14 +178,26 @@ static int xdl_prepare_ctx(unsigned int pass, mmfile_t *mf, long narec, xpparam_\n \n \txdl_parse_lines(mf, narec, xdf);\n \n+\tif ((xpp->flags & XDF_WHITESPACE_FLAGS) == 0) {\n+\t\tfor (usize i = 0; i < (usize) xdf->nrec; i++) {\n+\t\t\txrecord_t *rec = xdf->recs[i];\n+\t\t\trec->ha = xxh3_64(rec->ptr, rec->size);\n+\t\t}\n+\t} else {\n+\t\tfor (usize i = 0; i < (usize) xdf->nrec; i++) {\n+\t\t\txrecord_t *rec = xdf->recs[i];\n+\t\t\tchar const* dump = (char const*) rec->ptr;\n+\t\t\trec->ha = xdl_hash_record(&dump, (char const*) (rec->ptr + rec->size), xpp->flags);\n+\t\t}\n+\t}\n+\n \tfor (usize i = 0; i < (usize) xdf->nrec; i++) {\n \t\txrecord_t *rec = xdf->recs[i];\n-\t\tchar const* dump = (char const*) rec->ptr;\n-\t\trec->ha = xdl_hash_record(&dump, (char const*) (rec->ptr + rec->size), xpp->flags);\n \t\txdl_classify_record(pass, cf, rec);\n \t}\n \n \n+\n \tif (!XDL_CALLOC_ARRAY(rchg, xdf->nrec + 2))\n \t\tgoto abort;\n \n-- \ngitgitgadget\n\n"},{"id":"524212","messageId":"7dc241e668293ec1048d93c5b9649f7eb1ce1104.1755220973.git.gitgitgadget@gmail.com","threadId":"63804","inReplyTo":"pull.1980.v2.git.git.1755220973.gitgitgadget@gmail.com","subject":"[PATCH v2 08/17] github workflows: install rust","fromName":"Ezekiel Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2025-08-15T01:22:43Z","receivedAt":"2025-08-15T01:23:04Z","isPatch":true,"sender":{"key":"ezekielnewren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/18313324?v=4"},"body":"From: Ezekiel Newren \n\nSince we have introduced rust, it needs to be installed for the\ncontinuous integration build targets. Create an install script\n(build_rust.sh) that needs to be run as the same user that builds git.\nBecause of the limitations of meson, create build_rust.sh which makes\nit easy to centralize how rust is built between meson and make.\n\nThere are 2 interesting decisions worth calling out in this commit:\n\n* The 'output' field of custom_target() does not allow specifying a\n file nested inside the build directory. Thus create build_rust.sh to\n build rust with all of its parameters and then moves libxdiff.a to\n the root of the build directory.\n\n* Install curl, to facilitate the rustup install script.\n\nSigned-off-by: Ezekiel Newren \n---\n .github/workflows/main.yml | 1 +\n Makefile | 46 +++++++++++++++++++----------\n build_rust.sh | 59 ++++++++++++++++++++++++++++++++++++++\n ci/install-dependencies.sh | 14 ++++-----\n ci/install-rust.sh | 33 +++++++++++++++++++++\n ci/lib.sh | 8 ++++++\n ci/make-test-artifacts.sh | 7 +++++\n ci/run-build-and-tests.sh | 10 +++++++\n meson.build | 40 +++++++++++---------------\n 9 files changed, 172 insertions(+), 46 deletions(-)\n create mode 100755 build_rust.sh\n create mode 100644 ci/install-rust.sh\n\ndiff --git a/.github/workflows/main.yml b/.github/workflows/main.yml\nindex 7dbf9f7f123c..8aac18a6ba45 100644\n--- a/.github/workflows/main.yml\n+++ b/.github/workflows/main.yml\n@@ -4,6 +4,7 @@ on: [push, pull_request]\n \n env:\n DEVELOPER: 1\n+ RUST_VERSION: 1.87.0\n \n # If more than one workflow run is triggered for the very same commit hash\n # (which happens when multiple branches pointing to the same commit), only\ndiff --git a/Makefile b/Makefile\nindex db39e6e1c28e..e659b6eefe82 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -919,11 +919,29 @@ TEST_SHELL_PATH = $(SHELL_PATH)\n \n LIB_FILE = libgit.a\n XDIFF_LIB = xdiff/lib.a\n+\n+EXTLIBS =\n+\n ifeq ($(DEBUG), 1)\n-RUST_LIB = rust/target/debug/libxdiff.a\n+ RUST_BUILD_MODE = debug\n else\n-RUST_LIB = rust/target/release/libxdiff.a\n+ RUST_BUILD_MODE = release\n+endif\n+\n+RUST_TARGET_DIR = rust/target/$(RUST_BUILD_MODE)\n+RUST_FLAGS_FOR_C = -L$(RUST_TARGET_DIR)\n+\n+.PHONY: compile_rust\n+compile_rust:\n+\t./build_rust.sh . $(RUST_BUILD_MODE) xdiff\n+\n+EXTLIBS += ./$(RUST_TARGET_DIR)/libxdiff.a\n+\n+UNAME_S := $(shell uname -s)\n+ifeq ($(UNAME_S),Linux)\n+ EXTLIBS += -ldl\n endif\n+\n REFTABLE_LIB = reftable/libreftable.a\n \n GENERATED_H += command-list.h\n@@ -1395,9 +1413,7 @@ UNIT_TEST_OBJS += $(UNIT_TEST_DIR)/lib-reftable.o\n \n # xdiff and reftable libs may in turn depend on what is in libgit.a\n GITLIBS = common-main.o $(LIB_FILE) $(XDIFF_LIB) $(REFTABLE_LIB) $(LIB_FILE)\n-EXTLIBS =\n \n-GITLIBS += $(RUST_LIB)\n \n GIT_USER_AGENT = git/$(GIT_VERSION)\n \n@@ -2548,7 +2564,7 @@ git.sp git.s git.o: EXTRA_CPPFLAGS = \\\n \t'-DGIT_MAN_PATH=\"$(mandir_relative_SQ)\"' \\\n \t'-DGIT_INFO_PATH=\"$(infodir_relative_SQ)\"'\n \n-git$X: git.o GIT-LDFLAGS $(BUILTIN_OBJS) $(GITLIBS)\n+git$X: git.o GIT-LDFLAGS $(BUILTIN_OBJS) $(GITLIBS) compile_rust\n \t$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) \\\n \t\t$(filter %.o,$^) $(LIBS)\n \n@@ -2898,17 +2914,17 @@ headless-git.o: compat/win32/headless.c GIT-CFLAGS\n headless-git$X: headless-git.o git.res GIT-LDFLAGS\n \t$(QUIET_LINK)$(CC) $(ALL_CFLAGS) $(ALL_LDFLAGS) -mwindows -o $@ $< git.res\n \n-git-%$X: %.o GIT-LDFLAGS $(GITLIBS)\n+git-%$X: %.o GIT-LDFLAGS $(GITLIBS) compile_rust\n \t$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) $(LIBS)\n \n-git-imap-send$X: imap-send.o $(IMAP_SEND_BUILDDEPS) GIT-LDFLAGS $(GITLIBS)\n+git-imap-send$X: imap-send.o $(IMAP_SEND_BUILDDEPS) GIT-LDFLAGS $(GITLIBS) compile_rust\n \t$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \\\n \t\t$(IMAP_SEND_LDFLAGS) $(LIBS)\n \n-git-http-fetch$X: http.o http-walker.o http-fetch.o GIT-LDFLAGS $(GITLIBS)\n+git-http-fetch$X: http.o http-walker.o http-fetch.o GIT-LDFLAGS $(GITLIBS) compile_rust\n \t$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \\\n \t\t$(CURL_LIBCURL) $(LIBS)\n-git-http-push$X: http.o http-push.o GIT-LDFLAGS $(GITLIBS)\n+git-http-push$X: http.o http-push.o GIT-LDFLAGS $(GITLIBS) compile_rust\n \t$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \\\n \t\t$(CURL_LIBCURL) $(EXPAT_LIBEXPAT) $(LIBS)\n \n@@ -2918,11 +2934,11 @@ $(REMOTE_CURL_ALIASES): $(REMOTE_CURL_PRIMARY)\n \tln -s $< $@ 2>/dev/null || \\\n \tcp $< $@\n \n-$(REMOTE_CURL_PRIMARY): remote-curl.o http.o http-walker.o GIT-LDFLAGS $(GITLIBS)\n+$(REMOTE_CURL_PRIMARY): remote-curl.o http.o http-walker.o GIT-LDFLAGS $(GITLIBS) compile_rust\n \t$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \\\n \t\t$(CURL_LIBCURL) $(EXPAT_LIBEXPAT) $(LIBS)\n \n-scalar$X: scalar.o GIT-LDFLAGS $(GITLIBS)\n+scalar$X: scalar.o GIT-LDFLAGS $(GITLIBS) compile_rust\n \t$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) \\\n \t\t$(filter %.o,$^) $(LIBS)\n \n@@ -3309,7 +3325,7 @@ perf: all\n \n t/helper/test-tool$X: $(patsubst %,t/helper/%,$(TEST_BUILTINS_OBJS)) $(UNIT_TEST_DIR)/test-lib.o\n \n-t/helper/test-%$X: t/helper/test-%.o GIT-LDFLAGS $(GITLIBS)\n+t/helper/test-%$X: t/helper/test-%.o GIT-LDFLAGS $(GITLIBS) compile_rust\n \t$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) $(filter %.a,$^) $(LIBS)\n \n check-sha1:: t/helper/test-tool$X\n@@ -3929,13 +3945,13 @@ FUZZ_CXXFLAGS ?= $(ALL_CFLAGS)\n .PHONY: fuzz-all\n fuzz-all: $(FUZZ_PROGRAMS)\n \n-$(FUZZ_PROGRAMS): %: %.o oss-fuzz/dummy-cmd-main.o $(GITLIBS) GIT-LDFLAGS\n+$(FUZZ_PROGRAMS): %: %.o oss-fuzz/dummy-cmd-main.o $(GITLIBS) GIT-LDFLAGS compile_rust\n \t$(QUIET_LINK)$(FUZZ_CXX) $(FUZZ_CXXFLAGS) -o $@ $(ALL_LDFLAGS) \\\n \t\t-Wl,--allow-multiple-definition \\\n \t\t$(filter %.o,$^) $(filter %.a,$^) $(LIBS) $(LIB_FUZZING_ENGINE)\n \n $(UNIT_TEST_PROGS): $(UNIT_TEST_BIN)/%$X: $(UNIT_TEST_DIR)/%.o $(UNIT_TEST_OBJS) \\\n-\t$(GITLIBS) GIT-LDFLAGS\n+\t$(GITLIBS) GIT-LDFLAGS compile_rust\n \t$(call mkdir_p_parent_template)\n \t$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) \\\n \t\t$(filter %.o,$^) $(filter %.a,$^) $(LIBS)\n@@ -3954,7 +3970,7 @@ $(UNIT_TEST_DIR)/clar.suite: $(UNIT_TEST_DIR)/clar-decls.h $(UNIT_TEST_DIR)/gene\n $(UNIT_TEST_DIR)/clar/clar.o: $(UNIT_TEST_DIR)/clar.suite\n $(CLAR_TEST_OBJS): $(UNIT_TEST_DIR)/clar-decls.h\n $(CLAR_TEST_OBJS): EXTRA_CPPFLAGS = -I$(UNIT_TEST_DIR)\n-$(CLAR_TEST_PROG): $(UNIT_TEST_DIR)/clar.suite $(CLAR_TEST_OBJS) $(GITLIBS) GIT-LDFLAGS\n+$(CLAR_TEST_PROG): $(UNIT_TEST_DIR)/clar.suite $(CLAR_TEST_OBJS) $(GITLIBS) GIT-LDFLAGS compile_rust\n \t$(call mkdir_p_parent_template)\n \t$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) $(LIBS)\n \ndiff --git a/build_rust.sh b/build_rust.sh\nnew file mode 100755\nindex 000000000000..4c12135cd205\n--- /dev/null\n+++ b/build_rust.sh\n@@ -0,0 +1,59 @@\n+#!/bin/sh\n+\n+if [ -z \"$CARGO_HOME\" ]; then\n+ export CARGO_HOME=$HOME/.cargo\n+ echo >&2 \"::warning:: CARGO_HOME is not set\"\n+fi\n+echo \"CARGO_HOME=$CARGO_HOME\"\n+\n+rustc -vV\n+cargo --version\n+\n+dir_git_root=${0%/*}\n+dir_build=$1\n+rust_target=$2\n+crate=$3\n+\n+dir_rust=$dir_git_root/rust\n+\n+if [ \"$dir_git_root\" = \"\" ]; then\n+ echo \"did not specify the directory for the root of git\"\n+ exit 1\n+fi\n+\n+if [ \"$dir_build\" = \"\" ]; then\n+ echo \"did not specify the build directory\"\n+ exit 1\n+fi\n+\n+if [ \"$rust_target\" = \"\" ]; then\n+ echo \"did not specify the rust_target\"\n+ exit 1\n+fi\n+\n+if [ \"$rust_target\" = \"release\" ]; then\n+ rust_args=\"--release\"\n+ export RUSTFLAGS='-Aunused_imports -Adead_code'\n+elif [ \"$rust_target\" = \"debug\" ]; then\n+ rust_args=\"\"\n+ export RUSTFLAGS='-Aunused_imports -Adead_code -C debuginfo=2 -C opt-level=1 -C force-frame-pointers=yes'\n+else\n+ echo \"illegal rust_target value $rust_target\"\n+ exit 1\n+fi\n+\n+cd $dir_rust && cargo clean && pwd && cargo build -p $crate $rust_args; cd ..\n+\n+libfile=\"lib${crate}.a\"\n+dst=$dir_build/$libfile\n+\n+if [ \"$dir_git_root\" != \"$dir_build\" ]; then\n+ src=$dir_rust/target/$rust_target/$libfile\n+ if [ ! -f $src ]; then\n+ echo >&2 \"::error:: cannot find path of static library\"\n+ exit 5\n+ fi\n+\n+ rm $dst 2>/dev/null\n+ mv $src $dst\n+fi\ndiff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh\nindex d061a4729339..7801075821ba 100755\n--- a/ci/install-dependencies.sh\n+++ b/ci/install-dependencies.sh\n@@ -24,14 +24,14 @@ fi\n \n case \"$distro\" in\n alpine-*)\n-\tapk add --update shadow sudo meson ninja-build gcc libc-dev curl-dev openssl-dev expat-dev gettext \\\n+\tapk add --update shadow sudo meson ninja-build gcc libc-dev curl curl-dev openssl-dev expat-dev gettext \\\n \t\tzlib-ng-dev pcre2-dev python3 musl-libintl perl-utils ncurses \\\n \t\tapache2 apache2-http2 apache2-proxy apache2-ssl apache2-webdav apr-util-dbd_sqlite3 \\\n \t\tbash cvs gnupg perl-cgi perl-dbd-sqlite perl-io-tty >/dev/null\n \t;;\n fedora-*|almalinux-*)\n \tdnf -yq update >/dev/null &&\n-\tdnf -yq install shadow-utils sudo make gcc findutils diffutils perl python3 gawk gettext zlib-devel expat-devel openssl-devel curl-devel pcre2-devel >/dev/null\n+\tdnf -yq install shadow-utils sudo make gcc findutils diffutils perl python3 gawk gettext zlib-devel expat-devel openssl-devel curl curl-devel pcre2-devel >/dev/null\n \t;;\n ubuntu-*|i386/ubuntu-*|debian-*)\n \t# Required so that apt doesn't wait for user input on certain packages.\n@@ -55,8 +55,8 @@ ubuntu-*|i386/ubuntu-*|debian-*)\n \tsudo apt-get -q update\n \tsudo apt-get -q -y install \\\n \t\t$LANGUAGES apache2 cvs cvsps git gnupg $SVN \\\n-\t\tmake libssl-dev libcurl4-openssl-dev libexpat-dev wget sudo default-jre \\\n-\t\ttcl tk gettext zlib1g-dev perl-modules liberror-perl libauthen-sasl-perl \\\n+\t\tmake libssl-dev curl libcurl4-openssl-dev libexpat-dev wget sudo default-jre \\\n+\t\ttcl tk gettext zlib1g zlib1g-dev perl-modules liberror-perl libauthen-sasl-perl \\\n \t\tlibemail-valid-perl libio-pty-perl libio-socket-ssl-perl libnet-smtp-ssl-perl libdbd-sqlite3-perl libcgi-pm-perl \\\n \t\tlibsecret-1-dev libpcre2-dev meson ninja-build pkg-config \\\n \t\t${CC_PACKAGE:-${CC:-gcc}} $PYTHON_PACKAGE\n@@ -121,13 +121,13 @@ ClangFormat)\n \t;;\n StaticAnalysis)\n \tsudo apt-get -q update\n-\tsudo apt-get -q -y install coccinelle libcurl4-openssl-dev libssl-dev \\\n+\tsudo apt-get -q -y install coccinelle curl libcurl4-openssl-dev libssl-dev \\\n \t\tlibexpat-dev gettext make\n \t;;\n sparse)\n \tsudo apt-get -q update -q\n-\tsudo apt-get -q -y install libssl-dev libcurl4-openssl-dev \\\n-\t\tlibexpat-dev gettext zlib1g-dev sparse\n+\tsudo apt-get -q -y install libssl-dev curl libcurl4-openssl-dev \\\n+\t\tlibexpat-dev gettext zlib1g zlib1g-dev sparse\n \t;;\n Documentation)\n \tsudo apt-get -q update\ndiff --git a/ci/install-rust.sh b/ci/install-rust.sh\nnew file mode 100644\nindex 000000000000..141ceddb17cf\n--- /dev/null\n+++ b/ci/install-rust.sh\n@@ -0,0 +1,33 @@\n+#!/bin/sh\n+\n+if [ \"$(id -u)\" -eq 0 ]; then\n+ echo >&2 \"::warning:: installing rust as root\"\n+fi\n+\n+if [ \"$CARGO_HOME\" = \"\" ]; then\n+ echo >&2 \"::warning:: CARGO_HOME is not set\"\n+ export CARGO_HOME=$HOME/.cargo\n+fi\n+\n+export RUSTUP_HOME=$CARGO_HOME\n+\n+if [ \"$RUST_VERSION\" = \"\" ]; then\n+ echo >&2 \"::error:: RUST_VERSION is not set\"\n+ exit 2\n+fi\n+\n+## install rustup\n+curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- --default-toolchain none -y\n+if [ ! -f $CARGO_HOME/env ]; then\n+ echo \"PATH=$CARGO_HOME/bin:\\$PATH\" > $CARGO_HOME/env\n+fi\n+## install a specific version of rust\n+if [ \"$BITNESS\" = \"32\" ]; then\n+ $CARGO_HOME/bin/rustup set default-host i686-unknown-linux-gnu || exit $?\n+ $CARGO_HOME/bin/rustup install $RUST_VERSION || exit $?\n+ $CARGO_HOME/bin/rustup default --force-non-host $RUST_VERSION || exit $?\n+else\n+ $CARGO_HOME/bin/rustup default $RUST_VERSION || exit $?\n+fi\n+\n+. $CARGO_HOME/env\ndiff --git a/ci/lib.sh b/ci/lib.sh\nindex f561884d4016..ad0e49a68dcb 100755\n--- a/ci/lib.sh\n+++ b/ci/lib.sh\n@@ -1,5 +1,13 @@\n # Library of functions shared by all CI scripts\n \n+\n+export BITNESS=\"64\"\n+if command -v getconf >/dev/null && [ \"$(getconf LONG_BIT 2>/dev/null)\" = \"32\" ]; then\n+ export BITNESS=\"32\"\n+fi\n+echo \"BITNESS=$BITNESS\"\n+\n+\n if test true = \"$GITHUB_ACTIONS\"\n then\n \tbegin_group () {\ndiff --git a/ci/make-test-artifacts.sh b/ci/make-test-artifacts.sh\nindex 74141af0cc74..56aa7efb1d53 100755\n--- a/ci/make-test-artifacts.sh\n+++ b/ci/make-test-artifacts.sh\n@@ -7,6 +7,13 @@ mkdir -p \"$1\" # in case ci/lib.sh decides to quit early\n \n . ${0%/*}/lib.sh\n \n+## install rust per user rather than system wide\n+. ${0%/*}/install-rust.sh\n+\n group Build make artifacts-tar ARTIFACTS_DIRECTORY=\"$1\"\n \n+if [ -d \"$CARGO_HOME\" ]; then\n+ rm -rf $CARGO_HOME\n+fi\n+\n check_unignored_build_artifacts\ndiff --git a/ci/run-build-and-tests.sh b/ci/run-build-and-tests.sh\nindex 01823fd0f140..dbab1cb2f936 100755\n--- a/ci/run-build-and-tests.sh\n+++ b/ci/run-build-and-tests.sh\n@@ -5,6 +5,12 @@\n \n . ${0%/*}/lib.sh\n \n+## install rust per user rather than system wide\n+. ${0%/*}/install-rust.sh\n+\n+rustc -vV\n+cargo --version || exit $?\n+\n run_tests=t\n \n case \"$jobname\" in\n@@ -72,5 +78,9 @@ case \"$jobname\" in\n \t;;\n esac\n \n+if [ -d \"$CARGO_HOME\" ]; then\n+ rm -rf $CARGO_HOME\n+fi\n+\n check_unignored_build_artifacts\n save_good_tree\ndiff --git a/meson.build b/meson.build\nindex 2d8da17f6515..047d7e5b6630 100644\n--- a/meson.build\n+++ b/meson.build\n@@ -277,26 +277,17 @@ else\n rustflags = '-Aunused_imports -Adead_code -C debuginfo=2 -C opt-level=1 -C force-frame-pointers=yes'\n endif\n \n-\n-rust_leaf = custom_target('rust_leaf',\n+rust_build_xdiff = custom_target('rust_build_xdiff',\n output: 'libxdiff.a',\n build_by_default: true,\n build_always_stale: true,\n- command: ['cargo', 'build',\n- '--manifest-path', meson.project_source_root() / 'rust/Cargo.toml'\n- ] + rust_args,\n- env: {\n- 'RUSTFLAGS': rustflags,\n- },\n+ command: [\n+ meson.project_source_root() / 'build_rust.sh',\n+ meson.current_build_dir(), rust_target, 'xdiff',\n+ ],\n install: false,\n )\n \n-rust_xdiff_dep = declare_dependency(\n- link_args: ['-L' + meson.project_source_root() / 'rust/target' / rust_target, '-lxdiff'],\n-# include_directories: include_directories('xdiff/include'), # Adjust if you expose headers\n-)\n-\n-\n compiler = meson.get_compiler('c')\n \n libgit_sources = [\n@@ -1707,17 +1698,18 @@ version_def_h = custom_target(\n )\n libgit_sources += version_def_h\n \n-libgit_dependencies += rust_xdiff_dep\n-\n libgit = declare_dependency(\n- link_with: static_library('git',\n- sources: libgit_sources,\n- c_args: libgit_c_args + [\n- '-DGIT_VERSION_H=\"' + version_def_h.full_path() + '\"',\n- ],\n- dependencies: libgit_dependencies,\n- include_directories: libgit_include_directories,\n- ),\n+ link_with: [\n+ static_library('git',\n+ sources: libgit_sources,\n+ c_args: libgit_c_args + [\n+ '-DGIT_VERSION_H=\"' + version_def_h.full_path() + '\"',\n+ ],\n+ dependencies: libgit_dependencies,\n+ include_directories: libgit_include_directories,\n+ ),\n+ rust_build_xdiff,\n+ ],\n compile_args: libgit_c_args,\n dependencies: libgit_dependencies,\n include_directories: libgit_include_directories,\n-- \ngitgitgadget\n\n"},{"id":"524213","messageId":"96041a10d545e0e431d05b93544771c6bdfc06f1.1755220973.git.gitgitgadget@gmail.com","threadId":"63804","inReplyTo":"pull.1980.v2.git.git.1755220973.gitgitgadget@gmail.com","subject":"[PATCH v2 09/17] Do support Windows again after requiring Rust","fromName":"Johannes Schindelin via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2025-08-15T01:22:44Z","receivedAt":"2025-08-15T01:23:05Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"From: Johannes Schindelin \n\nBy default, Rust wants to build MS Visual C-compatible libraries on\nWindows, because that is _the_ native C compiler.\n\nGit is historically lacking in its MSVC support, and the official Git\nfor Windows versions are built using GCC instead. As a consequence, a\n(subset of a) GCC toolchain is installed as part of the `windows-build`\njob of every CI build.\n\nNaturally, this requires adjustments in how Rust is called, most\nimportantly it requires installing support for a GCC-compatible build\ntarget.\n\nLet's make the necessary adjustment both in the CI-specific code that\ninstalls Rust as well as in the Windows-specific configuration in\n`config.mak.uname`.\n\nSigned-off-by: Johannes Schindelin \n[en: Moved lib userenv handling to a later patch]\nSigned-off-by: Ezekiel Newren \n---\n ci/install-rust.sh | 3 +++\n config.mak.uname | 7 +++++++\n 2 files changed, 10 insertions(+)\n\ndiff --git a/ci/install-rust.sh b/ci/install-rust.sh\nindex 141ceddb17cf..c22baa629ceb 100644\n--- a/ci/install-rust.sh\n+++ b/ci/install-rust.sh\n@@ -28,6 +28,9 @@ if [ \"$BITNESS\" = \"32\" ]; then\n $CARGO_HOME/bin/rustup default --force-non-host $RUST_VERSION || exit $?\n else\n $CARGO_HOME/bin/rustup default $RUST_VERSION || exit $?\n+ if [ \"$CI_OS_NAME\" = \"windows\" ]; then\n+ $CARGO_HOME/bin/rustup target add x86_64-pc-windows-gnu || exit $?\n+ fi\n fi\n \n . $CARGO_HOME/env\ndiff --git a/config.mak.uname b/config.mak.uname\nindex 3e26bb074a4b..a22703284b56 100644\n--- a/config.mak.uname\n+++ b/config.mak.uname\n@@ -727,19 +727,26 @@ ifeq ($(uname_S),MINGW)\n \t\tprefix = /mingw32\n \t\tHOST_CPU = i686\n \t\tBASIC_LDFLAGS += -Wl,--pic-executable,-e,_mainCRTStartup\n+\t\tCARGO_BUILD_TARGET = i686-pc-windows-gnu\n endif\n ifeq (MINGW64,$(MSYSTEM))\n \t\tprefix = /mingw64\n \t\tHOST_CPU = x86_64\n \t\tBASIC_LDFLAGS += -Wl,--pic-executable,-e,mainCRTStartup\n+\t\tCARGO_BUILD_TARGET = x86_64-pc-windows-gnu\n else ifeq (CLANGARM64,$(MSYSTEM))\n \t\tprefix = /clangarm64\n \t\tHOST_CPU = aarch64\n \t\tBASIC_LDFLAGS += -Wl,--pic-executable,-e,mainCRTStartup\n+\t\tCARGO_BUILD_TARGET = aarch64-pc-windows-gnu\n else\n \t\tCOMPAT_CFLAGS += -D_USE_32BIT_TIME_T\n \t\tBASIC_LDFLAGS += -Wl,--large-address-aware\n endif\n+\n+\texport CARGO_BUILD_TARGET\n+\tRUST_TARGET_DIR = rust/target/$(CARGO_BUILD_TARGET)/$(RUST_BUILD_MODE)\n+\n \tCC = gcc\n \tCOMPAT_CFLAGS += -D__USE_MINGW_ANSI_STDIO=0 -DDETECT_MSYS_TTY \\\n \t\t-fstack-protector-strong\n-- \ngitgitgadget\n\n"},{"id":"524214","messageId":"1194de3f39c76da76b7cbf045162c9c9c8cd5be4.1755220973.git.gitgitgadget@gmail.com","threadId":"63804","inReplyTo":"pull.1980.v2.git.git.1755220973.gitgitgadget@gmail.com","subject":"[PATCH v2 10/17] win+Meson: allow for xdiff to be compiled with MSVC","fromName":"Johannes Schindelin via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2025-08-15T01:22:45Z","receivedAt":"2025-08-15T01:23:05Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"From: Johannes Schindelin \n\nThe `build_rust.sh` script is quite opinionated about the naming scheme\nof the C compiler: It assumes that the xdiff library file will be named\n`libxdiff.a`.\n\nHowever, MS Visual C generates `xdiff.lib` files instead; This naming\nscheme has been in use in a very, very long time.\n\nLet's allow for that.\n\nSigned-off-by: Johannes Schindelin \n---\n build_rust.sh | 7 ++++++-\n meson.build | 12 +++++++++---\n 2 files changed, 15 insertions(+), 4 deletions(-)\n\ndiff --git a/build_rust.sh b/build_rust.sh\nindex 4c12135cd205..694d48d857a5 100755\n--- a/build_rust.sh\n+++ b/build_rust.sh\n@@ -44,7 +44,12 @@ fi\n \n cd $dir_rust && cargo clean && pwd && cargo build -p $crate $rust_args; cd ..\n \n-libfile=\"lib${crate}.a\"\n+if grep x86_64-pc-windows-msvc rust/target/.rustc_info.json\n+then\n+ libfile=\"${crate}.lib\"\n+else\n+ libfile=\"lib${crate}.a\"\n+fi\n dst=$dir_build/$libfile\n \n if [ \"$dir_git_root\" != \"$dir_build\" ]; then\ndiff --git a/meson.build b/meson.build\nindex 047d7e5b6630..5e89a5dd0e00 100644\n--- a/meson.build\n+++ b/meson.build\n@@ -277,8 +277,16 @@ else\n rustflags = '-Aunused_imports -Adead_code -C debuginfo=2 -C opt-level=1 -C force-frame-pointers=yes'\n endif\n \n+compiler = meson.get_compiler('c')\n+\n+if compiler.get_id() == 'msvc'\n+ xdiff_lib_filename = 'xdiff.lib'\n+else\n+ xdiff_lib_filename = 'libxdiff.a'\n+endif\n+\n rust_build_xdiff = custom_target('rust_build_xdiff',\n- output: 'libxdiff.a',\n+ output: xdiff_lib_filename,\n build_by_default: true,\n build_always_stale: true,\n command: [\n@@ -288,8 +296,6 @@ rust_build_xdiff = custom_target('rust_build_xdiff',\n install: false,\n )\n \n-compiler = meson.get_compiler('c')\n-\n libgit_sources = [\n 'abspath.c',\n 'add-interactive.c',\n-- \ngitgitgadget\n\n"},{"id":"524215","messageId":"382067a09e34f63ca33d7ba40d828e8236405fba.1755220973.git.gitgitgadget@gmail.com","threadId":"63804","inReplyTo":"pull.1980.v2.git.git.1755220973.gitgitgadget@gmail.com","subject":"[PATCH v2 11/17] win+Meson: do allow linking with the Rust-built xdiff","fromName":"Johannes Schindelin via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2025-08-15T01:22:46Z","receivedAt":"2025-08-15T01:23:06Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"From: Johannes Schindelin \n\nWhen linking against the Rust-built `xdiff`, there is now a new required\ndependency: Without _also_ linking to the system library `userenv`, the\ncompile would fail with this error message:\n\n xdiff.lib(std-c85e9beb7923f636.std.df32d1bc89881d89-cgu.0.rcgu.o) :\n error LNK2019: unresolved external symbol __imp_GetUserProfileDirectoryW\n referenced in function _ZN3std3env8home_dir17hfd1c3b6676cd78f6E\n\nTherefore, just like we do in case of Makefile-based builds on Windows,\nwe now also link to that library when building with Meson.\n\nNote that if we only have Rust depend upon libuserenv then at link time\nGCC would complain about:\n\n undefined reference to `GetUserProfileDirectoryW'\n\nApparently there is _some_ closure that gets compiled in that requires\nthis function, and that in turn forces Git to link to libuserenv.\n\nThis is a new requirement, and therefore has not been made part of the\n\"minimal Git for Windows SDK\".\n\nIn the near future, I intend to include it, but for now let's just\nensure that the file is added manually if it is missing.\n\nSigned-off-by: Johannes Schindelin \n[en: Squashed a few of Johannes's patches, and moved lib userenv\n handling from an earlier patch]\nSigned-off-by: Ezekiel Newren \n---\n .github/workflows/main.yml | 8 ++++++++\n config.mak.uname | 2 ++\n meson.build | 1 +\n 3 files changed, 11 insertions(+)\n\ndiff --git a/.github/workflows/main.yml b/.github/workflows/main.yml\nindex 8aac18a6ba45..aa18742f08c4 100644\n--- a/.github/workflows/main.yml\n+++ b/.github/workflows/main.yml\n@@ -115,6 +115,14 @@ jobs:\n steps:\n - uses: actions/checkout@v4\n - uses: git-for-windows/setup-git-for-windows-sdk@v1\n+ - name: ensure that libuserenv.a is present\n+ shell: bash\n+ run: |\n+ cd /mingw64/lib && {\n+ test -f libuserenv.a ||\n+ /c/Program\\ Files/Git/mingw64/bin/curl -Lo libuserenv.a \\\n+ https://github.com/git-for-windows/git-sdk-64/raw/HEAD/mingw64/lib/libuserenv.a\n+ }\n - name: build\n shell: bash\n env:\ndiff --git a/config.mak.uname b/config.mak.uname\nindex a22703284b56..fbe7cebf40ed 100644\n--- a/config.mak.uname\n+++ b/config.mak.uname\n@@ -746,6 +746,8 @@ ifeq ($(uname_S),MINGW)\n \n \texport CARGO_BUILD_TARGET\n \tRUST_TARGET_DIR = rust/target/$(CARGO_BUILD_TARGET)/$(RUST_BUILD_MODE)\n+\t# Unfortunately now needed because of Rust\n+\tEXTLIBS += -luserenv\n \n \tCC = gcc\n \tCOMPAT_CFLAGS += -D__USE_MINGW_ANSI_STDIO=0 -DDETECT_MSYS_TTY \\\ndiff --git a/meson.build b/meson.build\nindex 5e89a5dd0e00..af015f04763f 100644\n--- a/meson.build\n+++ b/meson.build\n@@ -1260,6 +1260,7 @@ elif host_machine.system() == 'windows'\n ]\n \n libgit_dependencies += compiler.find_library('ntdll')\n+ libgit_dependencies += compiler.find_library('userenv')\n libgit_include_directories += 'compat/win32'\n if compiler.get_id() == 'msvc'\n libgit_include_directories += 'compat/vcbuild/include'\n-- \ngitgitgadget\n\n"},{"id":"524216","messageId":"fffdb326710897fcc952dafad3d8cf600e5d7ff7.1755220973.git.gitgitgadget@gmail.com","threadId":"63804","inReplyTo":"pull.1980.v2.git.git.1755220973.gitgitgadget@gmail.com","subject":"[PATCH v2 12/17] github workflows: define rust versions and targets in the same place","fromName":"Ezekiel Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2025-08-15T01:22:47Z","receivedAt":"2025-08-15T01:23:08Z","isPatch":true,"sender":{"key":"ezekielnewren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/18313324?v=4"},"body":"From: Ezekiel Newren \n\nConsolidate the Rust toolchain definitions in main.yaml. Prefer using\nactions-rs/toolchain@v1 where possible, but for docker targets use\na script to install the Rust toolchain. Four overrides are used in\nmain.yaml:\n\n * On Windows: Rust didn't resolve the bcrypt library on Windows\n correctly until version 1.78.0. Also since rustup mis-identifies\n the Rust toolchain, the Rust target triple must be set to\n x86_64-pc-windows-gnu.\n * On musl: libc differences, such as ftruncate64 vs ftruncate, were\n not accounted for until Rust version 1.72.0. No older version of\n Rust will work on musl for our needs.\n * In a 32-bit docker container running on a 64-bit host, we need to\n override the Rust target triple. This is because rustup asks the\n kernel for the bitness of the system and it says 64, even though\n the container will only run 32-bit. This also allows us to remove\n the BITNESS environment variable in ci/lib.sh.\n\nSigned-off-by: Ezekiel Newren \n---\n .github/workflows/main.yml | 34 +++++++++++++++++++++++++++++++++-\n build_rust.sh | 11 +++--------\n ci/install-rust.sh | 33 +++++++++++++++++----------------\n ci/lib.sh | 7 -------\n ci/make-test-artifacts.sh | 12 ++++++------\n ci/run-build-and-tests.sh | 8 +++++---\n meson.build | 1 +\n 7 files changed, 65 insertions(+), 41 deletions(-)\n mode change 100644 => 100755 ci/install-rust.sh\n\ndiff --git a/.github/workflows/main.yml b/.github/workflows/main.yml\nindex aa18742f08c4..ef4d6348edcd 100644\n--- a/.github/workflows/main.yml\n+++ b/.github/workflows/main.yml\n@@ -4,7 +4,6 @@ on: [push, pull_request]\n \n env:\n DEVELOPER: 1\n- RUST_VERSION: 1.87.0\n \n # If more than one workflow run is triggered for the very same commit hash\n # (which happens when multiple branches pointing to the same commit), only\n@@ -27,6 +26,12 @@ jobs:\n outputs:\n enabled: ${{ steps.check-ref.outputs.enabled }}${{ steps.skip-if-redundant.outputs.enabled }}\n skip_concurrent: ${{ steps.check-ref.outputs.skip_concurrent }}\n+ rust_version_minimum: 1.61.0\n+ rust_version_windows: 1.78.0\n+ rust_version_musl: 1.72.0\n+ ## the rust target is inferred by rustup unless specified\n+ rust_target_windows: x86_64-pc-windows-gnu\n+ rust_target_32bit_linux: i686-unknown-linux-gnu\n steps:\n - name: try to clone ci-config branch\n run: |\n@@ -123,11 +128,19 @@ jobs:\n /c/Program\\ Files/Git/mingw64/bin/curl -Lo libuserenv.a \\\n https://github.com/git-for-windows/git-sdk-64/raw/HEAD/mingw64/lib/libuserenv.a\n }\n+ - name: Install Rust\n+ uses: actions-rs/toolchain@v1\n+ with:\n+ toolchain: ${{ needs.ci-config.outputs.rust_version_windows }}\n+ target: ${{ needs.ci-config.outputs.rust_target_windows }}\n+ profile: minimal\n+ override: true\n - name: build\n shell: bash\n env:\n HOME: ${{runner.workspace}}\n NO_PERL: 1\n+ CARGO_HOME: \"/c/Users/runneradmin/.cargo\"\n run: . /etc/profile && ci/make-test-artifacts.sh artifacts\n - name: zip up tracked files\n run: git archive -o artifacts/tracked.tar.gz HEAD\n@@ -269,6 +282,13 @@ jobs:\n steps:\n - uses: actions/checkout@v4\n - uses: actions/setup-python@v5\n+ - name: Install Rust\n+ uses: actions-rs/toolchain@v1\n+ with:\n+ toolchain: ${{ needs.ci-config.outputs.rust_version_windows }}\n+ target: ${{ needs.ci-config.outputs.rust_target_windows }}\n+ profile: minimal\n+ override: true\n - name: Set up dependencies\n shell: pwsh\n run: pip install meson ninja\n@@ -342,6 +362,12 @@ jobs:\n steps:\n - uses: actions/checkout@v4\n - run: ci/install-dependencies.sh\n+ - name: Install Rust\n+ uses: actions-rs/toolchain@v1\n+ with:\n+ toolchain: ${{ needs.ci-config.outputs.rust_version_minimum }}\n+ profile: minimal\n+ override: true\n - run: ci/run-build-and-tests.sh\n - name: print test failures\n if: failure() && env.FAILED_TEST_ARTIFACTS != ''\n@@ -402,9 +428,11 @@ jobs:\n cc: gcc\n - jobname: linux-musl-meson\n image: alpine:latest\n+ rust_version_override: ${{ needs.ci-config.outputs.rust_version_musl }}\n # Supported until 2025-04-02.\n - jobname: linux32\n image: i386/ubuntu:focal\n+ rust_target_override: ${{ needs.ci-config.outputs.rust_target_32bit_linux }}\n - jobname: pedantic\n image: fedora:latest\n # A RHEL 8 compatible distro. Supported until 2029-05-31.\n@@ -417,7 +445,11 @@ jobs:\n jobname: ${{matrix.vector.jobname}}\n CC: ${{matrix.vector.cc}}\n CI_JOB_IMAGE: ${{matrix.vector.image}}\n+ CI_IS_DOCKER: \"true\"\n CUSTOM_PATH: /custom\n+ RUST_VERSION: ${{ matrix.vector.rust_version_override || needs.ci-config.outputs.rust_version_minimum }}\n+ RUST_TARGET: ${{ matrix.vector.rust_target_override || '' }}\n+ CARGO_HOME: /home/builder/.cargo\n runs-on: ubuntu-latest\n container: ${{matrix.vector.image}}\n steps:\ndiff --git a/build_rust.sh b/build_rust.sh\nindex 694d48d857a5..80ce7eae3b00 100755\n--- a/build_rust.sh\n+++ b/build_rust.sh\n@@ -1,13 +1,8 @@\n #!/bin/sh\n \n-if [ -z \"$CARGO_HOME\" ]; then\n- export CARGO_HOME=$HOME/.cargo\n- echo >&2 \"::warning:: CARGO_HOME is not set\"\n-fi\n-echo \"CARGO_HOME=$CARGO_HOME\"\n \n-rustc -vV\n-cargo --version\n+rustc -vV || exit $?\n+cargo --version || exit $?\n \n dir_git_root=${0%/*}\n dir_build=$1\n@@ -55,7 +50,7 @@ dst=$dir_build/$libfile\n if [ \"$dir_git_root\" != \"$dir_build\" ]; then\n src=$dir_rust/target/$rust_target/$libfile\n if [ ! -f $src ]; then\n- echo >&2 \"::error:: cannot find path of static library\"\n+ echo >&2 \"::error:: cannot find path of static library $src is not a file or does not exist\"\n exit 5\n fi\n \ndiff --git a/ci/install-rust.sh b/ci/install-rust.sh\nold mode 100644\nnew mode 100755\nindex c22baa629ceb..133aa8cc878a\n--- a/ci/install-rust.sh\n+++ b/ci/install-rust.sh\n@@ -1,36 +1,37 @@\n #!/bin/sh\n \n+## github workflows actions-rs/toolchain@v1 doesn't work for docker\n+## targets. This script should only be used if the ci pipeline\n+## doesn't support installing rust on a particular target.\n+\n if [ \"$(id -u)\" -eq 0 ]; then\n echo >&2 \"::warning:: installing rust as root\"\n fi\n \n-if [ \"$CARGO_HOME\" = \"\" ]; then\n- echo >&2 \"::warning:: CARGO_HOME is not set\"\n- export CARGO_HOME=$HOME/.cargo\n-fi\n-\n-export RUSTUP_HOME=$CARGO_HOME\n-\n if [ \"$RUST_VERSION\" = \"\" ]; then\n echo >&2 \"::error:: RUST_VERSION is not set\"\n+ exit 1\n+fi\n+\n+if [ \"$CARGO_HOME\" = \"\" ]; then\n+ echo >&2 \"::error:: CARGO_HOME is not set\"\n exit 2\n fi\n \n+export RUSTUP_HOME=$CARGO_HOME\n+\n ## install rustup\n curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- --default-toolchain none -y\n if [ ! -f $CARGO_HOME/env ]; then\n echo \"PATH=$CARGO_HOME/bin:\\$PATH\" > $CARGO_HOME/env\n fi\n+. $CARGO_HOME/env\n+\n ## install a specific version of rust\n-if [ \"$BITNESS\" = \"32\" ]; then\n- $CARGO_HOME/bin/rustup set default-host i686-unknown-linux-gnu || exit $?\n- $CARGO_HOME/bin/rustup install $RUST_VERSION || exit $?\n- $CARGO_HOME/bin/rustup default --force-non-host $RUST_VERSION || exit $?\n+if [ \"$RUST_TARGET\" != \"\" ]; then\n+ rustup default --force-non-host \"$RUST_VERSION-$RUST_TARGET\" || exit $?\n else\n- $CARGO_HOME/bin/rustup default $RUST_VERSION || exit $?\n- if [ \"$CI_OS_NAME\" = \"windows\" ]; then\n- $CARGO_HOME/bin/rustup target add x86_64-pc-windows-gnu || exit $?\n- fi\n+ rustup default \"$RUST_VERSION\" || exit $?\n fi\n \n-. $CARGO_HOME/env\n+rustc -vV || exit $?\ndiff --git a/ci/lib.sh b/ci/lib.sh\nindex ad0e49a68dcb..a7992b22fdc9 100755\n--- a/ci/lib.sh\n+++ b/ci/lib.sh\n@@ -1,13 +1,6 @@\n # Library of functions shared by all CI scripts\n \n \n-export BITNESS=\"64\"\n-if command -v getconf >/dev/null && [ \"$(getconf LONG_BIT 2>/dev/null)\" = \"32\" ]; then\n- export BITNESS=\"32\"\n-fi\n-echo \"BITNESS=$BITNESS\"\n-\n-\n if test true = \"$GITHUB_ACTIONS\"\n then\n \tbegin_group () {\ndiff --git a/ci/make-test-artifacts.sh b/ci/make-test-artifacts.sh\nindex 56aa7efb1d53..70d0dfbc0e8b 100755\n--- a/ci/make-test-artifacts.sh\n+++ b/ci/make-test-artifacts.sh\n@@ -7,13 +7,13 @@ mkdir -p \"$1\" # in case ci/lib.sh decides to quit early\n \n . ${0%/*}/lib.sh\n \n-## install rust per user rather than system wide\n-. ${0%/*}/install-rust.sh\n+if [ -z \"$CARGO_HOME\" ]; then\n+ echo >&2 \"::error:: CARGO_HOME is not set\"\n+ exit 1\n+fi\n \n-group Build make artifacts-tar ARTIFACTS_DIRECTORY=\"$1\"\n+export PATH=\"$CARGO_HOME/bin:$PATH\"\n \n-if [ -d \"$CARGO_HOME\" ]; then\n- rm -rf $CARGO_HOME\n-fi\n+group Build make artifacts-tar ARTIFACTS_DIRECTORY=\"$1\"\n \n check_unignored_build_artifacts\ndiff --git a/ci/run-build-and-tests.sh b/ci/run-build-and-tests.sh\nindex dbab1cb2f936..0f1f704d583e 100755\n--- a/ci/run-build-and-tests.sh\n+++ b/ci/run-build-and-tests.sh\n@@ -5,10 +5,12 @@\n \n . ${0%/*}/lib.sh\n \n-## install rust per user rather than system wide\n-. ${0%/*}/install-rust.sh\n+## actions-rs/toolchain@v1 doesn't work for docker targets.\n+if [ \"$CI_IS_DOCKER\" = \"true\" ]; then\n+ . ${0%/*}/install-rust.sh\n+fi\n \n-rustc -vV\n+rustc -vV || exit $?\n cargo --version || exit $?\n \n run_tests=t\ndiff --git a/meson.build b/meson.build\nindex af015f04763f..a2f9f063bef2 100644\n--- a/meson.build\n+++ b/meson.build\n@@ -293,6 +293,7 @@ rust_build_xdiff = custom_target('rust_build_xdiff',\n meson.project_source_root() / 'build_rust.sh',\n meson.current_build_dir(), rust_target, 'xdiff',\n ],\n+ env: script_environment,\n install: false,\n )\n \n-- \ngitgitgadget\n\n"},{"id":"524217","messageId":"44784f0d672f15df238d5232ca098df3d13733ce.1755220973.git.gitgitgadget@gmail.com","threadId":"63804","inReplyTo":"pull.1980.v2.git.git.1755220973.gitgitgadget@gmail.com","subject":"[PATCH v2 13/17] github workflows: upload Cargo.lock","fromName":"Ezekiel Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2025-08-15T01:22:48Z","receivedAt":"2025-08-15T01:23:09Z","isPatch":true,"sender":{"key":"ezekielnewren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/18313324?v=4"},"body":"From: Ezekiel Newren \n\nMake each ci workflow upload its Cargo.lock file as a build artifact so\nthat we can audit build dependencies.\n\nSigned-off-by: Ezekiel Newren \n---\n .github/workflows/main.yml | 20 ++++++++++++++++++++\n 1 file changed, 20 insertions(+)\n\ndiff --git a/.github/workflows/main.yml b/.github/workflows/main.yml\nindex ef4d6348edcd..ba61bd516639 100644\n--- a/.github/workflows/main.yml\n+++ b/.github/workflows/main.yml\n@@ -149,6 +149,11 @@ jobs:\n with:\n name: windows-artifacts\n path: artifacts\n+ - name: upload Cargo.lock\n+ uses: actions/upload-artifact@v4\n+ with:\n+ name: cargo-lock-windows\n+ path: rust/Cargo.lock\n windows-test:\n name: win test\n runs-on: windows-latest\n@@ -303,6 +308,11 @@ jobs:\n with:\n name: windows-meson-artifacts\n path: build\n+ - name: Upload Cargo.lock\n+ uses: actions/upload-artifact@v4\n+ with:\n+ name: cargo-lock-windows-meson\n+ path: rust/Cargo.lock\n windows-meson-test:\n name: win+Meson test\n runs-on: windows-latest\n@@ -378,6 +388,11 @@ jobs:\n with:\n name: failed-tests-${{matrix.vector.jobname}}\n path: ${{env.FAILED_TEST_ARTIFACTS}}\n+ - name: Upload Cargo.lock\n+ uses: actions/upload-artifact@v4\n+ with:\n+ name: cargo-lock-${{matrix.vector.jobname}}\n+ path: rust/Cargo.lock\n fuzz-smoke-test:\n name: fuzz smoke test\n needs: ci-config\n@@ -484,6 +499,11 @@ jobs:\n with:\n name: failed-tests-${{matrix.vector.jobname}}\n path: ${{env.FAILED_TEST_ARTIFACTS}}\n+ - name: Upload Cargo.lock\n+ uses: actions/upload-artifact@v4\n+ with:\n+ name: cargo-lock-${{matrix.vector.jobname}}\n+ path: rust/Cargo.lock\n static-analysis:\n needs: ci-config\n if: needs.ci-config.outputs.enabled == 'yes'\n-- \ngitgitgadget\n\n"},{"id":"524219","messageId":"f20efdff7aa0a02ca2585de0cd9de21bbff22d58.1755220973.git.gitgitgadget@gmail.com","threadId":"63804","inReplyTo":"pull.1980.v2.git.git.1755220973.gitgitgadget@gmail.com","subject":"[PATCH v2 14/17] xdiff: implement a white space iterator in Rust","fromName":"Ezekiel Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2025-08-15T01:22:49Z","receivedAt":"2025-08-15T01:23:09Z","isPatch":true,"sender":{"key":"ezekielnewren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/18313324?v=4"},"body":"From: Ezekiel Newren \n\nXdiff has traditionally implemented the logic for iterating over\nwhitespace in every location that needed to do so. Create a consolidated\niterator in Rust that we can call from each location. Write Rust unit\ntests to ensure the correctness of the Rust whitespace iterator and the\nchunked_iter_equal() function.\n\nSigned-off-by: Ezekiel Newren \n---\n rust/xdiff/src/lib.rs | 10 ++\n rust/xdiff/src/xutils.rs | 292 +++++++++++++++++++++++++++++++++++++++\n 2 files changed, 302 insertions(+)\n create mode 100644 rust/xdiff/src/xutils.rs\n\ndiff --git a/rust/xdiff/src/lib.rs b/rust/xdiff/src/lib.rs\nindex 96975975a1ba..9cf0462bcdb9 100644\n--- a/rust/xdiff/src/lib.rs\n+++ b/rust/xdiff/src/lib.rs\n@@ -1,3 +1,13 @@\n+pub mod xutils;\n+\n+pub const XDF_IGNORE_WHITESPACE: u64 = 1 << 1;\n+pub const XDF_IGNORE_WHITESPACE_CHANGE: u64 = 1 << 2;\n+pub const XDF_IGNORE_WHITESPACE_AT_EOL: u64 = 1 << 3;\n+pub const XDF_IGNORE_CR_AT_EOL: u64 = 1 << 4;\n+pub const XDF_WHITESPACE_FLAGS: u64 = XDF_IGNORE_WHITESPACE |\n+ XDF_IGNORE_WHITESPACE_CHANGE |\n+ XDF_IGNORE_WHITESPACE_AT_EOL |\n+ XDF_IGNORE_CR_AT_EOL;\n \n \n #[no_mangle]\ndiff --git a/rust/xdiff/src/xutils.rs b/rust/xdiff/src/xutils.rs\nnew file mode 100644\nindex 000000000000..38126b47292f\n--- /dev/null\n+++ b/rust/xdiff/src/xutils.rs\n@@ -0,0 +1,292 @@\n+use crate::*;\n+\n+pub(crate) fn xdl_isspace(v: u8) -> bool {\n+ match v {\n+ b'\\t' | b'\\n' | b'\\r' | b' ' => true,\n+ _ => false,\n+ }\n+}\n+\n+pub struct WhitespaceIter<'a> {\n+ line: &'a [u8],\n+ index: usize,\n+ flags: u64,\n+}\n+\n+\n+impl<'a> WhitespaceIter<'a> {\n+ pub fn new(line: &'a [u8], flags: u64) -> Self {\n+ Self {\n+ line,\n+ index: 0,\n+ flags,\n+ }\n+ }\n+}\n+\n+impl<'a> Iterator for WhitespaceIter<'a> {\n+ type Item = &'a [u8];\n+\n+ fn next(&mut self) -> Option {\n+ if self.index >= self.line.len() {\n+ return None;\n+ }\n+\n+ loop {\n+ let start = self.index;\n+ if self.index == self.line.len() {\n+ return None;\n+ }\n+\n+ /* return contiguous run of not space bytes */\n+ while self.index < self.line.len() {\n+ if xdl_isspace(self.line[self.index]) {\n+ break;\n+ }\n+ self.index += 1;\n+ }\n+ if self.index > start {\n+ return Some(&self.line[start..self.index]);\n+ }\n+ /* the current byte had better be a space */\n+ if !xdl_isspace(self.line[self.index]) {\n+ panic!(\"xdl_line_iter_next xdl_isspace() is false\")\n+ }\n+\n+ while self.index < self.line.len() && xdl_isspace(self.line[self.index]) {\n+ self.index += 1;\n+ }\n+\n+\n+ if self.index <= start {\n+ panic!(\"xdl_isspace() cannot simultaneously be true and false\");\n+ }\n+\n+ if (self.flags & XDF_IGNORE_WHITESPACE_AT_EOL) != 0\n+ && self.index == self.line.len()\n+ {\n+ return None;\n+ }\n+ if (self.flags & XDF_IGNORE_WHITESPACE) != 0 {\n+ continue;\n+ }\n+ if (self.flags & XDF_IGNORE_WHITESPACE_CHANGE) != 0 {\n+ if self.index == self.line.len() {\n+ continue;\n+ }\n+ return Some(\" \".as_bytes());\n+ }\n+ if (self.flags & XDF_IGNORE_CR_AT_EOL) != 0 {\n+ if start < self.line.len() && self.index == self.line.len() {\n+ let mut end = self.line.len();\n+ if end > 0 && self.line[end - 1] == b'\\n' {\n+ if end - start == 1 {\n+ return Some(&self.line[start..end]);\n+ } else {\n+ end -= 1;\n+ }\n+ if end > 0 && self.line[end - 1] == b'\\r' {\n+ self.index = end;\n+ end -= 1;\n+ if end - start == 0 {\n+ continue;\n+ }\n+ return Some(&self.line[start..end]);\n+ }\n+ }\n+ }\n+ }\n+ return Some(&self.line[start..self.index]);\n+ }\n+ }\n+}\n+\n+pub fn chunked_iter_equal<'a, T, IT0, IT1>(mut it0: IT0, mut it1: IT1) -> bool\n+where\n+ T: Eq + 'a,\n+ IT0: Iterator,\n+ IT1: Iterator,\n+{\n+ let mut run_option0: Option<&[T]> = it0.next();\n+ let mut run_option1: Option<&[T]> = it1.next();\n+ let mut i0 = 0;\n+ let mut i1 = 0;\n+\n+ while let (Some(run0), Some(run1)) = (run_option0, run_option1) {\n+ while i0 < run0.len() && i1 < run1.len() {\n+ if run0[i0] != run1[i1] {\n+ return false;\n+ }\n+\n+ i0 += 1;\n+ i1 += 1;\n+ }\n+\n+ if i0 == run0.len() {\n+ i0 = 0;\n+ run_option0 = it0.next();\n+ }\n+ if i1 == run1.len() {\n+ i1 = 0;\n+ run_option1 = it1.next();\n+ }\n+ }\n+\n+ while let Some(run0) = run_option0 {\n+ if run0.len() == 0 {\n+ run_option0 = it0.next();\n+ } else {\n+ break;\n+ }\n+ }\n+\n+ while let Some(run1) = run_option1 {\n+ if run1.len() == 0 {\n+ run_option1 = it1.next();\n+ } else {\n+ break;\n+ }\n+ }\n+\n+ run_option0.is_none() && run_option1.is_none()\n+}\n+\n+#[cfg(test)]\n+mod tests {\n+ use crate::*;\n+ use crate::xutils::{chunked_iter_equal, WhitespaceIter};\n+\n+ fn extract_string<'a>(line: &[u8], flags: u64, buffer: &'a mut Vec) -> &'a str {\n+ let it = WhitespaceIter::new(line, flags);\n+ buffer.clear();\n+ for run in it {\n+ #[cfg(test)]\n+ let _view = unsafe { std::str::from_utf8_unchecked(run) };\n+ buffer.extend_from_slice(run);\n+ }\n+ unsafe { std::str::from_utf8_unchecked(buffer.as_slice()) }\n+ }\n+\n+ fn get_str_it<'a>(slice: &'a [&'a str]) -> impl Iterator + 'a {\n+ slice.iter().map(|v| (*v).as_bytes())\n+ }\n+\n+ #[test]\n+ fn test_ignore_space() {\n+ let tv_individual = vec![\n+ (\"ab\\r\", \"ab\\r\", XDF_IGNORE_CR_AT_EOL),\n+ (\"ab \\r\", \"ab \\r\", XDF_IGNORE_CR_AT_EOL),\n+ (\"\\r \\t a \\r\", \"\\r \\t a \\r\", XDF_IGNORE_CR_AT_EOL),\n+ (\"\\r a \\r\", \"\\r a \\r\", XDF_IGNORE_CR_AT_EOL),\n+ (\"\\r\", \"\\r\", XDF_IGNORE_CR_AT_EOL),\n+ (\"\", \"\", XDF_IGNORE_CR_AT_EOL),\n+ (\"\\r a \\r\", \"\\r a \\r\", XDF_IGNORE_CR_AT_EOL),\n+\n+ (\"\\r \\t a \\n\", \"\\r \\t a \\r\\n\", XDF_IGNORE_CR_AT_EOL),\n+ (\"\\r a \\n\", \"\\r a \\r\\n\", XDF_IGNORE_CR_AT_EOL),\n+ (\"\\n\", \"\\r\\n\", XDF_IGNORE_CR_AT_EOL),\n+ (\"\\n\", \"\\n\", XDF_IGNORE_CR_AT_EOL),\n+ (\"\\r a \\n\", \"\\r a \\n\", XDF_IGNORE_CR_AT_EOL),\n+\n+ (\"1\\n\", \"1\\r\\n\", XDF_IGNORE_CR_AT_EOL),\n+ (\"1\", \"1\\r\\n\", XDF_IGNORE_WHITESPACE_CHANGE),\n+\n+ (\"\\r \\t a \\r\\n\", \"\\r \\t a \\r\\n\", 0),\n+ (\"\\r a \\r\\n\", \"\\r a \\r\\n\", 0),\n+ (\"\\r\\n\", \"\\r\\n\", 0),\n+ (\"\\n\", \"\\n\", 0),\n+ (\"\\r a \\n\", \"\\r a \\n\", 0),\n+ (\" \\n\", \" \\n\", 0),\n+ (\"a \\n\", \"a \\n\", 0),\n+ (\" a \\t asdf \\t \\r\\n\", \" a \\t asdf \\t \\r\\n\", 0),\n+ (\"\\t a b \\t \\n\", \"\\t a b \\t \\n\", 0),\n+ (\" a b \\t \\r\\n\", \" a b \\t \\r\\n\", 0),\n+ (\"\\t a \\n\", \"\\t a \\n\", 0),\n+ (\"\\t\\t\\ta\\t\\n\", \"\\t\\t\\ta\\t\\n\", 0),\n+ (\"a\\n\", \"a\\n\", 0),\n+ (\"\\ta\\n\", \"\\ta\\n\", 0),\n+\n+ (\"a\", \"\\r \\t a \\r\\n\", XDF_IGNORE_WHITESPACE),\n+ (\"a\", \"\\r a \\r\\n\", XDF_IGNORE_WHITESPACE),\n+ (\"\", \"\\r\\n\", XDF_IGNORE_WHITESPACE),\n+ (\"\", \"\\n\", XDF_IGNORE_WHITESPACE),\n+ (\"a\", \"\\r a \\n\", XDF_IGNORE_WHITESPACE),\n+ (\"\", \" \\n\", XDF_IGNORE_WHITESPACE),\n+ (\"a\", \"a \\n\", XDF_IGNORE_WHITESPACE),\n+ (\"aasdf\", \" a \\t asdf \\t \\r\\n\", XDF_IGNORE_WHITESPACE),\n+ (\"ab\", \"\\t a b \\t \\n\", XDF_IGNORE_WHITESPACE),\n+ (\"ab\", \" a b \\t \\r\\n\", XDF_IGNORE_WHITESPACE),\n+ (\"a\", \"\\t a \\n\", XDF_IGNORE_WHITESPACE),\n+ (\"a\", \"\\t\\t\\ta\\t\\n\", XDF_IGNORE_WHITESPACE),\n+ (\"a\", \"a\\n\", XDF_IGNORE_WHITESPACE),\n+ (\"a\", \"\\ta\\n\", XDF_IGNORE_WHITESPACE),\n+\n+ (\"\", \" \\n\", XDF_IGNORE_WHITESPACE_AT_EOL),\n+ (\"a\", \"a \\n\", XDF_IGNORE_WHITESPACE_AT_EOL),\n+ (\" a \\t asdf\", \" a \\t asdf \\t \\r\\n\", XDF_IGNORE_WHITESPACE_AT_EOL),\n+ (\"\\t a b\", \"\\t a b \\t \\n\", XDF_IGNORE_WHITESPACE_AT_EOL),\n+\n+ (\" a b\", \" a b \\t \\r\\n\", XDF_IGNORE_WHITESPACE_CHANGE),\n+ (\" a\", \"\\t a \\n\", XDF_IGNORE_WHITESPACE_CHANGE),\n+ (\" a\", \"\\t\\t\\ta\\t\\n\", XDF_IGNORE_WHITESPACE_CHANGE),\n+ (\"a\", \"a\\n\", XDF_IGNORE_WHITESPACE_CHANGE),\n+ (\" a\", \"\\ta\\n\", XDF_IGNORE_WHITESPACE_CHANGE),\n+\n+ (\"ab\", \" a b \\t \\r\\n\", XDF_IGNORE_WHITESPACE | XDF_IGNORE_WHITESPACE_CHANGE),\n+ (\"a\", \"\\t a \\n\", XDF_IGNORE_WHITESPACE | XDF_IGNORE_WHITESPACE_CHANGE),\n+ (\"a\", \"\\t\\t\\ta\\t\\n\", XDF_IGNORE_WHITESPACE | XDF_IGNORE_WHITESPACE_CHANGE),\n+ (\"a\", \"a\\n\", XDF_IGNORE_WHITESPACE | XDF_IGNORE_WHITESPACE_CHANGE),\n+ (\"a\", \"\\ta\\n\", XDF_IGNORE_WHITESPACE | XDF_IGNORE_WHITESPACE_CHANGE),\n+ ];\n+\n+ let mut buffer = Vec::::new();\n+ for (expected, input, flags) in tv_individual {\n+ let actual = extract_string(input.as_bytes(), flags, &mut buffer);\n+ assert_eq!(expected, actual, \"input: {:?} flags: 0x{:x}\", input, flags);\n+ }\n+ }\n+\n+ #[test]\n+ fn test_chunked_iter_equal() {\n+ let tv_str: Vec<(Vec<&str>, Vec<&str>)> = vec![\n+ /* equal cases */\n+ (vec![\"\", \"\", \"abc\"], vec![\"\", \"abc\"]),\n+ (vec![\"c\", \"\", \"a\"], vec![\"c\", \"a\"]),\n+ (vec![\"a\", \"\", \"b\", \"\", \"c\"], vec![\"a\", \"b\", \"c\"]),\n+ (vec![\"\", \"\", \"a\"], vec![\"a\"]),\n+ (vec![\"\", \"a\"], vec![\"a\"]),\n+ (vec![\"\"], vec![]),\n+ (vec![\"\", \"\"], vec![\"\"]),\n+ (vec![\"a\"], vec![\"\", \"\", \"a\"]),\n+ (vec![\"a\"], vec![\"\", \"a\"]),\n+ (vec![], vec![\"\"]),\n+ (vec![\"\"], vec![\"\", \"\"]),\n+ (vec![\"hello \", \"world\"], vec![\"hel\", \"lo wo\", \"rld\"]),\n+ (vec![\"hel\", \"lo wo\", \"rld\"], vec![\"hello \", \"world\"]),\n+ (vec![\"hello world\"], vec![\"hello world\"]),\n+ (vec![\"abc\", \"def\"], vec![\"def\", \"abc\"]),\n+ (vec![], vec![]),\n+\n+ /* different cases */\n+ (vec![\"abc\"], vec![]),\n+ (vec![\"\", \"\", \"\"], vec![\"\", \"a\"]),\n+ (vec![\"\", \"a\"], vec![\"b\", \"\"]),\n+ (vec![\"abc\"], vec![\"abc\", \"de\"]),\n+ (vec![\"abc\", \"de\"], vec![\"abc\"]),\n+ (vec![], vec![\"a\"]),\n+ (vec![\"a\"], vec![]),\n+ (vec![\"abc\", \"kj\"], vec![\"abc\", \"de\"]),\n+ ];\n+\n+ for (lhs, rhs) in tv_str.iter() {\n+ let a: Vec = get_str_it(lhs).flatten().copied().collect();\n+ let b: Vec = get_str_it(rhs).flatten().copied().collect();\n+ let expected = a.as_slice() == b.as_slice();\n+\n+ let it0 = get_str_it(lhs);\n+ let it1 = get_str_it(rhs);\n+ let actual = chunked_iter_equal(it0, it1);\n+ assert_eq!(expected, actual);\n+ }\n+ }\n+}\n-- \ngitgitgadget\n\n"},{"id":"524218","messageId":"c8d411732742d774d2ce6418369170b9d1e9c0fc.1755220973.git.gitgitgadget@gmail.com","threadId":"63804","inReplyTo":"pull.1980.v2.git.git.1755220973.gitgitgadget@gmail.com","subject":"[PATCH v2 15/17] xdiff: create line_hash() and line_equal()","fromName":"Ezekiel Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2025-08-15T01:22:50Z","receivedAt":"2025-08-15T01:23:10Z","isPatch":true,"sender":{"key":"ezekielnewren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/18313324?v=4"},"body":"From: Ezekiel Newren \n\nThese functions use the whitespace iterator, when applicable, to hash,\nand compare lines.\n\nSigned-off-by: Ezekiel Newren \n---\n rust/xdiff/src/lib.rs | 19 +++++++++++++++++++\n rust/xdiff/src/xutils.rs | 28 ++++++++++++++++++++++++++++\n 2 files changed, 47 insertions(+)\n\ndiff --git a/rust/xdiff/src/lib.rs b/rust/xdiff/src/lib.rs\nindex 9cf0462bcdb9..809c5573c6e7 100644\n--- a/rust/xdiff/src/lib.rs\n+++ b/rust/xdiff/src/lib.rs\n@@ -1,3 +1,7 @@\n+use std::hash::Hasher;\n+use xxhash_rust::xxh3::Xxh3Default;\n+use crate::xutils::*;\n+\n pub mod xutils;\n \n pub const XDF_IGNORE_WHITESPACE: u64 = 1 << 1;\n@@ -15,3 +19,18 @@ unsafe extern \"C\" fn xxh3_64(ptr: *const u8, size: usize) -> u64 {\n let slice = std::slice::from_raw_parts(ptr, size);\n xxhash_rust::xxh3::xxh3_64(slice)\n }\n+\n+#[no_mangle]\n+unsafe extern \"C\" fn xdl_line_hash(ptr: *const u8, size: usize, flags: u64) -> u64 {\n+ let line = std::slice::from_raw_parts(ptr, size);\n+\n+ line_hash(line, flags)\n+}\n+\n+#[no_mangle]\n+unsafe extern \"C\" fn xdl_line_equal(lhs: *const u8, lhs_len: usize, rhs: *const u8, rhs_len: usize, flags: u64) -> bool {\n+ let lhs_line = std::slice::from_raw_parts(lhs, lhs_len);\n+ let rhs_line = std::slice::from_raw_parts(rhs, rhs_len);\n+\n+ line_equal(lhs_line, rhs_line, flags)\n+}\ndiff --git a/rust/xdiff/src/xutils.rs b/rust/xdiff/src/xutils.rs\nindex 38126b47292f..796a5708b6bf 100644\n--- a/rust/xdiff/src/xutils.rs\n+++ b/rust/xdiff/src/xutils.rs\n@@ -1,4 +1,5 @@\n use crate::*;\n+use xxhash_rust::xxh3::xxh3_64;\n \n pub(crate) fn xdl_isspace(v: u8) -> bool {\n match v {\n@@ -151,6 +152,33 @@ where\n run_option0.is_none() && run_option1.is_none()\n }\n \n+\n+pub fn line_hash(line: &[u8], flags: u64) -> u64 {\n+ if (flags & XDF_WHITESPACE_FLAGS) == 0 {\n+ return xxh3_64(line);\n+ }\n+\n+ let mut hasher = Xxh3Default::new();\n+ for chunk in WhitespaceIter::new(line, flags) {\n+ hasher.update(chunk);\n+ }\n+\n+ hasher.finish()\n+}\n+\n+\n+pub fn line_equal(lhs: &[u8], rhs: &[u8], flags: u64) -> bool {\n+ if (flags & XDF_WHITESPACE_FLAGS) == 0 {\n+ return lhs == rhs;\n+ }\n+\n+ let lhs_it = WhitespaceIter::new(lhs, flags);\n+ let rhs_it = WhitespaceIter::new(rhs, flags);\n+\n+ chunked_iter_equal(lhs_it, rhs_it)\n+}\n+\n+\n #[cfg(test)]\n mod tests {\n use crate::*;\n-- \ngitgitgadget\n\n"},{"id":"524220","messageId":"f7829c558715022258486bc265e04bd093665e2d.1755220973.git.gitgitgadget@gmail.com","threadId":"63804","inReplyTo":"pull.1980.v2.git.git.1755220973.gitgitgadget@gmail.com","subject":"[PATCH v2 16/17] xdiff: optimize case where --ignore-cr-at-eol is the only whitespace flag","fromName":"Ezekiel Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2025-08-15T01:22:51Z","receivedAt":"2025-08-15T01:23:11Z","isPatch":true,"sender":{"key":"ezekielnewren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/18313324?v=4"},"body":"From: Ezekiel Newren \n\nCurrently the whitespace iterator is slower than git's C implementation\nso we skip using the whitespace iterator if there are no whitespace\nflags. Special case the --ignore-cr-at-eol similarly to make it\nperformant. For the rest of the whitespace flags they will be slower\nfor now, but as more of Xdiff is translated into Rust it'll be easier\nto revisit and optimize whitespace processing. Optimizing the other\nwhitespace flags now would be difficult because:\n\n * Xxhash uses chunk based processing.\n * The same iterator is used for hashing and equality, which means the\n iterator could be optimized for returning large chunks for fast\n hashing or could return each byte making equality testing faster.\n I opted for faster hashing. The data structures in C need to be\n cleaned up before they're interoperable with Rust. Once that's done\n I believe a faster method of whitespace processing will be possible.\n * Trying to make heavliy optimized code between 2 languages that aren't\n easily interoperable in their current state makes the code either\n fast or easy to maintain. But once enough of Xdiff is written in\n Rust I believe that a fast and maintainable method can be\n implemented.\n\nSigned-off-by: Ezekiel Newren \n---\n rust/xdiff/src/xutils.rs | 34 ++++++++++++++++++++++++++++++++++\n 1 file changed, 34 insertions(+)\n\ndiff --git a/rust/xdiff/src/xutils.rs b/rust/xdiff/src/xutils.rs\nindex 796a5708b6bf..1ea9cfa02db5 100644\n--- a/rust/xdiff/src/xutils.rs\n+++ b/rust/xdiff/src/xutils.rs\n@@ -33,6 +33,18 @@ impl<'a> Iterator for WhitespaceIter<'a> {\n return None;\n }\n \n+ // optimize case where --ignore-cr-at-eol is the only whitespace flag\n+ if (self.flags & XDF_WHITESPACE_FLAGS) == XDF_IGNORE_CR_AT_EOL {\n+ if self.index == 0 && self.line.ends_with(b\"\\r\\n\") {\n+ self.index = self.line.len() - 1;\n+ return Some(&self.line[..self.line.len() - 2])\n+ } else {\n+ let off = self.index;\n+ self.index = self.line.len();\n+ return Some(&self.line[off..])\n+ }\n+ }\n+\n loop {\n let start = self.index;\n if self.index == self.line.len() {\n@@ -172,6 +184,28 @@ pub fn line_equal(lhs: &[u8], rhs: &[u8], flags: u64) -> bool {\n return lhs == rhs;\n }\n \n+ // optimize case where --ignore-cr-at-eol is the only whitespace flag\n+ if (flags & XDF_WHITESPACE_FLAGS) == XDF_IGNORE_CR_AT_EOL {\n+ let a = lhs.ends_with(b\"\\r\\n\");\n+ let b = rhs.ends_with(b\"\\r\\n\");\n+\n+ if !(a ^ b) {\n+ return lhs == rhs;\n+ } else {\n+ let lm = if a { 1 } else { 0 };\n+ let rm = if b { 1 } else { 0 };\n+\n+ if lhs.len() - lm != rhs.len() - rm {\n+ return false;\n+ } else if &lhs[..lhs.len() - 1 - lm] != &rhs[..rhs.len() - 1 - rm] {\n+ return false;\n+ } else if lhs[lhs.len() - 1] != rhs[rhs.len() - 1] {\n+ return false;\n+ }\n+ return true;\n+ }\n+ }\n+\n let lhs_it = WhitespaceIter::new(lhs, flags);\n let rhs_it = WhitespaceIter::new(rhs, flags);\n \n-- \ngitgitgadget\n\n"},{"id":"524221","messageId":"395609aff4bc32e09c48204cdc2974e614675297.1755220973.git.gitgitgadget@gmail.com","threadId":"63804","inReplyTo":"pull.1980.v2.git.git.1755220973.gitgitgadget@gmail.com","subject":"[PATCH v2 17/17] xdiff: use rust's version of whitespace processing","fromName":"Ezekiel Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2025-08-15T01:22:52Z","receivedAt":"2025-08-15T01:23:12Z","isPatch":true,"sender":{"key":"ezekielnewren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/18313324?v=4"},"body":"From: Ezekiel Newren \n\nDelete xdl_hash_record() and xdl_recmatch() in favor of xdl_line_hash()\nand xdl_line_equal().\n\nSigned-off-by: Ezekiel Newren \n---\n rust/xdiff/src/lib.rs | 6 --\n xdiff-interface.c | 4 +-\n xdiff/xmerge.c | 8 +--\n xdiff/xprepare.c | 29 ++------\n xdiff/xutils.c | 158 ------------------------------------------\n xdiff/xutils.h | 4 +-\n 6 files changed, 15 insertions(+), 194 deletions(-)\n\ndiff --git a/rust/xdiff/src/lib.rs b/rust/xdiff/src/lib.rs\nindex 809c5573c6e7..634b453a21b6 100644\n--- a/rust/xdiff/src/lib.rs\n+++ b/rust/xdiff/src/lib.rs\n@@ -14,12 +14,6 @@ pub const XDF_WHITESPACE_FLAGS: u64 = XDF_IGNORE_WHITESPACE |\n XDF_IGNORE_CR_AT_EOL;\n \n \n-#[no_mangle]\n-unsafe extern \"C\" fn xxh3_64(ptr: *const u8, size: usize) -> u64 {\n- let slice = std::slice::from_raw_parts(ptr, size);\n- xxhash_rust::xxh3::xxh3_64(slice)\n-}\n-\n #[no_mangle]\n unsafe extern \"C\" fn xdl_line_hash(ptr: *const u8, size: usize, flags: u64) -> u64 {\n let line = std::slice::from_raw_parts(ptr, size);\ndiff --git a/xdiff-interface.c b/xdiff-interface.c\nindex 1edcd319e6ef..71ddccf2cc15 100644\n--- a/xdiff-interface.c\n+++ b/xdiff-interface.c\n@@ -299,13 +299,13 @@ void xdiff_clear_find_func(xdemitconf_t *xecfg)\n \n unsigned long xdiff_hash_string(const char *s, size_t len, long flags)\n {\n-\treturn xdl_hash_record(&s, s + len, flags);\n+\treturn xdl_line_hash((u8 const*) s, len, flags);\n }\n \n int xdiff_compare_lines(const char *l1, long s1,\n \t\t\tconst char *l2, long s2, long flags)\n {\n-\treturn xdl_recmatch(l1, s1, l2, s2, flags);\n+\treturn xdl_line_equal((u8 const*) l1, s1, (u8 const*) l2, s2, flags);\n }\n \n int parse_conflict_style_name(const char *value)\ndiff --git a/xdiff/xmerge.c b/xdiff/xmerge.c\nindex 6fa6ea61a208..2f64651a839b 100644\n--- a/xdiff/xmerge.c\n+++ b/xdiff/xmerge.c\n@@ -101,8 +101,8 @@ static int xdl_merge_cmp_lines(xdfenv_t *xe1, int i1, xdfenv_t *xe2, int i2,\n \txrecord_t **rec2 = xe2->xdf2.recs + i2;\n \n \tfor (i = 0; i < line_count; i++) {\n-\t\tint result = xdl_recmatch((const char*) rec1[i]->ptr, rec1[i]->size,\n-\t\t\t(const char*) rec2[i]->ptr, rec2[i]->size, flags);\n+\t\tbool result = xdl_line_equal(rec1[i]->ptr, rec1[i]->size,\n+\t\t\trec2[i]->ptr, rec2[i]->size, flags);\n \t\tif (!result)\n \t\t\treturn -1;\n \t}\n@@ -324,8 +324,8 @@ static int xdl_fill_merge_buffer(xdfenv_t *xe1, const char *name1,\n \n static int recmatch(xrecord_t *rec1, xrecord_t *rec2, unsigned long flags)\n {\n-\treturn xdl_recmatch((char const*) rec1->ptr, rec1->size,\n-\t\t\t (char const*) rec2->ptr, rec2->size, flags);\n+\treturn xdl_line_equal(rec1->ptr, rec1->size,\n+\t\t\t rec2->ptr, rec2->size, flags);\n }\n \n /*\ndiff --git a/xdiff/xprepare.c b/xdiff/xprepare.c\nindex c0463bacd94b..b9f12184b1bb 100644\n--- a/xdiff/xprepare.c\n+++ b/xdiff/xprepare.c\n@@ -33,8 +33,8 @@\n typedef struct s_xdlclass {\n \tstruct s_xdlclass *next;\n \tu64 ha;\n-\tchar const *line;\n-\tlong size;\n+\tu8 const *line;\n+\tusize size;\n \tlong idx;\n \tlong len1, len2;\n } xdlclass_t;\n@@ -93,15 +93,15 @@ static void xdl_free_classifier(xdlclassifier_t *cf) {\n \n static int xdl_classify_record(unsigned int pass, xdlclassifier_t *cf, xrecord_t *rec) {\n \tlong hi;\n-\tchar const *line;\n+\tu8 const *line;\n \txdlclass_t *rcrec;\n \n-\tline = (char const*) rec->ptr;\n+\tline = rec->ptr;\n \thi = (long) XDL_HASHLONG(rec->ha, cf->hbits);\n \tfor (rcrec = cf->rchash[hi]; rcrec; rcrec = rcrec->next)\n \t\tif (rcrec->ha == rec->ha &&\n-\t\t\t\txdl_recmatch(rcrec->line, rcrec->size,\n-\t\t\t\t\t(const char*) rec->ptr, rec->size, cf->flags))\n+\t\t\t\txdl_line_equal(rcrec->line, rcrec->size,\n+\t\t\t\t\trec->ptr, rec->size, cf->flags))\n \t\t\tbreak;\n \n \tif (!rcrec) {\n@@ -160,9 +160,6 @@ static void xdl_parse_lines(mmfile_t *mf, long narec, xdfile_t *xdf) {\n }\n \n \n-extern u64 xxh3_64(u8 const* ptr, usize size);\n-\n-\n static int xdl_prepare_ctx(unsigned int pass, mmfile_t *mf, long narec, xpparam_t const *xpp,\n \t\t\t xdlclassifier_t *cf, xdfile_t *xdf) {\n \tunsigned long *ha;\n@@ -178,21 +175,9 @@ static int xdl_prepare_ctx(unsigned int pass, mmfile_t *mf, long narec, xpparam_\n \n \txdl_parse_lines(mf, narec, xdf);\n \n-\tif ((xpp->flags & XDF_WHITESPACE_FLAGS) == 0) {\n-\t\tfor (usize i = 0; i < (usize) xdf->nrec; i++) {\n-\t\t\txrecord_t *rec = xdf->recs[i];\n-\t\t\trec->ha = xxh3_64(rec->ptr, rec->size);\n-\t\t}\n-\t} else {\n-\t\tfor (usize i = 0; i < (usize) xdf->nrec; i++) {\n-\t\t\txrecord_t *rec = xdf->recs[i];\n-\t\t\tchar const* dump = (char const*) rec->ptr;\n-\t\t\trec->ha = xdl_hash_record(&dump, (char const*) (rec->ptr + rec->size), xpp->flags);\n-\t\t}\n-\t}\n-\n \tfor (usize i = 0; i < (usize) xdf->nrec; i++) {\n \t\txrecord_t *rec = xdf->recs[i];\n+\t\trec->ha = xdl_line_hash(rec->ptr, rec->size, xpp->flags);\n \t\txdl_classify_record(pass, cf, rec);\n \t}\n \ndiff --git a/xdiff/xutils.c b/xdiff/xutils.c\nindex 10e4f20b7c31..29e240eb138b 100644\n--- a/xdiff/xutils.c\n+++ b/xdiff/xutils.c\n@@ -152,164 +152,6 @@ int xdl_blankline(const char *line, long size, long flags)\n \treturn (i == size);\n }\n \n-/*\n- * Have we eaten everything on the line, except for an optional\n- * CR at the very end?\n- */\n-static int ends_with_optional_cr(const char *l, long s, long i)\n-{\n-\tint complete = s && l[s-1] == '\\n';\n-\n-\tif (complete)\n-\t\ts--;\n-\tif (s == i)\n-\t\treturn 1;\n-\t/* do not ignore CR at the end of an incomplete line */\n-\tif (complete && s == i + 1 && l[i] == '\\r')\n-\t\treturn 1;\n-\treturn 0;\n-}\n-\n-int xdl_recmatch(const char *l1, long s1, const char *l2, long s2, long flags)\n-{\n-\tint i1, i2;\n-\n-\tif (s1 == s2 && !memcmp(l1, l2, s1))\n-\t\treturn 1;\n-\tif (!(flags & XDF_WHITESPACE_FLAGS))\n-\t\treturn 0;\n-\n-\ti1 = 0;\n-\ti2 = 0;\n-\n-\t/*\n-\t * -w matches everything that matches with -b, and -b in turn\n-\t * matches everything that matches with --ignore-space-at-eol,\n-\t * which in turn matches everything that matches with --ignore-cr-at-eol.\n-\t *\n-\t * Each flavor of ignoring needs different logic to skip whitespaces\n-\t * while we have both sides to compare.\n-\t */\n-\tif (flags & XDF_IGNORE_WHITESPACE) {\n-\t\tgoto skip_ws;\n-\t\twhile (i1 < s1 && i2 < s2) {\n-\t\t\tif (l1[i1++] != l2[i2++])\n-\t\t\t\treturn 0;\n-\t\tskip_ws:\n-\t\t\twhile (i1 < s1 && XDL_ISSPACE(l1[i1]))\n-\t\t\t\ti1++;\n-\t\t\twhile (i2 < s2 && XDL_ISSPACE(l2[i2]))\n-\t\t\t\ti2++;\n-\t\t}\n-\t} else if (flags & XDF_IGNORE_WHITESPACE_CHANGE) {\n-\t\twhile (i1 < s1 && i2 < s2) {\n-\t\t\tif (XDL_ISSPACE(l1[i1]) && XDL_ISSPACE(l2[i2])) {\n-\t\t\t\t/* Skip matching spaces and try again */\n-\t\t\t\twhile (i1 < s1 && XDL_ISSPACE(l1[i1]))\n-\t\t\t\t\ti1++;\n-\t\t\t\twhile (i2 < s2 && XDL_ISSPACE(l2[i2]))\n-\t\t\t\t\ti2++;\n-\t\t\t\tcontinue;\n-\t\t\t}\n-\t\t\tif (l1[i1++] != l2[i2++])\n-\t\t\t\treturn 0;\n-\t\t}\n-\t} else if (flags & XDF_IGNORE_WHITESPACE_AT_EOL) {\n-\t\twhile (i1 < s1 && i2 < s2 && l1[i1] == l2[i2]) {\n-\t\t\ti1++;\n-\t\t\ti2++;\n-\t\t}\n-\t} else if (flags & XDF_IGNORE_CR_AT_EOL) {\n-\t\t/* Find the first difference and see how the line ends */\n-\t\twhile (i1 < s1 && i2 < s2 && l1[i1] == l2[i2]) {\n-\t\t\ti1++;\n-\t\t\ti2++;\n-\t\t}\n-\t\treturn (ends_with_optional_cr(l1, s1, i1) &&\n-\t\t\tends_with_optional_cr(l2, s2, i2));\n-\t}\n-\n-\t/*\n-\t * After running out of one side, the remaining side must have\n-\t * nothing but whitespace for the lines to match. Note that\n-\t * ignore-whitespace-at-eol case may break out of the loop\n-\t * while there still are characters remaining on both lines.\n-\t */\n-\tif (i1 < s1) {\n-\t\twhile (i1 < s1 && XDL_ISSPACE(l1[i1]))\n-\t\t\ti1++;\n-\t\tif (s1 != i1)\n-\t\t\treturn 0;\n-\t}\n-\tif (i2 < s2) {\n-\t\twhile (i2 < s2 && XDL_ISSPACE(l2[i2]))\n-\t\t\ti2++;\n-\t\treturn (s2 == i2);\n-\t}\n-\treturn 1;\n-}\n-\n-static unsigned long xdl_hash_record_with_whitespace(char const **data,\n-\t\tchar const *top, long flags) {\n-\tunsigned long ha = 5381;\n-\tchar const *ptr = *data;\n-\tint cr_at_eol_only = (flags & XDF_WHITESPACE_FLAGS) == XDF_IGNORE_CR_AT_EOL;\n-\n-\tfor (; ptr < top && *ptr != '\\n'; ptr++) {\n-\t\tif (cr_at_eol_only) {\n-\t\t\t/* do not ignore CR at the end of an incomplete line */\n-\t\t\tif (*ptr == '\\r' &&\n-\t\t\t (ptr + 1 < top && ptr[1] == '\\n'))\n-\t\t\t\tcontinue;\n-\t\t}\n-\t\telse if (XDL_ISSPACE(*ptr)) {\n-\t\t\tconst char *ptr2 = ptr;\n-\t\t\tint at_eol;\n-\t\t\twhile (ptr + 1 < top && XDL_ISSPACE(ptr[1])\n-\t\t\t\t\t&& ptr[1] != '\\n')\n-\t\t\t\tptr++;\n-\t\t\tat_eol = (top <= ptr + 1 || ptr[1] == '\\n');\n-\t\t\tif (flags & XDF_IGNORE_WHITESPACE)\n-\t\t\t\t; /* already handled */\n-\t\t\telse if (flags & XDF_IGNORE_WHITESPACE_CHANGE\n-\t\t\t\t && !at_eol) {\n-\t\t\t\tha += (ha << 5);\n-\t\t\t\tha ^= (unsigned long) ' ';\n-\t\t\t}\n-\t\t\telse if (flags & XDF_IGNORE_WHITESPACE_AT_EOL\n-\t\t\t\t && !at_eol) {\n-\t\t\t\twhile (ptr2 != ptr + 1) {\n-\t\t\t\t\tha += (ha << 5);\n-\t\t\t\t\tha ^= (unsigned long) *ptr2;\n-\t\t\t\t\tptr2++;\n-\t\t\t\t}\n-\t\t\t}\n-\t\t\tcontinue;\n-\t\t}\n-\t\tha += (ha << 5);\n-\t\tha ^= (unsigned long) *ptr;\n-\t}\n-\t*data = ptr < top ? ptr + 1: ptr;\n-\n-\treturn ha;\n-}\n-\n-unsigned long xdl_hash_record(char const **data, char const *top, long flags) {\n-\tunsigned long ha = 5381;\n-\tchar const *ptr = *data;\n-\n-\tif (flags & XDF_WHITESPACE_FLAGS)\n-\t\treturn xdl_hash_record_with_whitespace(data, top, flags);\n-\n-\tfor (; ptr < top && *ptr != '\\n'; ptr++) {\n-\t\tha += (ha << 5);\n-\t\tha ^= (unsigned long) *ptr;\n-\t}\n-\t*data = ptr < top ? ptr + 1: ptr;\n-\n-\treturn ha;\n-}\n-\n unsigned int xdl_hashbits(unsigned int size) {\n \tunsigned int val = 1, bits = 0;\n \ndiff --git a/xdiff/xutils.h b/xdiff/xutils.h\nindex fd0bba94e8b4..8f524b72c491 100644\n--- a/xdiff/xutils.h\n+++ b/xdiff/xutils.h\n@@ -33,8 +33,8 @@ void xdl_cha_free(chastore_t *cha);\n void *xdl_cha_alloc(chastore_t *cha);\n long xdl_guess_lines(mmfile_t *mf, long sample);\n int xdl_blankline(const char *line, long size, long flags);\n-int xdl_recmatch(const char *l1, long s1, const char *l2, long s2, long flags);\n-unsigned long xdl_hash_record(char const **data, char const *top, long flags);\n+u64 xdl_line_hash(u8 const* ptr, usize size, u64 flags);\n+bool xdl_line_equal(u8 const* lhs, usize lhs_len, u8 const* rhs, usize rhs_len, u64 flags);\n unsigned int xdl_hashbits(unsigned int size);\n int xdl_num_out(char *out, long val);\n int xdl_emit_hunk_hdr(long s1, long c1, long s2, long c2,\n-- \ngitgitgadget\n"},{"id":"524242","messageId":"2d5ae8f6-69f1-486b-bd38-337f0b54f737@ramsayjones.plus.com","threadId":"63804","inReplyTo":"pull.1980.v2.git.git.1755220973.gitgitgadget@gmail.com","subject":"Re: [-SPAM-] [PATCH v2 00/17] RFC: Accelerate xdiff and begin its rustification","fromName":"Ramsay Jones","fromEmail":"ramsay@ramsayjones.plus.com","sentAt":"2025-08-15T15:07:02Z","receivedAt":"2025-08-15T15:10:12Z","isPatch":true,"sender":{"key":"ramsay@ramsayjones.plus.com","avatar":"https://avatars.githubusercontent.com/u/33702710?v=4"},"body":"\n\nOn 15/08/2025 02:22, Ezekiel Newren via GitGitGadget wrote:\n> Changes in this second round of this RFC:\n> \n> * Now builds and passes tests on all platforms (example run:\n> https://github.com/ezekielnewren/git/actions/runs/16974821401). Special\n> thanks to Johannes Schindelin for patches to things for Windows and\n> linux32.\n\nHmm, builds on *all* platforms may be a bit optimistic (it doesn't on\ncygwin, for instance), so I'm guessing you mean all platforms which\nhave CI defined. Perhaps you could mention the platforms which you\nhave tested on. :)\n\nATB,\nRamsay Jones\n\n"},{"id":"524269","messageId":"DB9P250MB06923B01AACB69F02170B1E3A534A@DB9P250MB0692.EURP250.PROD.OUTLOOK.COM","threadId":"63804","inReplyTo":"75dfb40ead370e80dda423998f8220ac19c2ff46.1755220973.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v2 01/17] doc: add a policy for using Rust","fromName":"Matthias Aßhauer","fromEmail":"mha1993@live.de","sentAt":"2025-08-15T17:03:17Z","receivedAt":"2025-08-15T17:03:27Z","isPatch":true,"sender":{"key":"mha1993@live.de","avatar":"https://avatars.githubusercontent.com/u/6178234?v=4"},"body":"\n\nOn Fri, 15 Aug 2025, brian m. carlson via GitGitGadget wrote:\n\n> From: \"brian m. carlson\" \n>\n> Git has historically been written primarily in C, with some shell and\n> Perl. However, C is not memory safe, which makes it more likely that\n> security vulnerabilities or other bugs will be introduced, and it is\n> also more verbose and less ergonomic than other, more modern languages.\n>\n> One of the most common modern compiled languages which is easily\n> interoperable with C is Rust. It is popular (the most admired language\n> on the 2024 Stack Overflow Developer Survey), efficient, portable, and\n> robust.\n>\n> Introduce a document laying out the incremental introduction of Rust to\n> Git and provide a detailed rationale for doing so, including the points\n> above. Propose a design for this approach that addresses the needs of\n> downstreams and distributors, as well as contributors.\n>\n> Since we don't want to carry both a C and Rust version of code and want\n> to be able to add new features only in Rust, mention that Rust is a\n> required part of our platform support policy.\n>\n> It should be noted that a recent discussion at the Berlin Git Merge\n> Contributor Summit found widespread support for the addition of Rust to\n> Git. While of course not all contributors were represented, the\n> proposal appeared to have the support of a majority of active\n> contributors.\n>\n> Signed-off-by: brian m. carlson \n> Signed-off-by: Ezekiel Newren \n> ---\n> Documentation/Makefile | 1 +\n> Documentation/technical/platform-support.adoc | 2 +\n> Documentation/technical/rust-support.adoc | 119 ++++++++++++++++++\n> 3 files changed, 122 insertions(+)\n> create mode 100644 Documentation/technical/rust-support.adoc\n>\n> diff --git a/Documentation/Makefile b/Documentation/Makefile\n> index b109d25e9c80..066b761c01b9 100644\n> --- a/Documentation/Makefile\n> +++ b/Documentation/Makefile\n> @@ -127,6 +127,7 @@ TECH_DOCS += technical/parallel-checkout\n> TECH_DOCS += technical/partial-clone\n> TECH_DOCS += technical/platform-support\n> TECH_DOCS += technical/racy-git\n> +TECH_DOCS += technical/rust-support\n> TECH_DOCS += technical/reftable\n> TECH_DOCS += technical/scalar\n> TECH_DOCS += technical/send-pack-pipeline\n> diff --git a/Documentation/technical/platform-support.adoc b/Documentation/technical/platform-support.adoc\n> index 0a2fb28d6277..42b04b186105 100644\n> --- a/Documentation/technical/platform-support.adoc\n> +++ b/Documentation/technical/platform-support.adoc\n> @@ -33,6 +33,8 @@ meet the following minimum requirements:\n>\n> * Has active security support (taking security releases of dependencies, etc)\n>\n> +* Supports Rust and the toolchain version specified in link:rust-support.txt[].\n\ns/rust-support.txt/rust-support.adoc/\n\n> +\n> These requirements are a starting point, and not sufficient on their own for the\n> Git community to be enthusiastic about supporting your platform. Maintainers of\n> platforms which do meet these requirements can follow the steps below to make it\n> diff --git a/Documentation/technical/rust-support.adoc b/Documentation/technical/rust-support.adoc\n> new file mode 100644\n> index 000000000000..a63327ebc575\n> --- /dev/null\n> +++ b/Documentation/technical/rust-support.adoc\n> @@ -0,0 +1,119 @@\n> +Usage of Rust in Git\n> +====================\n> +\n> +Objective\n> +---------\n> +Introduce Rust into Git incrementally to improve security and maintainability.\n> +\n> +Background\n> +----------\n> +Git has historically been written primarily in C, with some portions in shell,\n> +Perl, or other languages. At the time it was originally written, this was\n> +important for portability and was a logical choice for software development.\n> +\n> +:0: link:https://security.googleblog.com/2024/09/eliminating-memory-safety-vulnerabilities-Android.html\n> +:1: link:https://www.cisa.gov/resources-tools/resources/product-security-bad-practices\n> +\n> +However, as time has progressed, we've seen an increased concern with memory\n> +safety vulnerabilities and the development of newer languages, such as Rust,\n> +that substantially limit or eliminate this class of vulnerabilities.\n> +Development in a variety of projects has found that memory safety\n> +vulnerabilities constitute about 70% of vulnerabilities of software in\n> +languages that are not memory safe. For instance, {0}[one survey of Android]\n> +found that memory safety vulnerabilities decreased from 76% to 24% over six\n> +years due to an increase in memory safe code. Similarly, the U.S. government\n> +is {1}[proposing to classify development in memory unsafe languages as a\n> +Product Security Bad Practice\"].\n> +\n> +These risks are even more substantial when we consider the fact that Git is a\n> +network-facing service. Many organizations run Git servers internally or use a\n> +cloud-based forge, and the risk of accidental exposure or compromise of user\n> +data is substantial. It's important to ensure that Git, whether it's used\n> +locally or remotely, is robustly secure.\n> +\n> +In addition, C is a difficult language to write well and concisely. While it\n> +is of course possible to do anything with C, it lacks built-in support for\n> +niceties found in modern languages, such as hash tables, generics, typed\n> +errors, and automatic destruction, and most modern language offer shorter, more\n> +ergonomic syntax for expressing code. This is valuable functionality that can\n> +allow Git to be developed more rapidly, more easily, by more developers of a\n> +variety of levels, and with more confidence in the correctness of the code.\n> +\n> +For these reasons, adding Rust to Git is a sensible and prudent move that will\n> +allow us to improve the quality of the code and potentially attract new developers.\n> +\n> +Goals\n> +-----\n> +1. Git continues to build, run, and pass tests on a wide variety of operating\n> + systems and architectures.\n> +2. Transition from C to Rust is incremental; that is, code can be ported as it\n> + is convenient and Git does not need to transition all at once.\n> +3. Git continues to support older operating systems in conformance with the\n> + platform support policy.\n> +\n> +Non-Goals\n> +---------\n> +1. Support for every possible operating system and architecture. Git already\n> + has a platform support policy which defines what is supported and we already\n> + exclude some operating systems for various reasons (e.g., lacking enough POSIX\n> + tools to pass the test suite).\n> +2. Implementing C-only versions of Rust code or compiling a C-only Git. This\n> + would be difficult to maintain and would not offer the ergonomic benefits we\n> + desire.\n> +\n> +Design\n> +------\n> +Git will adopt Rust incrementally. This transition will start with the\n> +creation of a static library that can be linked into the existing Git binaries.\n> +At some point, we may wish to expose a dynamic library and compile the Git\n> +binaries themselves using Rust. Using an incremental approach allows us to\n> +determine as we go along how to structure our code in the best way for the\n> +project and avoids the need to make hard, potentially disruptive, transitions\n> +caused by porting a binary wholesale from one language to another that might\n> +introduce bugs.\n> +\n> +We will use the `bindgen` and `cbindgen` crates for handling C-compatible\n> +bindings and the `rustix` crate for POSIX-compatible interfaces. The `libc`\n> +crate, which is used by `rustix`, does not expose safe interfaces and does not\n> +handle differences between platforms, such as differing 64-bit `stat` call\n> +names, and so is less desirable as a target than `rustix`. We may still choose\n> +to use it in some cases if `rustix` does not offer suitable interfaces.\n> +\n> +Rust upstream releases every six weeks and only supports the latest stable\n> +release. While it is nice that upstream is active, we would like our software\n> +releases to have a lifespan exceeding six weeks. To allow compiling our code\n> +on a variety of systems, we will support the version of Rust in Debian stable,\n> +plus, for a year after a new Debian stable is released, the version in Debian\n> +oldstable.\n> +\n> +This provides an approximately three-year lifespan of support for a Rust\n> +release and allows us to support a variety of operating systems and\n> +architectures, including those for which Rust upstream does not build binaries.\n> +Debian stable is the benchmark distribution used by many Rust projects when\n> +determining supported Rust versions, and it is an extremely portable and\n> +popular free software operating system that is available to the public at no\n> +charge, which makes it a sensible choice for us as well.\n> +\n> +We may change this policy if the Rust project issues long-term support releases\n> +or the Rust community and distributors agree on releases to target as if they\n> +were long-term support releases.\n> +\n> +This version support policy necessitates that we be very careful about the\n> +dependencies we include, since many Rust projects support only the latest\n> +stable version. However, we typically have been careful about dependencies in\n> +the first place, so this should not be a major departure from existing policy,\n> +although it may be a change for some existing Rust developers.\n> +\n> +We will avoid including the `Cargo.lock` file in the repository and instead\n> +specify minimum dependency versions in the `Cargo.toml` file. We want to allow\n> +people to use newer versions of dependencies if necessary to support newer\n> +platforms without needing to force upgrades of dependencies on all users, and\n> +it provides additional flexibility for distribution maintainers.\n> +\n> +We do not plan to support beta or nightly versions of the Rust compiler. These\n> +versions may change rapidly and especially parts of the toolchain such as\n> +Clippy, the lint tool, can have false positives or add additional warnings with\n> +too great of a frequency to be supportable by the project. However, we do plan\n> +to support alternate compilers, such as the rust_codegen_gcc backend and gccrs\n> +when they are stable and support our desired release versions. This will\n> +provide greater support for more operating systems and architectures.\n> -- \n> gitgitgadget\n\nbest regards\n\nMatthias\n"},{"id":"524270","messageId":"DB9P250MB0692900F30A3E71E4F01DFFFA534A@DB9P250MB0692.EURP250.PROD.OUTLOOK.COM","threadId":"63804","inReplyTo":"96041a10d545e0e431d05b93544771c6bdfc06f1.1755220973.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v2 09/17] Do support Windows again after requiring Rust","fromName":"Matthias Aßhauer","fromEmail":"mha1993@live.de","sentAt":"2025-08-15T17:12:56Z","receivedAt":"2025-08-15T17:13:13Z","isPatch":true,"sender":{"key":"mha1993@live.de","avatar":"https://avatars.githubusercontent.com/u/6178234?v=4"},"body":"\n\nOn Fri, 15 Aug 2025, Johannes Schindelin via GitGitGadget wrote:\n\n> From: Johannes Schindelin \n>\n> By default, Rust wants to build MS Visual C-compatible libraries on\n> Windows, because that is _the_ native C compiler.\n>\n> Git is historically lacking in its MSVC support, and the official Git\n> for Windows versions are built using GCC instead. As a consequence, a\n> (subset of a) GCC toolchain is installed as part of the `windows-build`\n> job of every CI build.\n>\n> Naturally, this requires adjustments in how Rust is called, most\n> importantly it requires installing support for a GCC-compatible build\n> target.\n>\n> Let's make the necessary adjustment both in the CI-specific code that\n> installs Rust as well as in the Windows-specific configuration in\n> `config.mak.uname`.\n>\n> Signed-off-by: Johannes Schindelin \n> [en: Moved lib userenv handling to a later patch]\n> Signed-off-by: Ezekiel Newren \n> ---\n> ci/install-rust.sh | 3 +++\n> config.mak.uname | 7 +++++++\n> 2 files changed, 10 insertions(+)\n>\n> diff --git a/ci/install-rust.sh b/ci/install-rust.sh\n> index 141ceddb17cf..c22baa629ceb 100644\n> --- a/ci/install-rust.sh\n> +++ b/ci/install-rust.sh\n> @@ -28,6 +28,9 @@ if [ \"$BITNESS\" = \"32\" ]; then\n> $CARGO_HOME/bin/rustup default --force-non-host $RUST_VERSION || exit $?\n> else\n> $CARGO_HOME/bin/rustup default $RUST_VERSION || exit $?\n> + if [ \"$CI_OS_NAME\" = \"windows\" ]; then\n> + $CARGO_HOME/bin/rustup target add x86_64-pc-windows-gnu || exit $?\n> + fi\n> fi\n>\n> . $CARGO_HOME/env\n> diff --git a/config.mak.uname b/config.mak.uname\n> index 3e26bb074a4b..a22703284b56 100644\n> --- a/config.mak.uname\n> +++ b/config.mak.uname\n> @@ -727,19 +727,26 @@ ifeq ($(uname_S),MINGW)\n> \t\tprefix = /mingw32\n> \t\tHOST_CPU = i686\n> \t\tBASIC_LDFLAGS += -Wl,--pic-executable,-e,_mainCRTStartup\n> +\t\tCARGO_BUILD_TARGET = i686-pc-windows-gnu\n> endif\n> ifeq (MINGW64,$(MSYSTEM))\n> \t\tprefix = /mingw64\n> \t\tHOST_CPU = x86_64\n> \t\tBASIC_LDFLAGS += -Wl,--pic-executable,-e,mainCRTStartup\n> +\t\tCARGO_BUILD_TARGET = x86_64-pc-windows-gnu\n\nI've said it when Johannes originally sent this patch[1], but it bears \nrepeating: The *-pc-windows-gnu targets will pass CI, but would mean \nraising the required Windows version from 8.1 to 10. We'd want to use\nthe *-win7-windows-gnu targets[2] to keep Windows 8.1 supported.\n\n[1] \nhttps://lore.kernel.org/git/pull.1980.git.git.1752784344.gitgitgadget@gmail.com/T/#ma10be2ed0a0e776b0af2fdd0de63d51ba51609e4\n[2] \nhttps://doc.rust-lang.org/nightly/rustc/platform-support/win7-windows-gnu.html\n\n> else ifeq (CLANGARM64,$(MSYSTEM))\n> \t\tprefix = /clangarm64\n> \t\tHOST_CPU = aarch64\n> \t\tBASIC_LDFLAGS += -Wl,--pic-executable,-e,mainCRTStartup\n> +\t\tCARGO_BUILD_TARGET = aarch64-pc-windows-gnu\n\nAs I've also mentioned before [1], this target doesn't seem to exist. The \ncorrect target seems to be aarch64-pc-windows-gnullvm. [3]\n\n[3] https://doc.rust-lang.org/rustc/platform-support/windows-gnullvm.html\n\n> else\n> \t\tCOMPAT_CFLAGS += -D_USE_32BIT_TIME_T\n> \t\tBASIC_LDFLAGS += -Wl,--large-address-aware\n> endif\n> +\n> +\texport CARGO_BUILD_TARGET\n> +\tRUST_TARGET_DIR = rust/target/$(CARGO_BUILD_TARGET)/$(RUST_BUILD_MODE)\n> +\n> \tCC = gcc\n> \tCOMPAT_CFLAGS += -D__USE_MINGW_ANSI_STDIO=0 -DDETECT_MSYS_TTY \\\n> \t\t-fstack-protector-strong\n> -- \n> gitgitgadget\n\nBest regards\n\nMatthias\n"},{"id":"524279","messageId":"xmqqcy8wnuga.fsf@gitster.g","threadId":"63804","inReplyTo":"DB9P250MB06923B01AACB69F02170B1E3A534A@DB9P250MB0692.EURP250.PROD.OUTLOOK.COM","subject":"Re: [PATCH v2 01/17] doc: add a policy for using Rust","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-08-15T21:31:01Z","receivedAt":"2025-08-15T21:31:05Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Matthias Aßhauer writes:\n\n>> diff --git a/Documentation/technical/platform-support.adoc b/Documentation/technical/platform-support.adoc\n>> index 0a2fb28d6277..42b04b186105 100644\n>> --- a/Documentation/technical/platform-support.adoc\n>> +++ b/Documentation/technical/platform-support.adoc\n>> @@ -33,6 +33,8 @@ meet the following minimum requirements:\n>>\n>> * Has active security support (taking security releases of dependencies, etc)\n>>\n>> +* Supports Rust and the toolchain version specified in link:rust-support.txt[].\n>\n> s/rust-support.txt/rust-support.adoc/\n\nYour review is very much appreciated, but ...\n\n>> +\n>> These requirements are a starting point, and not sufficient on their own for the\n>> Git community to be enthusiastic about supporting your platform. Maintainers of\n>> platforms which do meet these requirements can follow the steps below to make it\n\n...could you trim your quotes to relevant parts that is needed to\nhelp readers understand the point? It is a bit brutal to force\nreaders wade through 200 lines of text only to find this \"you got\n.txt suffix for a document with .adoc suffix\" comment.\n\nThanks.\n"},{"id":"524280","messageId":"xmqq349sntms.fsf@gitster.g","threadId":"63804","inReplyTo":"DB9P250MB0692900F30A3E71E4F01DFFFA534A@DB9P250MB0692.EURP250.PROD.OUTLOOK.COM","subject":"Re: [PATCH v2 09/17] Do support Windows again after requiring Rust","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-08-15T21:48:43Z","receivedAt":"2025-08-15T21:48:47Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Matthias Aßhauer writes:\n\n>> ifeq (MINGW64,$(MSYSTEM))\n>> \t\tprefix = /mingw64\n>> \t\tHOST_CPU = x86_64\n>> \t\tBASIC_LDFLAGS += -Wl,--pic-executable,-e,mainCRTStartup\n>> +\t\tCARGO_BUILD_TARGET = x86_64-pc-windows-gnu\n>\n> I've said it when Johannes originally sent this patch[1], but it bears\n> repeating: The *-pc-windows-gnu targets will pass CI, but would mean\n> raising the required Windows version from 8.1 to 10. We'd want to use\n> the *-win7-windows-gnu targets[2] to keep Windows 8.1 supported.\n\nIt seems that Dscho did not respond on the list to your initial\nobjection in the discussion you cited.\n\nI do not think we spell out which releases of various platforms are\nstill supported by us (we do list requirements for platforms in the\nPlatform Support Policy document, though), but in general we should\nnot be attempting to give extended support to systems that the\nvendor no longer supports. As Windows 8.1 is no longer supported by\nMicrosoft since Jan 2023, and Windows 10 will go out of support in a\nfew month after Oct 2025, if I am reading the table correctly, so as\nlong as we document our intention of dropping a commercial system\nthat is no longer supported by its vender clearly, I do not mind the\nabove that discards 8.1 [*].\n\nBut I may be biased, as I do not live in the Microsoft ecosystem.\n\n\n* https://learn.microsoft.com/en-us/lifecycle/products/windows-81\n* https://learn.microsoft.com/en-us/lifecycle/products/windows-10-home-and-pro\n* https://learn.microsoft.com/en-us/lifecycle/products/windows-10-enterprise-and-education\n"},{"id":"524282","messageId":"2ce3f7ee-62d0-9ddc-761e-31dc30109db5@gmx.de","threadId":"63804","inReplyTo":"xmqq349sntms.fsf@gitster.g","subject":"Re: [PATCH v2 09/17] Do support Windows again after requiring Rust","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2025-08-15T22:11:46Z","receivedAt":"2025-08-15T22:12:09Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Fri, 15 Aug 2025, Junio C Hamano wrote:\n\n> Matthias Aßhauer writes:\n> \n> >> ifeq (MINGW64,$(MSYSTEM))\n> >> \t\tprefix = /mingw64\n> >> \t\tHOST_CPU = x86_64\n> >> \t\tBASIC_LDFLAGS += -Wl,--pic-executable,-e,mainCRTStartup\n> >> +\t\tCARGO_BUILD_TARGET = x86_64-pc-windows-gnu\n> >\n> > I've said it when Johannes originally sent this patch[1], but it bears\n> > repeating: The *-pc-windows-gnu targets will pass CI, but would mean\n> > raising the required Windows version from 8.1 to 10. We'd want to use\n> > the *-win7-windows-gnu targets[2] to keep Windows 8.1 supported.\n> \n> It seems that Dscho did not respond on the list to your initial\n> objection in the discussion you cited.\n\nI would have hoped that it is clear by now that Matthias is as much to be\ntrusted with Git for Windows concerns as I am (just like the other active\nGit for Windows contributors, if you can get them onto this here mailing\nlist). Just in case that it really needs my explicit ACK: What he said\nabout Windows 8.1 support in Git for Windows is accurate.\n\n> I do not think we spell out which releases of various platforms are\n> still supported by us (we do list requirements for platforms in the\n> Platform Support Policy document, though), but in general we should\n> not be attempting to give extended support to systems that the\n> vendor no longer supports. As Windows 8.1 is no longer supported by\n> Microsoft since Jan 2023, and Windows 10 will go out of support in a\n> few month after Oct 2025, if I am reading the table correctly, so as\n> long as we document our intention of dropping a commercial system\n> that is no longer supported by its vender clearly, I do not mind the\n> above that discards 8.1 [*].\n\nWhile there is obviously some connection between the official EOL of\nWindows versions (see https://endoflife.date/windows) and which versions\nGit for Windows supports, the balance we try to strike (and by \"we\" I\ndon't apply the pluralis majestatis, it is very much a consensus between\nall active Git for Windows contributors, including Matthias and myself) is\nto support older Windows versions as much as can be done with a reasonable\namount of effort (where \"reasonable\" is obviously as subjective as the\ndefinition of \"taste\").\n\nThe consensus of what Windows versions can be reasonably supported is\ndocumented at https://gitforwindows.org/requirements.html#windows-version.\nCurrently that is — you may have guessed it — as Matthias has stated: Git\nfor Windows will support Windows 8.1 for the time being. The hope is that\nwe will be able to notify users when support for that version will be\nphased out well in advance, much as we did for Windows 7 and 8, where\ndeprecation notices were included in the release notes of several Git for\nWindows versions prior to v2.46.2, which was the last Git for Windows\nversion to support Windows 7 and 8.\n\n> But I may be biased, as I do not live in the Microsoft ecosystem.\n\nYou do point that out frequently, so I believe that you made the point.\n\nPersonally, I would like to see a more open-minded approach here.\n\nCiao,\nJohannes\n"},{"id":"524283","messageId":"xmqq349sma0s.fsf@gitster.g","threadId":"63804","inReplyTo":"xmqq349sntms.fsf@gitster.g","subject":"Re: [PATCH v2 09/17] Do support Windows again after requiring Rust","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-08-15T23:37:39Z","receivedAt":"2025-08-15T23:37:43Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Junio C Hamano writes:\n\n> I do not think we spell out which releases of various platforms are\n> still supported by us (we do list requirements for platforms in the\n> Platform Support Policy document, though), but in general we should\n> not be attempting to give extended support to systems that the\n> vendor no longer supports.\n> ... so as\n> long as we document our intention of dropping a commercial system\n> that is no longer supported by its vender clearly, I do not mind the\n> above that discards 8.1 [*].\n\nApologies to authors of the PSP document. We do have this as part\nof \"minimum requirement\":\n\n * Has active security support (taking security releases of dependencies, etc)\n\nSo, being implicit about dropping 8.1, while it may be less than\nnice as we could, is perhaps fine.\n\nIf we wanted to support a tad older releases that are still used\nwidely, that is fine as well. I didn't check what additional\ndocuments and policies GfW (Git for Windows) project says about this\nissue, so perhaps it is all documented there, in which case our PSP\ndocument is fine as-is, too. In other words, if GfW project takes\nresponsibility of supporting ports for an older release that is out\nof support, what our PSP document says does not matter.\n\n"},{"id":"524284","messageId":"xmqqy0rkkvg9.fsf@gitster.g","threadId":"63804","inReplyTo":"2ce3f7ee-62d0-9ddc-761e-31dc30109db5@gmx.de","subject":"Re: [PATCH v2 09/17] Do support Windows again after requiring Rust","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-08-15T23:37:42Z","receivedAt":"2025-08-15T23:37:45Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Johannes Schindelin writes:\n\n>> But I may be biased, as I do not live in the Microsoft ecosystem.\n>\n> You do point that out frequently, so I believe that you made the point.\n>\n> Personally, I would like to see a more open-minded approach here.\n\nI gave it as an explanation for the reason why my conclusion may be\ndifferent from what those in the Microsoft ecosystem decided to keep\nsupporting, and I have no reason to object what they want to do.\n\nIt has nothing to do with open-mindedness and such a comment was\nuncalled for.\n"},{"id":"524288","messageId":"DB9P250MB06923623511AA4E1361F1600A537A@DB9P250MB0692.EURP250.PROD.OUTLOOK.COM","threadId":"63804","inReplyTo":"xmqqcy8wnuga.fsf@gitster.g","subject":"Re: [PATCH v2 01/17] doc: add a policy for using Rust","fromName":"Matthias Aßhauer","fromEmail":"mha1993@live.de","sentAt":"2025-08-16T08:06:57Z","receivedAt":"2025-08-16T08:07:08Z","isPatch":true,"sender":{"key":"mha1993@live.de","avatar":"https://avatars.githubusercontent.com/u/6178234?v=4"},"body":"\n\nOn Fri, 15 Aug 2025, Junio C Hamano wrote:\n\n> ...could you trim your quotes to relevant parts that is needed to\n> help readers understand the point? It is a bit brutal to force\n> readers wade through 200 lines of text only to find this \"you got\n> .txt suffix for a document with .adoc suffix\" comment.\n>\n> Thanks.\n>\n\nSorry, I'll try to be more mindful of trimming my mails.\n\nBest regards\n\nMatthias\n"},{"id":"524289","messageId":"DB9P250MB0692F39EA259A4B31845C6B2A537A@DB9P250MB0692.EURP250.PROD.OUTLOOK.COM","threadId":"63804","inReplyTo":"xmqq349sntms.fsf@gitster.g","subject":"Re: [PATCH v2 09/17] Do support Windows again after requiring Rust","fromName":"Matthias Aßhauer","fromEmail":"mha1993@live.de","sentAt":"2025-08-16T08:53:59Z","receivedAt":"2025-08-16T08:54:08Z","isPatch":true,"sender":{"key":"mha1993@live.de","avatar":"https://avatars.githubusercontent.com/u/6178234?v=4"},"body":"\n\nOn Fri, 15 Aug 2025, Junio C Hamano wrote:\n\n> Matthias Aßhauer writes:\n>\n>>> ifeq (MINGW64,$(MSYSTEM))\n>>> \t\tprefix = /mingw64\n>>> \t\tHOST_CPU = x86_64\n>>> \t\tBASIC_LDFLAGS += -Wl,--pic-executable,-e,mainCRTStartup\n>>> +\t\tCARGO_BUILD_TARGET = x86_64-pc-windows-gnu\n>>\n>> I've said it when Johannes originally sent this patch[1], but it bears\n>> repeating: The *-pc-windows-gnu targets will pass CI, but would mean\n>> raising the required Windows version from 8.1 to 10. We'd want to use\n>> the *-win7-windows-gnu targets[2] to keep Windows 8.1 supported.\n>\n> It seems that Dscho did not respond on the list to your initial\n> objection in the discussion you cited.\n\nHe didn't, but from various interactions surrounding Git for Windows, I do \nthink he's currently in favour of keeping Windows 8.1 supported in Git for \nWindows.\n\n> I do not think we spell out which releases of various platforms are\n> still supported by us (we do list requirements for platforms in the\n> Platform Support Policy document, though),\n\nWe don't do that in git.git, no. Git for Windows very explicitly spells \nout which versions of Windows are supported (though usually we just \nmention the Desktop versions and imply the corresponding Windows Server \nversions). Since 2.47.0 that is Windows 8.1 and newer Desktop releases [1] \n(Windows 11 on ARM64). We even tend to announce in advance when we intend \nto drop support for a Windows version.\n\n[1] \nhttps://gitforwindows.org/faq.html#which-versions-of-windows-are-supported\n\n> but in general we should not be attempting to give extended support to\n> systems that the vendor no longer supports. As Windows 8.1 is no longer\n> supported by Microsoft since Jan 2023, and Windows 10 will go out of\n> support in a few month after Oct 2025, if I am reading the table correctly,\n\nGit for Windows has historically supported Windows Versions beyond this \ndate.\n\n* XP was supported for 2 years beyond the official extended EOL. [1][2]\n* Vista was supported for 5 years beyond the official extended EOL [1][3]\n* 7 was supported for 4 years beyond the official extended EOL [1][4]\n* 8 was supported for 8 years beyond the official extended EOL [1][5]\n\ngit.git has historically roughly followed Git for Windows in this.\n\nYou're reading the tables correctly, but there are so called LTSC releases \nof Windows 10 with support until 2026/2029. [6]\n\n[2] https://learn.microsoft.com/en-us/lifecycle/products/windows-xp\n[3] https://learn.microsoft.com/en-us/lifecycle/products/windows-vista\n[4] https://learn.microsoft.com/en-us/lifecycle/products/windows-7\n[5] https://learn.microsoft.com/en-us/lifecycle/products/windows-8\n[6] \nhttps://learn.microsoft.com/en-us/windows/whats-new/ltsc/whats-new-windows-10-2021#lifecycle\n\n> so as long as we document our intention of dropping a commercial system\n> that is no longer supported by its vender clearly, I do not mind the\n> above that discards 8.1 [*].\n\nI'm not completely opposed, but I do think it should be a concious \ndecision and not an unintended side effect of some change that our CI\ndidn't catch.\n\nBest regards\n\nMatthias"},{"id":"524320","messageId":"xmqqjz32kkjx.fsf@gitster.g","threadId":"63804","inReplyTo":"DB9P250MB0692F39EA259A4B31845C6B2A537A@DB9P250MB0692.EURP250.PROD.OUTLOOK.COM","subject":"Re: [PATCH v2 09/17] Do support Windows again after requiring Rust","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-08-17T15:57:38Z","receivedAt":"2025-08-17T15:57:42Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Matthias Aßhauer writes:\n\n>> It seems that Dscho did not respond on the list to your initial\n>> objection in the discussion you cited.\n>\n> He didn't, but from various interactions surrounding Git for Windows,\n> I do think he's currently in favour of keeping Windows 8.1 supported\n> in Git for Windows.\n\nOK, as long as folks with stakes in Git for Windows are in\nagreement, I have no problem (except that in principle we should\navoid doing disservice to the end user population by doing things\nthat encourage their prolonged use of out-of-security-support\nplatforms).\n\n> We don't do that in git.git, no. Git for Windows very explicitly\n> spells out which versions of Windows are supported (though usually we\n> just mention the Desktop versions and imply the corresponding Windows\n> Server versions).\n\nYup, thanks for clarifying it to me. Could you do the same for\nfuture readers of the updated version of the commit 09/17 by telling\nthe author about that in your review comment, so that the log\nmessage can talk about the reasons why a specific CARGO_BUILD_TARGET\nwas chosen (e.g. \"as described in Git for Windows documentation at\n$URL, we support Windows versions X or newer, so we use this cargo\nbuild target to ensure we still work with that version\").\n\n> I'm not completely opposed, but I do think it should be a concious\n> decision and not an unintended side effect of some change that our CI\n> didn't catch.\n\nOh, absolutely.\n\nThanks for clarification.\n"},{"id":"524375","messageId":"xmqqldnggt2v.fsf@gitster.g","threadId":"63804","inReplyTo":"pull.1980.v2.git.git.1755220973.gitgitgadget@gmail.com","subject":"Re: [PATCH v2 00/17] RFC: Accelerate xdiff and begin its rustification","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-08-18T22:31:36Z","receivedAt":"2025-08-18T22:31:41Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Ezekiel Newren via GitGitGadget\" writes:\n\n> * Code style: Should we adopt a Rust code style of some sort? Perhaps have\n> the code always be formatted by rustfmt in its default configuration?\n\nSounds sensible. I'll let folks with more Rust inclination to\nfigure out what _the_ style should be, but having _a_ style we all\nstick to is good.\n\n> * Rust version: We are not using the same Rust version on all platforms in\n> CI; 32-bit builds and Windows builds require a newer Rust version to\n> successfully build.\n\nAs long as we do not have to bend backwards on the code with \"if\nusing version X or older, use this alternative codepath\" all over\nthe place, \"pick a version that works on each platform\" that results\nin \"due to the quality of ports, some platform's older port is\nunusable and newer version is required\" is not too bad, especially\nfor a system that is still rapidly getting improved and a bit on the\nunstable side, I think.\n\n> * Performance with whitepsace flags: I originally intended to leave out the\n> whitespace handling because I knew it was slower,...\n\nIf the Rust guinea pig were different from how each line is hashed\nin xdiff, which is targetted by am/xdiff-hash-tweak topic, then we\ncan leave out the whitespace-ignoring hashing from this topic\naltogether.\n\nQuite honestly, I do not like throwing away the other optimization\nefforts that can be reviewed and integrated trivially, but it is\npractically impossible to do so while still have a \"let's start\nplaying with Rust\" topic that targets exactly the same area. Yes,\nthis topic licked the same corner of the cake first, but still,\nI was hoping that the second iteration of this series would use a\ndifferent code paths as a Rust guinea pig.\n\nAfter all, the primary objective of our first Rust topic is to set\nthe framework right (like the platform and version support policies,\nhow foreign interfaces like type systems get impedance-matched, what\nthe impact to our build infrature looks like, etc.). It would be a\nhuge plus if it can at the same time demonstrate how much safer code\nwe can write with less effort if we switched writing some (and\ngradually larger, posibly) parts in the language.\n\nThe result this cover letter has in its title, 'accelerate xdiff',\nis not primarily due to use of Rust, is it? As the other topic\ndemonstrates, it is to use an implementation of a faster hash\nfunction (we can consider it to be an impressive technology\ndemonstration that a rust reimplementation of original C code can be\ndone in a very performant way). And nobody is expecting that we\nwould be using Rust for speed anyway, no?\n"},{"id":"524379","messageId":"2FE193EB-125F-443E-926C-E9460A1CD5BD@gmail.com","threadId":"63804","inReplyTo":"xmqqldnggt2v.fsf@gitster.g","subject":"Re: [PATCH v2 00/17] RFC: Accelerate xdiff and begin its rustification","fromName":"Ben Knoble","fromEmail":"ben.knoble@gmail.com","sentAt":"2025-08-18T23:52:13Z","receivedAt":"2025-08-18T23:52:25Z","isPatch":true,"sender":{"key":"ben.knoble@gmail.com","avatar":"https://avatars.githubusercontent.com/u/22802209?v=4"},"body":"\n> Le 18 août 2025 à 18:31, Junio C Hamano a écrit :\n> \n> \"Ezekiel Newren via GitGitGadget\" writes:\n> \n>> * Code style: Should we adopt a Rust code style of some sort? Perhaps have\n>> the code always be formatted by rustfmt in its default configuration?\n> \n> Sounds sensible. I'll let folks with more Rust inclination to\n> figure out what _the_ style should be, but having _a_ style we all\n> stick to is good.\n\nWhile there can be room for configuring the formatter if we have particularly idiosyncratic needs, I’d second going with cargo fmt / rustfmt in default configurations to start (the former is a shortcut for the latter over a whole crate AIUI)."},{"id":"524382","messageId":"CABPp-BGvQdrft62S_0_-pdReZCV_rdy=2X0Uebi4oa+-emW6mw@mail.gmail.com","threadId":"63804","inReplyTo":"xmqqldnggt2v.fsf@gitster.g","subject":"Re: [PATCH v2 00/17] RFC: Accelerate xdiff and begin its rustification","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2025-08-19T01:52:06Z","receivedAt":"2025-08-19T01:52:18Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"On Mon, Aug 18, 2025 at 3:31 PM Junio C Hamano wrote:\n>\n> \"Ezekiel Newren via GitGitGadget\" writes:\n>\n> > * Code style: Should we adopt a Rust code style of some sort? Perhaps have\n> > the code always be formatted by rustfmt in its default configuration?\n>\n> Sounds sensible. I'll let folks with more Rust inclination to\n> figure out what _the_ style should be, but having _a_ style we all\n> stick to is good.\n>\n> > * Rust version: We are not using the same Rust version on all platforms in\n> > CI; 32-bit builds and Windows builds require a newer Rust version to\n> > successfully build.\n>\n> As long as we do not have to bend backwards on the code with \"if\n> using version X or older, use this alternative codepath\" all over\n> the place, \"pick a version that works on each platform\" that results\n> in \"due to the quality of ports, some platform's older port is\n> unusable and newer version is required\" is not too bad, especially\n> for a system that is still rapidly getting improved and a bit on the\n> unstable side, I think.\n>\n> > * Performance with whitepsace flags: I originally intended to leave out the\n> > whitespace handling because I knew it was slower,...\n>\n> If the Rust guinea pig were different from how each line is hashed\n> in xdiff, which is targetted by am/xdiff-hash-tweak topic, then we\n> can leave out the whitespace-ignoring hashing from this topic\n> altogether.\n>\n> Quite honestly, I do not like throwing away the other optimization\n> efforts that can be reviewed and integrated trivially, but it is\n> practically impossible to do so while still have a \"let's start\n> playing with Rust\" topic that targets exactly the same area. Yes,\n> this topic licked the same corner of the cake first, but still,\n> I was hoping that the second iteration of this series would use a\n> different code paths as a Rust guinea pig.\n\nI agree it would be nice to merge those down first. One possibility\nhere would be having Ezekiel rebase his work on top of\nam/xdiff-hash-tweak...\n\n> After all, the primary objective of our first Rust topic is to set\n> the framework right (like the platform and version support policies,\n> how foreign interfaces like type systems get impedance-matched, what\n> the impact to our build infrature looks like, etc.). It would be a\n> huge plus if it can at the same time demonstrate how much safer code\n> we can write with less effort if we switched writing some (and\n> gradually larger, posibly) parts in the language.\n>\n> The result this cover letter has in its title, 'accelerate xdiff',\n> is not primarily due to use of Rust, is it? As the other topic\n> demonstrates, it is to use an implementation of a faster hash\n> function (we can consider it to be an impressive technology\n> demonstration that a rust reimplementation of original C code can be\n> done in a very performant way). And nobody is expecting that we\n> would be using Rust for speed anyway, no?\n\nYou are correct that it is not due to Rust. My original objective\nthat I tried to trick/coax/nerd-snipe/whatever Ezekiel into looking\ninto was cleaning up xdiff, to allow various features in `git replay`\nand rebasing merges. Ezekiel was interested in Rust and in a\nchallenge. xdiff is quite a knot to untangle, and Ezekiel's been at\nwork on it for quite some time. But, he happened to notice this\nspeedup, and found a way to turn it into a short series without all\nhis other patches.\n\nWe could perhaps shift focus, but I'm curious if you're wanting the\nxdiff work to be thrown away or shelved in favor of some completely\ndifferent area of the code, or if perhaps some other aspects of the\nxdiff code would still be amenable.\n\nOne big challenge in finding another area, whether in xdiff or\nelsewhere, is that Ezekiel really wants to showcase how nice Rust's\nunittesting is, but that only works if we start at a low-level and\nbuild up. If we make Rust code call C code, that'd either not be\nreadily unit-testable, or would require us stubbing out the entire\nimplementation behind that C interface or doing something more\ncomplex.\n\nWhat if Ezekiel rebased his series on am/xdiff-hash-tweak, and then\ninstead of further modifying the hashing in the first series, he:\n - introduced brian's patch with the platform support\n - setup the CI builds to test building with Rust (including Johannes' patches)\n - started working on transitioning xdfile_t data structure to be FFI friendly\n\nOne issue here is that it probably wouldn't be too long before we'd\nwant to rip out the xdlclassifier struct (mostly a glorified\nhashtable), which is kind of tied up in a knot with the hashing and\nline equality, so it would probably only be a few more series down the\nroad before we'd want to start tweaking the code in\nam/xdiff-hash-tweak to make use of the new data structures. Would\nthat be agreeable?\n"},{"id":"524384","messageId":"CABPp-BEOVBUa7_sTJybgFsgcwAUMeFFhNJEDVnYp_6TYnqu2rg@mail.gmail.com","threadId":"63804","inReplyTo":"2d5ae8f6-69f1-486b-bd38-337f0b54f737@ramsayjones.plus.com","subject":"Re: [-SPAM-] [PATCH v2 00/17] RFC: Accelerate xdiff and begin its rustification","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2025-08-19T02:00:16Z","receivedAt":"2025-08-19T02:00:28Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"On Fri, Aug 15, 2025 at 8:10 AM Ramsay Jones\n wrote:\n>\n> On 15/08/2025 02:22, Ezekiel Newren via GitGitGadget wrote:\n> > Changes in this second round of this RFC:\n> >\n> > * Now builds and passes tests on all platforms (example run:\n> > https://github.com/ezekielnewren/git/actions/runs/16974821401). Special\n> > thanks to Johannes Schindelin for patches to things for Windows and\n> > linux32.\n>\n> Hmm, builds on *all* platforms may be a bit optimistic (it doesn't on\n> cygwin, for instance), so I'm guessing you mean all platforms which\n> have CI defined. Perhaps you could mention the platforms which you\n> have tested on. :)\n\nEzekiel says this email didn't show up in his inbox (no idea why), but\nyes what was meant was all platforms where gitgitgadget CI runs. If\nyou follow the github.com link in the text that you quoted, you can\nsee all those platforms (various windows flavors, various osx builds,\nmusl, sparse, static analysis, etc.).\n"},{"id":"524385","messageId":"CAH=ZcbBmFGt3hVoPe2NSp=4Ew+kFF0-PWFQFey0fXhYBXB6Yrw@mail.gmail.com","threadId":"63804","inReplyTo":"DB9P250MB06923B01AACB69F02170B1E3A534A@DB9P250MB0692.EURP250.PROD.OUTLOOK.COM","subject":"Re: [PATCH v2 01/17] doc: add a policy for using Rust","fromName":"Ezekiel Newren","fromEmail":"ezekielnewren@gmail.com","sentAt":"2025-08-19T02:06:47Z","receivedAt":"2025-08-19T02:07:00Z","isPatch":true,"sender":{"key":"ezekielnewren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/18313324?v=4"},"body":"On Fri, Aug 15, 2025 at 11:03 AM Matthias Aßhauer wrote:\n>\n>> +* Supports Rust and the toolchain version specified in link:rust-support.txt[].\n>\n> s/rust-support.txt/rust-support.adoc/\n\nThanks for spotting that, I'll fix it up.\n"},{"id":"524386","messageId":"CAH=ZcbDc+Hi28Cu2roE1gezwPWbarxBj6JOjTb2ytnrYS72uTQ@mail.gmail.com","threadId":"63804","inReplyTo":"DB9P250MB0692900F30A3E71E4F01DFFFA534A@DB9P250MB0692.EURP250.PROD.OUTLOOK.COM","subject":"Re: [PATCH v2 09/17] Do support Windows again after requiring Rust","fromName":"Ezekiel Newren","fromEmail":"ezekielnewren@gmail.com","sentAt":"2025-08-19T02:22:30Z","receivedAt":"2025-08-19T02:22:43Z","isPatch":true,"sender":{"key":"ezekielnewren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/18313324?v=4"},"body":"On Fri, Aug 15, 2025 at 11:13 AM Matthias Aßhauer wrote:\n> > diff --git a/ci/install-rust.sh b/ci/install-rust.sh\n> > index 141ceddb17cf..c22baa629ceb 100644\n> > --- a/ci/install-rust.sh\n> > +++ b/ci/install-rust.sh\n> > @@ -28,6 +28,9 @@ if [ \"$BITNESS\" = \"32\" ]; then\n> > $CARGO_HOME/bin/rustup default --force-non-host $RUST_VERSION || exit $?\n> > else\n> > $CARGO_HOME/bin/rustup default $RUST_VERSION || exit $?\n> > + if [ \"$CI_OS_NAME\" = \"windows\" ]; then\n> > + $CARGO_HOME/bin/rustup target add x86_64-pc-windows-gnu || exit $?\n> > + fi\n> > fi\n> >\n> > . $CARGO_HOME/env\n> > diff --git a/config.mak.uname b/config.mak.uname\n> > index 3e26bb074a4b..a22703284b56 100644\n> > --- a/config.mak.uname\n> > +++ b/config.mak.uname\n> > @@ -727,19 +727,26 @@ ifeq ($(uname_S),MINGW)\n> > prefix = /mingw32\n> > HOST_CPU = i686\n> > BASIC_LDFLAGS += -Wl,--pic-executable,-e,_mainCRTStartup\n> > + CARGO_BUILD_TARGET = i686-pc-windows-gnu\n> > endif\n> > ifeq (MINGW64,$(MSYSTEM))\n> > prefix = /mingw64\n> > HOST_CPU = x86_64\n> > BASIC_LDFLAGS += -Wl,--pic-executable,-e,mainCRTStartup\n> > + CARGO_BUILD_TARGET = x86_64-pc-windows-gnu\n>\n> I've said it when Johannes originally sent this patch[1], but it bears\n> repeating: The *-pc-windows-gnu targets will pass CI, but would mean\n> raising the required Windows version from 8.1 to 10. We'd want to use\n> the *-win7-windows-gnu targets[2] to keep Windows 8.1 supported.\n>\n> [1]\n> https://lore.kernel.org/git/pull.1980.git.git.1752784344.gitgitgadget@gmail.com/T/#ma10be2ed0a0e776b0af2fdd0de63d51ba51609e4\n> [2]\n> https://doc.rust-lang.org/nightly/rustc/platform-support/win7-windows-gnu.html\n>\n> > else ifeq (CLANGARM64,$(MSYSTEM))\n> > prefix = /clangarm64\n> > HOST_CPU = aarch64\n> > BASIC_LDFLAGS += -Wl,--pic-executable,-e,mainCRTStartup\n> > + CARGO_BUILD_TARGET = aarch64-pc-windows-gnu\n>\n> As I've also mentioned before [1], this target doesn't seem to exist. The\n> correct target seems to be aarch64-pc-windows-gnullvm. [3]\n>\n> [3] https://doc.rust-lang.org/rustc/platform-support/windows-gnullvm.html\n\nI'll be happy to make that change for the next round.\n"},{"id":"524420","messageId":"xmqqzfbvfxs6.fsf@gitster.g","threadId":"63804","inReplyTo":"CABPp-BGvQdrft62S_0_-pdReZCV_rdy=2X0Uebi4oa+-emW6mw@mail.gmail.com","subject":"Re: [PATCH v2 00/17] RFC: Accelerate xdiff and begin its rustification","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-08-19T09:47:37Z","receivedAt":"2025-08-19T09:47:42Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Elijah Newren writes:\n\n> What if Ezekiel rebased his series on am/xdiff-hash-tweak, and then\n> instead of further modifying the hashing in the first series, he:\n> - introduced brian's patch with the platform support\n> - setup the CI builds to test building with Rust (including Johannes' patches)\n> - started working on transitioning xdfile_t data structure to be FFI friendly\n\nYup, that matches my understanding of what our first Rust topic\nwould want to achieve, i.e. get the framework right.\n\n> One issue here is that it probably wouldn't be too long before we'd\n> want to rip out the xdlclassifier struct (mostly a glorified\n> hashtable), which is kind of tied up in a knot with the hashing and\n> line equality, so it would probably only be a few more series down the\n> road before we'd want to start tweaking the code in\n> am/xdiff-hash-tweak to make use of the new data structures.\n\nI understand that at this point we do not expect to import any\n(security or otherwise) fixes to xdiff code from \"upstream\", as we\nare practically the upstream for other folks? For our consumption,\nthat would allow us to take a quite different stance from our\nhistorical attitude, which was to keep the modification to the\nminimum, and apply whatever clean-ups and optimizations only to suit\nour needs. So what you outline does make certain sense to me.\n\nI am however not sure if we owe anything to our downstream projects,\nthough (e.g., I understand that libgit2 extracted xdiff part from\nour source, so if we have serious security fixes in ours, they would\nwant to be able to import them?).\n\nThanks. \n"},{"id":"524751","messageId":"pull.1980.v3.git.git.1755921356.gitgitgadget@gmail.com","threadId":"63804","inReplyTo":"pull.1980.v2.git.git.1755220973.gitgitgadget@gmail.com","subject":"[PATCH v3 00/15] RFC: Cleanup xdiff and begin its rustification","fromName":"Ezekiel Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2025-08-23T03:55:41Z","receivedAt":"2025-08-23T03:56:02Z","isPatch":true,"sender":{"key":"ezekielnewren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/18313324?v=4"},"body":"In order to facilitate the quick adoption of am/xdiff-hash-tweak, this round\ndrops the changes to hashing in xdiff and instead modifies another part of\nxdiff.\n\nA high level overview of v3:\n\n * patch 1: add a policy for using Rust (brian's patch, with a small tweak)\n * patch 2: introduce Rust to the codebase\n * patches 3-5: adapt CI (github workflows) to build Git with Rust\n * patch 6: introduce the ivec type\n * patches 7-14: xdiff code cleanup in preparation for translating to Rust\n * patch 15: translate a C function into Rust and call it from C\n\nI'm particularly interested in what folks think of the new ivec type for\nsharing data across the language barrier. Thoughts?\n\nBuild results for these changes:\nhttps://github.com/git/git/actions/runs/17170212383\n\nLinks to older versions, which focused on hashing in xdiff:\n\n * v1:\n https://lore.kernel.org/git/pull.1980.git.git.1752784344.gitgitgadget@gmail.com/\n * v2:\n https://lore.kernel.org/git/pull.1980.v2.git.git.1755220973.gitgitgadget@gmail.com/\n\nEzekiel Newren (13):\n xdiff: introduce rust\n github workflows: install rust\n github workflows: upload Cargo.lock\n ivec: create a vector type that is interoperable between C and Rust\n xdiff/xprepare: remove superfluous forward declarations\n xdiff: delete unnecessary fields from xrecord_t and xdfile_t\n xdiff: make fields of xrecord_t Rust friendly\n xdiff: use one definition for freeing xdfile_t\n xdiff: replace chastore with an ivec in xdfile_t\n xdiff: delete nrec field from xdfile_t\n xdiff: delete recs field from xdfile_t\n xdiff: make xdfile_t more rust friendly\n xdiff: implement xdl_trim_ends() in Rust\n\nJohannes Schindelin (1):\n win+Meson: do allow linking with the Rust-built xdiff\n\nbrian m. carlson (1):\n doc: add a policy for using Rust\n\n .github/workflows/main.yml | 89 +++-\n .gitignore | 3 +\n Documentation/Makefile | 1 +\n Documentation/technical/platform-support.adoc | 2 +\n Documentation/technical/rust-support.adoc | 142 ++++++\n Makefile | 69 ++-\n build_rust.sh | 57 +++\n ci/install-dependencies.sh | 14 +-\n ci/install-rust-toolchain.sh | 30 ++\n ci/install-rustup.sh | 25 +\n ci/lib.sh | 1 +\n ci/make-test-artifacts.sh | 9 +\n ci/run-build-and-tests.sh | 13 +\n config.mak.uname | 4 +\n git-compat-util.h | 17 +\n interop/ivec.c | 151 ++++++\n interop/ivec.h | 52 ++\n meson.build | 54 +-\n rust/Cargo.toml | 6 +\n rust/interop/Cargo.toml | 14 +\n rust/interop/src/ivec.rs | 462 ++++++++++++++++++\n rust/interop/src/lib.rs | 10 +\n rust/xdiff/Cargo.toml | 15 +\n rust/xdiff/src/lib.rs | 15 +\n rust/xdiff/src/xprepare.rs | 27 +\n rust/xdiff/src/xtypes.rs | 19 +\n xdiff/xdiffi.c | 60 +--\n xdiff/xdiffi.h | 8 +-\n xdiff/xemit.c | 24 +-\n xdiff/xhistogram.c | 2 +-\n xdiff/xmerge.c | 72 +--\n xdiff/xpatience.c | 16 +-\n xdiff/xprepare.c | 271 ++++------\n xdiff/xtypes.h | 27 +-\n xdiff/xutils.c | 12 +-\n 35 files changed, 1474 insertions(+), 319 deletions(-)\n create mode 100644 Documentation/technical/rust-support.adoc\n create mode 100755 build_rust.sh\n create mode 100755 ci/install-rust-toolchain.sh\n create mode 100755 ci/install-rustup.sh\n create mode 100644 interop/ivec.c\n create mode 100644 interop/ivec.h\n create mode 100644 rust/Cargo.toml\n create mode 100644 rust/interop/Cargo.toml\n create mode 100644 rust/interop/src/ivec.rs\n create mode 100644 rust/interop/src/lib.rs\n create mode 100644 rust/xdiff/Cargo.toml\n create mode 100644 rust/xdiff/src/lib.rs\n create mode 100644 rust/xdiff/src/xprepare.rs\n create mode 100644 rust/xdiff/src/xtypes.rs\n\n\nbase-commit: 16bd9f20a403117f2e0d9bcda6c6e621d3763e77\nPublished-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-1980%2Fezekielnewren%2Fxdiff_rust_speedup-v3\nFetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-1980/ezekielnewren/xdiff_rust_speedup-v3\nPull-Request: https://github.com/git/git/pull/1980\n\nRange-diff vs v2:\n\n 1: 75dfb40ead3 ! 1: 6d065f550fe doc: add a policy for using Rust\n @@ Commit message\n contributors.\n \n Signed-off-by: brian m. carlson \n + [en: Added some comments about types, and changed the recommondations\n + about cbindgen, bindgen, rustix, libc.]\n Signed-off-by: Ezekiel Newren \n \n ## Documentation/Makefile ##\n @@ Documentation/technical/platform-support.adoc: meet the following minimum requir\n \n * Has active security support (taking security releases of dependencies, etc)\n \n -+* Supports Rust and the toolchain version specified in link:rust-support.txt[].\n ++* Supports Rust and the toolchain version specified in link:rust-support.adoc[].\n +\n These requirements are a starting point, and not sufficient on their own for the\n Git community to be enthusiastic about supporting your platform. Maintainers of\n @@ Documentation/technical/rust-support.adoc (new)\n +caused by porting a binary wholesale from one language to another that might\n +introduce bugs.\n +\n -+We will use the `bindgen` and `cbindgen` crates for handling C-compatible\n -+bindings and the `rustix` crate for POSIX-compatible interfaces. The `libc`\n -+crate, which is used by `rustix`, does not expose safe interfaces and does not\n -+handle differences between platforms, such as differing 64-bit `stat` call\n -+names, and so is less desirable as a target than `rustix`. We may still choose\n -+to use it in some cases if `rustix` does not offer suitable interfaces.\n ++Crates like libc or rustix define types like c_long, but in ways that are not\n ++safe across platforms.\n ++From https://docs.rs/rustix/latest/rustix/ffi/type.c_long.html:\n ++\n ++ This type will always be i32 or i64. Most notably, many Linux-based\n ++ systems assume an i64, but Windows assumes i32. The C standard technically\n ++ only requires that this type be a signed integer that is at least 32 bits\n ++ and at least the size of an int, although in practice, no system would\n ++ have a long that is neither an i32 nor i64.\n ++\n ++Also, note that other locations, such as\n ++https://docs.rs/libc/latest/libc/type.c_long.html, just hardcode c_long as i64\n ++even though C may mean i32 on some platforms.\n ++\n ++As such, using the c_long type would give us portability issues, and\n ++perpetuate some of the bugs git has faced across platforms. Avoid using C's\n ++types (long, unsigned, char, etc.), and switch to unambiguous types (e.g. i32\n ++or i64) before trying to make C and Rust interoperate.\n ++\n ++Crates like libc and rustix may have also traditionally aided interoperability\n ++with older versions of Rust (e.g. when worrying about stat[64] system calls),\n ++but the Rust standard library in newer versions of Rust handle these concerns\n ++in a platform agnostic way. There may arise cases where we need to consider\n ++these crates, but for now we omit them.\n ++\n ++Tools like bindgen and cbindgen create C-styled unsafe Rust code rather than\n ++idiomatic Rust; where possible, we prefer to switch to idiomatic Rust. Any\n ++standard C library functions that are needed can be manually wrapped on the\n ++Rust side.\n +\n +Rust upstream releases every six weeks and only supports the latest stable\n +release. While it is nice that upstream is active, we would like our software\n 2: 7709e5eddba < -: ----------- xdiff: introduce rust\n 8: 7dc241e6682 ! 2: 03939951256 github workflows: install rust\n @@ Metadata\n Author: Ezekiel Newren \n \n ## Commit message ##\n - github workflows: install rust\n + xdiff: introduce rust\n \n - Since we have introduced rust, it needs to be installed for the\n - continuous integration build targets. Create an install script\n - (build_rust.sh) that needs to be run as the same user that builds git.\n - Because of the limitations of meson, create build_rust.sh which makes\n - it easy to centralize how rust is built between meson and make.\n + Upcoming patches will simplify xdiff, while also porting parts of it to\n + Rust. In preparation, add some stubs and setup the Rust build. For now,\n + it is easier to let cargo build rust and have make or meson merely link\n + against the static library that cargo builds. In line with ongoing\n + libification efforts, use multiple crates to allow more modularity on\n + the Rust side. xdiff is the crate that this series will focus on, but\n + we also introduce the interop crate for future patch series.\n \n - There are 2 interesting decisions worth calling out in this commit:\n -\n - * The 'output' field of custom_target() does not allow specifying a\n - file nested inside the build directory. Thus create build_rust.sh to\n - build rust with all of its parameters and then moves libxdiff.a to\n - the root of the build directory.\n -\n - * Install curl, to facilitate the rustup install script.\n + In order to facilitate interoperability between C and Rust, introduce C\n + definitions for Rust primitive types in git-compat-util.h.\n \n Signed-off-by: Ezekiel Newren \n \n - ## .github/workflows/main.yml ##\n -@@ .github/workflows/main.yml: on: [push, pull_request]\n - \n - env:\n - DEVELOPER: 1\n -+ RUST_VERSION: 1.87.0\n - \n - # If more than one workflow run is triggered for the very same commit hash\n - # (which happens when multiple branches pointing to the same commit), only\n + ## .gitignore ##\n +@@ .gitignore: Release/\n + /contrib/buildsystems/out\n + /contrib/libgit-rs/target\n + /contrib/libgit-sys/target\n ++/.idea/\n ++/rust/target/\n ++/rust/Cargo.lock\n \n ## Makefile ##\n @@ Makefile: TEST_SHELL_PATH = $(SHELL_PATH)\n @@ Makefile: TEST_SHELL_PATH = $(SHELL_PATH)\n +\n +EXTLIBS =\n +\n - ifeq ($(DEBUG), 1)\n --RUST_LIB = rust/target/debug/libxdiff.a\n ++ifeq ($(DEBUG), 1)\n + RUST_BUILD_MODE = debug\n - else\n --RUST_LIB = rust/target/release/libxdiff.a\n ++else\n + RUST_BUILD_MODE = release\n +endif\n +\n @@ Makefile: TEST_SHELL_PATH = $(SHELL_PATH)\n +UNAME_S := $(shell uname -s)\n +ifeq ($(UNAME_S),Linux)\n + EXTLIBS += -ldl\n - endif\n ++endif\n +\n REFTABLE_LIB = reftable/libreftable.a\n \n @@ Makefile: UNIT_TEST_OBJS += $(UNIT_TEST_DIR)/lib-reftable.o\n # xdiff and reftable libs may in turn depend on what is in libgit.a\n GITLIBS = common-main.o $(LIB_FILE) $(XDIFF_LIB) $(REFTABLE_LIB) $(LIB_FILE)\n -EXTLIBS =\n - \n --GITLIBS += $(RUST_LIB)\n ++\n \n GIT_USER_AGENT = git/$(GIT_VERSION)\n \n @@ Makefile: $(REMOTE_CURL_ALIASES): $(REMOTE_CURL_PRIMARY)\n \t$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) \\\n \t\t$(filter %.o,$^) $(LIBS)\n \n +@@ Makefile: $(LIB_FILE): $(LIB_OBJS)\n + $(XDIFF_LIB): $(XDIFF_OBJS)\n + \t$(QUIET_AR)$(RM) $@ && $(AR) $(ARFLAGS) $@ $^\n + \n ++\n + $(REFTABLE_LIB): $(REFTABLE_OBJS)\n + \t$(QUIET_AR)$(RM) $@ && $(AR) $(ARFLAGS) $@ $^\n + \n @@ Makefile: perf: all\n \n t/helper/test-tool$X: $(patsubst %,t/helper/%,$(TEST_BUILTINS_OBJS)) $(UNIT_TEST_DIR)/test-lib.o\n @@ Makefile: perf: all\n \t$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) $(filter %.a,$^) $(LIBS)\n \n check-sha1:: t/helper/test-tool$X\n +@@ Makefile: cocciclean:\n + \t$(RM) -r .build/contrib/coccinelle\n + \t$(RM) contrib/coccinelle/*.cocci.patch\n + \n +-clean: profile-clean coverage-clean cocciclean\n ++rustclean:\n ++\tcd rust && cargo clean\n ++\n ++clean: profile-clean coverage-clean cocciclean rustclean\n + \t$(RM) -r .build $(UNIT_TEST_BIN)\n + \t$(RM) GIT-TEST-SUITES\n + \t$(RM) po/git.pot po/git-core.pot\n @@ Makefile: FUZZ_CXXFLAGS ?= $(ALL_CFLAGS)\n .PHONY: fuzz-all\n fuzz-all: $(FUZZ_PROGRAMS)\n @@ build_rust.sh (new)\n @@\n +#!/bin/sh\n +\n -+if [ -z \"$CARGO_HOME\" ]; then\n -+ export CARGO_HOME=$HOME/.cargo\n -+ echo >&2 \"::warning:: CARGO_HOME is not set\"\n -+fi\n -+echo \"CARGO_HOME=$CARGO_HOME\"\n +\n -+rustc -vV\n -+cargo --version\n ++rustc -vV || exit $?\n ++cargo --version || exit $?\n +\n +dir_git_root=${0%/*}\n +dir_build=$1\n -+rust_target=$2\n ++rust_build_profile=$2\n +crate=$3\n +\n +dir_rust=$dir_git_root/rust\n @@ build_rust.sh (new)\n + exit 1\n +fi\n +\n -+if [ \"$rust_target\" = \"\" ]; then\n -+ echo \"did not specify the rust_target\"\n ++if [ \"$rust_build_profile\" = \"\" ]; then\n ++ echo \"did not specify the rust_build_profile\"\n + exit 1\n +fi\n +\n -+if [ \"$rust_target\" = \"release\" ]; then\n ++if [ \"$rust_build_profile\" = \"release\" ]; then\n + rust_args=\"--release\"\n -+ export RUSTFLAGS='-Aunused_imports -Adead_code'\n -+elif [ \"$rust_target\" = \"debug\" ]; then\n ++ export RUSTFLAGS=''\n ++elif [ \"$rust_build_profile\" = \"debug\" ]; then\n + rust_args=\"\"\n -+ export RUSTFLAGS='-Aunused_imports -Adead_code -C debuginfo=2 -C opt-level=1 -C force-frame-pointers=yes'\n ++ export RUSTFLAGS='-C debuginfo=2 -C opt-level=1 -C force-frame-pointers=yes'\n +else\n -+ echo \"illegal rust_target value $rust_target\"\n ++ echo \"illegal rust_build_profile value $rust_build_profile\"\n + exit 1\n +fi\n +\n -+cd $dir_rust && cargo clean && pwd && cargo build -p $crate $rust_args; cd ..\n ++cd $dir_rust && cargo clean && pwd && cargo build -p $crate $rust_args; cd $dir_git_root\n +\n +libfile=\"lib${crate}.a\"\n ++if rustup show active-toolchain | grep windows-msvc; then\n ++ libfile=\"${crate}.lib\"\n ++fi\n +dst=$dir_build/$libfile\n +\n +if [ \"$dir_git_root\" != \"$dir_build\" ]; then\n -+ src=$dir_rust/target/$rust_target/$libfile\n ++ src=$dir_rust/target/$rust_build_profile/$libfile\n + if [ ! -f $src ]; then\n -+ echo >&2 \"::error:: cannot find path of static library\"\n ++ echo >&2 \"::error:: cannot find path of static library $src is not a file or does not exist\"\n + exit 5\n + fi\n +\n @@ build_rust.sh (new)\n + mv $src $dst\n +fi\n \n - ## ci/install-dependencies.sh ##\n -@@ ci/install-dependencies.sh: fi\n - \n - case \"$distro\" in\n - alpine-*)\n --\tapk add --update shadow sudo meson ninja-build gcc libc-dev curl-dev openssl-dev expat-dev gettext \\\n -+\tapk add --update shadow sudo meson ninja-build gcc libc-dev curl curl-dev openssl-dev expat-dev gettext \\\n - \t\tzlib-ng-dev pcre2-dev python3 musl-libintl perl-utils ncurses \\\n - \t\tapache2 apache2-http2 apache2-proxy apache2-ssl apache2-webdav apr-util-dbd_sqlite3 \\\n - \t\tbash cvs gnupg perl-cgi perl-dbd-sqlite perl-io-tty >/dev/null\n - \t;;\n - fedora-*|almalinux-*)\n - \tdnf -yq update >/dev/null &&\n --\tdnf -yq install shadow-utils sudo make gcc findutils diffutils perl python3 gawk gettext zlib-devel expat-devel openssl-devel curl-devel pcre2-devel >/dev/null\n -+\tdnf -yq install shadow-utils sudo make gcc findutils diffutils perl python3 gawk gettext zlib-devel expat-devel openssl-devel curl curl-devel pcre2-devel >/dev/null\n - \t;;\n - ubuntu-*|i386/ubuntu-*|debian-*)\n - \t# Required so that apt doesn't wait for user input on certain packages.\n -@@ ci/install-dependencies.sh: ubuntu-*|i386/ubuntu-*|debian-*)\n - \tsudo apt-get -q update\n - \tsudo apt-get -q -y install \\\n - \t\t$LANGUAGES apache2 cvs cvsps git gnupg $SVN \\\n --\t\tmake libssl-dev libcurl4-openssl-dev libexpat-dev wget sudo default-jre \\\n --\t\ttcl tk gettext zlib1g-dev perl-modules liberror-perl libauthen-sasl-perl \\\n -+\t\tmake libssl-dev curl libcurl4-openssl-dev libexpat-dev wget sudo default-jre \\\n -+\t\ttcl tk gettext zlib1g zlib1g-dev perl-modules liberror-perl libauthen-sasl-perl \\\n - \t\tlibemail-valid-perl libio-pty-perl libio-socket-ssl-perl libnet-smtp-ssl-perl libdbd-sqlite3-perl libcgi-pm-perl \\\n - \t\tlibsecret-1-dev libpcre2-dev meson ninja-build pkg-config \\\n - \t\t${CC_PACKAGE:-${CC:-gcc}} $PYTHON_PACKAGE\n -@@ ci/install-dependencies.sh: ClangFormat)\n - \t;;\n - StaticAnalysis)\n - \tsudo apt-get -q update\n --\tsudo apt-get -q -y install coccinelle libcurl4-openssl-dev libssl-dev \\\n -+\tsudo apt-get -q -y install coccinelle curl libcurl4-openssl-dev libssl-dev \\\n - \t\tlibexpat-dev gettext make\n - \t;;\n - sparse)\n - \tsudo apt-get -q update -q\n --\tsudo apt-get -q -y install libssl-dev libcurl4-openssl-dev \\\n --\t\tlibexpat-dev gettext zlib1g-dev sparse\n -+\tsudo apt-get -q -y install libssl-dev curl libcurl4-openssl-dev \\\n -+\t\tlibexpat-dev gettext zlib1g zlib1g-dev sparse\n - \t;;\n - Documentation)\n - \tsudo apt-get -q update\n + ## git-compat-util.h ##\n +@@ git-compat-util.h: static inline int is_xplatform_dir_sep(int c)\n + #include \"compat/msvc.h\"\n + #endif\n + \n ++/* rust types */\n ++typedef uint8_t u8;\n ++typedef uint16_t u16;\n ++typedef uint32_t u32;\n ++typedef uint64_t u64;\n ++\n ++typedef int8_t i8;\n ++typedef int16_t i16;\n ++typedef int32_t i32;\n ++typedef int64_t i64;\n ++\n ++typedef float f32;\n ++typedef double f64;\n ++\n ++typedef size_t usize;\n ++typedef ptrdiff_t isize;\n ++\n + /* used on Mac OS X */\n + #ifdef PRECOMPOSE_UNICODE\n + #include \"compat/precompose_utf8.h\"\n \n - ## ci/install-rust.sh (new) ##\n -@@\n -+#!/bin/sh\n -+\n -+if [ \"$(id -u)\" -eq 0 ]; then\n -+ echo >&2 \"::warning:: installing rust as root\"\n -+fi\n -+\n -+if [ \"$CARGO_HOME\" = \"\" ]; then\n -+ echo >&2 \"::warning:: CARGO_HOME is not set\"\n -+ export CARGO_HOME=$HOME/.cargo\n -+fi\n -+\n -+export RUSTUP_HOME=$CARGO_HOME\n -+\n -+if [ \"$RUST_VERSION\" = \"\" ]; then\n -+ echo >&2 \"::error:: RUST_VERSION is not set\"\n -+ exit 2\n -+fi\n -+\n -+## install rustup\n -+curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- --default-toolchain none -y\n -+if [ ! -f $CARGO_HOME/env ]; then\n -+ echo \"PATH=$CARGO_HOME/bin:\\$PATH\" > $CARGO_HOME/env\n -+fi\n -+## install a specific version of rust\n -+if [ \"$BITNESS\" = \"32\" ]; then\n -+ $CARGO_HOME/bin/rustup set default-host i686-unknown-linux-gnu || exit $?\n -+ $CARGO_HOME/bin/rustup install $RUST_VERSION || exit $?\n -+ $CARGO_HOME/bin/rustup default --force-non-host $RUST_VERSION || exit $?\n -+else\n -+ $CARGO_HOME/bin/rustup default $RUST_VERSION || exit $?\n -+fi\n -+\n -+. $CARGO_HOME/env\n -\n - ## ci/lib.sh ##\n -@@\n - # Library of functions shared by all CI scripts\n - \n -+\n -+export BITNESS=\"64\"\n -+if command -v getconf >/dev/null && [ \"$(getconf LONG_BIT 2>/dev/null)\" = \"32\" ]; then\n -+ export BITNESS=\"32\"\n -+fi\n -+echo \"BITNESS=$BITNESS\"\n -+\n -+\n - if test true = \"$GITHUB_ACTIONS\"\n - then\n - \tbegin_group () {\n -\n - ## ci/make-test-artifacts.sh ##\n -@@ ci/make-test-artifacts.sh: mkdir -p \"$1\" # in case ci/lib.sh decides to quit early\n - \n - . ${0%/*}/lib.sh\n - \n -+## install rust per user rather than system wide\n -+. ${0%/*}/install-rust.sh\n -+\n - group Build make artifacts-tar ARTIFACTS_DIRECTORY=\"$1\"\n - \n -+if [ -d \"$CARGO_HOME\" ]; then\n -+ rm -rf $CARGO_HOME\n -+fi\n -+\n - check_unignored_build_artifacts\n -\n - ## ci/run-build-and-tests.sh ##\n -@@\n - \n - . ${0%/*}/lib.sh\n + ## meson.build ##\n +@@ meson.build: version_gen_environment.set('GIT_DATE', get_option('build_date'))\n + version_gen_environment.set('GIT_USER_AGENT', get_option('user_agent'))\n + version_gen_environment.set('GIT_VERSION', get_option('version'))\n \n -+## install rust per user rather than system wide\n -+. ${0%/*}/install-rust.sh\n ++if get_option('optimization') in ['2', '3', 's', 'z']\n ++ rust_build_profile = 'release'\n ++else\n ++ rust_build_profile = 'debug'\n ++endif\n +\n -+rustc -vV\n -+cargo --version || exit $?\n ++# Run `rustup show active-toolchain` and capture output\n ++rustup_out = run_command('rustup', 'show', 'active-toolchain',\n ++ check: true).stdout().strip()\n ++\n ++rust_crates = ['xdiff']\n ++rust_builds = []\n ++\n ++foreach crate : rust_crates\n ++ if rustup_out.contains('windows-msvc')\n ++ libfile = crate + '.lib'\n ++ else\n ++ libfile = 'lib' + crate + '.a'\n ++ endif\n ++\n ++ rust_builds += custom_target(\n ++ 'rust_build_'+crate,\n ++ output: libfile,\n ++ build_by_default: true,\n ++ build_always_stale: true,\n ++ command: [\n ++ meson.project_source_root() / 'build_rust.sh',\n ++ meson.current_build_dir(), rust_build_profile, crate,\n ++ ],\n ++ install: false,\n ++ )\n ++endforeach\n +\n - run_tests=t\n - \n - case \"$jobname\" in\n -@@ ci/run-build-and-tests.sh: case \"$jobname\" in\n - \t;;\n - esac\n - \n -+if [ -d \"$CARGO_HOME\" ]; then\n -+ rm -rf $CARGO_HOME\n -+fi\n +\n - check_unignored_build_artifacts\n - save_good_tree\n -\n - ## meson.build ##\n -@@ meson.build: else\n - rustflags = '-Aunused_imports -Adead_code -C debuginfo=2 -C opt-level=1 -C force-frame-pointers=yes'\n - endif\n - \n --\n --rust_leaf = custom_target('rust_leaf',\n -+rust_build_xdiff = custom_target('rust_build_xdiff',\n - output: 'libxdiff.a',\n - build_by_default: true,\n - build_always_stale: true,\n -- command: ['cargo', 'build',\n -- '--manifest-path', meson.project_source_root() / 'rust/Cargo.toml'\n -- ] + rust_args,\n -- env: {\n -- 'RUSTFLAGS': rustflags,\n -- },\n -+ command: [\n -+ meson.project_source_root() / 'build_rust.sh',\n -+ meson.current_build_dir(), rust_target, 'xdiff',\n -+ ],\n - install: false,\n - )\n - \n --rust_xdiff_dep = declare_dependency(\n -- link_args: ['-L' + meson.project_source_root() / 'rust/target' / rust_target, '-lxdiff'],\n --# include_directories: include_directories('xdiff/include'), # Adjust if you expose headers\n --)\n --\n --\n compiler = meson.get_compiler('c')\n \n libgit_sources = [\n @@ meson.build: version_def_h = custom_target(\n - )\n libgit_sources += version_def_h\n \n --libgit_dependencies += rust_xdiff_dep\n --\n libgit = declare_dependency(\n - link_with: static_library('git',\n - sources: libgit_sources,\n @@ meson.build: version_def_h = custom_target(\n + dependencies: libgit_dependencies,\n + include_directories: libgit_include_directories,\n + ),\n -+ rust_build_xdiff,\n -+ ],\n ++ ] + rust_builds,\n compile_args: libgit_c_args,\n dependencies: libgit_dependencies,\n include_directories: libgit_include_directories,\n +\n + ## rust/Cargo.toml (new) ##\n +@@\n ++[workspace]\n ++members = [\n ++ \"xdiff\",\n ++ \"interop\",\n ++]\n ++resolver = \"2\"\n +\n + ## rust/interop/Cargo.toml (new) ##\n +@@\n ++[package]\n ++name = \"interop\"\n ++version = \"0.1.0\"\n ++edition = \"2021\"\n ++\n ++[lib]\n ++name = \"interop\"\n ++path = \"src/lib.rs\"\n ++## staticlib to generate xdiff.a for use by gcc\n ++## cdylib (optional) to generate xdiff.so for use by gcc\n ++## rlib is required by the rust unit tests\n ++crate-type = [\"staticlib\", \"rlib\"]\n ++\n ++[dependencies]\n +\n + ## rust/interop/src/lib.rs (new) ##\n +\n + ## rust/xdiff/Cargo.toml (new) ##\n +@@\n ++[package]\n ++name = \"xdiff\"\n ++version = \"0.1.0\"\n ++edition = \"2021\"\n ++\n ++[lib]\n ++name = \"xdiff\"\n ++path = \"src/lib.rs\"\n ++## staticlib to generate xdiff.a for use by gcc\n ++## cdylib (optional) to generate xdiff.so for use by gcc\n ++## rlib is required by the rust unit tests\n ++crate-type = [\"staticlib\", \"rlib\"]\n ++\n ++[dependencies]\n ++interop = { path = \"../interop\" }\n +\n + ## rust/xdiff/src/lib.rs (new) ##\n 12: fffdb326710 ! 3: a98d9e4d21b github workflows: define rust versions and targets in the same place\n @@ Metadata\n Author: Ezekiel Newren \n \n ## Commit message ##\n - github workflows: define rust versions and targets in the same place\n + github workflows: install rust\n \n - Consolidate the Rust toolchain definitions in main.yaml. Prefer using\n - actions-rs/toolchain@v1 where possible, but for docker targets use\n - a script to install the Rust toolchain. Four overrides are used in\n + Prefer using actions-rs/toolchain@v1 where possible to install rustup,\n + but for docker targets use a script to install rustup. Consolidate the\n + Rust toolchain definitions in main.yaml. Use install-rust-toolchain.sh\n + to ensure the correct toolchain is used. Five overrides are used in\n main.yaml:\n \n * On Windows: Rust didn't resolve the bcrypt library on Windows\n correctly until version 1.78.0. Also since rustup mis-identifies\n the Rust toolchain, the Rust target triple must be set to\n - x86_64-pc-windows-gnu.\n + x86_64-pc-windows-gnu for make (win build), and\n + x86_64-pc-windows-msvc for meson (win+Meson build).\n * On musl: libc differences, such as ftruncate64 vs ftruncate, were\n not accounted for until Rust version 1.72.0. No older version of\n Rust will work on musl for our needs.\n * In a 32-bit docker container running on a 64-bit host, we need to\n override the Rust target triple. This is because rustup asks the\n kernel for the bitness of the system and it says 64, even though\n - the container will only run 32-bit. This also allows us to remove\n - the BITNESS environment variable in ci/lib.sh.\n + the container is 32-bit. This also allows us to remove the\n + BITNESS environment variable in ci/lib.sh.\n \n + The logic for selecting library names was initially provided in a patch\n + from Johannes, but was reworked and squashed into this commit.\n +\n + Helped-by: Johannes Schindelin \n Signed-off-by: Ezekiel Newren \n \n ## .github/workflows/main.yml ##\n -@@ .github/workflows/main.yml: on: [push, pull_request]\n - \n - env:\n - DEVELOPER: 1\n -- RUST_VERSION: 1.87.0\n - \n - # If more than one workflow run is triggered for the very same commit hash\n - # (which happens when multiple branches pointing to the same commit), only\n @@ .github/workflows/main.yml: jobs:\n outputs:\n enabled: ${{ steps.check-ref.outputs.enabled }}${{ steps.skip-if-redundant.outputs.enabled }}\n @@ .github/workflows/main.yml: jobs:\n + rust_version_windows: 1.78.0\n + rust_version_musl: 1.72.0\n + ## the rust target is inferred by rustup unless specified\n -+ rust_target_windows: x86_64-pc-windows-gnu\n ++ rust_target_windows_make: x86_64-pc-windows-gnu\n ++ rust_target_windows_meson: x86_64-pc-windows-msvc\n + rust_target_32bit_linux: i686-unknown-linux-gnu\n steps:\n - name: try to clone ci-config branch\n run: |\n @@ .github/workflows/main.yml: jobs:\n - /c/Program\\ Files/Git/mingw64/bin/curl -Lo libuserenv.a \\\n - https://github.com/git-for-windows/git-sdk-64/raw/HEAD/mingw64/lib/libuserenv.a\n - }\n -+ - name: Install Rust\n + needs: ci-config\n + if: needs.ci-config.outputs.enabled == 'yes'\n + runs-on: windows-latest\n ++ env:\n ++ CARGO_HOME: \"/c/Users/runneradmin/.cargo\"\n + concurrency:\n + group: windows-build-${{ github.ref }}\n + cancel-in-progress: ${{ needs.ci-config.outputs.skip_concurrent == 'yes' }}\n + steps:\n + - uses: actions/checkout@v4\n + - uses: git-for-windows/setup-git-for-windows-sdk@v1\n ++ - name: Install rustup via github actions\n + uses: actions-rs/toolchain@v1\n + with:\n -+ toolchain: ${{ needs.ci-config.outputs.rust_version_windows }}\n -+ target: ${{ needs.ci-config.outputs.rust_target_windows }}\n ++ toolchain: stable\n + profile: minimal\n -+ override: true\n ++ override: false\n ++ - name: Install Rust toolchain\n ++ shell: bash\n ++ env:\n ++ RUST_VERSION: ${{ needs.ci-config.outputs.rust_version_windows }}\n ++ RUST_TARGET: ${{ needs.ci-config.outputs.rust_target_windows_make }}\n ++ run: ci/install-rust-toolchain.sh\n - name: build\n shell: bash\n env:\n - HOME: ${{runner.workspace}}\n - NO_PERL: 1\n -+ CARGO_HOME: \"/c/Users/runneradmin/.cargo\"\n - run: . /etc/profile && ci/make-test-artifacts.sh artifacts\n - - name: zip up tracked files\n - run: git archive -o artifacts/tracked.tar.gz HEAD\n @@ .github/workflows/main.yml: jobs:\n + needs: ci-config\n + if: needs.ci-config.outputs.enabled == 'yes'\n + runs-on: windows-latest\n ++ env:\n ++ CARGO_HOME: \"/c/Users/runneradmin/.cargo\"\n + concurrency:\n + group: windows-meson-build-${{ github.ref }}\n + cancel-in-progress: ${{ needs.ci-config.outputs.skip_concurrent == 'yes' }}\n steps:\n - uses: actions/checkout@v4\n - uses: actions/setup-python@v5\n -+ - name: Install Rust\n ++ - name: Install rustup via github actions\n + uses: actions-rs/toolchain@v1\n + with:\n -+ toolchain: ${{ needs.ci-config.outputs.rust_version_windows }}\n -+ target: ${{ needs.ci-config.outputs.rust_target_windows }}\n ++ toolchain: stable\n + profile: minimal\n -+ override: true\n ++ override: false\n ++ - name: Install Rust toolchain\n ++ shell: bash\n ++ env:\n ++ RUST_VERSION: ${{ needs.ci-config.outputs.rust_version_windows }}\n ++ RUST_TARGET: ${{ needs.ci-config.outputs.rust_target_windows_meson }}\n ++ run: ci/install-rust-toolchain.sh\n - name: Set up dependencies\n shell: pwsh\n run: pip install meson ninja\n @@ .github/workflows/main.yml: jobs:\n + jobname: ${{matrix.vector.jobname}}\n + CI_JOB_IMAGE: ${{matrix.vector.pool}}\n + TEST_OUTPUT_DIRECTORY: ${{github.workspace}}/t\n ++ CARGO_HOME: \"/Users/runner/.cargo\"\n + runs-on: ${{matrix.vector.pool}}\n steps:\n - uses: actions/checkout@v4\n - run: ci/install-dependencies.sh\n -+ - name: Install Rust\n +- - run: ci/run-build-and-tests.sh\n ++ - name: Install rustup via github actions\n + uses: actions-rs/toolchain@v1\n + with:\n -+ toolchain: ${{ needs.ci-config.outputs.rust_version_minimum }}\n ++ toolchain: stable\n + profile: minimal\n -+ override: true\n - - run: ci/run-build-and-tests.sh\n ++ override: false\n ++ - name: Install Rust toolchain\n ++ shell: bash\n ++ env:\n ++ RUST_VERSION: ${{ needs.ci-config.outputs.rust_version_minimum }}\n ++ run: ci/install-rust-toolchain.sh\n ++ - name: Run build and tests\n ++ run: ci/run-build-and-tests.sh\n - name: print test failures\n if: failure() && env.FAILED_TEST_ARTIFACTS != ''\n + run: ci/print-test-failures.sh\n @@ .github/workflows/main.yml: jobs:\n cc: gcc\n - jobname: linux-musl-meson\n @@ .github/workflows/main.yml: jobs:\n CI_JOB_IMAGE: ${{matrix.vector.image}}\n + CI_IS_DOCKER: \"true\"\n CUSTOM_PATH: /custom\n -+ RUST_VERSION: ${{ matrix.vector.rust_version_override || needs.ci-config.outputs.rust_version_minimum }}\n -+ RUST_TARGET: ${{ matrix.vector.rust_target_override || '' }}\n + CARGO_HOME: /home/builder/.cargo\n runs-on: ubuntu-latest\n container: ${{matrix.vector.image}}\n steps:\n +@@ .github/workflows/main.yml: jobs:\n + - run: ci/install-dependencies.sh\n + - run: useradd builder --create-home\n + - run: chown -R builder .\n ++ - name: Install rustup via script\n ++ run: sudo --preserve-env --set-home --user=builder ci/install-rustup.sh\n ++ - name: Install Rust toolchain\n ++ env:\n ++ RUST_VERSION: ${{ matrix.vector.rust_version_override || needs.ci-config.outputs.rust_version_minimum }}\n ++ RUST_TARGET: ${{ matrix.vector.rust_target_override || '' }}\n ++ run: sudo --preserve-env --set-home --user=builder ci/install-rust-toolchain.sh\n + - run: sudo --preserve-env --set-home --user=builder ci/run-build-and-tests.sh\n + - name: print test failures\n + if: failure() && env.FAILED_TEST_ARTIFACTS != ''\n \n - ## build_rust.sh ##\n -@@\n - #!/bin/sh\n - \n --if [ -z \"$CARGO_HOME\" ]; then\n -- export CARGO_HOME=$HOME/.cargo\n -- echo >&2 \"::warning:: CARGO_HOME is not set\"\n --fi\n --echo \"CARGO_HOME=$CARGO_HOME\"\n + ## ci/install-dependencies.sh ##\n +@@ ci/install-dependencies.sh: fi\n \n --rustc -vV\n --cargo --version\n + case \"$distro\" in\n + alpine-*)\n +-\tapk add --update shadow sudo meson ninja-build gcc libc-dev curl-dev openssl-dev expat-dev gettext \\\n ++\tapk add --update shadow sudo meson ninja-build gcc libc-dev curl curl-dev openssl-dev expat-dev gettext \\\n + \t\tzlib-ng-dev pcre2-dev python3 musl-libintl perl-utils ncurses \\\n + \t\tapache2 apache2-http2 apache2-proxy apache2-ssl apache2-webdav apr-util-dbd_sqlite3 \\\n + \t\tbash cvs gnupg perl-cgi perl-dbd-sqlite perl-io-tty >/dev/null\n + \t;;\n + fedora-*|almalinux-*)\n + \tdnf -yq update >/dev/null &&\n +-\tdnf -yq install shadow-utils sudo make gcc findutils diffutils perl python3 gawk gettext zlib-devel expat-devel openssl-devel curl-devel pcre2-devel >/dev/null\n ++\tdnf -yq install shadow-utils sudo make gcc findutils diffutils perl python3 gawk gettext zlib-devel expat-devel openssl-devel curl curl-devel pcre2-devel >/dev/null\n + \t;;\n + ubuntu-*|i386/ubuntu-*|debian-*)\n + \t# Required so that apt doesn't wait for user input on certain packages.\n +@@ ci/install-dependencies.sh: ubuntu-*|i386/ubuntu-*|debian-*)\n + \tsudo apt-get -q update\n + \tsudo apt-get -q -y install \\\n + \t\t$LANGUAGES apache2 cvs cvsps git gnupg $SVN \\\n +-\t\tmake libssl-dev libcurl4-openssl-dev libexpat-dev wget sudo default-jre \\\n +-\t\ttcl tk gettext zlib1g-dev perl-modules liberror-perl libauthen-sasl-perl \\\n ++\t\tmake libssl-dev curl libcurl4-openssl-dev libexpat-dev wget sudo default-jre \\\n ++\t\ttcl tk gettext zlib1g zlib1g-dev perl-modules liberror-perl libauthen-sasl-perl \\\n + \t\tlibemail-valid-perl libio-pty-perl libio-socket-ssl-perl libnet-smtp-ssl-perl libdbd-sqlite3-perl libcgi-pm-perl \\\n + \t\tlibsecret-1-dev libpcre2-dev meson ninja-build pkg-config \\\n + \t\t${CC_PACKAGE:-${CC:-gcc}} $PYTHON_PACKAGE\n +@@ ci/install-dependencies.sh: ClangFormat)\n + \t;;\n + StaticAnalysis)\n + \tsudo apt-get -q update\n +-\tsudo apt-get -q -y install coccinelle libcurl4-openssl-dev libssl-dev \\\n ++\tsudo apt-get -q -y install coccinelle curl libcurl4-openssl-dev libssl-dev \\\n + \t\tlibexpat-dev gettext make\n + \t;;\n + sparse)\n + \tsudo apt-get -q update -q\n +-\tsudo apt-get -q -y install libssl-dev libcurl4-openssl-dev \\\n +-\t\tlibexpat-dev gettext zlib1g-dev sparse\n ++\tsudo apt-get -q -y install libssl-dev curl libcurl4-openssl-dev \\\n ++\t\tlibexpat-dev gettext zlib1g zlib1g-dev sparse\n + \t;;\n + Documentation)\n + \tsudo apt-get -q update\n +\n + ## ci/install-rust-toolchain.sh (new) ##\n +@@\n ++#!/bin/sh\n ++\n ++if [ \"$CARGO_HOME\" = \"\" ]; then\n ++ echo >&2 \"::error:: CARGO_HOME is not set\"\n ++ exit 2\n ++fi\n ++export PATH=\"$CARGO_HOME/bin:$PATH\"\n ++rustup -vV || exit $?\n ++\n ++## Enforce the correct Rust toolchain\n ++rustup override unset || true\n ++\n ++## install a specific version of rust\n ++if [ \"$RUST_TARGET\" != \"\" ]; then\n ++ rustup default --force-non-host \"$RUST_VERSION-$RUST_TARGET\" || exit $?\n ++else\n ++ rustup default \"$RUST_VERSION\" || exit $?\n ++fi\n ++\n +rustc -vV || exit $?\n -+cargo --version || exit $?\n - \n - dir_git_root=${0%/*}\n - dir_build=$1\n -@@ build_rust.sh: dst=$dir_build/$libfile\n - if [ \"$dir_git_root\" != \"$dir_build\" ]; then\n - src=$dir_rust/target/$rust_target/$libfile\n - if [ ! -f $src ]; then\n -- echo >&2 \"::error:: cannot find path of static library\"\n -+ echo >&2 \"::error:: cannot find path of static library $src is not a file or does not exist\"\n - exit 5\n - fi\n - \n ++\n ++RE_RUST_TARGET=\"$RUST_TARGET\"\n ++if [ \"$RUST_TARGET\" = \"\" ]; then\n ++ RE_RUST_TARGET=\"[^ ]+\"\n ++fi\n ++\n ++if ! rustup show active-toolchain | grep -E \"^$RUST_VERSION-$RE_RUST_TARGET \\(default\\)$\"; then\n ++ echo >&2 \"::error:: wrong Rust toolchain, active-toolchain: $(rustup show active-toolchain)\"\n ++ exit 3\n ++fi\n \n - ## ci/install-rust.sh (mode change 100644 => 100755) ##\n + ## ci/install-rustup.sh (new) ##\n @@\n - #!/bin/sh\n - \n ++#!/bin/sh\n ++\n +## github workflows actions-rs/toolchain@v1 doesn't work for docker\n +## targets. This script should only be used if the ci pipeline\n +## doesn't support installing rust on a particular target.\n +\n - if [ \"$(id -u)\" -eq 0 ]; then\n - echo >&2 \"::warning:: installing rust as root\"\n - fi\n - \n --if [ \"$CARGO_HOME\" = \"\" ]; then\n -- echo >&2 \"::warning:: CARGO_HOME is not set\"\n -- export CARGO_HOME=$HOME/.cargo\n --fi\n --\n --export RUSTUP_HOME=$CARGO_HOME\n --\n - if [ \"$RUST_VERSION\" = \"\" ]; then\n - echo >&2 \"::error:: RUST_VERSION is not set\"\n -+ exit 1\n ++if [ \"$(id -u)\" -eq 0 ]; then\n ++ echo >&2 \"::warning:: installing rust as root\"\n +fi\n +\n +if [ \"$CARGO_HOME\" = \"\" ]; then\n + echo >&2 \"::error:: CARGO_HOME is not set\"\n - exit 2\n - fi\n - \n ++ exit 2\n ++fi\n ++\n +export RUSTUP_HOME=$CARGO_HOME\n +\n - ## install rustup\n - curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- --default-toolchain none -y\n - if [ ! -f $CARGO_HOME/env ]; then\n - echo \"PATH=$CARGO_HOME/bin:\\$PATH\" > $CARGO_HOME/env\n - fi\n ++## install rustup\n ++curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- --default-toolchain none -y\n ++if [ ! -f $CARGO_HOME/env ]; then\n ++ echo \"PATH=$CARGO_HOME/bin:\\$PATH\" > $CARGO_HOME/env\n ++fi\n +. $CARGO_HOME/env\n +\n - ## install a specific version of rust\n --if [ \"$BITNESS\" = \"32\" ]; then\n -- $CARGO_HOME/bin/rustup set default-host i686-unknown-linux-gnu || exit $?\n -- $CARGO_HOME/bin/rustup install $RUST_VERSION || exit $?\n -- $CARGO_HOME/bin/rustup default --force-non-host $RUST_VERSION || exit $?\n -+if [ \"$RUST_TARGET\" != \"\" ]; then\n -+ rustup default --force-non-host \"$RUST_VERSION-$RUST_TARGET\" || exit $?\n - else\n -- $CARGO_HOME/bin/rustup default $RUST_VERSION || exit $?\n -- if [ \"$CI_OS_NAME\" = \"windows\" ]; then\n -- $CARGO_HOME/bin/rustup target add x86_64-pc-windows-gnu || exit $?\n -- fi\n -+ rustup default \"$RUST_VERSION\" || exit $?\n - fi\n - \n --. $CARGO_HOME/env\n -+rustc -vV || exit $?\n ++rustup -vV\n \n ## ci/lib.sh ##\n @@\n # Library of functions shared by all CI scripts\n \n - \n --export BITNESS=\"64\"\n --if command -v getconf >/dev/null && [ \"$(getconf LONG_BIT 2>/dev/null)\" = \"32\" ]; then\n -- export BITNESS=\"32\"\n --fi\n --echo \"BITNESS=$BITNESS\"\n --\n --\n ++\n if test true = \"$GITHUB_ACTIONS\"\n then\n \tbegin_group () {\n @@ ci/make-test-artifacts.sh: mkdir -p \"$1\" # in case ci/lib.sh decides to quit ear\n \n . ${0%/*}/lib.sh\n \n --## install rust per user rather than system wide\n --. ${0%/*}/install-rust.sh\n -+if [ -z \"$CARGO_HOME\" ]; then\n ++## ensure rustup is in the PATH variable\n ++if [ \"$CARGO_HOME\" = \"\" ]; then\n + echo >&2 \"::error:: CARGO_HOME is not set\"\n -+ exit 1\n ++ exit 2\n +fi\n - \n --group Build make artifacts-tar ARTIFACTS_DIRECTORY=\"$1\"\n +export PATH=\"$CARGO_HOME/bin:$PATH\"\n - \n --if [ -d \"$CARGO_HOME\" ]; then\n -- rm -rf $CARGO_HOME\n --fi\n -+group Build make artifacts-tar ARTIFACTS_DIRECTORY=\"$1\"\n ++\n ++rustc -vV\n ++\n + group Build make artifacts-tar ARTIFACTS_DIRECTORY=\"$1\"\n \n check_unignored_build_artifacts\n \n @@ ci/run-build-and-tests.sh\n \n . ${0%/*}/lib.sh\n \n --## install rust per user rather than system wide\n --. ${0%/*}/install-rust.sh\n -+## actions-rs/toolchain@v1 doesn't work for docker targets.\n -+if [ \"$CI_IS_DOCKER\" = \"true\" ]; then\n -+ . ${0%/*}/install-rust.sh\n ++## ensure rustup is in the PATH variable\n ++if [ \"$CARGO_HOME\" = \"\" ]; then\n ++ echo >&2 \"::error:: CARGO_HOME is not set\"\n ++ exit 2\n +fi\n - \n --rustc -vV\n ++. $CARGO_HOME/env\n ++\n +rustc -vV || exit $?\n - cargo --version || exit $?\n - \n ++\n run_tests=t\n -\n - ## meson.build ##\n -@@ meson.build: rust_build_xdiff = custom_target('rust_build_xdiff',\n - meson.project_source_root() / 'build_rust.sh',\n - meson.current_build_dir(), rust_target, 'xdiff',\n - ],\n -+ env: script_environment,\n - install: false,\n - )\n \n + case \"$jobname\" in\n +@@ ci/run-build-and-tests.sh: case \"$jobname\" in\n + \t;;\n + esac\n + \n ++if [ -d \"$CARGO_HOME\" ]; then\n ++ rm -rf $CARGO_HOME\n ++fi\n ++\n + check_unignored_build_artifacts\n + save_good_tree\n 11: 382067a09e3 ! 4: 0d2b39c3e03 win+Meson: do allow linking with the Rust-built xdiff\n @@ .github/workflows/main.yml: jobs:\n + /c/Program\\ Files/Git/mingw64/bin/curl -Lo libuserenv.a \\\n + https://github.com/git-for-windows/git-sdk-64/raw/HEAD/mingw64/lib/libuserenv.a\n + }\n - - name: build\n - shell: bash\n - env:\n + - name: Install rustup via github actions\n + uses: actions-rs/toolchain@v1\n + with:\n \n ## config.mak.uname ##\n @@ config.mak.uname: ifeq ($(uname_S),MINGW)\n - \n - \texport CARGO_BUILD_TARGET\n - \tRUST_TARGET_DIR = rust/target/$(CARGO_BUILD_TARGET)/$(RUST_BUILD_MODE)\n + \t\tCOMPAT_CFLAGS += -D_USE_32BIT_TIME_T\n + \t\tBASIC_LDFLAGS += -Wl,--large-address-aware\n + endif\n ++\n +\t# Unfortunately now needed because of Rust\n +\tEXTLIBS += -luserenv\n - \n ++\n \tCC = gcc\n \tCOMPAT_CFLAGS += -D__USE_MINGW_ANSI_STDIO=0 -DDETECT_MSYS_TTY \\\n + \t\t-fstack-protector-strong\n \n ## meson.build ##\n @@ meson.build: elif host_machine.system() == 'windows'\n 13: 44784f0d672 = 5: e65488ab993 github workflows: upload Cargo.lock\n -: ----------- > 6: db5d22b1887 ivec: create a vector type that is interoperable between C and Rust\n 3: 56c96d35554 = 7: d4bed954632 xdiff/xprepare: remove superfluous forward declarations\n 4: ebec3689dce = 8: 7c68ce5349c xdiff: delete unnecessary fields from xrecord_t and xdfile_t\n 5: 769d1a5b9d2 = 9: e516ccc8c0a xdiff: make fields of xrecord_t Rust friendly\n 6: 87623495994 ! 10: 21bfb9f0883 xdiff: separate parsing lines from hashing them\n @@ Metadata\n Author: Ezekiel Newren \n \n ## Commit message ##\n - xdiff: separate parsing lines from hashing them\n + xdiff: use one definition for freeing xdfile_t\n \n - We want to use xxhash for faster hashing. To facilitate that\n - and to simplify the code. Separate the concerns of parsing\n - and hashing into discrete steps. This makes swapping the hash\n - function much easier. Since xdl_hash_record() both parses and\n - hashses lines, this requires some slight code restructuring.\n + Simplify xdl_prepare_ctx() by using xdl_free_ctx() instead of using\n + local variables with hand rolled memory management.\n \n Signed-off-by: Ezekiel Newren \n \n @@ xdiff/xprepare.c: static int xdl_classify_record(unsigned int pass, xdlclassifie\n }\n \n \n -+static void xdl_parse_lines(mmfile_t *mf, long narec, xdfile_t *xdf) {\n -+\tu8 const* ptr = (u8 const*) mf->ptr;\n -+\tusize len = (usize) mf->size;\n -+\n -+\txdf->recs = NULL;\n -+\txdf->nrec = 0;\n -+\tXDL_ALLOC_ARRAY(xdf->recs, narec);\n -+\n -+\twhile (len > 0) {\n -+\t\txrecord_t *rec = NULL;\n -+\t\tusize length;\n -+\t\tu8 const* result = memchr(ptr, '\\n', len);\n -+\t\tif (result) {\n -+\t\t\tlength = result - ptr + 1;\n -+\t\t} else {\n -+\t\t\tlength = len;\n -+\t\t}\n -+\t\tif (XDL_ALLOC_GROW(xdf->recs, xdf->nrec + 1, narec))\n -+\t\t\tdie(\"XDL_ALLOC_GROW failed\");\n -+\t\trec = xdl_cha_alloc(&xdf->rcha);\n -+\t\trec->ptr = ptr;\n -+\t\trec->size = length;\n -+\t\trec->ha = 0;\n -+\t\txdf->recs[xdf->nrec++] = rec;\n -+\t\tptr += length;\n -+\t\tlen -= length;\n -+\t}\n -+\n ++static void xdl_free_ctx(xdfile_t *xdf) {\n ++\txdl_free(xdf->rindex);\n ++\txdl_free(xdf->rchg - 1);\n ++\txdl_free(xdf->ha);\n ++\txdl_free(xdf->recs);\n ++\txdl_cha_free(&xdf->rcha);\n +}\n +\n +\n static int xdl_prepare_ctx(unsigned int pass, mmfile_t *mf, long narec, xpparam_t const *xpp,\n \t\t\t xdlclassifier_t *cf, xdfile_t *xdf) {\n -\tlong nrec, bsize;\n --\tunsigned long hav;\n --\tchar const *blk, *cur, *top, *prev;\n --\txrecord_t *crec;\n ++\tlong bsize;\n + \tunsigned long hav;\n + \tchar const *blk, *cur, *top, *prev;\n + \txrecord_t *crec;\n -\txrecord_t **recs;\n - \tunsigned long *ha;\n - \tchar *rchg;\n - \tlong *rindex;\n -@@ xdiff/xprepare.c: static int xdl_prepare_ctx(unsigned int pass, mmfile_t *mf, long narec, xpparam_\n - \tha = NULL;\n - \trindex = NULL;\n - \trchg = NULL;\n +-\tunsigned long *ha;\n +-\tchar *rchg;\n +-\tlong *rindex;\n + \n +-\tha = NULL;\n +-\trindex = NULL;\n +-\trchg = NULL;\n -\trecs = NULL;\n ++\txdf->ha = NULL;\n ++\txdf->rindex = NULL;\n ++\txdf->rchg = NULL;\n ++\txdf->recs = NULL;\n ++\txdf->nrec = 0;\n \n \tif (xdl_cha_init(&xdf->rcha, sizeof(xrecord_t), narec / 4 + 1) < 0)\n \t\tgoto abort;\n -\tif (!XDL_ALLOC_ARRAY(recs, narec))\n --\t\tgoto abort;\n ++\tif (!XDL_ALLOC_ARRAY(xdf->recs, narec))\n + \t\tgoto abort;\n \n -\tnrec = 0;\n --\tif ((cur = blk = xdl_mmfile_first(mf, &bsize))) {\n --\t\tfor (top = blk + bsize; cur < top; ) {\n --\t\t\tprev = cur;\n --\t\t\thav = xdl_hash_record(&cur, top, xpp->flags);\n + \tif ((cur = blk = xdl_mmfile_first(mf, &bsize))) {\n + \t\tfor (top = blk + bsize; cur < top; ) {\n + \t\t\tprev = cur;\n + \t\t\thav = xdl_hash_record(&cur, top, xpp->flags);\n -\t\t\tif (XDL_ALLOC_GROW(recs, nrec + 1, narec))\n --\t\t\t\tgoto abort;\n --\t\t\tif (!(crec = xdl_cha_alloc(&xdf->rcha)))\n --\t\t\t\tgoto abort;\n --\t\t\tcrec->ptr = (u8 const*) prev;\n --\t\t\tcrec->size = (long) (cur - prev);\n --\t\t\tcrec->ha = hav;\n ++\t\t\tif (XDL_ALLOC_GROW(xdf->recs, xdf->nrec + 1, narec))\n + \t\t\t\tgoto abort;\n + \t\t\tif (!(crec = xdl_cha_alloc(&xdf->rcha)))\n + \t\t\t\tgoto abort;\n + \t\t\tcrec->ptr = (u8 const*) prev;\n + \t\t\tcrec->size = (long) (cur - prev);\n + \t\t\tcrec->ha = hav;\n -\t\t\trecs[nrec++] = crec;\n --\t\t\tif (xdl_classify_record(pass, cf, crec) < 0)\n --\t\t\t\tgoto abort;\n --\t\t}\n -+\txdl_parse_lines(mf, narec, xdf);\n -+\n -+\tfor (usize i = 0; i < (usize) xdf->nrec; i++) {\n -+\t\txrecord_t *rec = xdf->recs[i];\n -+\t\tchar const* dump = (char const*) rec->ptr;\n -+\t\trec->ha = xdl_hash_record(&dump, (char const*) (rec->ptr + rec->size), xpp->flags);\n -+\t\txdl_classify_record(pass, cf, rec);\n ++\t\t\txdf->recs[xdf->nrec++] = crec;\n + \t\t\tif (xdl_classify_record(pass, cf, crec) < 0)\n + \t\t\t\tgoto abort;\n + \t\t}\n \t}\n \n -\tif (!XDL_CALLOC_ARRAY(rchg, nrec + 2))\n -+\n -+\tif (!XDL_CALLOC_ARRAY(rchg, xdf->nrec + 2))\n ++\tif (!XDL_CALLOC_ARRAY(xdf->rchg, xdf->nrec + 2))\n \t\tgoto abort;\n \n \tif ((XDF_DIFF_ALG(xpp->flags) != XDF_PATIENCE_DIFF) &&\n \t (XDF_DIFF_ALG(xpp->flags) != XDF_HISTOGRAM_DIFF)) {\n -\t\tif (!XDL_ALLOC_ARRAY(rindex, nrec + 1))\n -+\t\tif (!XDL_ALLOC_ARRAY(rindex, xdf->nrec + 1))\n ++\t\tif (!XDL_ALLOC_ARRAY(xdf->rindex, xdf->nrec + 1))\n \t\t\tgoto abort;\n -\t\tif (!XDL_ALLOC_ARRAY(ha, nrec + 1))\n -+\t\tif (!XDL_ALLOC_ARRAY(ha, xdf->nrec + 1))\n ++\t\tif (!XDL_ALLOC_ARRAY(xdf->ha, xdf->nrec + 1))\n \t\t\tgoto abort;\n \t}\n \n -\txdf->nrec = nrec;\n -\txdf->recs = recs;\n - \txdf->rchg = rchg + 1;\n - \txdf->rindex = rindex;\n +-\txdf->rchg = rchg + 1;\n +-\txdf->rindex = rindex;\n ++\txdf->rchg += 1;\n \txdf->nreff = 0;\n - \txdf->ha = ha;\n +-\txdf->ha = ha;\n \txdf->dstart = 0;\n -\txdf->dend = nrec - 1;\n +\txdf->dend = xdf->nrec - 1;\n \n \treturn 0;\n \n -@@ xdiff/xprepare.c: abort:\n - \txdl_free(ha);\n - \txdl_free(rindex);\n - \txdl_free(rchg);\n + abort:\n +-\txdl_free(ha);\n +-\txdl_free(rindex);\n +-\txdl_free(rchg);\n -\txdl_free(recs);\n -+\txdl_free(xdf->recs);\n - \txdl_cha_free(&xdf->rcha);\n +-\txdl_cha_free(&xdf->rcha);\n ++\txdl_free_ctx(xdf);\n \treturn -1;\n }\n + \n + \n +-static void xdl_free_ctx(xdfile_t *xdf) {\n +-\txdl_free(xdf->rindex);\n +-\txdl_free(xdf->rchg - 1);\n +-\txdl_free(xdf->ha);\n +-\txdl_free(xdf->recs);\n +-\txdl_cha_free(&xdf->rcha);\n +-}\n +-\n +-\n + void xdl_free_env(xdfenv_t *xe) {\n + \n + \txdl_free_ctx(&xe->xdf2);\n 7: d74fd4ef67a < -: ----------- xdiff: conditionally use Rust's implementation of xxhash\n 9: 96041a10d54 < -: ----------- Do support Windows again after requiring Rust\n 10: 1194de3f39c < -: ----------- win+Meson: allow for xdiff to be compiled with MSVC\n 14: f20efdff7aa < -: ----------- xdiff: implement a white space iterator in Rust\n -: ----------- > 11: 6ce0e252b38 xdiff: replace chastore with an ivec in xdfile_t\n -: ----------- > 12: 0cfc6cf26b7 xdiff: delete nrec field from xdfile_t\n -: ----------- > 13: cf0387d851c xdiff: delete recs field from xdfile_t\n -: ----------- > 14: ea699135f95 xdiff: make xdfile_t more rust friendly\n 15: c8d41173274 ! 15: b18544b74f3 xdiff: create line_hash() and line_equal()\n @@ Metadata\n Author: Ezekiel Newren \n \n ## Commit message ##\n - xdiff: create line_hash() and line_equal()\n + xdiff: implement xdl_trim_ends() in Rust\n \n - These functions use the whitespace iterator, when applicable, to hash,\n - and compare lines.\n + Replace the C implementation of xdl_trim_ends() with a Rust\n + implementation.\n \n Signed-off-by: Ezekiel Newren \n \n ## rust/xdiff/src/lib.rs ##\n @@\n -+use std::hash::Hasher;\n -+use xxhash_rust::xxh3::Xxh3Default;\n -+use crate::xutils::*;\n -+\n - pub mod xutils;\n ++pub mod xprepare;\n ++pub mod xtypes;\n \n - pub const XDF_IGNORE_WHITESPACE: u64 = 1 << 1;\n -@@ rust/xdiff/src/lib.rs: unsafe extern \"C\" fn xxh3_64(ptr: *const u8, size: usize) -> u64 {\n - let slice = std::slice::from_raw_parts(ptr, size);\n - xxhash_rust::xxh3::xxh3_64(slice)\n - }\n ++use crate::xprepare::trim_ends;\n ++use crate::xtypes::xdfile;\n +\n +#[no_mangle]\n -+unsafe extern \"C\" fn xdl_line_hash(ptr: *const u8, size: usize, flags: u64) -> u64 {\n -+ let line = std::slice::from_raw_parts(ptr, size);\n -+\n -+ line_hash(line, flags)\n -+}\n ++unsafe extern \"C\" fn xdl_trim_ends(xdf1: *mut xdfile, xdf2: *mut xdfile) -> i32 {\n ++ let xdf1 = xdf1.as_mut().expect(\"null pointer\");\n ++ let xdf2 = xdf2.as_mut().expect(\"null pointer\");\n +\n -+#[no_mangle]\n -+unsafe extern \"C\" fn xdl_line_equal(lhs: *const u8, lhs_len: usize, rhs: *const u8, rhs_len: usize, flags: u64) -> bool {\n -+ let lhs_line = std::slice::from_raw_parts(lhs, lhs_len);\n -+ let rhs_line = std::slice::from_raw_parts(rhs, rhs_len);\n ++ trim_ends(xdf1, xdf2);\n +\n -+ line_equal(lhs_line, rhs_line, flags)\n ++ 0\n +}\n \n - ## rust/xdiff/src/xutils.rs ##\n + ## rust/xdiff/src/xprepare.rs (new) ##\n @@\n - use crate::*;\n -+use xxhash_rust::xxh3::xxh3_64;\n - \n - pub(crate) fn xdl_isspace(v: u8) -> bool {\n - match v {\n -@@ rust/xdiff/src/xutils.rs: where\n - run_option0.is_none() && run_option1.is_none()\n - }\n - \n ++use crate::xtypes::xdfile;\n +\n -+pub fn line_hash(line: &[u8], flags: u64) -> u64 {\n -+ if (flags & XDF_WHITESPACE_FLAGS) == 0 {\n -+ return xxh3_64(line);\n -+ }\n ++///\n ++/// Early trim initial and terminal matching records.\n ++///\n ++pub(crate) fn trim_ends(xdf1: &mut xdfile, xdf2: &mut xdfile) {\n ++ let mut lim = std::cmp::min(xdf1.record.len(), xdf2.record.len());\n +\n -+ let mut hasher = Xxh3Default::new();\n -+ for chunk in WhitespaceIter::new(line, flags) {\n -+ hasher.update(chunk);\n ++ for i in 0..lim {\n ++ if xdf1.record[i].ha != xdf2.record[i].ha {\n ++ xdf1.dstart = i as isize;\n ++ xdf2.dstart = i as isize;\n ++ lim -= i;\n ++ break;\n ++ }\n + }\n +\n -+ hasher.finish()\n -+}\n -+\n -+\n -+pub fn line_equal(lhs: &[u8], rhs: &[u8], flags: u64) -> bool {\n -+ if (flags & XDF_WHITESPACE_FLAGS) == 0 {\n -+ return lhs == rhs;\n ++ for i in 0..lim {\n ++ let f1i = xdf1.record.len() - 1 - i;\n ++ let f2i = xdf2.record.len() - 1 - i;\n ++ if xdf1.record[f1i].ha != xdf2.record[f2i].ha {\n ++ xdf1.dend = f1i as isize;\n ++ xdf2.dend = f2i as isize;\n ++ break;\n ++ }\n + }\n -+\n -+ let lhs_it = WhitespaceIter::new(lhs, flags);\n -+ let rhs_it = WhitespaceIter::new(rhs, flags);\n -+\n -+ chunked_iter_equal(lhs_it, rhs_it)\n +}\n +\n + ## rust/xdiff/src/xtypes.rs (new) ##\n +@@\n ++use interop::ivec::IVec;\n +\n ++#[repr(C)]\n ++pub(crate) struct xrecord {\n ++ pub(crate) ptr: *const u8,\n ++ pub(crate) size: usize,\n ++ pub(crate) ha: u64,\n ++}\n +\n - #[cfg(test)]\n - mod tests {\n - use crate::*;\n ++#[repr(C)]\n ++pub(crate) struct xdfile {\n ++ pub(crate) record: IVec,\n ++ pub(crate) dstart: isize,\n ++ pub(crate) dend: isize,\n ++ pub(crate) rchg: *mut u8,\n ++ pub(crate) rindex: *mut usize,\n ++ pub(crate) nreff: usize,\n ++ pub(crate) ha: *mut u64,\n ++}\n +\n + ## xdiff/xprepare.c ##\n +@@ xdiff/xprepare.c: static int xdl_cleanup_records(xdlclassifier_t *cf, xdfile_t *xdf1, xdfile_t *xd\n + }\n + \n + \n +-/*\n +- * Early trim initial and terminal matching records.\n +- */\n +-static int xdl_trim_ends(xdfile_t *xdf1, xdfile_t *xdf2) {\n +-\tlong i, lim;\n +-\txrecord_t *recs1, *recs2;\n +-\n +-\trecs1 = xdf1->record.ptr;\n +-\trecs2 = xdf2->record.ptr;\n +-\tfor (i = 0, lim = XDL_MIN(xdf1->record.length, xdf2->record.length); i < lim;\n +-\t i++, recs1++, recs2++)\n +-\t\tif (recs1->ha != recs2->ha)\n +-\t\t\tbreak;\n +-\n +-\txdf1->dstart = xdf2->dstart = i;\n +-\n +-\trecs1 = xdf1->record.ptr + xdf1->record.length - 1;\n +-\trecs2 = xdf2->record.ptr + xdf2->record.length - 1;\n +-\tfor (lim -= i, i = 0; i < lim; i++, recs1--, recs2--)\n +-\t\tif (recs1->ha != recs2->ha)\n +-\t\t\tbreak;\n +-\n +-\txdf1->dend = xdf1->record.length - i - 1;\n +-\txdf2->dend = xdf2->record.length - i - 1;\n +-\n +-\treturn 0;\n +-}\n ++extern i32 xdl_trim_ends(xdfile_t *xdf1, xdfile_t *xdf2);\n + \n + \n + static int xdl_optimize_ctxs(xdlclassifier_t *cf, xdfile_t *xdf1, xdfile_t *xdf2) {\n 16: f7829c55871 < -: ----------- xdiff: optimize case where --ignore-cr-at-eol is the only whitespace flag\n 17: 395609aff4b < -: ----------- xdiff: use rust's version of whitespace processing\n\n-- \ngitgitgadget\n"},{"id":"524749","messageId":"6d065f550fe871cf010409f7bd2a63438cf52723.1755921357.git.gitgitgadget@gmail.com","threadId":"63804","inReplyTo":"pull.1980.v3.git.git.1755921356.gitgitgadget@gmail.com","subject":"[PATCH v3 01/15] doc: add a policy for using Rust","fromName":"brian m. carlson via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2025-08-23T03:55:42Z","receivedAt":"2025-08-23T03:56:03Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"From: \"brian m. carlson\" \n\nGit has historically been written primarily in C, with some shell and\nPerl. However, C is not memory safe, which makes it more likely that\nsecurity vulnerabilities or other bugs will be introduced, and it is\nalso more verbose and less ergonomic than other, more modern languages.\n\nOne of the most common modern compiled languages which is easily\ninteroperable with C is Rust. It is popular (the most admired language\non the 2024 Stack Overflow Developer Survey), efficient, portable, and\nrobust.\n\nIntroduce a document laying out the incremental introduction of Rust to\nGit and provide a detailed rationale for doing so, including the points\nabove. Propose a design for this approach that addresses the needs of\ndownstreams and distributors, as well as contributors.\n\nSince we don't want to carry both a C and Rust version of code and want\nto be able to add new features only in Rust, mention that Rust is a\nrequired part of our platform support policy.\n\nIt should be noted that a recent discussion at the Berlin Git Merge\nContributor Summit found widespread support for the addition of Rust to\nGit. While of course not all contributors were represented, the\nproposal appeared to have the support of a majority of active\ncontributors.\n\nSigned-off-by: brian m. carlson \n[en: Added some comments about types, and changed the recommondations\n about cbindgen, bindgen, rustix, libc.]\nSigned-off-by: Ezekiel Newren \n---\n Documentation/Makefile | 1 +\n Documentation/technical/platform-support.adoc | 2 +\n Documentation/technical/rust-support.adoc | 142 ++++++++++++++++++\n 3 files changed, 145 insertions(+)\n create mode 100644 Documentation/technical/rust-support.adoc\n\ndiff --git a/Documentation/Makefile b/Documentation/Makefile\nindex b109d25e9c80..066b761c01b9 100644\n--- a/Documentation/Makefile\n+++ b/Documentation/Makefile\n@@ -127,6 +127,7 @@ TECH_DOCS += technical/parallel-checkout\n TECH_DOCS += technical/partial-clone\n TECH_DOCS += technical/platform-support\n TECH_DOCS += technical/racy-git\n+TECH_DOCS += technical/rust-support\n TECH_DOCS += technical/reftable\n TECH_DOCS += technical/scalar\n TECH_DOCS += technical/send-pack-pipeline\ndiff --git a/Documentation/technical/platform-support.adoc b/Documentation/technical/platform-support.adoc\nindex 0a2fb28d6277..dc71672dcb57 100644\n--- a/Documentation/technical/platform-support.adoc\n+++ b/Documentation/technical/platform-support.adoc\n@@ -33,6 +33,8 @@ meet the following minimum requirements:\n \n * Has active security support (taking security releases of dependencies, etc)\n \n+* Supports Rust and the toolchain version specified in link:rust-support.adoc[].\n+\n These requirements are a starting point, and not sufficient on their own for the\n Git community to be enthusiastic about supporting your platform. Maintainers of\n platforms which do meet these requirements can follow the steps below to make it\ndiff --git a/Documentation/technical/rust-support.adoc b/Documentation/technical/rust-support.adoc\nnew file mode 100644\nindex 000000000000..57a001fa2d7b\n--- /dev/null\n+++ b/Documentation/technical/rust-support.adoc\n@@ -0,0 +1,142 @@\n+Usage of Rust in Git\n+====================\n+\n+Objective\n+---------\n+Introduce Rust into Git incrementally to improve security and maintainability.\n+\n+Background\n+----------\n+Git has historically been written primarily in C, with some portions in shell,\n+Perl, or other languages. At the time it was originally written, this was\n+important for portability and was a logical choice for software development.\n+\n+:0: link:https://security.googleblog.com/2024/09/eliminating-memory-safety-vulnerabilities-Android.html\n+:1: link:https://www.cisa.gov/resources-tools/resources/product-security-bad-practices\n+\n+However, as time has progressed, we've seen an increased concern with memory\n+safety vulnerabilities and the development of newer languages, such as Rust,\n+that substantially limit or eliminate this class of vulnerabilities.\n+Development in a variety of projects has found that memory safety\n+vulnerabilities constitute about 70% of vulnerabilities of software in\n+languages that are not memory safe. For instance, {0}[one survey of Android]\n+found that memory safety vulnerabilities decreased from 76% to 24% over six\n+years due to an increase in memory safe code. Similarly, the U.S. government\n+is {1}[proposing to classify development in memory unsafe languages as a\n+Product Security Bad Practice\"].\n+\n+These risks are even more substantial when we consider the fact that Git is a\n+network-facing service. Many organizations run Git servers internally or use a\n+cloud-based forge, and the risk of accidental exposure or compromise of user\n+data is substantial. It's important to ensure that Git, whether it's used\n+locally or remotely, is robustly secure.\n+\n+In addition, C is a difficult language to write well and concisely. While it\n+is of course possible to do anything with C, it lacks built-in support for\n+niceties found in modern languages, such as hash tables, generics, typed\n+errors, and automatic destruction, and most modern language offer shorter, more\n+ergonomic syntax for expressing code. This is valuable functionality that can\n+allow Git to be developed more rapidly, more easily, by more developers of a\n+variety of levels, and with more confidence in the correctness of the code.\n+\n+For these reasons, adding Rust to Git is a sensible and prudent move that will\n+allow us to improve the quality of the code and potentially attract new developers.\n+\n+Goals\n+-----\n+1. Git continues to build, run, and pass tests on a wide variety of operating\n+ systems and architectures.\n+2. Transition from C to Rust is incremental; that is, code can be ported as it\n+ is convenient and Git does not need to transition all at once.\n+3. Git continues to support older operating systems in conformance with the\n+ platform support policy.\n+\n+Non-Goals\n+---------\n+1. Support for every possible operating system and architecture. Git already\n+ has a platform support policy which defines what is supported and we already\n+ exclude some operating systems for various reasons (e.g., lacking enough POSIX\n+ tools to pass the test suite).\n+2. Implementing C-only versions of Rust code or compiling a C-only Git. This\n+ would be difficult to maintain and would not offer the ergonomic benefits we\n+ desire.\n+\n+Design\n+------\n+Git will adopt Rust incrementally. This transition will start with the\n+creation of a static library that can be linked into the existing Git binaries.\n+At some point, we may wish to expose a dynamic library and compile the Git\n+binaries themselves using Rust. Using an incremental approach allows us to\n+determine as we go along how to structure our code in the best way for the\n+project and avoids the need to make hard, potentially disruptive, transitions\n+caused by porting a binary wholesale from one language to another that might\n+introduce bugs.\n+\n+Crates like libc or rustix define types like c_long, but in ways that are not\n+safe across platforms.\n+From https://docs.rs/rustix/latest/rustix/ffi/type.c_long.html:\n+\n+ This type will always be i32 or i64. Most notably, many Linux-based\n+ systems assume an i64, but Windows assumes i32. The C standard technically\n+ only requires that this type be a signed integer that is at least 32 bits\n+ and at least the size of an int, although in practice, no system would\n+ have a long that is neither an i32 nor i64.\n+\n+Also, note that other locations, such as\n+https://docs.rs/libc/latest/libc/type.c_long.html, just hardcode c_long as i64\n+even though C may mean i32 on some platforms.\n+\n+As such, using the c_long type would give us portability issues, and\n+perpetuate some of the bugs git has faced across platforms. Avoid using C's\n+types (long, unsigned, char, etc.), and switch to unambiguous types (e.g. i32\n+or i64) before trying to make C and Rust interoperate.\n+\n+Crates like libc and rustix may have also traditionally aided interoperability\n+with older versions of Rust (e.g. when worrying about stat[64] system calls),\n+but the Rust standard library in newer versions of Rust handle these concerns\n+in a platform agnostic way. There may arise cases where we need to consider\n+these crates, but for now we omit them.\n+\n+Tools like bindgen and cbindgen create C-styled unsafe Rust code rather than\n+idiomatic Rust; where possible, we prefer to switch to idiomatic Rust. Any\n+standard C library functions that are needed can be manually wrapped on the\n+Rust side.\n+\n+Rust upstream releases every six weeks and only supports the latest stable\n+release. While it is nice that upstream is active, we would like our software\n+releases to have a lifespan exceeding six weeks. To allow compiling our code\n+on a variety of systems, we will support the version of Rust in Debian stable,\n+plus, for a year after a new Debian stable is released, the version in Debian\n+oldstable.\n+\n+This provides an approximately three-year lifespan of support for a Rust\n+release and allows us to support a variety of operating systems and\n+architectures, including those for which Rust upstream does not build binaries.\n+Debian stable is the benchmark distribution used by many Rust projects when\n+determining supported Rust versions, and it is an extremely portable and\n+popular free software operating system that is available to the public at no\n+charge, which makes it a sensible choice for us as well.\n+\n+We may change this policy if the Rust project issues long-term support releases\n+or the Rust community and distributors agree on releases to target as if they\n+were long-term support releases.\n+\n+This version support policy necessitates that we be very careful about the\n+dependencies we include, since many Rust projects support only the latest\n+stable version. However, we typically have been careful about dependencies in\n+the first place, so this should not be a major departure from existing policy,\n+although it may be a change for some existing Rust developers.\n+\n+We will avoid including the `Cargo.lock` file in the repository and instead\n+specify minimum dependency versions in the `Cargo.toml` file. We want to allow\n+people to use newer versions of dependencies if necessary to support newer\n+platforms without needing to force upgrades of dependencies on all users, and\n+it provides additional flexibility for distribution maintainers.\n+\n+We do not plan to support beta or nightly versions of the Rust compiler. These\n+versions may change rapidly and especially parts of the toolchain such as\n+Clippy, the lint tool, can have false positives or add additional warnings with\n+too great of a frequency to be supportable by the project. However, we do plan\n+to support alternate compilers, such as the rust_codegen_gcc backend and gccrs\n+when they are stable and support our desired release versions. This will\n+provide greater support for more operating systems and architectures.\n-- \ngitgitgadget\n\n"},{"id":"524750","messageId":"03939951256baaaec3fcc690cfa38ee12fb553ce.1755921357.git.gitgitgadget@gmail.com","threadId":"63804","inReplyTo":"pull.1980.v3.git.git.1755921356.gitgitgadget@gmail.com","subject":"[PATCH v3 02/15] xdiff: introduce rust","fromName":"Ezekiel Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2025-08-23T03:55:43Z","receivedAt":"2025-08-23T03:56:03Z","isPatch":true,"sender":{"key":"ezekielnewren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/18313324?v=4"},"body":"From: Ezekiel Newren \n\nUpcoming patches will simplify xdiff, while also porting parts of it to\nRust. In preparation, add some stubs and setup the Rust build. For now,\nit is easier to let cargo build rust and have make or meson merely link\nagainst the static library that cargo builds. In line with ongoing\nlibification efforts, use multiple crates to allow more modularity on\nthe Rust side. xdiff is the crate that this series will focus on, but\nwe also introduce the interop crate for future patch series.\n\nIn order to facilitate interoperability between C and Rust, introduce C\ndefinitions for Rust primitive types in git-compat-util.h.\n\nSigned-off-by: Ezekiel Newren \n---\n .gitignore | 3 +++\n Makefile | 53 ++++++++++++++++++++++++++++----------\n build_rust.sh | 57 +++++++++++++++++++++++++++++++++++++++++\n git-compat-util.h | 17 ++++++++++++\n meson.build | 52 +++++++++++++++++++++++++++++++------\n rust/Cargo.toml | 6 +++++\n rust/interop/Cargo.toml | 14 ++++++++++\n rust/interop/src/lib.rs | 0\n rust/xdiff/Cargo.toml | 15 +++++++++++\n rust/xdiff/src/lib.rs | 0\n 10 files changed, 196 insertions(+), 21 deletions(-)\n create mode 100755 build_rust.sh\n create mode 100644 rust/Cargo.toml\n create mode 100644 rust/interop/Cargo.toml\n create mode 100644 rust/interop/src/lib.rs\n create mode 100644 rust/xdiff/Cargo.toml\n create mode 100644 rust/xdiff/src/lib.rs\n\ndiff --git a/.gitignore b/.gitignore\nindex 04c444404e4b..ff81e3580c4e 100644\n--- a/.gitignore\n+++ b/.gitignore\n@@ -254,3 +254,6 @@ Release/\n /contrib/buildsystems/out\n /contrib/libgit-rs/target\n /contrib/libgit-sys/target\n+/.idea/\n+/rust/target/\n+/rust/Cargo.lock\ndiff --git a/Makefile b/Makefile\nindex 70d1543b6b86..1ec0c1ee6603 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -919,6 +919,29 @@ TEST_SHELL_PATH = $(SHELL_PATH)\n \n LIB_FILE = libgit.a\n XDIFF_LIB = xdiff/lib.a\n+\n+EXTLIBS =\n+\n+ifeq ($(DEBUG), 1)\n+ RUST_BUILD_MODE = debug\n+else\n+ RUST_BUILD_MODE = release\n+endif\n+\n+RUST_TARGET_DIR = rust/target/$(RUST_BUILD_MODE)\n+RUST_FLAGS_FOR_C = -L$(RUST_TARGET_DIR)\n+\n+.PHONY: compile_rust\n+compile_rust:\n+\t./build_rust.sh . $(RUST_BUILD_MODE) xdiff\n+\n+EXTLIBS += ./$(RUST_TARGET_DIR)/libxdiff.a\n+\n+UNAME_S := $(shell uname -s)\n+ifeq ($(UNAME_S),Linux)\n+ EXTLIBS += -ldl\n+endif\n+\n REFTABLE_LIB = reftable/libreftable.a\n \n GENERATED_H += command-list.h\n@@ -1390,7 +1413,7 @@ UNIT_TEST_OBJS += $(UNIT_TEST_DIR)/lib-reftable.o\n \n # xdiff and reftable libs may in turn depend on what is in libgit.a\n GITLIBS = common-main.o $(LIB_FILE) $(XDIFF_LIB) $(REFTABLE_LIB) $(LIB_FILE)\n-EXTLIBS =\n+\n \n GIT_USER_AGENT = git/$(GIT_VERSION)\n \n@@ -2541,7 +2564,7 @@ git.sp git.s git.o: EXTRA_CPPFLAGS = \\\n \t'-DGIT_MAN_PATH=\"$(mandir_relative_SQ)\"' \\\n \t'-DGIT_INFO_PATH=\"$(infodir_relative_SQ)\"'\n \n-git$X: git.o GIT-LDFLAGS $(BUILTIN_OBJS) $(GITLIBS)\n+git$X: git.o GIT-LDFLAGS $(BUILTIN_OBJS) $(GITLIBS) compile_rust\n \t$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) \\\n \t\t$(filter %.o,$^) $(LIBS)\n \n@@ -2891,17 +2914,17 @@ headless-git.o: compat/win32/headless.c GIT-CFLAGS\n headless-git$X: headless-git.o git.res GIT-LDFLAGS\n \t$(QUIET_LINK)$(CC) $(ALL_CFLAGS) $(ALL_LDFLAGS) -mwindows -o $@ $< git.res\n \n-git-%$X: %.o GIT-LDFLAGS $(GITLIBS)\n+git-%$X: %.o GIT-LDFLAGS $(GITLIBS) compile_rust\n \t$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) $(LIBS)\n \n-git-imap-send$X: imap-send.o $(IMAP_SEND_BUILDDEPS) GIT-LDFLAGS $(GITLIBS)\n+git-imap-send$X: imap-send.o $(IMAP_SEND_BUILDDEPS) GIT-LDFLAGS $(GITLIBS) compile_rust\n \t$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \\\n \t\t$(IMAP_SEND_LDFLAGS) $(LIBS)\n \n-git-http-fetch$X: http.o http-walker.o http-fetch.o GIT-LDFLAGS $(GITLIBS)\n+git-http-fetch$X: http.o http-walker.o http-fetch.o GIT-LDFLAGS $(GITLIBS) compile_rust\n \t$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \\\n \t\t$(CURL_LIBCURL) $(LIBS)\n-git-http-push$X: http.o http-push.o GIT-LDFLAGS $(GITLIBS)\n+git-http-push$X: http.o http-push.o GIT-LDFLAGS $(GITLIBS) compile_rust\n \t$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \\\n \t\t$(CURL_LIBCURL) $(EXPAT_LIBEXPAT) $(LIBS)\n \n@@ -2911,11 +2934,11 @@ $(REMOTE_CURL_ALIASES): $(REMOTE_CURL_PRIMARY)\n \tln -s $< $@ 2>/dev/null || \\\n \tcp $< $@\n \n-$(REMOTE_CURL_PRIMARY): remote-curl.o http.o http-walker.o GIT-LDFLAGS $(GITLIBS)\n+$(REMOTE_CURL_PRIMARY): remote-curl.o http.o http-walker.o GIT-LDFLAGS $(GITLIBS) compile_rust\n \t$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \\\n \t\t$(CURL_LIBCURL) $(EXPAT_LIBEXPAT) $(LIBS)\n \n-scalar$X: scalar.o GIT-LDFLAGS $(GITLIBS)\n+scalar$X: scalar.o GIT-LDFLAGS $(GITLIBS) compile_rust\n \t$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) \\\n \t\t$(filter %.o,$^) $(LIBS)\n \n@@ -2925,6 +2948,7 @@ $(LIB_FILE): $(LIB_OBJS)\n $(XDIFF_LIB): $(XDIFF_OBJS)\n \t$(QUIET_AR)$(RM) $@ && $(AR) $(ARFLAGS) $@ $^\n \n+\n $(REFTABLE_LIB): $(REFTABLE_OBJS)\n \t$(QUIET_AR)$(RM) $@ && $(AR) $(ARFLAGS) $@ $^\n \n@@ -3294,7 +3318,7 @@ perf: all\n \n t/helper/test-tool$X: $(patsubst %,t/helper/%,$(TEST_BUILTINS_OBJS)) $(UNIT_TEST_DIR)/test-lib.o\n \n-t/helper/test-%$X: t/helper/test-%.o GIT-LDFLAGS $(GITLIBS)\n+t/helper/test-%$X: t/helper/test-%.o GIT-LDFLAGS $(GITLIBS) compile_rust\n \t$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) $(filter %.a,$^) $(LIBS)\n \n check-sha1:: t/helper/test-tool$X\n@@ -3756,7 +3780,10 @@ cocciclean:\n \t$(RM) -r .build/contrib/coccinelle\n \t$(RM) contrib/coccinelle/*.cocci.patch\n \n-clean: profile-clean coverage-clean cocciclean\n+rustclean:\n+\tcd rust && cargo clean\n+\n+clean: profile-clean coverage-clean cocciclean rustclean\n \t$(RM) -r .build $(UNIT_TEST_BIN)\n \t$(RM) GIT-TEST-SUITES\n \t$(RM) po/git.pot po/git-core.pot\n@@ -3911,13 +3938,13 @@ FUZZ_CXXFLAGS ?= $(ALL_CFLAGS)\n .PHONY: fuzz-all\n fuzz-all: $(FUZZ_PROGRAMS)\n \n-$(FUZZ_PROGRAMS): %: %.o oss-fuzz/dummy-cmd-main.o $(GITLIBS) GIT-LDFLAGS\n+$(FUZZ_PROGRAMS): %: %.o oss-fuzz/dummy-cmd-main.o $(GITLIBS) GIT-LDFLAGS compile_rust\n \t$(QUIET_LINK)$(FUZZ_CXX) $(FUZZ_CXXFLAGS) -o $@ $(ALL_LDFLAGS) \\\n \t\t-Wl,--allow-multiple-definition \\\n \t\t$(filter %.o,$^) $(filter %.a,$^) $(LIBS) $(LIB_FUZZING_ENGINE)\n \n $(UNIT_TEST_PROGS): $(UNIT_TEST_BIN)/%$X: $(UNIT_TEST_DIR)/%.o $(UNIT_TEST_OBJS) \\\n-\t$(GITLIBS) GIT-LDFLAGS\n+\t$(GITLIBS) GIT-LDFLAGS compile_rust\n \t$(call mkdir_p_parent_template)\n \t$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) \\\n \t\t$(filter %.o,$^) $(filter %.a,$^) $(LIBS)\n@@ -3936,7 +3963,7 @@ $(UNIT_TEST_DIR)/clar.suite: $(UNIT_TEST_DIR)/clar-decls.h $(UNIT_TEST_DIR)/gene\n $(UNIT_TEST_DIR)/clar/clar.o: $(UNIT_TEST_DIR)/clar.suite\n $(CLAR_TEST_OBJS): $(UNIT_TEST_DIR)/clar-decls.h\n $(CLAR_TEST_OBJS): EXTRA_CPPFLAGS = -I$(UNIT_TEST_DIR)\n-$(CLAR_TEST_PROG): $(UNIT_TEST_DIR)/clar.suite $(CLAR_TEST_OBJS) $(GITLIBS) GIT-LDFLAGS\n+$(CLAR_TEST_PROG): $(UNIT_TEST_DIR)/clar.suite $(CLAR_TEST_OBJS) $(GITLIBS) GIT-LDFLAGS compile_rust\n \t$(call mkdir_p_parent_template)\n \t$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) $(LIBS)\n \ndiff --git a/build_rust.sh b/build_rust.sh\nnew file mode 100755\nindex 000000000000..192385a1d961\n--- /dev/null\n+++ b/build_rust.sh\n@@ -0,0 +1,57 @@\n+#!/bin/sh\n+\n+\n+rustc -vV || exit $?\n+cargo --version || exit $?\n+\n+dir_git_root=${0%/*}\n+dir_build=$1\n+rust_build_profile=$2\n+crate=$3\n+\n+dir_rust=$dir_git_root/rust\n+\n+if [ \"$dir_git_root\" = \"\" ]; then\n+ echo \"did not specify the directory for the root of git\"\n+ exit 1\n+fi\n+\n+if [ \"$dir_build\" = \"\" ]; then\n+ echo \"did not specify the build directory\"\n+ exit 1\n+fi\n+\n+if [ \"$rust_build_profile\" = \"\" ]; then\n+ echo \"did not specify the rust_build_profile\"\n+ exit 1\n+fi\n+\n+if [ \"$rust_build_profile\" = \"release\" ]; then\n+ rust_args=\"--release\"\n+ export RUSTFLAGS=''\n+elif [ \"$rust_build_profile\" = \"debug\" ]; then\n+ rust_args=\"\"\n+ export RUSTFLAGS='-C debuginfo=2 -C opt-level=1 -C force-frame-pointers=yes'\n+else\n+ echo \"illegal rust_build_profile value $rust_build_profile\"\n+ exit 1\n+fi\n+\n+cd $dir_rust && cargo clean && pwd && cargo build -p $crate $rust_args; cd $dir_git_root\n+\n+libfile=\"lib${crate}.a\"\n+if rustup show active-toolchain | grep windows-msvc; then\n+ libfile=\"${crate}.lib\"\n+fi\n+dst=$dir_build/$libfile\n+\n+if [ \"$dir_git_root\" != \"$dir_build\" ]; then\n+ src=$dir_rust/target/$rust_build_profile/$libfile\n+ if [ ! -f $src ]; then\n+ echo >&2 \"::error:: cannot find path of static library $src is not a file or does not exist\"\n+ exit 5\n+ fi\n+\n+ rm $dst 2>/dev/null\n+ mv $src $dst\n+fi\ndiff --git a/git-compat-util.h b/git-compat-util.h\nindex 4678e21c4cb8..82dc99764ac0 100644\n--- a/git-compat-util.h\n+++ b/git-compat-util.h\n@@ -196,6 +196,23 @@ static inline int is_xplatform_dir_sep(int c)\n #include \"compat/msvc.h\"\n #endif\n \n+/* rust types */\n+typedef uint8_t u8;\n+typedef uint16_t u16;\n+typedef uint32_t u32;\n+typedef uint64_t u64;\n+\n+typedef int8_t i8;\n+typedef int16_t i16;\n+typedef int32_t i32;\n+typedef int64_t i64;\n+\n+typedef float f32;\n+typedef double f64;\n+\n+typedef size_t usize;\n+typedef ptrdiff_t isize;\n+\n /* used on Mac OS X */\n #ifdef PRECOMPOSE_UNICODE\n #include \"compat/precompose_utf8.h\"\ndiff --git a/meson.build b/meson.build\nindex 596f5ac7110e..324f968338b9 100644\n--- a/meson.build\n+++ b/meson.build\n@@ -267,6 +267,40 @@ version_gen_environment.set('GIT_DATE', get_option('build_date'))\n version_gen_environment.set('GIT_USER_AGENT', get_option('user_agent'))\n version_gen_environment.set('GIT_VERSION', get_option('version'))\n \n+if get_option('optimization') in ['2', '3', 's', 'z']\n+ rust_build_profile = 'release'\n+else\n+ rust_build_profile = 'debug'\n+endif\n+\n+# Run `rustup show active-toolchain` and capture output\n+rustup_out = run_command('rustup', 'show', 'active-toolchain',\n+ check: true).stdout().strip()\n+\n+rust_crates = ['xdiff']\n+rust_builds = []\n+\n+foreach crate : rust_crates\n+ if rustup_out.contains('windows-msvc')\n+ libfile = crate + '.lib'\n+ else\n+ libfile = 'lib' + crate + '.a'\n+ endif\n+\n+ rust_builds += custom_target(\n+ 'rust_build_'+crate,\n+ output: libfile,\n+ build_by_default: true,\n+ build_always_stale: true,\n+ command: [\n+ meson.project_source_root() / 'build_rust.sh',\n+ meson.current_build_dir(), rust_build_profile, crate,\n+ ],\n+ install: false,\n+ )\n+endforeach\n+\n+\n compiler = meson.get_compiler('c')\n \n libgit_sources = [\n@@ -1678,14 +1712,16 @@ version_def_h = custom_target(\n libgit_sources += version_def_h\n \n libgit = declare_dependency(\n- link_with: static_library('git',\n- sources: libgit_sources,\n- c_args: libgit_c_args + [\n- '-DGIT_VERSION_H=\"' + version_def_h.full_path() + '\"',\n- ],\n- dependencies: libgit_dependencies,\n- include_directories: libgit_include_directories,\n- ),\n+ link_with: [\n+ static_library('git',\n+ sources: libgit_sources,\n+ c_args: libgit_c_args + [\n+ '-DGIT_VERSION_H=\"' + version_def_h.full_path() + '\"',\n+ ],\n+ dependencies: libgit_dependencies,\n+ include_directories: libgit_include_directories,\n+ ),\n+ ] + rust_builds,\n compile_args: libgit_c_args,\n dependencies: libgit_dependencies,\n include_directories: libgit_include_directories,\ndiff --git a/rust/Cargo.toml b/rust/Cargo.toml\nnew file mode 100644\nindex 000000000000..ed3d79d7f827\n--- /dev/null\n+++ b/rust/Cargo.toml\n@@ -0,0 +1,6 @@\n+[workspace]\n+members = [\n+ \"xdiff\",\n+ \"interop\",\n+]\n+resolver = \"2\"\ndiff --git a/rust/interop/Cargo.toml b/rust/interop/Cargo.toml\nnew file mode 100644\nindex 000000000000..045e3b01cfad\n--- /dev/null\n+++ b/rust/interop/Cargo.toml\n@@ -0,0 +1,14 @@\n+[package]\n+name = \"interop\"\n+version = \"0.1.0\"\n+edition = \"2021\"\n+\n+[lib]\n+name = \"interop\"\n+path = \"src/lib.rs\"\n+## staticlib to generate xdiff.a for use by gcc\n+## cdylib (optional) to generate xdiff.so for use by gcc\n+## rlib is required by the rust unit tests\n+crate-type = [\"staticlib\", \"rlib\"]\n+\n+[dependencies]\ndiff --git a/rust/interop/src/lib.rs b/rust/interop/src/lib.rs\nnew file mode 100644\nindex 000000000000..e69de29bb2d1\ndiff --git a/rust/xdiff/Cargo.toml b/rust/xdiff/Cargo.toml\nnew file mode 100644\nindex 000000000000..eb7966aada64\n--- /dev/null\n+++ b/rust/xdiff/Cargo.toml\n@@ -0,0 +1,15 @@\n+[package]\n+name = \"xdiff\"\n+version = \"0.1.0\"\n+edition = \"2021\"\n+\n+[lib]\n+name = \"xdiff\"\n+path = \"src/lib.rs\"\n+## staticlib to generate xdiff.a for use by gcc\n+## cdylib (optional) to generate xdiff.so for use by gcc\n+## rlib is required by the rust unit tests\n+crate-type = [\"staticlib\", \"rlib\"]\n+\n+[dependencies]\n+interop = { path = \"../interop\" }\ndiff --git a/rust/xdiff/src/lib.rs b/rust/xdiff/src/lib.rs\nnew file mode 100644\nindex 000000000000..e69de29bb2d1\n-- \ngitgitgadget\n\n"},{"id":"524752","messageId":"a98d9e4d21baebb9c774413d6a9918e91691fe2c.1755921357.git.gitgitgadget@gmail.com","threadId":"63804","inReplyTo":"pull.1980.v3.git.git.1755921356.gitgitgadget@gmail.com","subject":"[PATCH v3 03/15] github workflows: install rust","fromName":"Ezekiel Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2025-08-23T03:55:44Z","receivedAt":"2025-08-23T03:56:05Z","isPatch":true,"sender":{"key":"ezekielnewren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/18313324?v=4"},"body":"From: Ezekiel Newren \n\nPrefer using actions-rs/toolchain@v1 where possible to install rustup,\nbut for docker targets use a script to install rustup. Consolidate the\nRust toolchain definitions in main.yaml. Use install-rust-toolchain.sh\nto ensure the correct toolchain is used. Five overrides are used in\nmain.yaml:\n\n * On Windows: Rust didn't resolve the bcrypt library on Windows\n correctly until version 1.78.0. Also since rustup mis-identifies\n the Rust toolchain, the Rust target triple must be set to\n x86_64-pc-windows-gnu for make (win build), and\n x86_64-pc-windows-msvc for meson (win+Meson build).\n * On musl: libc differences, such as ftruncate64 vs ftruncate, were\n not accounted for until Rust version 1.72.0. No older version of\n Rust will work on musl for our needs.\n * In a 32-bit docker container running on a 64-bit host, we need to\n override the Rust target triple. This is because rustup asks the\n kernel for the bitness of the system and it says 64, even though\n the container is 32-bit. This also allows us to remove the\n BITNESS environment variable in ci/lib.sh.\n\nThe logic for selecting library names was initially provided in a patch\nfrom Johannes, but was reworked and squashed into this commit.\n\nHelped-by: Johannes Schindelin \nSigned-off-by: Ezekiel Newren \n---\n .github/workflows/main.yml | 61 +++++++++++++++++++++++++++++++++++-\n ci/install-dependencies.sh | 14 ++++-----\n ci/install-rust-toolchain.sh | 30 ++++++++++++++++++\n ci/install-rustup.sh | 25 +++++++++++++++\n ci/lib.sh | 1 +\n ci/make-test-artifacts.sh | 9 ++++++\n ci/run-build-and-tests.sh | 13 ++++++++\n 7 files changed, 145 insertions(+), 8 deletions(-)\n create mode 100755 ci/install-rust-toolchain.sh\n create mode 100755 ci/install-rustup.sh\n\ndiff --git a/.github/workflows/main.yml b/.github/workflows/main.yml\nindex 7dbf9f7f123c..2fa5fab0fa83 100644\n--- a/.github/workflows/main.yml\n+++ b/.github/workflows/main.yml\n@@ -26,6 +26,13 @@ jobs:\n outputs:\n enabled: ${{ steps.check-ref.outputs.enabled }}${{ steps.skip-if-redundant.outputs.enabled }}\n skip_concurrent: ${{ steps.check-ref.outputs.skip_concurrent }}\n+ rust_version_minimum: 1.61.0\n+ rust_version_windows: 1.78.0\n+ rust_version_musl: 1.72.0\n+ ## the rust target is inferred by rustup unless specified\n+ rust_target_windows_make: x86_64-pc-windows-gnu\n+ rust_target_windows_meson: x86_64-pc-windows-msvc\n+ rust_target_32bit_linux: i686-unknown-linux-gnu\n steps:\n - name: try to clone ci-config branch\n run: |\n@@ -108,12 +115,26 @@ jobs:\n needs: ci-config\n if: needs.ci-config.outputs.enabled == 'yes'\n runs-on: windows-latest\n+ env:\n+ CARGO_HOME: \"/c/Users/runneradmin/.cargo\"\n concurrency:\n group: windows-build-${{ github.ref }}\n cancel-in-progress: ${{ needs.ci-config.outputs.skip_concurrent == 'yes' }}\n steps:\n - uses: actions/checkout@v4\n - uses: git-for-windows/setup-git-for-windows-sdk@v1\n+ - name: Install rustup via github actions\n+ uses: actions-rs/toolchain@v1\n+ with:\n+ toolchain: stable\n+ profile: minimal\n+ override: false\n+ - name: Install Rust toolchain\n+ shell: bash\n+ env:\n+ RUST_VERSION: ${{ needs.ci-config.outputs.rust_version_windows }}\n+ RUST_TARGET: ${{ needs.ci-config.outputs.rust_target_windows_make }}\n+ run: ci/install-rust-toolchain.sh\n - name: build\n shell: bash\n env:\n@@ -254,12 +275,26 @@ jobs:\n needs: ci-config\n if: needs.ci-config.outputs.enabled == 'yes'\n runs-on: windows-latest\n+ env:\n+ CARGO_HOME: \"/c/Users/runneradmin/.cargo\"\n concurrency:\n group: windows-meson-build-${{ github.ref }}\n cancel-in-progress: ${{ needs.ci-config.outputs.skip_concurrent == 'yes' }}\n steps:\n - uses: actions/checkout@v4\n - uses: actions/setup-python@v5\n+ - name: Install rustup via github actions\n+ uses: actions-rs/toolchain@v1\n+ with:\n+ toolchain: stable\n+ profile: minimal\n+ override: false\n+ - name: Install Rust toolchain\n+ shell: bash\n+ env:\n+ RUST_VERSION: ${{ needs.ci-config.outputs.rust_version_windows }}\n+ RUST_TARGET: ${{ needs.ci-config.outputs.rust_target_windows_meson }}\n+ run: ci/install-rust-toolchain.sh\n - name: Set up dependencies\n shell: pwsh\n run: pip install meson ninja\n@@ -329,11 +364,24 @@ jobs:\n jobname: ${{matrix.vector.jobname}}\n CI_JOB_IMAGE: ${{matrix.vector.pool}}\n TEST_OUTPUT_DIRECTORY: ${{github.workspace}}/t\n+ CARGO_HOME: \"/Users/runner/.cargo\"\n runs-on: ${{matrix.vector.pool}}\n steps:\n - uses: actions/checkout@v4\n - run: ci/install-dependencies.sh\n- - run: ci/run-build-and-tests.sh\n+ - name: Install rustup via github actions\n+ uses: actions-rs/toolchain@v1\n+ with:\n+ toolchain: stable\n+ profile: minimal\n+ override: false\n+ - name: Install Rust toolchain\n+ shell: bash\n+ env:\n+ RUST_VERSION: ${{ needs.ci-config.outputs.rust_version_minimum }}\n+ run: ci/install-rust-toolchain.sh\n+ - name: Run build and tests\n+ run: ci/run-build-and-tests.sh\n - name: print test failures\n if: failure() && env.FAILED_TEST_ARTIFACTS != ''\n run: ci/print-test-failures.sh\n@@ -393,9 +441,11 @@ jobs:\n cc: gcc\n - jobname: linux-musl-meson\n image: alpine:latest\n+ rust_version_override: ${{ needs.ci-config.outputs.rust_version_musl }}\n # Supported until 2025-04-02.\n - jobname: linux32\n image: i386/ubuntu:focal\n+ rust_target_override: ${{ needs.ci-config.outputs.rust_target_32bit_linux }}\n - jobname: pedantic\n image: fedora:latest\n # A RHEL 8 compatible distro. Supported until 2029-05-31.\n@@ -408,7 +458,9 @@ jobs:\n jobname: ${{matrix.vector.jobname}}\n CC: ${{matrix.vector.cc}}\n CI_JOB_IMAGE: ${{matrix.vector.image}}\n+ CI_IS_DOCKER: \"true\"\n CUSTOM_PATH: /custom\n+ CARGO_HOME: /home/builder/.cargo\n runs-on: ubuntu-latest\n container: ${{matrix.vector.image}}\n steps:\n@@ -433,6 +485,13 @@ jobs:\n - run: ci/install-dependencies.sh\n - run: useradd builder --create-home\n - run: chown -R builder .\n+ - name: Install rustup via script\n+ run: sudo --preserve-env --set-home --user=builder ci/install-rustup.sh\n+ - name: Install Rust toolchain\n+ env:\n+ RUST_VERSION: ${{ matrix.vector.rust_version_override || needs.ci-config.outputs.rust_version_minimum }}\n+ RUST_TARGET: ${{ matrix.vector.rust_target_override || '' }}\n+ run: sudo --preserve-env --set-home --user=builder ci/install-rust-toolchain.sh\n - run: sudo --preserve-env --set-home --user=builder ci/run-build-and-tests.sh\n - name: print test failures\n if: failure() && env.FAILED_TEST_ARTIFACTS != ''\ndiff --git a/ci/install-dependencies.sh b/ci/install-dependencies.sh\nindex d061a4729339..7801075821ba 100755\n--- a/ci/install-dependencies.sh\n+++ b/ci/install-dependencies.sh\n@@ -24,14 +24,14 @@ fi\n \n case \"$distro\" in\n alpine-*)\n-\tapk add --update shadow sudo meson ninja-build gcc libc-dev curl-dev openssl-dev expat-dev gettext \\\n+\tapk add --update shadow sudo meson ninja-build gcc libc-dev curl curl-dev openssl-dev expat-dev gettext \\\n \t\tzlib-ng-dev pcre2-dev python3 musl-libintl perl-utils ncurses \\\n \t\tapache2 apache2-http2 apache2-proxy apache2-ssl apache2-webdav apr-util-dbd_sqlite3 \\\n \t\tbash cvs gnupg perl-cgi perl-dbd-sqlite perl-io-tty >/dev/null\n \t;;\n fedora-*|almalinux-*)\n \tdnf -yq update >/dev/null &&\n-\tdnf -yq install shadow-utils sudo make gcc findutils diffutils perl python3 gawk gettext zlib-devel expat-devel openssl-devel curl-devel pcre2-devel >/dev/null\n+\tdnf -yq install shadow-utils sudo make gcc findutils diffutils perl python3 gawk gettext zlib-devel expat-devel openssl-devel curl curl-devel pcre2-devel >/dev/null\n \t;;\n ubuntu-*|i386/ubuntu-*|debian-*)\n \t# Required so that apt doesn't wait for user input on certain packages.\n@@ -55,8 +55,8 @@ ubuntu-*|i386/ubuntu-*|debian-*)\n \tsudo apt-get -q update\n \tsudo apt-get -q -y install \\\n \t\t$LANGUAGES apache2 cvs cvsps git gnupg $SVN \\\n-\t\tmake libssl-dev libcurl4-openssl-dev libexpat-dev wget sudo default-jre \\\n-\t\ttcl tk gettext zlib1g-dev perl-modules liberror-perl libauthen-sasl-perl \\\n+\t\tmake libssl-dev curl libcurl4-openssl-dev libexpat-dev wget sudo default-jre \\\n+\t\ttcl tk gettext zlib1g zlib1g-dev perl-modules liberror-perl libauthen-sasl-perl \\\n \t\tlibemail-valid-perl libio-pty-perl libio-socket-ssl-perl libnet-smtp-ssl-perl libdbd-sqlite3-perl libcgi-pm-perl \\\n \t\tlibsecret-1-dev libpcre2-dev meson ninja-build pkg-config \\\n \t\t${CC_PACKAGE:-${CC:-gcc}} $PYTHON_PACKAGE\n@@ -121,13 +121,13 @@ ClangFormat)\n \t;;\n StaticAnalysis)\n \tsudo apt-get -q update\n-\tsudo apt-get -q -y install coccinelle libcurl4-openssl-dev libssl-dev \\\n+\tsudo apt-get -q -y install coccinelle curl libcurl4-openssl-dev libssl-dev \\\n \t\tlibexpat-dev gettext make\n \t;;\n sparse)\n \tsudo apt-get -q update -q\n-\tsudo apt-get -q -y install libssl-dev libcurl4-openssl-dev \\\n-\t\tlibexpat-dev gettext zlib1g-dev sparse\n+\tsudo apt-get -q -y install libssl-dev curl libcurl4-openssl-dev \\\n+\t\tlibexpat-dev gettext zlib1g zlib1g-dev sparse\n \t;;\n Documentation)\n \tsudo apt-get -q update\ndiff --git a/ci/install-rust-toolchain.sh b/ci/install-rust-toolchain.sh\nnew file mode 100755\nindex 000000000000..06a29c4cfa17\n--- /dev/null\n+++ b/ci/install-rust-toolchain.sh\n@@ -0,0 +1,30 @@\n+#!/bin/sh\n+\n+if [ \"$CARGO_HOME\" = \"\" ]; then\n+ echo >&2 \"::error:: CARGO_HOME is not set\"\n+ exit 2\n+fi\n+export PATH=\"$CARGO_HOME/bin:$PATH\"\n+rustup -vV || exit $?\n+\n+## Enforce the correct Rust toolchain\n+rustup override unset || true\n+\n+## install a specific version of rust\n+if [ \"$RUST_TARGET\" != \"\" ]; then\n+ rustup default --force-non-host \"$RUST_VERSION-$RUST_TARGET\" || exit $?\n+else\n+ rustup default \"$RUST_VERSION\" || exit $?\n+fi\n+\n+rustc -vV || exit $?\n+\n+RE_RUST_TARGET=\"$RUST_TARGET\"\n+if [ \"$RUST_TARGET\" = \"\" ]; then\n+ RE_RUST_TARGET=\"[^ ]+\"\n+fi\n+\n+if ! rustup show active-toolchain | grep -E \"^$RUST_VERSION-$RE_RUST_TARGET \\(default\\)$\"; then\n+ echo >&2 \"::error:: wrong Rust toolchain, active-toolchain: $(rustup show active-toolchain)\"\n+ exit 3\n+fi\ndiff --git a/ci/install-rustup.sh b/ci/install-rustup.sh\nnew file mode 100755\nindex 000000000000..0036231aeea7\n--- /dev/null\n+++ b/ci/install-rustup.sh\n@@ -0,0 +1,25 @@\n+#!/bin/sh\n+\n+## github workflows actions-rs/toolchain@v1 doesn't work for docker\n+## targets. This script should only be used if the ci pipeline\n+## doesn't support installing rust on a particular target.\n+\n+if [ \"$(id -u)\" -eq 0 ]; then\n+ echo >&2 \"::warning:: installing rust as root\"\n+fi\n+\n+if [ \"$CARGO_HOME\" = \"\" ]; then\n+ echo >&2 \"::error:: CARGO_HOME is not set\"\n+ exit 2\n+fi\n+\n+export RUSTUP_HOME=$CARGO_HOME\n+\n+## install rustup\n+curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- --default-toolchain none -y\n+if [ ! -f $CARGO_HOME/env ]; then\n+ echo \"PATH=$CARGO_HOME/bin:\\$PATH\" > $CARGO_HOME/env\n+fi\n+. $CARGO_HOME/env\n+\n+rustup -vV\ndiff --git a/ci/lib.sh b/ci/lib.sh\nindex f561884d4016..a7992b22fdc9 100755\n--- a/ci/lib.sh\n+++ b/ci/lib.sh\n@@ -1,5 +1,6 @@\n # Library of functions shared by all CI scripts\n \n+\n if test true = \"$GITHUB_ACTIONS\"\n then\n \tbegin_group () {\ndiff --git a/ci/make-test-artifacts.sh b/ci/make-test-artifacts.sh\nindex 74141af0cc74..e37ed7030cdf 100755\n--- a/ci/make-test-artifacts.sh\n+++ b/ci/make-test-artifacts.sh\n@@ -7,6 +7,15 @@ mkdir -p \"$1\" # in case ci/lib.sh decides to quit early\n \n . ${0%/*}/lib.sh\n \n+## ensure rustup is in the PATH variable\n+if [ \"$CARGO_HOME\" = \"\" ]; then\n+ echo >&2 \"::error:: CARGO_HOME is not set\"\n+ exit 2\n+fi\n+export PATH=\"$CARGO_HOME/bin:$PATH\"\n+\n+rustc -vV\n+\n group Build make artifacts-tar ARTIFACTS_DIRECTORY=\"$1\"\n \n check_unignored_build_artifacts\ndiff --git a/ci/run-build-and-tests.sh b/ci/run-build-and-tests.sh\nindex 01823fd0f140..22b61e2812db 100755\n--- a/ci/run-build-and-tests.sh\n+++ b/ci/run-build-and-tests.sh\n@@ -5,6 +5,15 @@\n \n . ${0%/*}/lib.sh\n \n+## ensure rustup is in the PATH variable\n+if [ \"$CARGO_HOME\" = \"\" ]; then\n+ echo >&2 \"::error:: CARGO_HOME is not set\"\n+ exit 2\n+fi\n+. $CARGO_HOME/env\n+\n+rustc -vV || exit $?\n+\n run_tests=t\n \n case \"$jobname\" in\n@@ -72,5 +81,9 @@ case \"$jobname\" in\n \t;;\n esac\n \n+if [ -d \"$CARGO_HOME\" ]; then\n+ rm -rf $CARGO_HOME\n+fi\n+\n check_unignored_build_artifacts\n save_good_tree\n-- \ngitgitgadget\n\n"},{"id":"524753","messageId":"0d2b39c3e0324de129404035def6f54754cc6225.1755921357.git.gitgitgadget@gmail.com","threadId":"63804","inReplyTo":"pull.1980.v3.git.git.1755921356.gitgitgadget@gmail.com","subject":"[PATCH v3 04/15] win+Meson: do allow linking with the Rust-built xdiff","fromName":"Johannes Schindelin via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2025-08-23T03:55:45Z","receivedAt":"2025-08-23T03:56:06Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"From: Johannes Schindelin \n\nWhen linking against the Rust-built `xdiff`, there is now a new required\ndependency: Without _also_ linking to the system library `userenv`, the\ncompile would fail with this error message:\n\n xdiff.lib(std-c85e9beb7923f636.std.df32d1bc89881d89-cgu.0.rcgu.o) :\n error LNK2019: unresolved external symbol __imp_GetUserProfileDirectoryW\n referenced in function _ZN3std3env8home_dir17hfd1c3b6676cd78f6E\n\nTherefore, just like we do in case of Makefile-based builds on Windows,\nwe now also link to that library when building with Meson.\n\nNote that if we only have Rust depend upon libuserenv then at link time\nGCC would complain about:\n\n undefined reference to `GetUserProfileDirectoryW'\n\nApparently there is _some_ closure that gets compiled in that requires\nthis function, and that in turn forces Git to link to libuserenv.\n\nThis is a new requirement, and therefore has not been made part of the\n\"minimal Git for Windows SDK\".\n\nIn the near future, I intend to include it, but for now let's just\nensure that the file is added manually if it is missing.\n\nSigned-off-by: Johannes Schindelin \n[en: Squashed a few of Johannes's patches, and moved lib userenv\n handling from an earlier patch]\nSigned-off-by: Ezekiel Newren \n---\n .github/workflows/main.yml | 8 ++++++++\n config.mak.uname | 4 ++++\n meson.build | 1 +\n 3 files changed, 13 insertions(+)\n\ndiff --git a/.github/workflows/main.yml b/.github/workflows/main.yml\nindex 2fa5fab0fa83..0f7396621df8 100644\n--- a/.github/workflows/main.yml\n+++ b/.github/workflows/main.yml\n@@ -123,6 +123,14 @@ jobs:\n steps:\n - uses: actions/checkout@v4\n - uses: git-for-windows/setup-git-for-windows-sdk@v1\n+ - name: ensure that libuserenv.a is present\n+ shell: bash\n+ run: |\n+ cd /mingw64/lib && {\n+ test -f libuserenv.a ||\n+ /c/Program\\ Files/Git/mingw64/bin/curl -Lo libuserenv.a \\\n+ https://github.com/git-for-windows/git-sdk-64/raw/HEAD/mingw64/lib/libuserenv.a\n+ }\n - name: Install rustup via github actions\n uses: actions-rs/toolchain@v1\n with:\ndiff --git a/config.mak.uname b/config.mak.uname\nindex 3e26bb074a4b..6805e3778a16 100644\n--- a/config.mak.uname\n+++ b/config.mak.uname\n@@ -740,6 +740,10 @@ ifeq ($(uname_S),MINGW)\n \t\tCOMPAT_CFLAGS += -D_USE_32BIT_TIME_T\n \t\tBASIC_LDFLAGS += -Wl,--large-address-aware\n endif\n+\n+\t# Unfortunately now needed because of Rust\n+\tEXTLIBS += -luserenv\n+\n \tCC = gcc\n \tCOMPAT_CFLAGS += -D__USE_MINGW_ANSI_STDIO=0 -DDETECT_MSYS_TTY \\\n \t\t-fstack-protector-strong\ndiff --git a/meson.build b/meson.build\nindex 324f968338b9..5aa9901bfc0f 100644\n--- a/meson.build\n+++ b/meson.build\n@@ -1267,6 +1267,7 @@ elif host_machine.system() == 'windows'\n ]\n \n libgit_dependencies += compiler.find_library('ntdll')\n+ libgit_dependencies += compiler.find_library('userenv')\n libgit_include_directories += 'compat/win32'\n if compiler.get_id() == 'msvc'\n libgit_include_directories += 'compat/vcbuild/include'\n-- \ngitgitgadget\n\n"},{"id":"524754","messageId":"e65488ab993f429174b1f90bc5d810d64347ee6b.1755921357.git.gitgitgadget@gmail.com","threadId":"63804","inReplyTo":"pull.1980.v3.git.git.1755921356.gitgitgadget@gmail.com","subject":"[PATCH v3 05/15] github workflows: upload Cargo.lock","fromName":"Ezekiel Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2025-08-23T03:55:46Z","receivedAt":"2025-08-23T03:56:07Z","isPatch":true,"sender":{"key":"ezekielnewren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/18313324?v=4"},"body":"From: Ezekiel Newren \n\nMake each ci workflow upload its Cargo.lock file as a build artifact so\nthat we can audit build dependencies.\n\nSigned-off-by: Ezekiel Newren \n---\n .github/workflows/main.yml | 20 ++++++++++++++++++++\n 1 file changed, 20 insertions(+)\n\ndiff --git a/.github/workflows/main.yml b/.github/workflows/main.yml\nindex 0f7396621df8..0f8785a676c3 100644\n--- a/.github/workflows/main.yml\n+++ b/.github/workflows/main.yml\n@@ -156,6 +156,11 @@ jobs:\n with:\n name: windows-artifacts\n path: artifacts\n+ - name: upload Cargo.lock\n+ uses: actions/upload-artifact@v4\n+ with:\n+ name: cargo-lock-windows\n+ path: rust/Cargo.lock\n windows-test:\n name: win test\n runs-on: windows-latest\n@@ -317,6 +322,11 @@ jobs:\n with:\n name: windows-meson-artifacts\n path: build\n+ - name: Upload Cargo.lock\n+ uses: actions/upload-artifact@v4\n+ with:\n+ name: cargo-lock-windows-meson\n+ path: rust/Cargo.lock\n windows-meson-test:\n name: win+Meson test\n runs-on: windows-latest\n@@ -399,6 +409,11 @@ jobs:\n with:\n name: failed-tests-${{matrix.vector.jobname}}\n path: ${{env.FAILED_TEST_ARTIFACTS}}\n+ - name: Upload Cargo.lock\n+ uses: actions/upload-artifact@v4\n+ with:\n+ name: cargo-lock-${{matrix.vector.jobname}}\n+ path: rust/Cargo.lock\n fuzz-smoke-test:\n name: fuzz smoke test\n needs: ci-config\n@@ -510,6 +525,11 @@ jobs:\n with:\n name: failed-tests-${{matrix.vector.jobname}}\n path: ${{env.FAILED_TEST_ARTIFACTS}}\n+ - name: Upload Cargo.lock\n+ uses: actions/upload-artifact@v4\n+ with:\n+ name: cargo-lock-${{matrix.vector.jobname}}\n+ path: rust/Cargo.lock\n static-analysis:\n needs: ci-config\n if: needs.ci-config.outputs.enabled == 'yes'\n-- \ngitgitgadget\n\n"},{"id":"524755","messageId":"db5d22b188740bcb830e4ccf7f19dcc4e6b557bd.1755921357.git.gitgitgadget@gmail.com","threadId":"63804","inReplyTo":"pull.1980.v3.git.git.1755921356.gitgitgadget@gmail.com","subject":"[PATCH v3 06/15] ivec: create a vector type that is interoperable between C and Rust","fromName":"Ezekiel Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2025-08-23T03:55:47Z","receivedAt":"2025-08-23T03:56:08Z","isPatch":true,"sender":{"key":"ezekielnewren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/18313324?v=4"},"body":"From: Ezekiel Newren \n\nTrying to use Rust's Vec in C, or git's ALLOC_GROW() macros (via\nwrapper functions) in Rust is painful because:\n\n * C doing vector things the Rust way would require wrapper functions,\n and Rust doing vector things the C way would require wrapper\n functions, so ivec was created to ensure a consistent contract\n between the 2 languages for how to manipulate a vector.\n * Currently, Rust defines its own 'Vec' type that is generic, but its\n memory allocator and struct layout weren't designed for\n interoperability with C (or any language for that matter), meaning\n that the C side cannot push to or expand a 'Vec' without defining\n wrapper functions in Rust that C can call. Without special care,\n the two languages might use different allocators (malloc/free on\n the C side, and possibly something else in Rust), which would make\n it difficult for a function in one language to free elements\n allocated by a call from a function in the other language.\n * Similarly, git defines ALLOC_GROW() and related macros in\n git-compat-util.h. While we could add functions allowing Rust to\n invoke something similar to those macros, passing three variables\n (pointer, length, allocated_size) instead of a single variable\n (vector) across the language boundary requires more cognitive\n overhead for readers to keep track of and makes it easier to make\n mistakes. Further, for low-level components that we want to\n eventually convert to pure Rust, such triplets would feel very out\n of place.\n\nTo address these issue, introduce a new type, ivec -- short for\ninteroperable vector. (We refer to it as 'ivec' generally, though on\nthe Rust side the struct is called IVec to match Rust style.) This new\ntype is specifically designed for FFI purposes, so that both languages\nhandle the vector in the same way, though it could be used on either\nside independently. This type is designed such that it can easily be\nreplaced by a standard Rust 'Vec' once interoperability is no longer a\nconcern.\n\nOne particular item to note is that Git's macros to handle vec\noperations infer the amount that a vec needs to grow from the size of\na pointer, but that makes it somewhat specific to the macros used in C.\nTo avoid defining every ivec function as a macro I opted to also\ninclude an element_size field that allows concrete functions like\npush() to know how much to grow the memory. This element_size also\nhelps in verifying that the ivec is correct when passing from C to\nRust.\n\nSigned-off-by: Ezekiel Newren \n---\n Makefile | 16 +-\n interop/ivec.c | 151 +++++++++++++\n interop/ivec.h | 52 +++++\n meson.build | 1 +\n rust/interop/src/ivec.rs | 462 +++++++++++++++++++++++++++++++++++++++\n rust/interop/src/lib.rs | 10 +\n rust/xdiff/src/lib.rs | 1 +\n 7 files changed, 690 insertions(+), 3 deletions(-)\n create mode 100644 interop/ivec.c\n create mode 100644 interop/ivec.h\n create mode 100644 rust/interop/src/ivec.rs\n\ndiff --git a/Makefile b/Makefile\nindex 1ec0c1ee6603..29a53520fd28 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -672,6 +672,7 @@ BUILTIN_OBJS =\n BUILT_INS =\n COMPAT_CFLAGS =\n COMPAT_OBJS =\n+INTEROP_OBJS =\n XDIFF_OBJS =\n GENERATED_H =\n EXTRA_CPPFLAGS =\n@@ -918,6 +919,7 @@ export PYTHON_PATH\n TEST_SHELL_PATH = $(SHELL_PATH)\n \n LIB_FILE = libgit.a\n+INTEROP_LIB = interop/lib.a\n XDIFF_LIB = xdiff/lib.a\n \n EXTLIBS =\n@@ -1412,7 +1414,7 @@ UNIT_TEST_OBJS += $(UNIT_TEST_DIR)/test-lib.o\n UNIT_TEST_OBJS += $(UNIT_TEST_DIR)/lib-reftable.o\n \n # xdiff and reftable libs may in turn depend on what is in libgit.a\n-GITLIBS = common-main.o $(LIB_FILE) $(XDIFF_LIB) $(REFTABLE_LIB) $(LIB_FILE)\n+GITLIBS = common-main.o $(LIB_FILE) $(INTEROP_LIB) $(XDIFF_LIB) $(REFTABLE_LIB) $(LIB_FILE)\n \n \n GIT_USER_AGENT = git/$(GIT_VERSION)\n@@ -2747,6 +2749,10 @@ reconfigure config.mak.autogen: config.status\n .PHONY: reconfigure # This is a convenience target.\n endif\n \n+INTEROP_OBJS += interop/ivec.o\n+.PHONY: interop-objs\n+interop-objs: $(INTEROP_OBJS)\n+\n XDIFF_OBJS += xdiff/xdiffi.o\n XDIFF_OBJS += xdiff/xemit.o\n XDIFF_OBJS += xdiff/xhistogram.o\n@@ -2791,6 +2797,7 @@ OBJECTS += $(GIT_OBJS)\n OBJECTS += $(SCALAR_OBJS)\n OBJECTS += $(PROGRAM_OBJS)\n OBJECTS += $(TEST_OBJS)\n+OBJECTS += $(INTEROP_OBJS)\n OBJECTS += $(XDIFF_OBJS)\n OBJECTS += $(FUZZ_OBJS)\n OBJECTS += $(REFTABLE_OBJS) $(REFTABLE_TEST_OBJS)\n@@ -2945,7 +2952,10 @@ scalar$X: scalar.o GIT-LDFLAGS $(GITLIBS) compile_rust\n $(LIB_FILE): $(LIB_OBJS)\n \t$(QUIET_AR)$(RM) $@ && $(AR) $(ARFLAGS) $@ $^\n \n-$(XDIFF_LIB): $(XDIFF_OBJS)\n+$(INTEROP_LIB): $(INTEROP_OBJS)\n+\t$(QUIET_AR)$(RM) $@ && $(AR) $(ARFLAGS) $@ $^\n+\n+$(XDIFF_LIB): $(INTEROP_OBJS) $(XDIFF_OBJS)\n \t$(QUIET_AR)$(RM) $@ && $(AR) $(ARFLAGS) $@ $^\n \n \n@@ -3790,7 +3800,7 @@ clean: profile-clean coverage-clean cocciclean rustclean\n \t$(RM) git.rc git.res\n \t$(RM) $(OBJECTS)\n \t$(RM) headless-git.o\n-\t$(RM) $(LIB_FILE) $(XDIFF_LIB) $(REFTABLE_LIB)\n+\t$(RM) $(LIB_FILE) $(INTEROP_LIB) $(XDIFF_LIB) $(REFTABLE_LIB)\n \t$(RM) $(ALL_PROGRAMS) $(SCRIPT_LIB) $(BUILT_INS) $(OTHER_PROGRAMS)\n \t$(RM) $(TEST_PROGRAMS)\n \t$(RM) $(FUZZ_PROGRAMS)\ndiff --git a/interop/ivec.c b/interop/ivec.c\nnew file mode 100644\nindex 000000000000..9bc2258c04ad\n--- /dev/null\n+++ b/interop/ivec.c\n@@ -0,0 +1,151 @@\n+#include \"ivec.h\"\n+\n+static void ivec_set_capacity(void* self, usize new_capacity) {\n+\tstruct rawivec *this = self;\n+\tif (new_capacity == 0)\n+\t\tFREE_AND_NULL(this->ptr);\n+\telse\n+\t\tthis->ptr = xrealloc(this->ptr, new_capacity * this->element_size);\n+\tthis->capacity = new_capacity;\n+}\n+\n+void ivec_init(void* self, usize element_size) {\n+\tstruct rawivec *this = self;\n+\tthis->ptr = NULL;\n+\tthis->length = 0;\n+\tthis->capacity = 0;\n+\tthis->element_size = element_size;\n+}\n+\n+/*\n+ * MUST CALL IVEC_INIT() FIRST!!!\n+ * This function will free the ivec, set self.capacity and self.length\n+ * to the specified capacity, and then calloc self.capacity number of\n+ * elements.\n+ */\n+void ivec_zero(void* self, usize capacity) {\n+\tstruct rawivec *this = self;\n+\tif (this->ptr)\n+\t\tFREE_AND_NULL(this->ptr);\n+\tthis->capacity = this->length = capacity;\n+\tthis->ptr = xcalloc(this->capacity, this->element_size);\n+}\n+\n+void ivec_clear(void* self) {\n+\tstruct rawivec *this = self;\n+\tthis->length = 0;\n+}\n+\n+void ivec_reserve_exact(void* self, usize additional) {\n+\tstruct rawivec *this = self;\n+\tusize new_capacity = this->capacity + additional;\n+\tivec_set_capacity(self, new_capacity);\n+}\n+\n+void ivec_reserve(void* self, usize additional) {\n+\tstruct rawivec *this = self;\n+\tusize growby = 128;\n+\tif (this->capacity > growby) {\n+\t\tgrowby = this->capacity;\n+\t}\n+\tif (additional > growby) {\n+\t\tgrowby = additional;\n+\t}\n+\tivec_reserve_exact(self, growby);\n+}\n+\n+void ivec_shrink_to_fit(void* self) {\n+\tstruct rawivec *this = self;\n+\tivec_set_capacity(self, this->length);\n+}\n+\n+void ivec_resize(void* self, usize new_length, void* default_value) {\n+\tstruct rawivec *this = self;\n+\tisize additional = (isize) (new_length - this->capacity);\n+\tif (additional > 0) {\n+\t\tivec_reserve(self, additional);\n+\t}\n+\n+\tfor (usize i = this->length; i < new_length; i++) {\n+\t\tvoid* dst = (u8*) this->ptr + (this->length + i) * this->element_size;\n+\t\tmemcpy(dst, default_value, this->element_size);\n+\t}\n+\tthis->length = new_length;\n+}\n+\n+void ivec_push(void* self, void* value) {\n+\tstruct rawivec *this = self;\n+\tu8* dst;\n+\n+\tif (this->length == this->capacity) {\n+\t\tivec_reserve(self, 1);\n+\t}\n+\tdst = (u8*) this->ptr + this->length * this->element_size;\n+\tmemcpy(dst, value, this->element_size);\n+\tthis->length++;\n+}\n+\n+void ivec_extend_from_slice(void* self, void const* ptr, usize size) {\n+\tstruct rawivec *this = self;\n+\tu8* dst;\n+\n+\tif (size == 0)\n+\t\treturn;\n+\n+\tif (this->length + size > this->capacity) {\n+\t\tivec_reserve(self, this->capacity - this->length + size);\n+\t}\n+\tdst = (u8*) this->ptr + this->length * this->element_size;\n+\tmemcpy(dst, ptr, size * this->element_size);\n+\tthis->length += size;\n+}\n+\n+bool ivec_equal(void* self, void* other) {\n+\tstruct rawivec *lhs = self;\n+\tstruct rawivec *rhs = other;\n+\n+\tif (lhs->element_size != rhs->element_size) {\n+\t\treturn false;\n+\t}\n+\n+\tif (lhs->length != rhs->length) {\n+\t\treturn false;\n+\t}\n+\n+\n+\tfor (usize i = 0; i < lhs->length; i++) {\n+\t\tvoid* left = (u8 *) lhs->ptr + i * lhs->element_size;\n+\t\tvoid* right = (u8 *) rhs->ptr + i * rhs->element_size;\n+\t\tif (memcmp(left, right, lhs->element_size) != 0) {\n+\t\t\treturn false;\n+\t\t}\n+\t}\n+\n+\treturn true;\n+}\n+\n+\n+void ivec_free(void* self) {\n+\tstruct rawivec *this = self;\n+\tFREE_AND_NULL(this->ptr);\n+\tthis->length = 0;\n+\tthis->capacity = 0;\n+\t/* don't modify self->element_size */\n+}\n+\n+void ivec_move(void* source, void* destination) {\n+\tstruct rawivec *this = source;\n+\tstruct rawivec *that = destination;\n+\n+\tif (this->element_size != that->element_size)\n+\t\tBUG(\"mismatched element_size\");\n+\n+\tivec_free(destination);\n+\tthat->ptr = this->ptr;\n+\tthat->length = this->length;\n+\tthat->capacity = this->capacity;\n+\n+\tthis->ptr = NULL;\n+\tthis->length = 0;\n+\tthis->capacity = 0;\n+}\ndiff --git a/interop/ivec.h b/interop/ivec.h\nnew file mode 100644\nindex 000000000000..98be4bbeb54a\n--- /dev/null\n+++ b/interop/ivec.h\n@@ -0,0 +1,52 @@\n+#ifndef IVEC_H\n+#define IVEC_H\n+\n+#include \"../git-compat-util.h\"\n+\n+struct rawivec {\n+\tvoid* ptr;\n+\tusize length;\n+\tusize capacity;\n+\tusize element_size;\n+};\n+\n+#define DEFINE_IVEC_TYPE(type, suffix) \\\n+struct ivec_##suffix { \\\n+\ttype* ptr; \\\n+\tsize_t length; \\\n+\tsize_t capacity; \\\n+\tsize_t element_size; \\\n+}\n+\n+#define IVEC_INIT(variable) ivec_init(&(variable), sizeof(*(variable).ptr))\n+\n+DEFINE_IVEC_TYPE(u8, u8);\n+DEFINE_IVEC_TYPE(u16, u16);\n+DEFINE_IVEC_TYPE(u32, u32);\n+DEFINE_IVEC_TYPE(u64, u64);\n+\n+DEFINE_IVEC_TYPE(i8, i8);\n+DEFINE_IVEC_TYPE(i16, i16);\n+DEFINE_IVEC_TYPE(i32, i32);\n+DEFINE_IVEC_TYPE(i64, i64);\n+\n+DEFINE_IVEC_TYPE(f32, f32);\n+DEFINE_IVEC_TYPE(f64, f64);\n+\n+DEFINE_IVEC_TYPE(usize, usize);\n+DEFINE_IVEC_TYPE(isize, isize);\n+\n+void ivec_init(void* self, usize element_size);\n+void ivec_zero(void* self, usize capacity);\n+void ivec_clear(void* self);\n+void ivec_reserve_exact(void* self, usize additional);\n+void ivec_reserve(void* self, usize additional);\n+void ivec_shrink_to_fit(void* self);\n+void ivec_resize(void* self, usize new_length, void* default_value);\n+void ivec_push(void* self, void* value);\n+void ivec_extend_from_slice(void* self, void const* ptr, usize size);\n+bool ivec_equal(void* self, void* other);\n+void ivec_free(void* self);\n+void ivec_move(void* source, void* destination);\n+\n+#endif //IVEC_H\ndiff --git a/meson.build b/meson.build\nindex 5aa9901bfc0f..fc7c133f79d8 100644\n--- a/meson.build\n+++ b/meson.build\n@@ -395,6 +395,7 @@ libgit_sources = [\n 'hex-ll.c',\n 'hook.c',\n 'ident.c',\n+ 'interop/ivec.c',\n 'json-writer.c',\n 'kwset.c',\n 'levenshtein.c',\ndiff --git a/rust/interop/src/ivec.rs b/rust/interop/src/ivec.rs\nnew file mode 100644\nindex 000000000000..4e047572b922\n--- /dev/null\n+++ b/rust/interop/src/ivec.rs\n@@ -0,0 +1,462 @@\n+use crate::*;\n+use core::mem::{align_of, size_of};\n+use std::fmt::{Debug, Formatter};\n+use std::ops::{Index, IndexMut};\n+\n+#[repr(C)]\n+pub struct IVec {\n+ ptr: *mut T,\n+ length: usize,\n+ capacity: usize,\n+ element_size: usize,\n+}\n+\n+impl Default for IVec {\n+ fn default() -> Self {\n+ Self::new()\n+ }\n+}\n+\n+impl Drop for IVec {\n+ fn drop(&mut self) {\n+ unsafe {\n+ self._free();\n+ }\n+ }\n+}\n+\n+impl Clone for IVec {\n+ fn clone(&self) -> Self {\n+ let mut copy = Self::new();\n+ copy.reserve_exact(self.len());\n+ for i in 0..self.len() {\n+ copy.push(self[i].clone());\n+ }\n+\n+ copy\n+ }\n+}\n+\n+impl PartialEq for IVec {\n+ fn eq(&self, other: &Self) -> bool {\n+ if self.len() != other.len() {\n+ return false;\n+ }\n+\n+ let lhs = self.as_slice();\n+ let rhs = &other.as_slice()[..lhs.len()];\n+ for i in 0..lhs.len() {\n+ if lhs[i] != rhs[i] {\n+ return false;\n+ }\n+ }\n+\n+ true\n+ }\n+}\n+\n+impl Eq for IVec {}\n+\n+/*\n+ * constructors\n+ */\n+impl IVec {\n+ pub fn new() -> Self {\n+ Self {\n+ ptr: std::ptr::null_mut(),\n+ length: 0,\n+ capacity: 0,\n+ element_size: size_of::(),\n+ }\n+ }\n+\n+ /// uses calloc to create the IVec, it's unsafe because\n+ /// zeroed memory may not be a valid default value\n+ pub unsafe fn zero(capacity: usize) -> Self {\n+ Self {\n+ ptr: calloc(capacity, size_of::()) as *mut T,\n+ length: capacity,\n+ capacity,\n+ element_size: size_of::(),\n+ }\n+ }\n+\n+ pub fn with_capacity(capacity: usize) -> Self {\n+ let mut vec = Self::new();\n+ vec._set_capacity(capacity);\n+ vec\n+ }\n+\n+ pub fn with_capacity_and_default(capacity: usize, default_value: T) -> Self\n+ where\n+ T: Copy,\n+ {\n+ let mut vec = Self::new();\n+ vec._set_capacity(capacity);\n+ vec._buffer_mut().fill(default_value);\n+ vec\n+ }\n+\n+ pub unsafe fn from_raw_mut<'a>(raw: *mut Self) -> &'a mut Self {\n+ let vec = raw.as_mut().expect(\"null pointer\");\n+ #[cfg(debug_assertions)]\n+ vec.test_invariants();\n+ vec\n+ }\n+\n+ pub unsafe fn from_raw<'a>(raw: *const Self) -> &'a Self {\n+ let vec = &*raw.as_ref().expect(\"null pointer\");\n+ #[cfg(debug_assertions)]\n+ vec.test_invariants();\n+ vec\n+ }\n+}\n+\n+/*\n+ * private methods\n+ */\n+impl IVec {\n+ pub fn test_invariants(&self) {\n+ if !self.ptr.is_null() && (self.ptr as usize) % align_of::() != 0 {\n+ panic!(\n+ \"misaligned pointer: expected {:x}, got {:x}\",\n+ align_of::(),\n+ self.ptr as usize\n+ );\n+ }\n+ if self.ptr.is_null() && (self.length > 0 || self.capacity > 0) {\n+ panic!(\"ptr is null, but length or capacity is > 0\");\n+ }\n+ if !self.ptr.is_null() && self.capacity == 0 {\n+ panic!(\"ptr ISN'T null, but capacity == 0\");\n+ }\n+ if self.element_size != size_of::() {\n+ panic!(\n+ \"incorrect element size, should be: {}, but was: {}\",\n+ size_of::(),\n+ self.element_size\n+ );\n+ }\n+ if self.length > self.capacity {\n+ panic!(\"length: {} > capacity: {}\", self.length, self.capacity);\n+ }\n+ if self.capacity > usize::MAX / size_of::() {\n+ panic!(\n+ \"Capacity {} is too large, potential overflow detected\",\n+ self.capacity\n+ );\n+ }\n+ }\n+\n+ fn _zero(&mut self) {\n+ self.ptr = std::ptr::null_mut();\n+ self.length = 0;\n+ self.capacity = 0;\n+ // DO NOT MODIFY element_size!!!\n+ }\n+\n+ unsafe fn _free(&mut self) {\n+ free(self.ptr as *mut std::ffi::c_void);\n+ self._zero();\n+ }\n+\n+ fn _set_capacity(&mut self, new_capacity: usize) {\n+ unsafe {\n+ if new_capacity == self.capacity {\n+ return;\n+ }\n+ if new_capacity == 0 {\n+ self._free();\n+ } else {\n+ let t = realloc(\n+ self.ptr as *mut std::ffi::c_void,\n+ new_capacity * size_of::(),\n+ );\n+ if t.is_null() {\n+ panic!(\"out of memory\");\n+ }\n+ self.ptr = t as *mut T;\n+ }\n+ self.capacity = new_capacity;\n+ }\n+ }\n+\n+ fn _resize(&mut self, new_length: usize, default_value: T, exact: bool)\n+ where\n+ T: Copy,\n+ {\n+ if exact {\n+ self._set_capacity(new_length);\n+ } else if new_length > self.capacity {\n+ self.reserve(new_length - self.capacity);\n+ } else {\n+ /* capacity does not need to be changed */\n+ }\n+\n+ if new_length > self.length {\n+ let range = self.length..new_length;\n+ self._buffer_mut()[range].fill(default_value);\n+ }\n+\n+ self.length = new_length;\n+ }\n+\n+ fn _buffer_mut(&mut self) -> &mut [T] {\n+ if self.ptr.is_null() {\n+ &mut []\n+ } else {\n+ unsafe { std::slice::from_raw_parts_mut(self.ptr, self.capacity) }\n+ }\n+ }\n+\n+ fn _buffer(&self) -> &[T] {\n+ if self.ptr.is_null() {\n+ &[]\n+ } else {\n+ unsafe { std::slice::from_raw_parts(self.ptr, self.capacity) }\n+ }\n+ }\n+}\n+\n+/*\n+ * methods\n+ */\n+impl IVec {\n+ pub fn len(&self) -> usize {\n+ self.length\n+ }\n+\n+ pub unsafe fn set_len(&mut self, new_length: usize) {\n+ self.length = new_length;\n+ }\n+\n+ pub fn capacity(&self) -> usize {\n+ self.capacity\n+ }\n+\n+ pub fn reserve_exact(&mut self, additional: usize) {\n+ self._set_capacity(self.capacity + additional);\n+ }\n+\n+ pub fn reserve(&mut self, additional: usize) {\n+ let growby = std::cmp::max(128, self.capacity);\n+ self.reserve_exact(std::cmp::max(additional, growby));\n+ }\n+\n+ pub fn shrink_to_fit(&mut self) {\n+ self._set_capacity(self.length);\n+ }\n+\n+ pub fn resize(&mut self, new_length: usize, default_value: T)\n+ where\n+ T: Copy,\n+ {\n+ self._resize(new_length, default_value, false);\n+ }\n+\n+ pub fn resize_exact(&mut self, new_length: usize, default_value: T)\n+ where\n+ T: Copy,\n+ {\n+ self._resize(new_length, default_value, true);\n+ }\n+\n+ pub fn insert(&mut self, index: usize, value: T) {\n+ if self.length == self.capacity {\n+ self.reserve(1);\n+ }\n+\n+ unsafe {\n+ let src = &self._buffer()[index] as *const T;\n+ let dst = src.add(1) as *mut T;\n+ let len = self.length - index;\n+ std::ptr::copy(src, dst, len);\n+ std::ptr::write(self.ptr.add(index), value);\n+ }\n+ }\n+\n+ pub fn push(&mut self, value: T) {\n+ if self.length == self.capacity {\n+ self.reserve(1);\n+ }\n+\n+ let i = self.length;\n+ unsafe {\n+ std::ptr::write(self.ptr.add(i), value);\n+ }\n+ self.length += 1;\n+ }\n+\n+ pub fn extend_from_slice(&mut self, slice: &[T])\n+ where\n+ T: Clone,\n+ {\n+ for v in slice {\n+ self.push(v.clone());\n+ }\n+ }\n+\n+ pub fn clear(&mut self) {\n+ self.length = 0;\n+ }\n+\n+ pub fn as_ptr(&self) -> *const T {\n+ self.ptr\n+ }\n+\n+ pub fn as_mut_ptr(&self) -> *mut T {\n+ self.ptr\n+ }\n+\n+ pub fn as_slice(&self) -> &[T] {\n+ &self._buffer()[0..self.length]\n+ }\n+\n+ pub fn as_mut_slice(&mut self) -> &mut [T] {\n+ let range = 0..self.length;\n+ &mut self._buffer_mut()[range]\n+ }\n+}\n+\n+impl Extend for IVec {\n+ fn extend>(&mut self, iter: IT) {\n+ for v in iter {\n+ self.push(v);\n+ }\n+ }\n+}\n+\n+impl Index for IVec {\n+ type Output = T;\n+\n+ fn index(&self, index: usize) -> &Self::Output {\n+ &self.as_slice()[index]\n+ }\n+}\n+\n+impl IndexMut for IVec {\n+ fn index_mut(&mut self, index: usize) -> &mut Self::Output {\n+ &mut self.as_mut_slice()[index]\n+ }\n+}\n+\n+impl Debug for IVec {\n+ fn fmt(&self, f: &mut Formatter<'_>) -> std::fmt::Result {\n+ writeln!(\n+ f,\n+ \"ptr: {}, capacity: {}, len: {}, element_size: {}, content: {:?}\",\n+ self.ptr as usize,\n+ self.capacity,\n+ self.length,\n+ self.element_size,\n+ self.as_slice()\n+ )\n+ }\n+}\n+\n+impl std::fmt::Write for IVec {\n+ fn write_str(&mut self, s: &str) -> std::fmt::Result {\n+ Ok(self.extend_from_slice(s.as_bytes()))\n+ }\n+}\n+\n+impl std::io::Write for IVec {\n+ fn write(&mut self, buf: &[u8]) -> std::io::Result {\n+ self.extend_from_slice(buf);\n+ Ok(buf.len())\n+ }\n+\n+ fn flush(&mut self) -> std::io::Result<()> {\n+ Ok(())\n+ }\n+}\n+\n+#[cfg(test)]\n+mod tests {\n+ use crate::ivec::IVec;\n+ use std::panic;\n+\n+ #[test]\n+ fn test_panic_on_out_of_bounds() {\n+ type TestType = i16;\n+ let result = panic::catch_unwind(|| {\n+ let mut v = IVec::::with_capacity(1_000_000);\n+ v[0] = 55;\n+ });\n+\n+ match result {\n+ Ok(_) => assert!(false, \"index was out of bounds, but no panic was triggered\"),\n+ Err(_) => assert!(true),\n+ }\n+ }\n+\n+ #[test]\n+ fn test_push_clear_resize_then_shrink_to_fit() {\n+ let mut vec = IVec::::new();\n+ let mut monotonic = 1;\n+\n+ vec.reserve_exact(1);\n+ assert_eq!(1, vec.capacity);\n+\n+ // test push\n+ for _ in 0..10 {\n+ vec.push(monotonic);\n+ assert_eq!(monotonic as usize, vec.length);\n+ assert_eq!(monotonic, vec[(monotonic - 1) as usize]);\n+ assert!(vec.capacity >= vec.length);\n+ monotonic += 1;\n+ }\n+\n+ // test clear\n+ let expected = vec.capacity;\n+ vec.clear();\n+ assert_eq!(0, vec.length);\n+ assert_eq!(expected, vec.capacity);\n+\n+ // test resize\n+ let expected = vec.capacity + 10;\n+ let default_value = 19;\n+ vec.resize(expected, default_value);\n+ // assert_eq!(vec.capacity, vec.slice.len());\n+ assert_eq!(expected, vec.length);\n+ assert!(vec.capacity >= expected);\n+ for i in 0..vec.length {\n+ assert_eq!(default_value, vec[i]);\n+ }\n+\n+ vec.reserve(10);\n+ // assert_eq!(vec.capacity, vec.slice.len());\n+ assert!(vec.capacity > vec.length);\n+ let length_before = vec.length;\n+ vec.shrink_to_fit();\n+ assert_eq!(length_before, vec.length);\n+ assert_eq!(vec.length, vec.capacity);\n+ // assert_eq!(vec.capacity, vec.slice.len());\n+ }\n+\n+ #[test]\n+ fn test_struct_size() {\n+ let vec = IVec::::new();\n+\n+ assert_eq!(2, vec.element_size);\n+ assert_eq!(size_of::() * 4, size_of::>());\n+\n+ drop(vec);\n+\n+ let vec = IVec::::new();\n+ assert_eq!(16, vec.element_size);\n+ assert_eq!(size_of::() * 4, size_of::>());\n+ }\n+\n+ #[test]\n+ fn test_manual_free() {\n+ type TestType = i16;\n+ let mut vec = IVec::::new();\n+\n+ unsafe { vec._free() };\n+ assert!(vec.ptr.is_null());\n+ assert_eq!(0, vec.length);\n+ assert_eq!(0, vec.capacity);\n+ assert_eq!(size_of::(), vec.element_size);\n+ }\n+}\ndiff --git a/rust/interop/src/lib.rs b/rust/interop/src/lib.rs\nindex e69de29bb2d1..4850f66e5bd9 100644\n--- a/rust/interop/src/lib.rs\n+++ b/rust/interop/src/lib.rs\n@@ -0,0 +1,10 @@\n+pub mod ivec;\n+\n+use std::ffi::c_void;\n+\n+extern \"C\" {\n+ pub fn malloc(size: usize) -> *mut c_void;\n+ pub fn calloc(nmemb: usize, size: usize) -> *mut c_void;\n+ pub fn realloc(ptr: *mut c_void, size: usize) -> *mut c_void;\n+ pub fn free(ptr: *mut c_void);\n+}\ndiff --git a/rust/xdiff/src/lib.rs b/rust/xdiff/src/lib.rs\nindex e69de29bb2d1..8b137891791f 100644\n--- a/rust/xdiff/src/lib.rs\n+++ b/rust/xdiff/src/lib.rs\n@@ -0,0 +1 @@\n+\n-- \ngitgitgadget\n\n"},{"id":"524758","messageId":"d4bed95463216668e7c024e6186ca4574c7114dc.1755921357.git.gitgitgadget@gmail.com","threadId":"63804","inReplyTo":"pull.1980.v3.git.git.1755921356.gitgitgadget@gmail.com","subject":"[PATCH v3 07/15] xdiff/xprepare: remove superfluous forward declarations","fromName":"Ezekiel Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2025-08-23T03:55:48Z","receivedAt":"2025-08-23T03:56:09Z","isPatch":true,"sender":{"key":"ezekielnewren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/18313324?v=4"},"body":"From: Ezekiel Newren \n\nMove xdl_prepare_env() later in the file to avoid the need\nfor forward declarations.\n\nSigned-off-by: Ezekiel Newren \n---\n xdiff/xprepare.c | 116 ++++++++++++++++++++---------------------------\n 1 file changed, 50 insertions(+), 66 deletions(-)\n\ndiff --git a/xdiff/xprepare.c b/xdiff/xprepare.c\nindex e1d4017b2dde..a45c5ee208c8 100644\n--- a/xdiff/xprepare.c\n+++ b/xdiff/xprepare.c\n@@ -53,21 +53,6 @@ typedef struct s_xdlclassifier {\n \n \n \n-static int xdl_init_classifier(xdlclassifier_t *cf, long size, long flags);\n-static void xdl_free_classifier(xdlclassifier_t *cf);\n-static int xdl_classify_record(unsigned int pass, xdlclassifier_t *cf, xrecord_t **rhash,\n-\t\t\t unsigned int hbits, xrecord_t *rec);\n-static int xdl_prepare_ctx(unsigned int pass, mmfile_t *mf, long narec, xpparam_t const *xpp,\n-\t\t\t xdlclassifier_t *cf, xdfile_t *xdf);\n-static void xdl_free_ctx(xdfile_t *xdf);\n-static int xdl_clean_mmatch(char const *dis, long i, long s, long e);\n-static int xdl_cleanup_records(xdlclassifier_t *cf, xdfile_t *xdf1, xdfile_t *xdf2);\n-static int xdl_trim_ends(xdfile_t *xdf1, xdfile_t *xdf2);\n-static int xdl_optimize_ctxs(xdlclassifier_t *cf, xdfile_t *xdf1, xdfile_t *xdf2);\n-\n-\n-\n-\n static int xdl_init_classifier(xdlclassifier_t *cf, long size, long flags) {\n \tcf->flags = flags;\n \n@@ -242,57 +227,6 @@ static void xdl_free_ctx(xdfile_t *xdf) {\n }\n \n \n-int xdl_prepare_env(mmfile_t *mf1, mmfile_t *mf2, xpparam_t const *xpp,\n-\t\t xdfenv_t *xe) {\n-\tlong enl1, enl2, sample;\n-\txdlclassifier_t cf;\n-\n-\tmemset(&cf, 0, sizeof(cf));\n-\n-\t/*\n-\t * For histogram diff, we can afford a smaller sample size and\n-\t * thus a poorer estimate of the number of lines, as the hash\n-\t * table (rhash) won't be filled up/grown. The number of lines\n-\t * (nrecs) will be updated correctly anyway by\n-\t * xdl_prepare_ctx().\n-\t */\n-\tsample = (XDF_DIFF_ALG(xpp->flags) == XDF_HISTOGRAM_DIFF\n-\t\t ? XDL_GUESS_NLINES2 : XDL_GUESS_NLINES1);\n-\n-\tenl1 = xdl_guess_lines(mf1, sample) + 1;\n-\tenl2 = xdl_guess_lines(mf2, sample) + 1;\n-\n-\tif (xdl_init_classifier(&cf, enl1 + enl2 + 1, xpp->flags) < 0)\n-\t\treturn -1;\n-\n-\tif (xdl_prepare_ctx(1, mf1, enl1, xpp, &cf, &xe->xdf1) < 0) {\n-\n-\t\txdl_free_classifier(&cf);\n-\t\treturn -1;\n-\t}\n-\tif (xdl_prepare_ctx(2, mf2, enl2, xpp, &cf, &xe->xdf2) < 0) {\n-\n-\t\txdl_free_ctx(&xe->xdf1);\n-\t\txdl_free_classifier(&cf);\n-\t\treturn -1;\n-\t}\n-\n-\tif ((XDF_DIFF_ALG(xpp->flags) != XDF_PATIENCE_DIFF) &&\n-\t (XDF_DIFF_ALG(xpp->flags) != XDF_HISTOGRAM_DIFF) &&\n-\t xdl_optimize_ctxs(&cf, &xe->xdf1, &xe->xdf2) < 0) {\n-\n-\t\txdl_free_ctx(&xe->xdf2);\n-\t\txdl_free_ctx(&xe->xdf1);\n-\t\txdl_free_classifier(&cf);\n-\t\treturn -1;\n-\t}\n-\n-\txdl_free_classifier(&cf);\n-\n-\treturn 0;\n-}\n-\n-\n void xdl_free_env(xdfenv_t *xe) {\n \n \txdl_free_ctx(&xe->xdf2);\n@@ -460,3 +394,53 @@ static int xdl_optimize_ctxs(xdlclassifier_t *cf, xdfile_t *xdf1, xdfile_t *xdf2\n \n \treturn 0;\n }\n+\n+int xdl_prepare_env(mmfile_t *mf1, mmfile_t *mf2, xpparam_t const *xpp,\n+\t\t xdfenv_t *xe) {\n+\tlong enl1, enl2, sample;\n+\txdlclassifier_t cf;\n+\n+\tmemset(&cf, 0, sizeof(cf));\n+\n+\t/*\n+\t * For histogram diff, we can afford a smaller sample size and\n+\t * thus a poorer estimate of the number of lines, as the hash\n+\t * table (rhash) won't be filled up/grown. The number of lines\n+\t * (nrecs) will be updated correctly anyway by\n+\t * xdl_prepare_ctx().\n+\t */\n+\tsample = (XDF_DIFF_ALG(xpp->flags) == XDF_HISTOGRAM_DIFF\n+\t\t ? XDL_GUESS_NLINES2 : XDL_GUESS_NLINES1);\n+\n+\tenl1 = xdl_guess_lines(mf1, sample) + 1;\n+\tenl2 = xdl_guess_lines(mf2, sample) + 1;\n+\n+\tif (xdl_init_classifier(&cf, enl1 + enl2 + 1, xpp->flags) < 0)\n+\t\treturn -1;\n+\n+\tif (xdl_prepare_ctx(1, mf1, enl1, xpp, &cf, &xe->xdf1) < 0) {\n+\n+\t\txdl_free_classifier(&cf);\n+\t\treturn -1;\n+\t}\n+\tif (xdl_prepare_ctx(2, mf2, enl2, xpp, &cf, &xe->xdf2) < 0) {\n+\n+\t\txdl_free_ctx(&xe->xdf1);\n+\t\txdl_free_classifier(&cf);\n+\t\treturn -1;\n+\t}\n+\n+\tif ((XDF_DIFF_ALG(xpp->flags) != XDF_PATIENCE_DIFF) &&\n+\t (XDF_DIFF_ALG(xpp->flags) != XDF_HISTOGRAM_DIFF) &&\n+\t xdl_optimize_ctxs(&cf, &xe->xdf1, &xe->xdf2) < 0) {\n+\n+\t\txdl_free_ctx(&xe->xdf2);\n+\t\txdl_free_ctx(&xe->xdf1);\n+\t\txdl_free_classifier(&cf);\n+\t\treturn -1;\n+\t }\n+\n+\txdl_free_classifier(&cf);\n+\n+\treturn 0;\n+}\n-- \ngitgitgadget\n\n"},{"id":"524756","messageId":"7c68ce5349c0a77426f03e898ae15392a3b06b65.1755921357.git.gitgitgadget@gmail.com","threadId":"63804","inReplyTo":"pull.1980.v3.git.git.1755921356.gitgitgadget@gmail.com","subject":"[PATCH v3 08/15] xdiff: delete unnecessary fields from xrecord_t and xdfile_t","fromName":"Ezekiel Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2025-08-23T03:55:49Z","receivedAt":"2025-08-23T03:56:10Z","isPatch":true,"sender":{"key":"ezekielnewren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/18313324?v=4"},"body":"From: Ezekiel Newren \n\nxrecord_t.next, xdfile_t.hbits, xdfile_t.rhash are initialized,\nbut never used for anything by the code. Remove them.\n\nSigned-off-by: Ezekiel Newren \n---\n xdiff/xprepare.c | 24 +++---------------------\n xdiff/xtypes.h | 3 ---\n 2 files changed, 3 insertions(+), 24 deletions(-)\n\ndiff --git a/xdiff/xprepare.c b/xdiff/xprepare.c\nindex a45c5ee208c8..ad356281f939 100644\n--- a/xdiff/xprepare.c\n+++ b/xdiff/xprepare.c\n@@ -91,8 +91,7 @@ static void xdl_free_classifier(xdlclassifier_t *cf) {\n }\n \n \n-static int xdl_classify_record(unsigned int pass, xdlclassifier_t *cf, xrecord_t **rhash,\n-\t\t\t unsigned int hbits, xrecord_t *rec) {\n+static int xdl_classify_record(unsigned int pass, xdlclassifier_t *cf, xrecord_t *rec) {\n \tlong hi;\n \tchar const *line;\n \txdlclass_t *rcrec;\n@@ -126,23 +125,17 @@ static int xdl_classify_record(unsigned int pass, xdlclassifier_t *cf, xrecord_t\n \n \trec->ha = (unsigned long) rcrec->idx;\n \n-\thi = (long) XDL_HASHLONG(rec->ha, hbits);\n-\trec->next = rhash[hi];\n-\trhash[hi] = rec;\n-\n \treturn 0;\n }\n \n \n static int xdl_prepare_ctx(unsigned int pass, mmfile_t *mf, long narec, xpparam_t const *xpp,\n \t\t\t xdlclassifier_t *cf, xdfile_t *xdf) {\n-\tunsigned int hbits;\n-\tlong nrec, hsize, bsize;\n+\tlong nrec, bsize;\n \tunsigned long hav;\n \tchar const *blk, *cur, *top, *prev;\n \txrecord_t *crec;\n \txrecord_t **recs;\n-\txrecord_t **rhash;\n \tunsigned long *ha;\n \tchar *rchg;\n \tlong *rindex;\n@@ -150,7 +143,6 @@ static int xdl_prepare_ctx(unsigned int pass, mmfile_t *mf, long narec, xpparam_\n \tha = NULL;\n \trindex = NULL;\n \trchg = NULL;\n-\trhash = NULL;\n \trecs = NULL;\n \n \tif (xdl_cha_init(&xdf->rcha, sizeof(xrecord_t), narec / 4 + 1) < 0)\n@@ -158,11 +150,6 @@ static int xdl_prepare_ctx(unsigned int pass, mmfile_t *mf, long narec, xpparam_\n \tif (!XDL_ALLOC_ARRAY(recs, narec))\n \t\tgoto abort;\n \n-\thbits = xdl_hashbits((unsigned int) narec);\n-\thsize = 1 << hbits;\n-\tif (!XDL_CALLOC_ARRAY(rhash, hsize))\n-\t\tgoto abort;\n-\n \tnrec = 0;\n \tif ((cur = blk = xdl_mmfile_first(mf, &bsize))) {\n \t\tfor (top = blk + bsize; cur < top; ) {\n@@ -176,7 +163,7 @@ static int xdl_prepare_ctx(unsigned int pass, mmfile_t *mf, long narec, xpparam_\n \t\t\tcrec->size = (long) (cur - prev);\n \t\t\tcrec->ha = hav;\n \t\t\trecs[nrec++] = crec;\n-\t\t\tif (xdl_classify_record(pass, cf, rhash, hbits, crec) < 0)\n+\t\t\tif (xdl_classify_record(pass, cf, crec) < 0)\n \t\t\t\tgoto abort;\n \t\t}\n \t}\n@@ -194,8 +181,6 @@ static int xdl_prepare_ctx(unsigned int pass, mmfile_t *mf, long narec, xpparam_\n \n \txdf->nrec = nrec;\n \txdf->recs = recs;\n-\txdf->hbits = hbits;\n-\txdf->rhash = rhash;\n \txdf->rchg = rchg + 1;\n \txdf->rindex = rindex;\n \txdf->nreff = 0;\n@@ -209,7 +194,6 @@ abort:\n \txdl_free(ha);\n \txdl_free(rindex);\n \txdl_free(rchg);\n-\txdl_free(rhash);\n \txdl_free(recs);\n \txdl_cha_free(&xdf->rcha);\n \treturn -1;\n@@ -217,8 +201,6 @@ abort:\n \n \n static void xdl_free_ctx(xdfile_t *xdf) {\n-\n-\txdl_free(xdf->rhash);\n \txdl_free(xdf->rindex);\n \txdl_free(xdf->rchg - 1);\n \txdl_free(xdf->ha);\ndiff --git a/xdiff/xtypes.h b/xdiff/xtypes.h\nindex 8442bd436efe..8b8467360ecf 100644\n--- a/xdiff/xtypes.h\n+++ b/xdiff/xtypes.h\n@@ -39,7 +39,6 @@ typedef struct s_chastore {\n } chastore_t;\n \n typedef struct s_xrecord {\n-\tstruct s_xrecord *next;\n \tchar const *ptr;\n \tlong size;\n \tunsigned long ha;\n@@ -48,8 +47,6 @@ typedef struct s_xrecord {\n typedef struct s_xdfile {\n \tchastore_t rcha;\n \tlong nrec;\n-\tunsigned int hbits;\n-\txrecord_t **rhash;\n \tlong dstart, dend;\n \txrecord_t **recs;\n \tchar *rchg;\n-- \ngitgitgadget\n\n"},{"id":"524757","messageId":"e516ccc8c0abb1705f486c7d0b62eaace38f769f.1755921357.git.gitgitgadget@gmail.com","threadId":"63804","inReplyTo":"pull.1980.v3.git.git.1755921356.gitgitgadget@gmail.com","subject":"[PATCH v3 09/15] xdiff: make fields of xrecord_t Rust friendly","fromName":"Ezekiel Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2025-08-23T03:55:50Z","receivedAt":"2025-08-23T03:56:10Z","isPatch":true,"sender":{"key":"ezekielnewren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/18313324?v=4"},"body":"From: Ezekiel Newren \n\nA few commits ago, we added definitions for Rust primitive types,\nto facilitate interoperability between C and Rust. Switch a\nfew variables to use these types. Which, for now, will\nrequire adding some casts.\n\nAlso change xdlclass_t::ha to be u64 to match xrecord_t::ha, as\npointed out by Johannes.\n\nHelped-by: Johannes Schindelin \nSigned-off-by: Ezekiel Newren \n---\n xdiff/xdiffi.c | 8 ++++----\n xdiff/xemit.c | 2 +-\n xdiff/xmerge.c | 14 +++++++-------\n xdiff/xpatience.c | 2 +-\n xdiff/xprepare.c | 8 ++++----\n xdiff/xtypes.h | 6 +++---\n xdiff/xutils.c | 4 ++--\n 7 files changed, 22 insertions(+), 22 deletions(-)\n\ndiff --git a/xdiff/xdiffi.c b/xdiff/xdiffi.c\nindex 5a96e36dfbea..3b364c61f671 100644\n--- a/xdiff/xdiffi.c\n+++ b/xdiff/xdiffi.c\n@@ -418,7 +418,7 @@ static int get_indent(xrecord_t *rec)\n \tlong i;\n \tint ret = 0;\n \n-\tfor (i = 0; i < rec->size; i++) {\n+\tfor (i = 0; i < (long) rec->size; i++) {\n \t\tchar c = rec->ptr[i];\n \n \t\tif (!XDL_ISSPACE(c))\n@@ -1005,11 +1005,11 @@ static void xdl_mark_ignorable_lines(xdchange_t *xscr, xdfenv_t *xe, long flags)\n \n \t\trec = &xe->xdf1.recs[xch->i1];\n \t\tfor (i = 0; i < xch->chg1 && ignore; i++)\n-\t\t\tignore = xdl_blankline(rec[i]->ptr, rec[i]->size, flags);\n+\t\t\tignore = xdl_blankline((const char*) rec[i]->ptr, rec[i]->size, flags);\n \n \t\trec = &xe->xdf2.recs[xch->i2];\n \t\tfor (i = 0; i < xch->chg2 && ignore; i++)\n-\t\t\tignore = xdl_blankline(rec[i]->ptr, rec[i]->size, flags);\n+\t\t\tignore = xdl_blankline((const char*)rec[i]->ptr, rec[i]->size, flags);\n \n \t\txch->ignore = ignore;\n \t}\n@@ -1020,7 +1020,7 @@ static int record_matches_regex(xrecord_t *rec, xpparam_t const *xpp) {\n \tsize_t i;\n \n \tfor (i = 0; i < xpp->ignore_regex_nr; i++)\n-\t\tif (!regexec_buf(xpp->ignore_regex[i], rec->ptr, rec->size, 1,\n+\t\tif (!regexec_buf(xpp->ignore_regex[i], (const char*) rec->ptr, rec->size, 1,\n \t\t\t\t ®match, 0))\n \t\t\treturn 1;\n \ndiff --git a/xdiff/xemit.c b/xdiff/xemit.c\nindex 1d40c9cb4076..bbf7b7f8c862 100644\n--- a/xdiff/xemit.c\n+++ b/xdiff/xemit.c\n@@ -24,7 +24,7 @@\n \n static long xdl_get_rec(xdfile_t *xdf, long ri, char const **rec) {\n \n-\t*rec = xdf->recs[ri]->ptr;\n+\t*rec = (char const*) xdf->recs[ri]->ptr;\n \n \treturn xdf->recs[ri]->size;\n }\ndiff --git a/xdiff/xmerge.c b/xdiff/xmerge.c\nindex af40c88a5b36..6fa6ea61a208 100644\n--- a/xdiff/xmerge.c\n+++ b/xdiff/xmerge.c\n@@ -101,8 +101,8 @@ static int xdl_merge_cmp_lines(xdfenv_t *xe1, int i1, xdfenv_t *xe2, int i2,\n \txrecord_t **rec2 = xe2->xdf2.recs + i2;\n \n \tfor (i = 0; i < line_count; i++) {\n-\t\tint result = xdl_recmatch(rec1[i]->ptr, rec1[i]->size,\n-\t\t\trec2[i]->ptr, rec2[i]->size, flags);\n+\t\tint result = xdl_recmatch((const char*) rec1[i]->ptr, rec1[i]->size,\n+\t\t\t(const char*) rec2[i]->ptr, rec2[i]->size, flags);\n \t\tif (!result)\n \t\t\treturn -1;\n \t}\n@@ -324,8 +324,8 @@ static int xdl_fill_merge_buffer(xdfenv_t *xe1, const char *name1,\n \n static int recmatch(xrecord_t *rec1, xrecord_t *rec2, unsigned long flags)\n {\n-\treturn xdl_recmatch(rec1->ptr, rec1->size,\n-\t\t\t rec2->ptr, rec2->size, flags);\n+\treturn xdl_recmatch((char const*) rec1->ptr, rec1->size,\n+\t\t\t (char const*) rec2->ptr, rec2->size, flags);\n }\n \n /*\n@@ -383,10 +383,10 @@ static int xdl_refine_conflicts(xdfenv_t *xe1, xdfenv_t *xe2, xdmerge_t *m,\n \t\t */\n \t\tt1.ptr = (char *)xe1->xdf2.recs[m->i1]->ptr;\n \t\tt1.size = xe1->xdf2.recs[m->i1 + m->chg1 - 1]->ptr\n-\t\t\t+ xe1->xdf2.recs[m->i1 + m->chg1 - 1]->size - t1.ptr;\n+\t\t\t+ xe1->xdf2.recs[m->i1 + m->chg1 - 1]->size - (u8 const*) t1.ptr;\n \t\tt2.ptr = (char *)xe2->xdf2.recs[m->i2]->ptr;\n \t\tt2.size = xe2->xdf2.recs[m->i2 + m->chg2 - 1]->ptr\n-\t\t\t+ xe2->xdf2.recs[m->i2 + m->chg2 - 1]->size - t2.ptr;\n+\t\t\t+ xe2->xdf2.recs[m->i2 + m->chg2 - 1]->size - (u8 const*) t2.ptr;\n \t\tif (xdl_do_diff(&t1, &t2, xpp, &xe) < 0)\n \t\t\treturn -1;\n \t\tif (xdl_change_compact(&xe.xdf1, &xe.xdf2, xpp->flags) < 0 ||\n@@ -440,7 +440,7 @@ static int line_contains_alnum(const char *ptr, long size)\n static int lines_contain_alnum(xdfenv_t *xe, int i, int chg)\n {\n \tfor (; chg; chg--, i++)\n-\t\tif (line_contains_alnum(xe->xdf2.recs[i]->ptr,\n+\t\tif (line_contains_alnum((char const*) xe->xdf2.recs[i]->ptr,\n \t\t\t\txe->xdf2.recs[i]->size))\n \t\t\treturn 1;\n \treturn 0;\ndiff --git a/xdiff/xpatience.c b/xdiff/xpatience.c\nindex 77dc411d1937..986a3a3f749a 100644\n--- a/xdiff/xpatience.c\n+++ b/xdiff/xpatience.c\n@@ -121,7 +121,7 @@ static void insert_record(xpparam_t const *xpp, int line, struct hashmap *map,\n \t\treturn;\n \tmap->entries[index].line1 = line;\n \tmap->entries[index].hash = record->ha;\n-\tmap->entries[index].anchor = is_anchor(xpp, map->env->xdf1.recs[line - 1]->ptr);\n+\tmap->entries[index].anchor = is_anchor(xpp, (const char*) map->env->xdf1.recs[line - 1]->ptr);\n \tif (!map->first)\n \t\tmap->first = map->entries + index;\n \tif (map->last) {\ndiff --git a/xdiff/xprepare.c b/xdiff/xprepare.c\nindex ad356281f939..00cdf7d8a038 100644\n--- a/xdiff/xprepare.c\n+++ b/xdiff/xprepare.c\n@@ -32,7 +32,7 @@\n \n typedef struct s_xdlclass {\n \tstruct s_xdlclass *next;\n-\tunsigned long ha;\n+\tu64 ha;\n \tchar const *line;\n \tlong size;\n \tlong idx;\n@@ -96,12 +96,12 @@ static int xdl_classify_record(unsigned int pass, xdlclassifier_t *cf, xrecord_t\n \tchar const *line;\n \txdlclass_t *rcrec;\n \n-\tline = rec->ptr;\n+\tline = (char const*) rec->ptr;\n \thi = (long) XDL_HASHLONG(rec->ha, cf->hbits);\n \tfor (rcrec = cf->rchash[hi]; rcrec; rcrec = rcrec->next)\n \t\tif (rcrec->ha == rec->ha &&\n \t\t\t\txdl_recmatch(rcrec->line, rcrec->size,\n-\t\t\t\t\trec->ptr, rec->size, cf->flags))\n+\t\t\t\t\t(const char*) rec->ptr, rec->size, cf->flags))\n \t\t\tbreak;\n \n \tif (!rcrec) {\n@@ -159,7 +159,7 @@ static int xdl_prepare_ctx(unsigned int pass, mmfile_t *mf, long narec, xpparam_\n \t\t\t\tgoto abort;\n \t\t\tif (!(crec = xdl_cha_alloc(&xdf->rcha)))\n \t\t\t\tgoto abort;\n-\t\t\tcrec->ptr = prev;\n+\t\t\tcrec->ptr = (u8 const*) prev;\n \t\t\tcrec->size = (long) (cur - prev);\n \t\t\tcrec->ha = hav;\n \t\t\trecs[nrec++] = crec;\ndiff --git a/xdiff/xtypes.h b/xdiff/xtypes.h\nindex 8b8467360ecf..6e5f67ebf380 100644\n--- a/xdiff/xtypes.h\n+++ b/xdiff/xtypes.h\n@@ -39,9 +39,9 @@ typedef struct s_chastore {\n } chastore_t;\n \n typedef struct s_xrecord {\n-\tchar const *ptr;\n-\tlong size;\n-\tunsigned long ha;\n+\tu8 const* ptr;\n+\tusize size;\n+\tu64 ha;\n } xrecord_t;\n \n typedef struct s_xdfile {\ndiff --git a/xdiff/xutils.c b/xdiff/xutils.c\nindex 444a108f87c0..10e4f20b7c31 100644\n--- a/xdiff/xutils.c\n+++ b/xdiff/xutils.c\n@@ -418,10 +418,10 @@ int xdl_fall_back_diff(xdfenv_t *diff_env, xpparam_t const *xpp,\n \n \tsubfile1.ptr = (char *)diff_env->xdf1.recs[line1 - 1]->ptr;\n \tsubfile1.size = diff_env->xdf1.recs[line1 + count1 - 2]->ptr +\n-\t\tdiff_env->xdf1.recs[line1 + count1 - 2]->size - subfile1.ptr;\n+\t\tdiff_env->xdf1.recs[line1 + count1 - 2]->size - (u8 const*) subfile1.ptr;\n \tsubfile2.ptr = (char *)diff_env->xdf2.recs[line2 - 1]->ptr;\n \tsubfile2.size = diff_env->xdf2.recs[line2 + count2 - 2]->ptr +\n-\t\tdiff_env->xdf2.recs[line2 + count2 - 2]->size - subfile2.ptr;\n+\t\tdiff_env->xdf2.recs[line2 + count2 - 2]->size - (u8 const*) subfile2.ptr;\n \tif (xdl_do_diff(&subfile1, &subfile2, xpp, &env) < 0)\n \t\treturn -1;\n \n-- \ngitgitgadget\n\n"},{"id":"524759","messageId":"21bfb9f08836898c6e46564c3a2ce65cdcfa4471.1755921357.git.gitgitgadget@gmail.com","threadId":"63804","inReplyTo":"pull.1980.v3.git.git.1755921356.gitgitgadget@gmail.com","subject":"[PATCH v3 10/15] xdiff: use one definition for freeing xdfile_t","fromName":"Ezekiel Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2025-08-23T03:55:51Z","receivedAt":"2025-08-23T03:56:11Z","isPatch":true,"sender":{"key":"ezekielnewren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/18313324?v=4"},"body":"From: Ezekiel Newren \n\nSimplify xdl_prepare_ctx() by using xdl_free_ctx() instead of using\nlocal variables with hand rolled memory management.\n\nSigned-off-by: Ezekiel Newren \n---\n xdiff/xprepare.c | 60 +++++++++++++++++++-----------------------------\n 1 file changed, 24 insertions(+), 36 deletions(-)\n\ndiff --git a/xdiff/xprepare.c b/xdiff/xprepare.c\nindex 00cdf7d8a038..55e1cc308756 100644\n--- a/xdiff/xprepare.c\n+++ b/xdiff/xprepare.c\n@@ -129,86 +129,74 @@ static int xdl_classify_record(unsigned int pass, xdlclassifier_t *cf, xrecord_t\n }\n \n \n+static void xdl_free_ctx(xdfile_t *xdf) {\n+\txdl_free(xdf->rindex);\n+\txdl_free(xdf->rchg - 1);\n+\txdl_free(xdf->ha);\n+\txdl_free(xdf->recs);\n+\txdl_cha_free(&xdf->rcha);\n+}\n+\n+\n static int xdl_prepare_ctx(unsigned int pass, mmfile_t *mf, long narec, xpparam_t const *xpp,\n \t\t\t xdlclassifier_t *cf, xdfile_t *xdf) {\n-\tlong nrec, bsize;\n+\tlong bsize;\n \tunsigned long hav;\n \tchar const *blk, *cur, *top, *prev;\n \txrecord_t *crec;\n-\txrecord_t **recs;\n-\tunsigned long *ha;\n-\tchar *rchg;\n-\tlong *rindex;\n \n-\tha = NULL;\n-\trindex = NULL;\n-\trchg = NULL;\n-\trecs = NULL;\n+\txdf->ha = NULL;\n+\txdf->rindex = NULL;\n+\txdf->rchg = NULL;\n+\txdf->recs = NULL;\n+\txdf->nrec = 0;\n \n \tif (xdl_cha_init(&xdf->rcha, sizeof(xrecord_t), narec / 4 + 1) < 0)\n \t\tgoto abort;\n-\tif (!XDL_ALLOC_ARRAY(recs, narec))\n+\tif (!XDL_ALLOC_ARRAY(xdf->recs, narec))\n \t\tgoto abort;\n \n-\tnrec = 0;\n \tif ((cur = blk = xdl_mmfile_first(mf, &bsize))) {\n \t\tfor (top = blk + bsize; cur < top; ) {\n \t\t\tprev = cur;\n \t\t\thav = xdl_hash_record(&cur, top, xpp->flags);\n-\t\t\tif (XDL_ALLOC_GROW(recs, nrec + 1, narec))\n+\t\t\tif (XDL_ALLOC_GROW(xdf->recs, xdf->nrec + 1, narec))\n \t\t\t\tgoto abort;\n \t\t\tif (!(crec = xdl_cha_alloc(&xdf->rcha)))\n \t\t\t\tgoto abort;\n \t\t\tcrec->ptr = (u8 const*) prev;\n \t\t\tcrec->size = (long) (cur - prev);\n \t\t\tcrec->ha = hav;\n-\t\t\trecs[nrec++] = crec;\n+\t\t\txdf->recs[xdf->nrec++] = crec;\n \t\t\tif (xdl_classify_record(pass, cf, crec) < 0)\n \t\t\t\tgoto abort;\n \t\t}\n \t}\n \n-\tif (!XDL_CALLOC_ARRAY(rchg, nrec + 2))\n+\tif (!XDL_CALLOC_ARRAY(xdf->rchg, xdf->nrec + 2))\n \t\tgoto abort;\n \n \tif ((XDF_DIFF_ALG(xpp->flags) != XDF_PATIENCE_DIFF) &&\n \t (XDF_DIFF_ALG(xpp->flags) != XDF_HISTOGRAM_DIFF)) {\n-\t\tif (!XDL_ALLOC_ARRAY(rindex, nrec + 1))\n+\t\tif (!XDL_ALLOC_ARRAY(xdf->rindex, xdf->nrec + 1))\n \t\t\tgoto abort;\n-\t\tif (!XDL_ALLOC_ARRAY(ha, nrec + 1))\n+\t\tif (!XDL_ALLOC_ARRAY(xdf->ha, xdf->nrec + 1))\n \t\t\tgoto abort;\n \t}\n \n-\txdf->nrec = nrec;\n-\txdf->recs = recs;\n-\txdf->rchg = rchg + 1;\n-\txdf->rindex = rindex;\n+\txdf->rchg += 1;\n \txdf->nreff = 0;\n-\txdf->ha = ha;\n \txdf->dstart = 0;\n-\txdf->dend = nrec - 1;\n+\txdf->dend = xdf->nrec - 1;\n \n \treturn 0;\n \n abort:\n-\txdl_free(ha);\n-\txdl_free(rindex);\n-\txdl_free(rchg);\n-\txdl_free(recs);\n-\txdl_cha_free(&xdf->rcha);\n+\txdl_free_ctx(xdf);\n \treturn -1;\n }\n \n \n-static void xdl_free_ctx(xdfile_t *xdf) {\n-\txdl_free(xdf->rindex);\n-\txdl_free(xdf->rchg - 1);\n-\txdl_free(xdf->ha);\n-\txdl_free(xdf->recs);\n-\txdl_cha_free(&xdf->rcha);\n-}\n-\n-\n void xdl_free_env(xdfenv_t *xe) {\n \n \txdl_free_ctx(&xe->xdf2);\n-- \ngitgitgadget\n\n"},{"id":"524760","messageId":"6ce0e252b3802c63d1969c23a1fdfa5e80609cf9.1755921357.git.gitgitgadget@gmail.com","threadId":"63804","inReplyTo":"pull.1980.v3.git.git.1755921356.gitgitgadget@gmail.com","subject":"[PATCH v3 11/15] xdiff: replace chastore with an ivec in xdfile_t","fromName":"Ezekiel Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2025-08-23T03:55:52Z","receivedAt":"2025-08-23T03:56:12Z","isPatch":true,"sender":{"key":"ezekielnewren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/18313324?v=4"},"body":"From: Ezekiel Newren \n\nxdfile_t currently uses a chastore which functions as a memory pool and\na vector which maps to the alocations created by the chastore. It seems\nlike xrecord_t used to be a linked list until the recs and nrec fields\nwere added. I think that xrecord_t.next was meant to be removed, but\nwas overlooked. This dual data structure setup make the code somewhat\nconfusing.\n\nAdditionally the C type chastore_t isn't FFI friendly. While it could\nbe implemented in Rust, since the data structure is confusing anyway,\nreplace it with an ivec whose purpose is to be interoperable. This\nmakes the fields nrec and recs in xdfile_t redundant, which will be\nremoved in the next 2 commits.\n\nSigned-off-by: Ezekiel Newren \n---\n xdiff/xprepare.c | 34 +++++++++++++++++-----------------\n xdiff/xtypes.h | 6 ++++--\n 2 files changed, 21 insertions(+), 19 deletions(-)\n\ndiff --git a/xdiff/xprepare.c b/xdiff/xprepare.c\nindex 55e1cc308756..3b33186c15a3 100644\n--- a/xdiff/xprepare.c\n+++ b/xdiff/xprepare.c\n@@ -130,11 +130,11 @@ static int xdl_classify_record(unsigned int pass, xdlclassifier_t *cf, xrecord_t\n \n \n static void xdl_free_ctx(xdfile_t *xdf) {\n+\tivec_free(&xdf->record);\n \txdl_free(xdf->rindex);\n \txdl_free(xdf->rchg - 1);\n \txdl_free(xdf->ha);\n \txdl_free(xdf->recs);\n-\txdl_cha_free(&xdf->rcha);\n }\n \n \n@@ -143,35 +143,35 @@ static int xdl_prepare_ctx(unsigned int pass, mmfile_t *mf, long narec, xpparam_\n \tlong bsize;\n \tunsigned long hav;\n \tchar const *blk, *cur, *top, *prev;\n-\txrecord_t *crec;\n \n \txdf->ha = NULL;\n \txdf->rindex = NULL;\n \txdf->rchg = NULL;\n \txdf->recs = NULL;\n \txdf->nrec = 0;\n-\n-\tif (xdl_cha_init(&xdf->rcha, sizeof(xrecord_t), narec / 4 + 1) < 0)\n-\t\tgoto abort;\n-\tif (!XDL_ALLOC_ARRAY(xdf->recs, narec))\n-\t\tgoto abort;\n+\tIVEC_INIT(xdf->record);\n \n \tif ((cur = blk = xdl_mmfile_first(mf, &bsize))) {\n \t\tfor (top = blk + bsize; cur < top; ) {\n+\t\t\txrecord_t crec;\n \t\t\tprev = cur;\n \t\t\thav = xdl_hash_record(&cur, top, xpp->flags);\n-\t\t\tif (XDL_ALLOC_GROW(xdf->recs, xdf->nrec + 1, narec))\n-\t\t\t\tgoto abort;\n-\t\t\tif (!(crec = xdl_cha_alloc(&xdf->rcha)))\n-\t\t\t\tgoto abort;\n-\t\t\tcrec->ptr = (u8 const*) prev;\n-\t\t\tcrec->size = (long) (cur - prev);\n-\t\t\tcrec->ha = hav;\n-\t\t\txdf->recs[xdf->nrec++] = crec;\n-\t\t\tif (xdl_classify_record(pass, cf, crec) < 0)\n-\t\t\t\tgoto abort;\n+\t\t\tcrec.ptr = (u8 const*) prev;\n+\t\t\tcrec.size = cur - prev;\n+\t\t\tcrec.ha = hav;\n+\t\t\tivec_push(&xdf->record, &crec);\n \t\t}\n \t}\n+\tivec_shrink_to_fit(&xdf->record);\n+\n+\txdf->nrec = (long) xdf->record.length;\n+\tif (!XDL_ALLOC_ARRAY(xdf->recs, xdf->record.length))\n+\t\tgoto abort;\n+\tfor (usize i = 0; i < xdf->record.length; i++) {\n+\t\tif (xdl_classify_record(pass, cf, &xdf->record.ptr[i]) < 0)\n+\t\t\tgoto abort;\n+\t\txdf->recs[i] = &xdf->record.ptr[i];\n+\t}\n \n \tif (!XDL_CALLOC_ARRAY(xdf->rchg, xdf->nrec + 2))\n \t\tgoto abort;\ndiff --git a/xdiff/xtypes.h b/xdiff/xtypes.h\nindex 6e5f67ebf380..5028a70b2675 100644\n--- a/xdiff/xtypes.h\n+++ b/xdiff/xtypes.h\n@@ -23,7 +23,7 @@\n #if !defined(XTYPES_H)\n #define XTYPES_H\n \n-\n+#include \"../interop/ivec.h\"\n \n typedef struct s_chanode {\n \tstruct s_chanode *next;\n@@ -44,8 +44,10 @@ typedef struct s_xrecord {\n \tu64 ha;\n } xrecord_t;\n \n+DEFINE_IVEC_TYPE(xrecord_t, xrecord);\n+\n typedef struct s_xdfile {\n-\tchastore_t rcha;\n+\tstruct ivec_xrecord record;\n \tlong nrec;\n \tlong dstart, dend;\n \txrecord_t **recs;\n-- \ngitgitgadget\n\n"},{"id":"524761","messageId":"0cfc6cf26b75d32df676c65b1645c54d2df35002.1755921357.git.gitgitgadget@gmail.com","threadId":"63804","inReplyTo":"pull.1980.v3.git.git.1755921356.gitgitgadget@gmail.com","subject":"[PATCH v3 12/15] xdiff: delete nrec field from xdfile_t","fromName":"Ezekiel Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2025-08-23T03:55:53Z","receivedAt":"2025-08-23T03:56:14Z","isPatch":true,"sender":{"key":"ezekielnewren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/18313324?v=4"},"body":"From: Ezekiel Newren \n\nBecause of the data structure cleanup in the previous commit, the nrec\nfield is no longer necessary. Use record.length in place of nrec.\n\nThis commit is best viewed with --color-words.\n\nSigned-off-by: Ezekiel Newren \n---\n xdiff/xdiffi.c | 10 +++++-----\n xdiff/xemit.c | 20 ++++++++++----------\n xdiff/xmerge.c | 10 +++++-----\n xdiff/xpatience.c | 2 +-\n xdiff/xprepare.c | 34 ++++++++++++++++------------------\n xdiff/xtypes.h | 1 -\n 6 files changed, 37 insertions(+), 40 deletions(-)\n\ndiff --git a/xdiff/xdiffi.c b/xdiff/xdiffi.c\nindex 3b364c61f671..bcab2d7ae516 100644\n--- a/xdiff/xdiffi.c\n+++ b/xdiff/xdiffi.c\n@@ -496,7 +496,7 @@ static void measure_split(const xdfile_t *xdf, long split,\n {\n \tlong i;\n \n-\tif (split >= xdf->nrec) {\n+\tif (split >= (long) xdf->record.length) {\n \t\tm->end_of_file = 1;\n \t\tm->indent = -1;\n \t} else {\n@@ -519,7 +519,7 @@ static void measure_split(const xdfile_t *xdf, long split,\n \n \tm->post_blank = 0;\n \tm->post_indent = -1;\n-\tfor (i = split + 1; i < xdf->nrec; i++) {\n+\tfor (i = split + 1; i < (long) xdf->record.length; i++) {\n \t\tm->post_indent = get_indent(xdf->recs[i]);\n \t\tif (m->post_indent != -1)\n \t\t\tbreak;\n@@ -730,7 +730,7 @@ static void group_init(xdfile_t *xdf, struct xdlgroup *g)\n */\n static inline int group_next(xdfile_t *xdf, struct xdlgroup *g)\n {\n-\tif (g->end == xdf->nrec)\n+\tif (g->end == (long) xdf->record.length)\n \t\treturn -1;\n \n \tg->start = g->end + 1;\n@@ -763,7 +763,7 @@ static inline int group_previous(xdfile_t *xdf, struct xdlgroup *g)\n */\n static int group_slide_down(xdfile_t *xdf, struct xdlgroup *g)\n {\n-\tif (g->end < xdf->nrec &&\n+\tif (g->end < (long) xdf->record.length &&\n \t recs_match(xdf->recs[g->start], xdf->recs[g->end])) {\n \t\txdf->rchg[g->start++] = 0;\n \t\txdf->rchg[g->end++] = 1;\n@@ -950,7 +950,7 @@ int xdl_build_script(xdfenv_t *xe, xdchange_t **xscr) {\n \t/*\n \t * Trivial. Collects \"groups\" of changes and creates an edit script.\n \t */\n-\tfor (i1 = xe->xdf1.nrec, i2 = xe->xdf2.nrec; i1 >= 0 || i2 >= 0; i1--, i2--)\n+\tfor (i1 = xe->xdf1.record.length, i2 = xe->xdf2.record.length; i1 >= 0 || i2 >= 0; i1--, i2--)\n \t\tif (rchg1[i1 - 1] || rchg2[i2 - 1]) {\n \t\t\tfor (l1 = i1; rchg1[i1 - 1]; i1--);\n \t\t\tfor (l2 = i2; rchg2[i2 - 1]; i2--);\ndiff --git a/xdiff/xemit.c b/xdiff/xemit.c\nindex bbf7b7f8c862..11c2823ecab5 100644\n--- a/xdiff/xemit.c\n+++ b/xdiff/xemit.c\n@@ -147,7 +147,7 @@ static long get_func_line(xdfenv_t *xe, xdemitconf_t const *xecfg,\n \tbuf = func_line ? func_line->buf : dummy;\n \tsize = func_line ? sizeof(func_line->buf) : sizeof(dummy);\n \n-\tfor (l = start; l != limit && 0 <= l && l < xe->xdf1.nrec; l += step) {\n+\tfor (l = start; l != limit && 0 <= l && l < (long) xe->xdf1.record.length; l += step) {\n \t\tlong len = match_func_rec(&xe->xdf1, xecfg, l, buf, size);\n \t\tif (len >= 0) {\n \t\t\tif (func_line)\n@@ -191,14 +191,14 @@ pre_context_calculation:\n \t\t\tlong fs1, i1 = xch->i1;\n \n \t\t\t/* Appended chunk? */\n-\t\t\tif (i1 >= xe->xdf1.nrec) {\n+\t\t\tif (i1 >= (long) xe->xdf1.record.length) {\n \t\t\t\tlong i2 = xch->i2;\n \n \t\t\t\t/*\n \t\t\t\t * We don't need additional context if\n \t\t\t\t * a whole function was added.\n \t\t\t\t */\n-\t\t\t\twhile (i2 < xe->xdf2.nrec) {\n+\t\t\t\twhile (i2 < (long) xe->xdf2.record.length) {\n \t\t\t\t\tif (is_func_rec(&xe->xdf2, xecfg, i2))\n \t\t\t\t\t\tgoto post_context_calculation;\n \t\t\t\t\ti2++;\n@@ -208,7 +208,7 @@ pre_context_calculation:\n \t\t\t\t * Otherwise get more context from the\n \t\t\t\t * pre-image.\n \t\t\t\t */\n-\t\t\t\ti1 = xe->xdf1.nrec - 1;\n+\t\t\t\ti1 = xe->xdf1.record.length - 1;\n \t\t\t}\n \n \t\t\tfs1 = get_func_line(xe, xecfg, NULL, i1, -1);\n@@ -240,8 +240,8 @@ pre_context_calculation:\n \n post_context_calculation:\n \t\tlctx = xecfg->ctxlen;\n-\t\tlctx = XDL_MIN(lctx, xe->xdf1.nrec - (xche->i1 + xche->chg1));\n-\t\tlctx = XDL_MIN(lctx, xe->xdf2.nrec - (xche->i2 + xche->chg2));\n+\t\tlctx = XDL_MIN(lctx, (long) xe->xdf1.record.length - (xche->i1 + xche->chg1));\n+\t\tlctx = XDL_MIN(lctx, (long) xe->xdf2.record.length - (xche->i2 + xche->chg2));\n \n \t\te1 = xche->i1 + xche->chg1 + lctx;\n \t\te2 = xche->i2 + xche->chg2 + lctx;\n@@ -249,13 +249,13 @@ pre_context_calculation:\n \t\tif (xecfg->flags & XDL_EMIT_FUNCCONTEXT) {\n \t\t\tlong fe1 = get_func_line(xe, xecfg, NULL,\n \t\t\t\t\t\t xche->i1 + xche->chg1,\n-\t\t\t\t\t\t xe->xdf1.nrec);\n+\t\t\t\t\t\t xe->xdf1.record.length);\n \t\t\twhile (fe1 > 0 && is_empty_rec(&xe->xdf1, fe1 - 1))\n \t\t\t\tfe1--;\n \t\t\tif (fe1 < 0)\n-\t\t\t\tfe1 = xe->xdf1.nrec;\n+\t\t\t\tfe1 = xe->xdf1.record.length;\n \t\t\tif (fe1 > e1) {\n-\t\t\t\te2 = XDL_MIN(e2 + (fe1 - e1), xe->xdf2.nrec);\n+\t\t\t\te2 = XDL_MIN(e2 + (fe1 - e1), (long) xe->xdf2.record.length);\n \t\t\t\te1 = fe1;\n \t\t\t}\n \n@@ -266,7 +266,7 @@ pre_context_calculation:\n \t\t\t */\n \t\t\tif (xche->next) {\n \t\t\t\tlong l = XDL_MIN(xche->next->i1,\n-\t\t\t\t\t\t xe->xdf1.nrec - 1);\n+\t\t\t\t\t\t (long) xe->xdf1.record.length - 1);\n \t\t\t\tif (l - xecfg->ctxlen <= e1 ||\n \t\t\t\t get_func_line(xe, xecfg, NULL, l, e1) < 0) {\n \t\t\t\t\txche = xche->next;\ndiff --git a/xdiff/xmerge.c b/xdiff/xmerge.c\nindex 6fa6ea61a208..f48549605d09 100644\n--- a/xdiff/xmerge.c\n+++ b/xdiff/xmerge.c\n@@ -158,11 +158,11 @@ static int is_eol_crlf(xdfile_t *file, int i)\n {\n \tlong size;\n \n-\tif (i < file->nrec - 1)\n+\tif (i < (isize) file->record.length - 1)\n \t\t/* All lines before the last *must* end in LF */\n \t\treturn (size = file->recs[i]->size) > 1 &&\n \t\t\tfile->recs[i]->ptr[size - 2] == '\\r';\n-\tif (!file->nrec)\n+\tif (!file->record.length)\n \t\t/* Cannot determine eol style from empty file */\n \t\treturn -1;\n \tif ((size = file->recs[i]->size) &&\n@@ -317,7 +317,7 @@ static int xdl_fill_merge_buffer(xdfenv_t *xe1, const char *name1,\n \t\t\tcontinue;\n \t\ti = m->i1 + m->chg1;\n \t}\n-\tsize += xdl_recs_copy(xe1, i, xe1->xdf2.nrec - i, 0, 0,\n+\tsize += xdl_recs_copy(xe1, i, (int) xe1->xdf2.record.length - i, 0, 0,\n \t\t\t dest ? dest + size : NULL);\n \treturn size;\n }\n@@ -622,7 +622,7 @@ static int xdl_do_merge(xdfenv_t *xe1, xdchange_t *xscr1,\n \t\t\tchanges = c;\n \t\ti0 = xscr1->i1;\n \t\ti1 = xscr1->i2;\n-\t\ti2 = xscr1->i1 + xe2->xdf2.nrec - xe2->xdf1.nrec;\n+\t\ti2 = xscr1->i1 + xe2->xdf2.record.length - xe2->xdf1.record.length;\n \t\tchg0 = xscr1->chg1;\n \t\tchg1 = xscr1->chg2;\n \t\tchg2 = xscr1->chg1;\n@@ -637,7 +637,7 @@ static int xdl_do_merge(xdfenv_t *xe1, xdchange_t *xscr1,\n \t\tif (!changes)\n \t\t\tchanges = c;\n \t\ti0 = xscr2->i1;\n-\t\ti1 = xscr2->i1 + xe1->xdf2.nrec - xe1->xdf1.nrec;\n+\t\ti1 = xscr2->i1 + xe1->xdf2.record.length - xe1->xdf1.record.length;\n \t\ti2 = xscr2->i2;\n \t\tchg0 = xscr2->chg1;\n \t\tchg1 = xscr2->chg1;\ndiff --git a/xdiff/xpatience.c b/xdiff/xpatience.c\nindex 986a3a3f749a..e1ce9a399fbf 100644\n--- a/xdiff/xpatience.c\n+++ b/xdiff/xpatience.c\n@@ -370,5 +370,5 @@ static int patience_diff(xpparam_t const *xpp, xdfenv_t *env,\n \n int xdl_do_patience_diff(xpparam_t const *xpp, xdfenv_t *env)\n {\n-\treturn patience_diff(xpp, env, 1, env->xdf1.nrec, 1, env->xdf2.nrec);\n+\treturn patience_diff(xpp, env, 1, env->xdf1.record.length, 1, env->xdf2.record.length);\n }\ndiff --git a/xdiff/xprepare.c b/xdiff/xprepare.c\nindex 3b33186c15a3..9b46523afe97 100644\n--- a/xdiff/xprepare.c\n+++ b/xdiff/xprepare.c\n@@ -138,7 +138,7 @@ static void xdl_free_ctx(xdfile_t *xdf) {\n }\n \n \n-static int xdl_prepare_ctx(unsigned int pass, mmfile_t *mf, long narec, xpparam_t const *xpp,\n+static int xdl_prepare_ctx(unsigned int pass, mmfile_t *mf, xpparam_t const *xpp,\n \t\t\t xdlclassifier_t *cf, xdfile_t *xdf) {\n \tlong bsize;\n \tunsigned long hav;\n@@ -148,7 +148,6 @@ static int xdl_prepare_ctx(unsigned int pass, mmfile_t *mf, long narec, xpparam_\n \txdf->rindex = NULL;\n \txdf->rchg = NULL;\n \txdf->recs = NULL;\n-\txdf->nrec = 0;\n \tIVEC_INIT(xdf->record);\n \n \tif ((cur = blk = xdl_mmfile_first(mf, &bsize))) {\n@@ -164,7 +163,6 @@ static int xdl_prepare_ctx(unsigned int pass, mmfile_t *mf, long narec, xpparam_\n \t}\n \tivec_shrink_to_fit(&xdf->record);\n \n-\txdf->nrec = (long) xdf->record.length;\n \tif (!XDL_ALLOC_ARRAY(xdf->recs, xdf->record.length))\n \t\tgoto abort;\n \tfor (usize i = 0; i < xdf->record.length; i++) {\n@@ -173,21 +171,21 @@ static int xdl_prepare_ctx(unsigned int pass, mmfile_t *mf, long narec, xpparam_\n \t\txdf->recs[i] = &xdf->record.ptr[i];\n \t}\n \n-\tif (!XDL_CALLOC_ARRAY(xdf->rchg, xdf->nrec + 2))\n+\tif (!XDL_CALLOC_ARRAY(xdf->rchg, xdf->record.length + 2))\n \t\tgoto abort;\n \n \tif ((XDF_DIFF_ALG(xpp->flags) != XDF_PATIENCE_DIFF) &&\n \t (XDF_DIFF_ALG(xpp->flags) != XDF_HISTOGRAM_DIFF)) {\n-\t\tif (!XDL_ALLOC_ARRAY(xdf->rindex, xdf->nrec + 1))\n+\t\tif (!XDL_ALLOC_ARRAY(xdf->rindex, xdf->record.length + 1))\n \t\t\tgoto abort;\n-\t\tif (!XDL_ALLOC_ARRAY(xdf->ha, xdf->nrec + 1))\n+\t\tif (!XDL_ALLOC_ARRAY(xdf->ha, xdf->record.length + 1))\n \t\t\tgoto abort;\n \t}\n \n \txdf->rchg += 1;\n \txdf->nreff = 0;\n \txdf->dstart = 0;\n-\txdf->dend = xdf->nrec - 1;\n+\txdf->dend = xdf->record.length - 1;\n \n \treturn 0;\n \n@@ -274,12 +272,12 @@ static int xdl_cleanup_records(xdlclassifier_t *cf, xdfile_t *xdf1, xdfile_t *xd\n \tchar *dis, *dis1, *dis2;\n \tint need_min = !!(cf->flags & XDF_NEED_MINIMAL);\n \n-\tif (!XDL_CALLOC_ARRAY(dis, xdf1->nrec + xdf2->nrec + 2))\n+\tif (!XDL_CALLOC_ARRAY(dis, xdf1->record.length + xdf2->record.length + 2))\n \t\treturn -1;\n \tdis1 = dis;\n-\tdis2 = dis1 + xdf1->nrec + 1;\n+\tdis2 = dis1 + xdf1->record.length + 1;\n \n-\tif ((mlim = xdl_bogosqrt(xdf1->nrec)) > XDL_MAX_EQLIMIT)\n+\tif ((mlim = xdl_bogosqrt(xdf1->record.length)) > XDL_MAX_EQLIMIT)\n \t\tmlim = XDL_MAX_EQLIMIT;\n \tfor (i = xdf1->dstart, recs = &xdf1->recs[xdf1->dstart]; i <= xdf1->dend; i++, recs++) {\n \t\trcrec = cf->rcrecs[(*recs)->ha];\n@@ -287,7 +285,7 @@ static int xdl_cleanup_records(xdlclassifier_t *cf, xdfile_t *xdf1, xdfile_t *xd\n \t\tdis1[i] = (nm == 0) ? 0: (nm >= mlim && !need_min) ? 2: 1;\n \t}\n \n-\tif ((mlim = xdl_bogosqrt(xdf2->nrec)) > XDL_MAX_EQLIMIT)\n+\tif ((mlim = xdl_bogosqrt(xdf2->record.length)) > XDL_MAX_EQLIMIT)\n \t\tmlim = XDL_MAX_EQLIMIT;\n \tfor (i = xdf2->dstart, recs = &xdf2->recs[xdf2->dstart]; i <= xdf2->dend; i++, recs++) {\n \t\trcrec = cf->rcrecs[(*recs)->ha];\n@@ -334,21 +332,21 @@ static int xdl_trim_ends(xdfile_t *xdf1, xdfile_t *xdf2) {\n \n \trecs1 = xdf1->recs;\n \trecs2 = xdf2->recs;\n-\tfor (i = 0, lim = XDL_MIN(xdf1->nrec, xdf2->nrec); i < lim;\n+\tfor (i = 0, lim = XDL_MIN(xdf1->record.length, xdf2->record.length); i < lim;\n \t i++, recs1++, recs2++)\n \t\tif ((*recs1)->ha != (*recs2)->ha)\n \t\t\tbreak;\n \n \txdf1->dstart = xdf2->dstart = i;\n \n-\trecs1 = xdf1->recs + xdf1->nrec - 1;\n-\trecs2 = xdf2->recs + xdf2->nrec - 1;\n+\trecs1 = xdf1->recs + xdf1->record.length - 1;\n+\trecs2 = xdf2->recs + xdf2->record.length - 1;\n \tfor (lim -= i, i = 0; i < lim; i++, recs1--, recs2--)\n \t\tif ((*recs1)->ha != (*recs2)->ha)\n \t\t\tbreak;\n \n-\txdf1->dend = xdf1->nrec - i - 1;\n-\txdf2->dend = xdf2->nrec - i - 1;\n+\txdf1->dend = xdf1->record.length - i - 1;\n+\txdf2->dend = xdf2->record.length - i - 1;\n \n \treturn 0;\n }\n@@ -388,12 +386,12 @@ int xdl_prepare_env(mmfile_t *mf1, mmfile_t *mf2, xpparam_t const *xpp,\n \tif (xdl_init_classifier(&cf, enl1 + enl2 + 1, xpp->flags) < 0)\n \t\treturn -1;\n \n-\tif (xdl_prepare_ctx(1, mf1, enl1, xpp, &cf, &xe->xdf1) < 0) {\n+\tif (xdl_prepare_ctx(1, mf1, xpp, &cf, &xe->xdf1) < 0) {\n \n \t\txdl_free_classifier(&cf);\n \t\treturn -1;\n \t}\n-\tif (xdl_prepare_ctx(2, mf2, enl2, xpp, &cf, &xe->xdf2) < 0) {\n+\tif (xdl_prepare_ctx(2, mf2, xpp, &cf, &xe->xdf2) < 0) {\n \n \t\txdl_free_ctx(&xe->xdf1);\n \t\txdl_free_classifier(&cf);\ndiff --git a/xdiff/xtypes.h b/xdiff/xtypes.h\nindex 5028a70b2675..c322e62fbf06 100644\n--- a/xdiff/xtypes.h\n+++ b/xdiff/xtypes.h\n@@ -48,7 +48,6 @@ DEFINE_IVEC_TYPE(xrecord_t, xrecord);\n \n typedef struct s_xdfile {\n \tstruct ivec_xrecord record;\n-\tlong nrec;\n \tlong dstart, dend;\n \txrecord_t **recs;\n \tchar *rchg;\n-- \ngitgitgadget\n\n"},{"id":"524762","messageId":"ea699135f95283c05ff20ff47e0c73154ca60eca.1755921357.git.gitgitgadget@gmail.com","threadId":"63804","inReplyTo":"pull.1980.v3.git.git.1755921356.gitgitgadget@gmail.com","subject":"[PATCH v3 14/15] xdiff: make xdfile_t more rust friendly","fromName":"Ezekiel Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2025-08-23T03:55:55Z","receivedAt":"2025-08-23T03:56:15Z","isPatch":true,"sender":{"key":"ezekielnewren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/18313324?v=4"},"body":"From: Ezekiel Newren \n\nConvert the remaining ambiguous fields in xdfile_t from C types to Rust\ntypes for interoperability.\n\nSigned-off-by: Ezekiel Newren \n---\n xdiff/xdiffi.c | 16 ++++++++--------\n xdiff/xdiffi.h | 8 ++++----\n xdiff/xtypes.h | 10 +++++-----\n 3 files changed, 17 insertions(+), 17 deletions(-)\n\ndiff --git a/xdiff/xdiffi.c b/xdiff/xdiffi.c\nindex ebdb72432261..0509b4875996 100644\n--- a/xdiff/xdiffi.c\n+++ b/xdiff/xdiffi.c\n@@ -42,8 +42,8 @@ typedef struct s_xdpsplit {\n * using this algorithm, so a little bit of heuristic is needed to cut the\n * search and to return a suboptimal point.\n */\n-static long xdl_split(unsigned long const *ha1, long off1, long lim1,\n-\t\t unsigned long const *ha2, long off2, long lim2,\n+static long xdl_split(u64 const *ha1, long off1, long lim1,\n+\t\t u64 const *ha2, long off2, long lim2,\n \t\t long *kvdf, long *kvdb, int need_min, xdpsplit_t *spl,\n \t\t xdalgoenv_t *xenv) {\n \tlong dmin = off1 - lim2, dmax = lim1 - off2;\n@@ -260,7 +260,7 @@ static long xdl_split(unsigned long const *ha1, long off1, long lim1,\n int xdl_recs_cmp(diffdata_t *dd1, long off1, long lim1,\n \t\t diffdata_t *dd2, long off2, long lim2,\n \t\t long *kvdf, long *kvdb, int need_min, xdalgoenv_t *xenv) {\n-\tunsigned long const *ha1 = dd1->ha, *ha2 = dd2->ha;\n+\tu64 const *ha1 = dd1->ha, *ha2 = dd2->ha;\n \n \t/*\n \t * Shrink the box by walking through each diagonal snake (SW and NE).\n@@ -273,14 +273,14 @@ int xdl_recs_cmp(diffdata_t *dd1, long off1, long lim1,\n \t * be obviously changed.\n \t */\n \tif (off1 == lim1) {\n-\t\tchar *rchg2 = dd2->rchg;\n-\t\tlong *rindex2 = dd2->rindex;\n+\t\tu8 *rchg2 = dd2->rchg;\n+\t\tusize *rindex2 = dd2->rindex;\n \n \t\tfor (; off2 < lim2; off2++)\n \t\t\trchg2[rindex2[off2]] = 1;\n \t} else if (off2 == lim2) {\n-\t\tchar *rchg1 = dd1->rchg;\n-\t\tlong *rindex1 = dd1->rindex;\n+\t\tu8 *rchg1 = dd1->rchg;\n+\t\tusize *rindex1 = dd1->rindex;\n \n \t\tfor (; off1 < lim1; off1++)\n \t\t\trchg1[rindex1[off1]] = 1;\n@@ -944,7 +944,7 @@ int xdl_change_compact(xdfile_t *xdf, xdfile_t *xdfo, long flags) {\n \n int xdl_build_script(xdfenv_t *xe, xdchange_t **xscr) {\n \txdchange_t *cscr = NULL, *xch;\n-\tchar *rchg1 = xe->xdf1.rchg, *rchg2 = xe->xdf2.rchg;\n+\tu8 *rchg1 = xe->xdf1.rchg, *rchg2 = xe->xdf2.rchg;\n \tlong i1, i2, l1, l2;\n \n \t/*\ndiff --git a/xdiff/xdiffi.h b/xdiff/xdiffi.h\nindex 126c9d8ff4e4..c766ee115c99 100644\n--- a/xdiff/xdiffi.h\n+++ b/xdiff/xdiffi.h\n@@ -25,10 +25,10 @@\n \n \n typedef struct s_diffdata {\n-\tlong nrec;\n-\tunsigned long const *ha;\n-\tlong *rindex;\n-\tchar *rchg;\n+\tusize nrec;\n+\tu64 const *ha;\n+\tusize *rindex;\n+\tu8 *rchg;\n } diffdata_t;\n \n typedef struct s_xdalgoenv {\ndiff --git a/xdiff/xtypes.h b/xdiff/xtypes.h\nindex 849f218b3277..66b3dfae8bdf 100644\n--- a/xdiff/xtypes.h\n+++ b/xdiff/xtypes.h\n@@ -48,11 +48,11 @@ DEFINE_IVEC_TYPE(xrecord_t, xrecord);\n \n typedef struct s_xdfile {\n \tstruct ivec_xrecord record;\n-\tlong dstart, dend;\n-\tchar *rchg;\n-\tlong *rindex;\n-\tlong nreff;\n-\tunsigned long *ha;\n+\tisize dstart, dend;\n+\tu8 *rchg;\n+\tusize *rindex;\n+\tusize nreff;\n+\tu64 *ha;\n } xdfile_t;\n \n typedef struct s_xdfenv {\n-- \ngitgitgadget\n\n"},{"id":"524763","messageId":"cf0387d851c61f13ab6af9506201c8ad11583363.1755921357.git.gitgitgadget@gmail.com","threadId":"63804","inReplyTo":"pull.1980.v3.git.git.1755921356.gitgitgadget@gmail.com","subject":"[PATCH v3 13/15] xdiff: delete recs field from xdfile_t","fromName":"Ezekiel Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2025-08-23T03:55:54Z","receivedAt":"2025-08-23T03:56:15Z","isPatch":true,"sender":{"key":"ezekielnewren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/18313324?v=4"},"body":"From: Ezekiel Newren \n\nBecause of the change from chastore to ivec a few commits ago,\nrecs now points to record's elements in a 1:1 mapping.\nSince both recs and record are vectors, this additional mapping is\nsuperfluous. Remove recs.\n\nThis commit is best viewed with --color-words.\n\nSigned-off-by: Ezekiel Newren \n---\n xdiff/xdiffi.c | 30 ++++++++++++------------\n xdiff/xemit.c | 4 ++--\n xdiff/xhistogram.c | 2 +-\n xdiff/xmerge.c | 58 +++++++++++++++++++++++-----------------------\n xdiff/xpatience.c | 14 +++++------\n xdiff/xprepare.c | 37 +++++++++++++----------------\n xdiff/xtypes.h | 1 -\n xdiff/xutils.c | 12 +++++-----\n 8 files changed, 76 insertions(+), 82 deletions(-)\n\ndiff --git a/xdiff/xdiffi.c b/xdiff/xdiffi.c\nindex bcab2d7ae516..ebdb72432261 100644\n--- a/xdiff/xdiffi.c\n+++ b/xdiff/xdiffi.c\n@@ -501,13 +501,13 @@ static void measure_split(const xdfile_t *xdf, long split,\n \t\tm->indent = -1;\n \t} else {\n \t\tm->end_of_file = 0;\n-\t\tm->indent = get_indent(xdf->recs[split]);\n+\t\tm->indent = get_indent(&xdf->record.ptr[split]);\n \t}\n \n \tm->pre_blank = 0;\n \tm->pre_indent = -1;\n \tfor (i = split - 1; i >= 0; i--) {\n-\t\tm->pre_indent = get_indent(xdf->recs[i]);\n+\t\tm->pre_indent = get_indent(&xdf->record.ptr[i]);\n \t\tif (m->pre_indent != -1)\n \t\t\tbreak;\n \t\tm->pre_blank += 1;\n@@ -520,7 +520,7 @@ static void measure_split(const xdfile_t *xdf, long split,\n \tm->post_blank = 0;\n \tm->post_indent = -1;\n \tfor (i = split + 1; i < (long) xdf->record.length; i++) {\n-\t\tm->post_indent = get_indent(xdf->recs[i]);\n+\t\tm->post_indent = get_indent(&xdf->record.ptr[i]);\n \t\tif (m->post_indent != -1)\n \t\t\tbreak;\n \t\tm->post_blank += 1;\n@@ -764,7 +764,7 @@ static inline int group_previous(xdfile_t *xdf, struct xdlgroup *g)\n static int group_slide_down(xdfile_t *xdf, struct xdlgroup *g)\n {\n \tif (g->end < (long) xdf->record.length &&\n-\t recs_match(xdf->recs[g->start], xdf->recs[g->end])) {\n+\t recs_match(&xdf->record.ptr[g->start], &xdf->record.ptr[g->end])) {\n \t\txdf->rchg[g->start++] = 0;\n \t\txdf->rchg[g->end++] = 1;\n \n@@ -785,7 +785,7 @@ static int group_slide_down(xdfile_t *xdf, struct xdlgroup *g)\n static int group_slide_up(xdfile_t *xdf, struct xdlgroup *g)\n {\n \tif (g->start > 0 &&\n-\t recs_match(xdf->recs[g->start - 1], xdf->recs[g->end - 1])) {\n+\t recs_match(&xdf->record.ptr[g->start - 1], &xdf->record.ptr[g->end - 1])) {\n \t\txdf->rchg[--g->start] = 1;\n \t\txdf->rchg[--g->end] = 0;\n \n@@ -1000,16 +1000,16 @@ static void xdl_mark_ignorable_lines(xdchange_t *xscr, xdfenv_t *xe, long flags)\n \n \tfor (xch = xscr; xch; xch = xch->next) {\n \t\tint ignore = 1;\n-\t\txrecord_t **rec;\n+\t\txrecord_t *rec;\n \t\tlong i;\n \n-\t\trec = &xe->xdf1.recs[xch->i1];\n+\t\trec = &xe->xdf1.record.ptr[xch->i1];\n \t\tfor (i = 0; i < xch->chg1 && ignore; i++)\n-\t\t\tignore = xdl_blankline((const char*) rec[i]->ptr, rec[i]->size, flags);\n+\t\t\tignore = xdl_blankline((const char*) rec[i].ptr, rec[i].size, flags);\n \n-\t\trec = &xe->xdf2.recs[xch->i2];\n+\t\trec = &xe->xdf2.record.ptr[xch->i2];\n \t\tfor (i = 0; i < xch->chg2 && ignore; i++)\n-\t\t\tignore = xdl_blankline((const char*)rec[i]->ptr, rec[i]->size, flags);\n+\t\t\tignore = xdl_blankline((const char*)rec[i].ptr, rec[i].size, flags);\n \n \t\txch->ignore = ignore;\n \t}\n@@ -1033,7 +1033,7 @@ static void xdl_mark_ignorable_regex(xdchange_t *xscr, const xdfenv_t *xe,\n \txdchange_t *xch;\n \n \tfor (xch = xscr; xch; xch = xch->next) {\n-\t\txrecord_t **rec;\n+\t\txrecord_t *rec;\n \t\tint ignore = 1;\n \t\tlong i;\n \n@@ -1043,13 +1043,13 @@ static void xdl_mark_ignorable_regex(xdchange_t *xscr, const xdfenv_t *xe,\n \t\tif (xch->ignore)\n \t\t\tcontinue;\n \n-\t\trec = &xe->xdf1.recs[xch->i1];\n+\t\trec = &xe->xdf1.record.ptr[xch->i1];\n \t\tfor (i = 0; i < xch->chg1 && ignore; i++)\n-\t\t\tignore = record_matches_regex(rec[i], xpp);\n+\t\t\tignore = record_matches_regex(&rec[i], xpp);\n \n-\t\trec = &xe->xdf2.recs[xch->i2];\n+\t\trec = &xe->xdf2.record.ptr[xch->i2];\n \t\tfor (i = 0; i < xch->chg2 && ignore; i++)\n-\t\t\tignore = record_matches_regex(rec[i], xpp);\n+\t\t\tignore = record_matches_regex(&rec[i], xpp);\n \n \t\txch->ignore = ignore;\n \t}\ndiff --git a/xdiff/xemit.c b/xdiff/xemit.c\nindex 11c2823ecab5..0c9a12a5e828 100644\n--- a/xdiff/xemit.c\n+++ b/xdiff/xemit.c\n@@ -24,9 +24,9 @@\n \n static long xdl_get_rec(xdfile_t *xdf, long ri, char const **rec) {\n \n-\t*rec = (char const*) xdf->recs[ri]->ptr;\n+\t*rec = (char const*) xdf->record.ptr[ri].ptr;\n \n-\treturn xdf->recs[ri]->size;\n+\treturn xdf->record.ptr[ri].size;\n }\n \n \ndiff --git a/xdiff/xhistogram.c b/xdiff/xhistogram.c\nindex 040d81e0bc9f..643d1c8b7071 100644\n--- a/xdiff/xhistogram.c\n+++ b/xdiff/xhistogram.c\n@@ -86,7 +86,7 @@ struct region {\n \t((LINE_MAP(index, ptr))->cnt)\n \n #define REC(env, s, l) \\\n-\t(env->xdf##s.recs[l - 1])\n+\t(&env->xdf##s.record.ptr[l - 1])\n \n static int cmp_recs(xrecord_t *r1, xrecord_t *r2)\n {\ndiff --git a/xdiff/xmerge.c b/xdiff/xmerge.c\nindex f48549605d09..0a3e0f28ab84 100644\n--- a/xdiff/xmerge.c\n+++ b/xdiff/xmerge.c\n@@ -97,12 +97,12 @@ static int xdl_merge_cmp_lines(xdfenv_t *xe1, int i1, xdfenv_t *xe2, int i2,\n \t\tint line_count, long flags)\n {\n \tint i;\n-\txrecord_t **rec1 = xe1->xdf2.recs + i1;\n-\txrecord_t **rec2 = xe2->xdf2.recs + i2;\n+\txrecord_t *rec1 = xe1->xdf2.record.ptr + i1;\n+\txrecord_t *rec2 = xe2->xdf2.record.ptr + i2;\n \n \tfor (i = 0; i < line_count; i++) {\n-\t\tint result = xdl_recmatch((const char*) rec1[i]->ptr, rec1[i]->size,\n-\t\t\t(const char*) rec2[i]->ptr, rec2[i]->size, flags);\n+\t\tint result = xdl_recmatch((const char*) rec1[i].ptr, rec1[i].size,\n+\t\t\t(const char*) rec2[i].ptr, rec2[i].size, flags);\n \t\tif (!result)\n \t\t\treturn -1;\n \t}\n@@ -111,20 +111,20 @@ static int xdl_merge_cmp_lines(xdfenv_t *xe1, int i1, xdfenv_t *xe2, int i2,\n \n static int xdl_recs_copy_0(int use_orig, xdfenv_t *xe, int i, int count, int needs_cr, int add_nl, char *dest)\n {\n-\txrecord_t **recs;\n+\txrecord_t *recs;\n \tint size = 0;\n \n-\trecs = (use_orig ? xe->xdf1.recs : xe->xdf2.recs) + i;\n+\trecs = (use_orig ? xe->xdf1.record.ptr : xe->xdf2.record.ptr) + i;\n \n \tif (count < 1)\n \t\treturn 0;\n \n-\tfor (i = 0; i < count; size += recs[i++]->size)\n+\tfor (i = 0; i < count; size += recs[i++].size)\n \t\tif (dest)\n-\t\t\tmemcpy(dest + size, recs[i]->ptr, recs[i]->size);\n+\t\t\tmemcpy(dest + size, recs[i].ptr, recs[i].size);\n \tif (add_nl) {\n-\t\ti = recs[count - 1]->size;\n-\t\tif (i == 0 || recs[count - 1]->ptr[i - 1] != '\\n') {\n+\t\ti = recs[count - 1].size;\n+\t\tif (i == 0 || recs[count - 1].ptr[i - 1] != '\\n') {\n \t\t\tif (needs_cr) {\n \t\t\t\tif (dest)\n \t\t\t\t\tdest[size] = '\\r';\n@@ -160,22 +160,22 @@ static int is_eol_crlf(xdfile_t *file, int i)\n \n \tif (i < (isize) file->record.length - 1)\n \t\t/* All lines before the last *must* end in LF */\n-\t\treturn (size = file->recs[i]->size) > 1 &&\n-\t\t\tfile->recs[i]->ptr[size - 2] == '\\r';\n+\t\treturn (size = file->record.ptr[i].size) > 1 &&\n+\t\t\tfile->record.ptr[i].ptr[size - 2] == '\\r';\n \tif (!file->record.length)\n \t\t/* Cannot determine eol style from empty file */\n \t\treturn -1;\n-\tif ((size = file->recs[i]->size) &&\n-\t\t\tfile->recs[i]->ptr[size - 1] == '\\n')\n+\tif ((size = file->record.ptr[i].size) &&\n+\t\t\tfile->record.ptr[i].ptr[size - 1] == '\\n')\n \t\t/* Last line; ends in LF; Is it CR/LF? */\n \t\treturn size > 1 &&\n-\t\t\tfile->recs[i]->ptr[size - 2] == '\\r';\n+\t\t\tfile->record.ptr[i].ptr[size - 2] == '\\r';\n \tif (!i)\n \t\t/* The only line has no eol */\n \t\treturn -1;\n \t/* Determine eol from second-to-last line */\n-\treturn (size = file->recs[i - 1]->size) > 1 &&\n-\t\tfile->recs[i - 1]->ptr[size - 2] == '\\r';\n+\treturn (size = file->record.ptr[i - 1].size) > 1 &&\n+\t\tfile->record.ptr[i - 1].ptr[size - 2] == '\\r';\n }\n \n static int is_cr_needed(xdfenv_t *xe1, xdfenv_t *xe2, xdmerge_t *m)\n@@ -334,22 +334,22 @@ static int recmatch(xrecord_t *rec1, xrecord_t *rec2, unsigned long flags)\n static void xdl_refine_zdiff3_conflicts(xdfenv_t *xe1, xdfenv_t *xe2, xdmerge_t *m,\n \t\txpparam_t const *xpp)\n {\n-\txrecord_t **rec1 = xe1->xdf2.recs, **rec2 = xe2->xdf2.recs;\n+\txrecord_t *rec1 = xe1->xdf2.record.ptr, *rec2 = xe2->xdf2.record.ptr;\n \tfor (; m; m = m->next) {\n \t\t/* let's handle just the conflicts */\n \t\tif (m->mode)\n \t\t\tcontinue;\n \n \t\twhile(m->chg1 && m->chg2 &&\n-\t\t recmatch(rec1[m->i1], rec2[m->i2], xpp->flags)) {\n+\t\t recmatch(&rec1[m->i1], &rec2[m->i2], xpp->flags)) {\n \t\t\tm->chg1--;\n \t\t\tm->chg2--;\n \t\t\tm->i1++;\n \t\t\tm->i2++;\n \t\t}\n \t\twhile (m->chg1 && m->chg2 &&\n-\t\t recmatch(rec1[m->i1 + m->chg1 - 1],\n-\t\t\t\trec2[m->i2 + m->chg2 - 1], xpp->flags)) {\n+\t\t recmatch(&rec1[m->i1 + m->chg1 - 1],\n+\t\t\t\t&rec2[m->i2 + m->chg2 - 1], xpp->flags)) {\n \t\t\tm->chg1--;\n \t\t\tm->chg2--;\n \t\t}\n@@ -381,12 +381,12 @@ static int xdl_refine_conflicts(xdfenv_t *xe1, xdfenv_t *xe2, xdmerge_t *m,\n \t\t * This probably does not work outside git, since\n \t\t * we have a very simple mmfile structure.\n \t\t */\n-\t\tt1.ptr = (char *)xe1->xdf2.recs[m->i1]->ptr;\n-\t\tt1.size = xe1->xdf2.recs[m->i1 + m->chg1 - 1]->ptr\n-\t\t\t+ xe1->xdf2.recs[m->i1 + m->chg1 - 1]->size - (u8 const*) t1.ptr;\n-\t\tt2.ptr = (char *)xe2->xdf2.recs[m->i2]->ptr;\n-\t\tt2.size = xe2->xdf2.recs[m->i2 + m->chg2 - 1]->ptr\n-\t\t\t+ xe2->xdf2.recs[m->i2 + m->chg2 - 1]->size - (u8 const*) t2.ptr;\n+\t\tt1.ptr = (char *)xe1->xdf2.record.ptr[m->i1].ptr;\n+\t\tt1.size = xe1->xdf2.record.ptr[m->i1 + m->chg1 - 1].ptr\n+\t\t\t+ xe1->xdf2.record.ptr[m->i1 + m->chg1 - 1].size - (u8 const*) t1.ptr;\n+\t\tt2.ptr = (char *)xe2->xdf2.record.ptr[m->i2].ptr;\n+\t\tt2.size = xe2->xdf2.record.ptr[m->i2 + m->chg2 - 1].ptr\n+\t\t\t+ xe2->xdf2.record.ptr[m->i2 + m->chg2 - 1].size - (u8 const*) t2.ptr;\n \t\tif (xdl_do_diff(&t1, &t2, xpp, &xe) < 0)\n \t\t\treturn -1;\n \t\tif (xdl_change_compact(&xe.xdf1, &xe.xdf2, xpp->flags) < 0 ||\n@@ -440,8 +440,8 @@ static int line_contains_alnum(const char *ptr, long size)\n static int lines_contain_alnum(xdfenv_t *xe, int i, int chg)\n {\n \tfor (; chg; chg--, i++)\n-\t\tif (line_contains_alnum((char const*) xe->xdf2.recs[i]->ptr,\n-\t\t\t\txe->xdf2.recs[i]->size))\n+\t\tif (line_contains_alnum((char const*) xe->xdf2.record.ptr[i].ptr,\n+\t\t\t\txe->xdf2.record.ptr[i].size))\n \t\t\treturn 1;\n \treturn 0;\n }\ndiff --git a/xdiff/xpatience.c b/xdiff/xpatience.c\nindex e1ce9a399fbf..31b819ec58f0 100644\n--- a/xdiff/xpatience.c\n+++ b/xdiff/xpatience.c\n@@ -88,9 +88,9 @@ static int is_anchor(xpparam_t const *xpp, const char *line)\n static void insert_record(xpparam_t const *xpp, int line, struct hashmap *map,\n \t\t\t int pass)\n {\n-\txrecord_t **records = pass == 1 ?\n-\t\tmap->env->xdf1.recs : map->env->xdf2.recs;\n-\txrecord_t *record = records[line - 1];\n+\txrecord_t *records = pass == 1 ?\n+\t\tmap->env->xdf1.record.ptr : map->env->xdf2.record.ptr;\n+\txrecord_t *record = &records[line - 1];\n \t/*\n \t * After xdl_prepare_env() (or more precisely, due to\n \t * xdl_classify_record()), the \"ha\" member of the records (AKA lines)\n@@ -121,7 +121,7 @@ static void insert_record(xpparam_t const *xpp, int line, struct hashmap *map,\n \t\treturn;\n \tmap->entries[index].line1 = line;\n \tmap->entries[index].hash = record->ha;\n-\tmap->entries[index].anchor = is_anchor(xpp, (const char*) map->env->xdf1.recs[line - 1]->ptr);\n+\tmap->entries[index].anchor = is_anchor(xpp, (const char*) map->env->xdf1.record.ptr[line - 1].ptr);\n \tif (!map->first)\n \t\tmap->first = map->entries + index;\n \tif (map->last) {\n@@ -246,9 +246,9 @@ static int find_longest_common_sequence(struct hashmap *map, struct entry **res)\n \n static int match(struct hashmap *map, int line1, int line2)\n {\n-\txrecord_t *record1 = map->env->xdf1.recs[line1 - 1];\n-\txrecord_t *record2 = map->env->xdf2.recs[line2 - 1];\n-\treturn record1->ha == record2->ha;\n+\tu64 mph1 = map->env->xdf1.record.ptr[line1 - 1].ha;\n+\tu64 mph2 = map->env->xdf2.record.ptr[line2 - 1].ha;\n+\treturn mph1 == mph2;\n }\n \n static int patience_diff(xpparam_t const *xpp, xdfenv_t *env,\ndiff --git a/xdiff/xprepare.c b/xdiff/xprepare.c\nindex 9b46523afe97..93370f1c6db4 100644\n--- a/xdiff/xprepare.c\n+++ b/xdiff/xprepare.c\n@@ -134,7 +134,6 @@ static void xdl_free_ctx(xdfile_t *xdf) {\n \txdl_free(xdf->rindex);\n \txdl_free(xdf->rchg - 1);\n \txdl_free(xdf->ha);\n-\txdl_free(xdf->recs);\n }\n \n \n@@ -147,7 +146,6 @@ static int xdl_prepare_ctx(unsigned int pass, mmfile_t *mf, xpparam_t const *xpp\n \txdf->ha = NULL;\n \txdf->rindex = NULL;\n \txdf->rchg = NULL;\n-\txdf->recs = NULL;\n \tIVEC_INIT(xdf->record);\n \n \tif ((cur = blk = xdl_mmfile_first(mf, &bsize))) {\n@@ -163,12 +161,9 @@ static int xdl_prepare_ctx(unsigned int pass, mmfile_t *mf, xpparam_t const *xpp\n \t}\n \tivec_shrink_to_fit(&xdf->record);\n \n-\tif (!XDL_ALLOC_ARRAY(xdf->recs, xdf->record.length))\n-\t\tgoto abort;\n \tfor (usize i = 0; i < xdf->record.length; i++) {\n \t\tif (xdl_classify_record(pass, cf, &xdf->record.ptr[i]) < 0)\n \t\t\tgoto abort;\n-\t\txdf->recs[i] = &xdf->record.ptr[i];\n \t}\n \n \tif (!XDL_CALLOC_ARRAY(xdf->rchg, xdf->record.length + 2))\n@@ -267,7 +262,7 @@ static int xdl_clean_mmatch(char const *dis, long i, long s, long e) {\n */\n static int xdl_cleanup_records(xdlclassifier_t *cf, xdfile_t *xdf1, xdfile_t *xdf2) {\n \tlong i, nm, nreff, mlim;\n-\txrecord_t **recs;\n+\txrecord_t *recs;\n \txdlclass_t *rcrec;\n \tchar *dis, *dis1, *dis2;\n \tint need_min = !!(cf->flags & XDF_NEED_MINIMAL);\n@@ -279,38 +274,38 @@ static int xdl_cleanup_records(xdlclassifier_t *cf, xdfile_t *xdf1, xdfile_t *xd\n \n \tif ((mlim = xdl_bogosqrt(xdf1->record.length)) > XDL_MAX_EQLIMIT)\n \t\tmlim = XDL_MAX_EQLIMIT;\n-\tfor (i = xdf1->dstart, recs = &xdf1->recs[xdf1->dstart]; i <= xdf1->dend; i++, recs++) {\n-\t\trcrec = cf->rcrecs[(*recs)->ha];\n+\tfor (i = xdf1->dstart, recs = &xdf1->record.ptr[xdf1->dstart]; i <= xdf1->dend; i++, recs++) {\n+\t\trcrec = cf->rcrecs[recs->ha];\n \t\tnm = rcrec ? rcrec->len2 : 0;\n \t\tdis1[i] = (nm == 0) ? 0: (nm >= mlim && !need_min) ? 2: 1;\n \t}\n \n \tif ((mlim = xdl_bogosqrt(xdf2->record.length)) > XDL_MAX_EQLIMIT)\n \t\tmlim = XDL_MAX_EQLIMIT;\n-\tfor (i = xdf2->dstart, recs = &xdf2->recs[xdf2->dstart]; i <= xdf2->dend; i++, recs++) {\n-\t\trcrec = cf->rcrecs[(*recs)->ha];\n+\tfor (i = xdf2->dstart, recs = &xdf2->record.ptr[xdf2->dstart]; i <= xdf2->dend; i++, recs++) {\n+\t\trcrec = cf->rcrecs[recs->ha];\n \t\tnm = rcrec ? rcrec->len1 : 0;\n \t\tdis2[i] = (nm == 0) ? 0: (nm >= mlim && !need_min) ? 2: 1;\n \t}\n \n-\tfor (nreff = 0, i = xdf1->dstart, recs = &xdf1->recs[xdf1->dstart];\n+\tfor (nreff = 0, i = xdf1->dstart, recs = &xdf1->record.ptr[xdf1->dstart];\n \t i <= xdf1->dend; i++, recs++) {\n \t\tif (dis1[i] == 1 ||\n \t\t (dis1[i] == 2 && !xdl_clean_mmatch(dis1, i, xdf1->dstart, xdf1->dend))) {\n \t\t\txdf1->rindex[nreff] = i;\n-\t\t\txdf1->ha[nreff] = (*recs)->ha;\n+\t\t\txdf1->ha[nreff] = recs->ha;\n \t\t\tnreff++;\n \t\t} else\n \t\t\txdf1->rchg[i] = 1;\n \t}\n \txdf1->nreff = nreff;\n \n-\tfor (nreff = 0, i = xdf2->dstart, recs = &xdf2->recs[xdf2->dstart];\n+\tfor (nreff = 0, i = xdf2->dstart, recs = &xdf2->record.ptr[xdf2->dstart];\n \t i <= xdf2->dend; i++, recs++) {\n \t\tif (dis2[i] == 1 ||\n \t\t (dis2[i] == 2 && !xdl_clean_mmatch(dis2, i, xdf2->dstart, xdf2->dend))) {\n \t\t\txdf2->rindex[nreff] = i;\n-\t\t\txdf2->ha[nreff] = (*recs)->ha;\n+\t\t\txdf2->ha[nreff] = recs->ha;\n \t\t\tnreff++;\n \t\t} else\n \t\t\txdf2->rchg[i] = 1;\n@@ -328,21 +323,21 @@ static int xdl_cleanup_records(xdlclassifier_t *cf, xdfile_t *xdf1, xdfile_t *xd\n */\n static int xdl_trim_ends(xdfile_t *xdf1, xdfile_t *xdf2) {\n \tlong i, lim;\n-\txrecord_t **recs1, **recs2;\n+\txrecord_t *recs1, *recs2;\n \n-\trecs1 = xdf1->recs;\n-\trecs2 = xdf2->recs;\n+\trecs1 = xdf1->record.ptr;\n+\trecs2 = xdf2->record.ptr;\n \tfor (i = 0, lim = XDL_MIN(xdf1->record.length, xdf2->record.length); i < lim;\n \t i++, recs1++, recs2++)\n-\t\tif ((*recs1)->ha != (*recs2)->ha)\n+\t\tif (recs1->ha != recs2->ha)\n \t\t\tbreak;\n \n \txdf1->dstart = xdf2->dstart = i;\n \n-\trecs1 = xdf1->recs + xdf1->record.length - 1;\n-\trecs2 = xdf2->recs + xdf2->record.length - 1;\n+\trecs1 = xdf1->record.ptr + xdf1->record.length - 1;\n+\trecs2 = xdf2->record.ptr + xdf2->record.length - 1;\n \tfor (lim -= i, i = 0; i < lim; i++, recs1--, recs2--)\n-\t\tif ((*recs1)->ha != (*recs2)->ha)\n+\t\tif (recs1->ha != recs2->ha)\n \t\t\tbreak;\n \n \txdf1->dend = xdf1->record.length - i - 1;\ndiff --git a/xdiff/xtypes.h b/xdiff/xtypes.h\nindex c322e62fbf06..849f218b3277 100644\n--- a/xdiff/xtypes.h\n+++ b/xdiff/xtypes.h\n@@ -49,7 +49,6 @@ DEFINE_IVEC_TYPE(xrecord_t, xrecord);\n typedef struct s_xdfile {\n \tstruct ivec_xrecord record;\n \tlong dstart, dend;\n-\txrecord_t **recs;\n \tchar *rchg;\n \tlong *rindex;\n \tlong nreff;\ndiff --git a/xdiff/xutils.c b/xdiff/xutils.c\nindex 10e4f20b7c31..eed88ee6cbe2 100644\n--- a/xdiff/xutils.c\n+++ b/xdiff/xutils.c\n@@ -416,12 +416,12 @@ int xdl_fall_back_diff(xdfenv_t *diff_env, xpparam_t const *xpp,\n \tmmfile_t subfile1, subfile2;\n \txdfenv_t env;\n \n-\tsubfile1.ptr = (char *)diff_env->xdf1.recs[line1 - 1]->ptr;\n-\tsubfile1.size = diff_env->xdf1.recs[line1 + count1 - 2]->ptr +\n-\t\tdiff_env->xdf1.recs[line1 + count1 - 2]->size - (u8 const*) subfile1.ptr;\n-\tsubfile2.ptr = (char *)diff_env->xdf2.recs[line2 - 1]->ptr;\n-\tsubfile2.size = diff_env->xdf2.recs[line2 + count2 - 2]->ptr +\n-\t\tdiff_env->xdf2.recs[line2 + count2 - 2]->size - (u8 const*) subfile2.ptr;\n+\tsubfile1.ptr = (char *)diff_env->xdf1.record.ptr[line1 - 1].ptr;\n+\tsubfile1.size = diff_env->xdf1.record.ptr[line1 + count1 - 2].ptr +\n+\t\tdiff_env->xdf1.record.ptr[line1 + count1 - 2].size - (u8 const*) subfile1.ptr;\n+\tsubfile2.ptr = (char *)diff_env->xdf2.record.ptr[line2 - 1].ptr;\n+\tsubfile2.size = diff_env->xdf2.record.ptr[line2 + count2 - 2].ptr +\n+\t\tdiff_env->xdf2.record.ptr[line2 + count2 - 2].size - (u8 const*) subfile2.ptr;\n \tif (xdl_do_diff(&subfile1, &subfile2, xpp, &env) < 0)\n \t\treturn -1;\n \n-- \ngitgitgadget\n\n"},{"id":"524764","messageId":"b18544b74f3c677bfa023e8a83ed9d2682e0e30d.1755921357.git.gitgitgadget@gmail.com","threadId":"63804","inReplyTo":"pull.1980.v3.git.git.1755921356.gitgitgadget@gmail.com","subject":"[PATCH v3 15/15] xdiff: implement xdl_trim_ends() in Rust","fromName":"Ezekiel Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2025-08-23T03:55:56Z","receivedAt":"2025-08-23T03:56:17Z","isPatch":true,"sender":{"key":"ezekielnewren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/18313324?v=4"},"body":"From: Ezekiel Newren \n\nReplace the C implementation of xdl_trim_ends() with a Rust\nimplementation.\n\nSigned-off-by: Ezekiel Newren \n---\n rust/xdiff/src/lib.rs | 14 ++++++++++++++\n rust/xdiff/src/xprepare.rs | 27 +++++++++++++++++++++++++++\n rust/xdiff/src/xtypes.rs | 19 +++++++++++++++++++\n xdiff/xprepare.c | 28 +---------------------------\n 4 files changed, 61 insertions(+), 27 deletions(-)\n create mode 100644 rust/xdiff/src/xprepare.rs\n create mode 100644 rust/xdiff/src/xtypes.rs\n\ndiff --git a/rust/xdiff/src/lib.rs b/rust/xdiff/src/lib.rs\nindex 8b137891791f..4cc05a7e6b4b 100644\n--- a/rust/xdiff/src/lib.rs\n+++ b/rust/xdiff/src/lib.rs\n@@ -1 +1,15 @@\n+pub mod xprepare;\n+pub mod xtypes;\n \n+use crate::xprepare::trim_ends;\n+use crate::xtypes::xdfile;\n+\n+#[no_mangle]\n+unsafe extern \"C\" fn xdl_trim_ends(xdf1: *mut xdfile, xdf2: *mut xdfile) -> i32 {\n+ let xdf1 = xdf1.as_mut().expect(\"null pointer\");\n+ let xdf2 = xdf2.as_mut().expect(\"null pointer\");\n+\n+ trim_ends(xdf1, xdf2);\n+\n+ 0\n+}\ndiff --git a/rust/xdiff/src/xprepare.rs b/rust/xdiff/src/xprepare.rs\nnew file mode 100644\nindex 000000000000..f64f60c09965\n--- /dev/null\n+++ b/rust/xdiff/src/xprepare.rs\n@@ -0,0 +1,27 @@\n+use crate::xtypes::xdfile;\n+\n+///\n+/// Early trim initial and terminal matching records.\n+///\n+pub(crate) fn trim_ends(xdf1: &mut xdfile, xdf2: &mut xdfile) {\n+ let mut lim = std::cmp::min(xdf1.record.len(), xdf2.record.len());\n+\n+ for i in 0..lim {\n+ if xdf1.record[i].ha != xdf2.record[i].ha {\n+ xdf1.dstart = i as isize;\n+ xdf2.dstart = i as isize;\n+ lim -= i;\n+ break;\n+ }\n+ }\n+\n+ for i in 0..lim {\n+ let f1i = xdf1.record.len() - 1 - i;\n+ let f2i = xdf2.record.len() - 1 - i;\n+ if xdf1.record[f1i].ha != xdf2.record[f2i].ha {\n+ xdf1.dend = f1i as isize;\n+ xdf2.dend = f2i as isize;\n+ break;\n+ }\n+ }\n+}\ndiff --git a/rust/xdiff/src/xtypes.rs b/rust/xdiff/src/xtypes.rs\nnew file mode 100644\nindex 000000000000..3d1ce9742f28\n--- /dev/null\n+++ b/rust/xdiff/src/xtypes.rs\n@@ -0,0 +1,19 @@\n+use interop::ivec::IVec;\n+\n+#[repr(C)]\n+pub(crate) struct xrecord {\n+ pub(crate) ptr: *const u8,\n+ pub(crate) size: usize,\n+ pub(crate) ha: u64,\n+}\n+\n+#[repr(C)]\n+pub(crate) struct xdfile {\n+ pub(crate) record: IVec,\n+ pub(crate) dstart: isize,\n+ pub(crate) dend: isize,\n+ pub(crate) rchg: *mut u8,\n+ pub(crate) rindex: *mut usize,\n+ pub(crate) nreff: usize,\n+ pub(crate) ha: *mut u64,\n+}\ndiff --git a/xdiff/xprepare.c b/xdiff/xprepare.c\nindex 93370f1c6db4..2c7480875f9f 100644\n--- a/xdiff/xprepare.c\n+++ b/xdiff/xprepare.c\n@@ -318,33 +318,7 @@ static int xdl_cleanup_records(xdlclassifier_t *cf, xdfile_t *xdf1, xdfile_t *xd\n }\n \n \n-/*\n- * Early trim initial and terminal matching records.\n- */\n-static int xdl_trim_ends(xdfile_t *xdf1, xdfile_t *xdf2) {\n-\tlong i, lim;\n-\txrecord_t *recs1, *recs2;\n-\n-\trecs1 = xdf1->record.ptr;\n-\trecs2 = xdf2->record.ptr;\n-\tfor (i = 0, lim = XDL_MIN(xdf1->record.length, xdf2->record.length); i < lim;\n-\t i++, recs1++, recs2++)\n-\t\tif (recs1->ha != recs2->ha)\n-\t\t\tbreak;\n-\n-\txdf1->dstart = xdf2->dstart = i;\n-\n-\trecs1 = xdf1->record.ptr + xdf1->record.length - 1;\n-\trecs2 = xdf2->record.ptr + xdf2->record.length - 1;\n-\tfor (lim -= i, i = 0; i < lim; i++, recs1--, recs2--)\n-\t\tif (recs1->ha != recs2->ha)\n-\t\t\tbreak;\n-\n-\txdf1->dend = xdf1->record.length - i - 1;\n-\txdf2->dend = xdf2->record.length - i - 1;\n-\n-\treturn 0;\n-}\n+extern i32 xdl_trim_ends(xdfile_t *xdf1, xdfile_t *xdf2);\n \n \n static int xdl_optimize_ctxs(xdlclassifier_t *cf, xdfile_t *xdf1, xdfile_t *xdf2) {\n-- \ngitgitgadget\n"},{"id":"524771","messageId":"ed31658a-9241-4d75-a086-633448b711a4@app.fastmail.com","threadId":"63804","inReplyTo":"db5d22b188740bcb830e4ccf7f19dcc4e6b557bd.1755921357.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v3 06/15] ivec: create a vector type that is interoperable between C and Rust","fromName":"Kristoffer Haugsbakk","fromEmail":"kristofferhaugsbakk@fastmail.com","sentAt":"2025-08-23T08:12:59Z","receivedAt":"2025-08-23T08:13:22Z","isPatch":true,"sender":{"key":"kristofferhaugsbakk@fastmail.com","avatar":null},"body":"On Sat, Aug 23, 2025, at 05:55, Ezekiel Newren via GitGitGadget wrote:\n> From: Ezekiel Newren \n>\n> Trying to use Rust's Vec in C, or git's ALLOC_GROW() macros (via\n> wrapper functions) in Rust is painful because:\n\nnit: s/git's/Git's/\n\n> [snip]\n> diff --git a/rust/interop/src/ivec.rs b/rust/interop/src/ivec.rs\n> [snip]\n> + // assert_eq!(vec.capacity, vec.slice.len());\n\nWhy are there three commented-out assertions? (all capacity/length)\n\n> + assert_eq!(expected, vec.length);\n> + assert!(vec.capacity >= expected);\n> + for i in 0..vec.length {\n> + assert_eq!(default_value, vec[i]);\n> + }\n> [snip]\n\n-- \nKristoffer Haugsbakk\n"},{"id":"524774","messageId":"CAH=ZcbBCg8837kN9LjvdRtgVWL9vP=EDYw04wEPZxO6NLscbGg@mail.gmail.com","threadId":"63804","inReplyTo":"ed31658a-9241-4d75-a086-633448b711a4@app.fastmail.com","subject":"Re: [PATCH v3 06/15] ivec: create a vector type that is interoperable between C and Rust","fromName":"Ezekiel Newren","fromEmail":"ezekielnewren@gmail.com","sentAt":"2025-08-23T09:29:03Z","receivedAt":"2025-08-23T09:29:16Z","isPatch":true,"sender":{"key":"ezekielnewren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/18313324?v=4"},"body":"On Sat, Aug 23, 2025 at 2:13 AM Kristoffer Haugsbakk\n wrote:\n>\n> On Sat, Aug 23, 2025, at 05:55, Ezekiel Newren via GitGitGadget wrote:\n> > From: Ezekiel Newren \n> >\n> > Trying to use Rust's Vec in C, or git's ALLOC_GROW() macros (via\n> > wrapper functions) in Rust is painful because:\n>\n> nit: s/git's/Git's/\n>\n> > [snip]\n> > diff --git a/rust/interop/src/ivec.rs b/rust/interop/src/ivec.rs\n> > [snip]\n> > + // assert_eq!(vec.capacity, vec.slice.len());\n>\n> Why are there three commented-out assertions? (all capacity/length)\n>\n> > + assert_eq!(expected, vec.length);\n> > + assert!(vec.capacity >= expected);\n> > + for i in 0..vec.length {\n> > + assert_eq!(default_value, vec[i]);\n> > + }\n> > [snip]\n>\n> --\n> Kristoffer Haugsbakk\n\nGood catch, I should have removed those commented out lines. Looking\nback through the code I also missed calling std::ptr::drop_in_place()\nif the IVec shrinks. I'll apply those changes in the next version.\n"},{"id":"524778","messageId":"030a01dc1433$ee3e2510$caba6f30$@nexbridge.com","threadId":"63804","inReplyTo":"03939951256baaaec3fcc690cfa38ee12fb553ce.1755921357.git.gitgitgadget@gmail.com","subject":"RE: [PATCH v3 02/15] xdiff: introduce rust","fromName":"","fromEmail":"rsbecker@nexbridge.com","sentAt":"2025-08-23T13:43:29Z","receivedAt":"2025-08-23T13:44:27Z","isPatch":true,"sender":{"key":"randall.becker@nexbridge.ca","avatar":"https://avatars.githubusercontent.com/u/28956764?v=4"},"body":"On August 22, 2025 11:56 PM, Ezekiel Newren wrote:\n>From: Ezekiel Newren \n>\n>Upcoming patches will simplify xdiff, while also porting parts of it to Rust. In\n>preparation, add some stubs and setup the Rust build. For now, it is easier to let\n>cargo build rust and have make or meson merely link against the static library that\n>cargo builds. In line with ongoing libification efforts, use multiple crates to allow\n>more modularity on the Rust side. xdiff is the crate that this series will focus on, but\n>we also introduce the interop crate for future patch series.\n>\n>In order to facilitate interoperability between C and Rust, introduce C definitions for\n>Rust primitive types in git-compat-util.h.\n>\n>Signed-off-by: Ezekiel Newren \n>---\n> .gitignore | 3 +++\n> Makefile | 53 ++++++++++++++++++++++++++++----------\n> build_rust.sh | 57 +++++++++++++++++++++++++++++++++++++++++\n> git-compat-util.h | 17 ++++++++++++\n> meson.build | 52 +++++++++++++++++++++++++++++++------\n> rust/Cargo.toml | 6 +++++\n> rust/interop/Cargo.toml | 14 ++++++++++ rust/interop/src/lib.rs | 0\n> rust/xdiff/Cargo.toml | 15 +++++++++++\n> rust/xdiff/src/lib.rs | 0\n> 10 files changed, 196 insertions(+), 21 deletions(-) create mode 100755\n>build_rust.sh create mode 100644 rust/Cargo.toml create mode 100644\n>rust/interop/Cargo.toml create mode 100644 rust/interop/src/lib.rs create mode\n>100644 rust/xdiff/Cargo.toml create mode 100644 rust/xdiff/src/lib.rs\n>\n>diff --git a/.gitignore b/.gitignore\n>index 04c444404e4b..ff81e3580c4e 100644\n>--- a/.gitignore\n>+++ b/.gitignore\n>@@ -254,3 +254,6 @@ Release/\n> /contrib/buildsystems/out\n> /contrib/libgit-rs/target\n> /contrib/libgit-sys/target\n>+/.idea/\n>+/rust/target/\n>+/rust/Cargo.lock\n>diff --git a/Makefile b/Makefile\n>index 70d1543b6b86..1ec0c1ee6603 100644\n>--- a/Makefile\n>+++ b/Makefile\n>@@ -919,6 +919,29 @@ TEST_SHELL_PATH = $(SHELL_PATH)\n>\n> LIB_FILE = libgit.a\n> XDIFF_LIB = xdiff/lib.a\n>+\n>+EXTLIBS =\n>+\n>+ifeq ($(DEBUG), 1)\n>+ RUST_BUILD_MODE = debug\n>+else\n>+ RUST_BUILD_MODE = release\n>+endif\n>+\n>+RUST_TARGET_DIR = rust/target/$(RUST_BUILD_MODE) RUST_FLAGS_FOR_C =\n>+-L$(RUST_TARGET_DIR)\n>+\n>+.PHONY: compile_rust\n>+compile_rust:\n>+\t./build_rust.sh . $(RUST_BUILD_MODE) xdiff\n>+\n>+EXTLIBS += ./$(RUST_TARGET_DIR)/libxdiff.a\n>+\n>+UNAME_S := $(shell uname -s)\n>+ifeq ($(UNAME_S),Linux)\n>+ EXTLIBS += -ldl\n>+endif\n>+\n> REFTABLE_LIB = reftable/libreftable.a\n>\n> GENERATED_H += command-list.h\n>@@ -1390,7 +1413,7 @@ UNIT_TEST_OBJS += $(UNIT_TEST_DIR)/lib-reftable.o\n>\n> # xdiff and reftable libs may in turn depend on what is in libgit.a GITLIBS =\n>common-main.o $(LIB_FILE) $(XDIFF_LIB) $(REFTABLE_LIB) $(LIB_FILE) -EXTLIBS =\n>+\n>\n> GIT_USER_AGENT = git/$(GIT_VERSION)\n>\n>@@ -2541,7 +2564,7 @@ git.sp git.s git.o: EXTRA_CPPFLAGS = \\\n> \t'-DGIT_MAN_PATH=\"$(mandir_relative_SQ)\"' \\\n> \t'-DGIT_INFO_PATH=\"$(infodir_relative_SQ)\"'\n>\n>-git$X: git.o GIT-LDFLAGS $(BUILTIN_OBJS) $(GITLIBS)\n>+git$X: git.o GIT-LDFLAGS $(BUILTIN_OBJS) $(GITLIBS) compile_rust\n> \t$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) \\\n> \t\t$(filter %.o,$^) $(LIBS)\n>\n>@@ -2891,17 +2914,17 @@ headless-git.o: compat/win32/headless.c GIT-\n>CFLAGS\n> headless-git$X: headless-git.o git.res GIT-LDFLAGS\n> \t$(QUIET_LINK)$(CC) $(ALL_CFLAGS) $(ALL_LDFLAGS) -mwindows -o $@\n>$< git.res\n>\n>-git-%$X: %.o GIT-LDFLAGS $(GITLIBS)\n>+git-%$X: %.o GIT-LDFLAGS $(GITLIBS) compile_rust\n> \t$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter\n>%.o,$^) $(LIBS)\n>\n>-git-imap-send$X: imap-send.o $(IMAP_SEND_BUILDDEPS) GIT-LDFLAGS\n>$(GITLIBS)\n>+git-imap-send$X: imap-send.o $(IMAP_SEND_BUILDDEPS) GIT-LDFLAGS\n>+$(GITLIBS) compile_rust\n> \t$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter\n>%.o,$^) \\\n> \t\t$(IMAP_SEND_LDFLAGS) $(LIBS)\n>\n>-git-http-fetch$X: http.o http-walker.o http-fetch.o GIT-LDFLAGS $(GITLIBS)\n>+git-http-fetch$X: http.o http-walker.o http-fetch.o GIT-LDFLAGS\n>+$(GITLIBS) compile_rust\n> \t$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter\n>%.o,$^) \\\n> \t\t$(CURL_LIBCURL) $(LIBS)\n>-git-http-push$X: http.o http-push.o GIT-LDFLAGS $(GITLIBS)\n>+git-http-push$X: http.o http-push.o GIT-LDFLAGS $(GITLIBS) compile_rust\n> \t$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter\n>%.o,$^) \\\n> \t\t$(CURL_LIBCURL) $(EXPAT_LIBEXPAT) $(LIBS)\n>\n>@@ -2911,11 +2934,11 @@ $(REMOTE_CURL_ALIASES):\n>$(REMOTE_CURL_PRIMARY)\n> \tln -s $< $@ 2>/dev/null || \\\n> \tcp $< $@\n>\n>-$(REMOTE_CURL_PRIMARY): remote-curl.o http.o http-walker.o GIT-LDFLAGS\n>$(GITLIBS)\n>+$(REMOTE_CURL_PRIMARY): remote-curl.o http.o http-walker.o GIT-LDFLAGS\n>+$(GITLIBS) compile_rust\n> \t$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter\n>%.o,$^) \\\n> \t\t$(CURL_LIBCURL) $(EXPAT_LIBEXPAT) $(LIBS)\n>\n>-scalar$X: scalar.o GIT-LDFLAGS $(GITLIBS)\n>+scalar$X: scalar.o GIT-LDFLAGS $(GITLIBS) compile_rust\n> \t$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) \\\n> \t\t$(filter %.o,$^) $(LIBS)\n>\n>@@ -2925,6 +2948,7 @@ $(LIB_FILE): $(LIB_OBJS)\n> $(XDIFF_LIB): $(XDIFF_OBJS)\n> \t$(QUIET_AR)$(RM) $@ && $(AR) $(ARFLAGS) $@ $^\n>\n>+\n> $(REFTABLE_LIB): $(REFTABLE_OBJS)\n> \t$(QUIET_AR)$(RM) $@ && $(AR) $(ARFLAGS) $@ $^\n>\n>@@ -3294,7 +3318,7 @@ perf: all\n>\n> t/helper/test-tool$X: $(patsubst %,t/helper/%,$(TEST_BUILTINS_OBJS))\n>$(UNIT_TEST_DIR)/test-lib.o\n>\n>-t/helper/test-%$X: t/helper/test-%.o GIT-LDFLAGS $(GITLIBS)\n>+t/helper/test-%$X: t/helper/test-%.o GIT-LDFLAGS $(GITLIBS)\n>+compile_rust\n> \t$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter\n>%.o,$^) $(filter %.a,$^) $(LIBS)\n>\n> check-sha1:: t/helper/test-tool$X\n>@@ -3756,7 +3780,10 @@ cocciclean:\n> \t$(RM) -r .build/contrib/coccinelle\n> \t$(RM) contrib/coccinelle/*.cocci.patch\n>\n>-clean: profile-clean coverage-clean cocciclean\n>+rustclean:\n>+\tcd rust && cargo clean\n>+\n>+clean: profile-clean coverage-clean cocciclean rustclean\n> \t$(RM) -r .build $(UNIT_TEST_BIN)\n> \t$(RM) GIT-TEST-SUITES\n> \t$(RM) po/git.pot po/git-core.pot\n>@@ -3911,13 +3938,13 @@ FUZZ_CXXFLAGS ?= $(ALL_CFLAGS)\n> .PHONY: fuzz-all\n> fuzz-all: $(FUZZ_PROGRAMS)\n>\n>-$(FUZZ_PROGRAMS): %: %.o oss-fuzz/dummy-cmd-main.o $(GITLIBS) GIT-\n>LDFLAGS\n>+$(FUZZ_PROGRAMS): %: %.o oss-fuzz/dummy-cmd-main.o $(GITLIBS)\n>+GIT-LDFLAGS compile_rust\n> \t$(QUIET_LINK)$(FUZZ_CXX) $(FUZZ_CXXFLAGS) -o $@ $(ALL_LDFLAGS) \\\n> \t\t-Wl,--allow-multiple-definition \\\n> \t\t$(filter %.o,$^) $(filter %.a,$^) $(LIBS) $(LIB_FUZZING_ENGINE)\n>\n> $(UNIT_TEST_PROGS): $(UNIT_TEST_BIN)/%$X: $(UNIT_TEST_DIR)/%.o\n>$(UNIT_TEST_OBJS) \\\n>-\t$(GITLIBS) GIT-LDFLAGS\n>+\t$(GITLIBS) GIT-LDFLAGS compile_rust\n> \t$(call mkdir_p_parent_template)\n> \t$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) \\\n> \t\t$(filter %.o,$^) $(filter %.a,$^) $(LIBS) @@ -3936,7 +3963,7 @@\n>$(UNIT_TEST_DIR)/clar.suite: $(UNIT_TEST_DIR)/clar-decls.h\n>$(UNIT_TEST_DIR)/gene\n> $(UNIT_TEST_DIR)/clar/clar.o: $(UNIT_TEST_DIR)/clar.suite\n> $(CLAR_TEST_OBJS): $(UNIT_TEST_DIR)/clar-decls.h\n> $(CLAR_TEST_OBJS): EXTRA_CPPFLAGS = -I$(UNIT_TEST_DIR)\n>-$(CLAR_TEST_PROG): $(UNIT_TEST_DIR)/clar.suite $(CLAR_TEST_OBJS) $(GITLIBS)\n>GIT-LDFLAGS\n>+$(CLAR_TEST_PROG): $(UNIT_TEST_DIR)/clar.suite $(CLAR_TEST_OBJS)\n>+$(GITLIBS) GIT-LDFLAGS compile_rust\n> \t$(call mkdir_p_parent_template)\n> \t$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter\n>%.o,$^) $(LIBS)\n>\n>diff --git a/build_rust.sh b/build_rust.sh new file mode 100755 index\n>000000000000..192385a1d961\n>--- /dev/null\n>+++ b/build_rust.sh\n>@@ -0,0 +1,57 @@\n>+#!/bin/sh\n>+\n>+\n>+rustc -vV || exit $?\n>+cargo --version || exit $?\n>+\n>+dir_git_root=${0%/*}\n>+dir_build=$1\n>+rust_build_profile=$2\n>+crate=$3\n>+\n>+dir_rust=$dir_git_root/rust\n>+\n>+if [ \"$dir_git_root\" = \"\" ]; then\n>+ echo \"did not specify the directory for the root of git\"\n>+ exit 1\n>+fi\n>+\n>+if [ \"$dir_build\" = \"\" ]; then\n>+ echo \"did not specify the build directory\"\n>+ exit 1\n>+fi\n>+\n>+if [ \"$rust_build_profile\" = \"\" ]; then\n>+ echo \"did not specify the rust_build_profile\"\n>+ exit 1\n>+fi\n>+\n>+if [ \"$rust_build_profile\" = \"release\" ]; then\n>+ rust_args=\"--release\"\n>+ export RUSTFLAGS=''\n>+elif [ \"$rust_build_profile\" = \"debug\" ]; then\n>+ rust_args=\"\"\n>+ export RUSTFLAGS='-C debuginfo=2 -C opt-level=1 -C force-frame-pointers=yes'\n>+else\n>+ echo \"illegal rust_build_profile value $rust_build_profile\"\n>+ exit 1\n>+fi\n>+\n>+cd $dir_rust && cargo clean && pwd && cargo build -p $crate $rust_args;\n>+cd $dir_git_root\n>+\n>+libfile=\"lib${crate}.a\"\n>+if rustup show active-toolchain | grep windows-msvc; then\n>+ libfile=\"${crate}.lib\"\n>+fi\n>+dst=$dir_build/$libfile\n>+\n>+if [ \"$dir_git_root\" != \"$dir_build\" ]; then\n>+ src=$dir_rust/target/$rust_build_profile/$libfile\n>+ if [ ! -f $src ]; then\n>+ echo >&2 \"::error:: cannot find path of static library $src is not a file or does not\n>exist\"\n>+ exit 5\n>+ fi\n>+\n>+ rm $dst 2>/dev/null\n>+ mv $src $dst\n>+fi\n>diff --git a/git-compat-util.h b/git-compat-util.h index\n>4678e21c4cb8..82dc99764ac0 100644\n>--- a/git-compat-util.h\n>+++ b/git-compat-util.h\n>@@ -196,6 +196,23 @@ static inline int is_xplatform_dir_sep(int c) #include\n>\"compat/msvc.h\"\n> #endif\n>\n>+/* rust types */\n>+typedef uint8_t u8;\n>+typedef uint16_t u16;\n>+typedef uint32_t u32;\n>+typedef uint64_t u64;\n>+\n>+typedef int8_t i8;\n>+typedef int16_t i16;\n>+typedef int32_t i32;\n>+typedef int64_t i64;\n>+\n>+typedef float f32;\n>+typedef double f64;\n>+\n>+typedef size_t usize;\n>+typedef ptrdiff_t isize;\n>+\n> /* used on Mac OS X */\n> #ifdef PRECOMPOSE_UNICODE\n> #include \"compat/precompose_utf8.h\"\n>diff --git a/meson.build b/meson.build\n>index 596f5ac7110e..324f968338b9 100644\n>--- a/meson.build\n>+++ b/meson.build\n>@@ -267,6 +267,40 @@ version_gen_environment.set('GIT_DATE',\n>get_option('build_date')) version_gen_environment.set('GIT_USER_AGENT',\n>get_option('user_agent')) version_gen_environment.set('GIT_VERSION',\n>get_option('version'))\n>\n>+if get_option('optimization') in ['2', '3', 's', 'z']\n>+ rust_build_profile = 'release'\n>+else\n>+ rust_build_profile = 'debug'\n>+endif\n>+\n>+# Run `rustup show active-toolchain` and capture output rustup_out =\n>+run_command('rustup', 'show', 'active-toolchain',\n>+ check: true).stdout().strip()\n>+\n>+rust_crates = ['xdiff']\n>+rust_builds = []\n>+\n>+foreach crate : rust_crates\n>+ if rustup_out.contains('windows-msvc')\n>+ libfile = crate + '.lib'\n>+ else\n>+ libfile = 'lib' + crate + '.a'\n>+ endif\n>+\n>+ rust_builds += custom_target(\n>+ 'rust_build_'+crate,\n>+ output: libfile,\n>+ build_by_default: true,\n>+ build_always_stale: true,\n>+ command: [\n>+ meson.project_source_root() / 'build_rust.sh',\n>+ meson.current_build_dir(), rust_build_profile, crate,\n>+ ],\n>+ install: false,\n>+ )\n>+endforeach\n>+\n>+\n> compiler = meson.get_compiler('c')\n>\n> libgit_sources = [\n>@@ -1678,14 +1712,16 @@ version_def_h = custom_target( libgit_sources +=\n>version_def_h\n>\n> libgit = declare_dependency(\n>- link_with: static_library('git',\n>- sources: libgit_sources,\n>- c_args: libgit_c_args + [\n>- '-DGIT_VERSION_H=\"' + version_def_h.full_path() + '\"',\n>- ],\n>- dependencies: libgit_dependencies,\n>- include_directories: libgit_include_directories,\n>- ),\n>+ link_with: [\n>+ static_library('git',\n>+ sources: libgit_sources,\n>+ c_args: libgit_c_args + [\n>+ '-DGIT_VERSION_H=\"' + version_def_h.full_path() + '\"',\n>+ ],\n>+ dependencies: libgit_dependencies,\n>+ include_directories: libgit_include_directories,\n>+ ),\n>+ ] + rust_builds,\n> compile_args: libgit_c_args,\n> dependencies: libgit_dependencies,\n> include_directories: libgit_include_directories, diff --git a/rust/Cargo.toml\n>b/rust/Cargo.toml new file mode 100644 index 000000000000..ed3d79d7f827\n>--- /dev/null\n>+++ b/rust/Cargo.toml\n>@@ -0,0 +1,6 @@\n>+[workspace]\n>+members = [\n>+ \"xdiff\",\n>+ \"interop\",\n>+]\n>+resolver = \"2\"\n>diff --git a/rust/interop/Cargo.toml b/rust/interop/Cargo.toml new file mode\n>100644 index 000000000000..045e3b01cfad\n>--- /dev/null\n>+++ b/rust/interop/Cargo.toml\n>@@ -0,0 +1,14 @@\n>+[package]\n>+name = \"interop\"\n>+version = \"0.1.0\"\n>+edition = \"2021\"\n>+\n>+[lib]\n>+name = \"interop\"\n>+path = \"src/lib.rs\"\n>+## staticlib to generate xdiff.a for use by gcc ## cdylib (optional) to\n>+generate xdiff.so for use by gcc ## rlib is required by the rust unit\n>+tests crate-type = [\"staticlib\", \"rlib\"]\n>+\n>+[dependencies]\n>diff --git a/rust/interop/src/lib.rs b/rust/interop/src/lib.rs new file mode 100644\n>index 000000000000..e69de29bb2d1 diff --git a/rust/xdiff/Cargo.toml\n>b/rust/xdiff/Cargo.toml new file mode 100644 index\n>000000000000..eb7966aada64\n>--- /dev/null\n>+++ b/rust/xdiff/Cargo.toml\n>@@ -0,0 +1,15 @@\n>+[package]\n>+name = \"xdiff\"\n>+version = \"0.1.0\"\n>+edition = \"2021\"\n>+\n>+[lib]\n>+name = \"xdiff\"\n>+path = \"src/lib.rs\"\n>+## staticlib to generate xdiff.a for use by gcc ## cdylib (optional) to\n>+generate xdiff.so for use by gcc ## rlib is required by the rust unit\n>+tests crate-type = [\"staticlib\", \"rlib\"]\n>+\n>+[dependencies]\n>+interop = { path = \"../interop\" }\n>diff --git a/rust/xdiff/src/lib.rs b/rust/xdiff/src/lib.rs new file mode 100644 index\n>000000000000..e69de29bb2d1\n\nDoes this introduce Rust as a mandatory dependency for git? If so, it cuts out\nnumerous platforms.\n\n"},{"id":"524779","messageId":"4dffd698-9d3c-41c8-9d3f-0d3750e683d3@app.fastmail.com","threadId":"63804","inReplyTo":"030a01dc1433$ee3e2510$caba6f30$@nexbridge.com","subject":"Re: [PATCH v3 02/15] xdiff: introduce rust","fromName":"Kristoffer Haugsbakk","fromEmail":"kristofferhaugsbakk@fastmail.com","sentAt":"2025-08-23T14:26:15Z","receivedAt":"2025-08-23T14:26:53Z","isPatch":true,"sender":{"key":"kristofferhaugsbakk@fastmail.com","avatar":null},"body":"On Sat, Aug 23, 2025, at 15:43, rsbecker@nexbridge.com wrote:\n> On August 22, 2025 11:56 PM, Ezekiel Newren wrote:\n>>From: Ezekiel Newren \n>>\n>>Upcoming patches will simplify xdiff, while also porting parts of it to Rust. In\n>>preparation, add some stubs and setup the Rust build. For now, it is easier to let\n>>cargo build rust and have make or meson merely link against the static library that\n>>cargo builds. In line with ongoing libification efforts, use multiple crates to allow\n>>more modularity on the Rust side. xdiff is the crate that this series will focus on, but\n>>we also introduce the interop crate for future patch series.\n>>\n>>In order to facilitate interoperability between C and Rust, introduce C definitions for\n>>Rust primitive types in git-compat-util.h.\n>>\n>>Signed-off-by: Ezekiel Newren \n>>[snip snip]\n>\n> Does this introduce Rust as a mandatory dependency for git? If so, it cuts out\n> numerous platforms.\n\nThe proposed platform support policy is in patch 1.\n\nhttps://lore.kernel.org/git/6d065f550fe871cf010409f7bd2a63438cf52723.1755921357.git.gitgitgadget@gmail.com/\n"},{"id":"524780","messageId":"CAH=ZcbB87HCNscUh+1GithGxjnkTcQEGiPdzuAYMk8CZE8KELw@mail.gmail.com","threadId":"63804","inReplyTo":"030a01dc1433$ee3e2510$caba6f30$@nexbridge.com","subject":"Re: [PATCH v3 02/15] xdiff: introduce rust","fromName":"Ezekiel Newren","fromEmail":"ezekielnewren@gmail.com","sentAt":"2025-08-23T14:29:08Z","receivedAt":"2025-08-23T14:29:21Z","isPatch":true,"sender":{"key":"ezekielnewren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/18313324?v=4"},"body":"On Sat, Aug 23, 2025 at 7:44 AM wrote:\n\n> Does this introduce Rust as a mandatory dependency for git? If so, it cuts out\n> numerous platforms.\n\nYes it does.\n"},{"id":"524781","messageId":"031601dc143f$7a9a25d0$6fce7170$@nexbridge.com","threadId":"63804","inReplyTo":"4dffd698-9d3c-41c8-9d3f-0d3750e683d3@app.fastmail.com","subject":"RE: [PATCH v3 02/15] xdiff: introduce rust","fromName":"","fromEmail":"rsbecker@nexbridge.com","sentAt":"2025-08-23T15:06:10Z","receivedAt":"2025-08-23T15:06:50Z","isPatch":true,"sender":{"key":"randall.becker@nexbridge.ca","avatar":"https://avatars.githubusercontent.com/u/28956764?v=4"},"body":"On August 23, 2025 10:26 AM, Kristoffer Haugsbakk wrote:\n>On Sat, Aug 23, 2025, at 15:43, rsbecker@nexbridge.com wrote:\n>> On August 22, 2025 11:56 PM, Ezekiel Newren wrote:\n>>>From: Ezekiel Newren \n>>>\n>>>Upcoming patches will simplify xdiff, while also porting parts of it\n>>>to Rust. In preparation, add some stubs and setup the Rust build. For\n>>>now, it is easier to let cargo build rust and have make or meson\n>>>merely link against the static library that cargo builds. In line with\n>>>ongoing libification efforts, use multiple crates to allow more\n>>>modularity on the Rust side. xdiff is the crate that this series will focus on, but we\n>also introduce the interop crate for future patch series.\n>>>\n>>>In order to facilitate interoperability between C and Rust, introduce\n>>>C definitions for Rust primitive types in git-compat-util.h.\n>>>\n>>>Signed-off-by: Ezekiel Newren [snip snip]\n>>\n>> Does this introduce Rust as a mandatory dependency for git? If so, it\n>> cuts out numerous platforms.\n>\n>The proposed platform support policy is in patch 1.\n>\n>https://lore.kernel.org/git/6d065f550fe871cf010409f7bd2a63438cf52723.1755\n>921357.git.gitgitgadget@gmail.com/\n\nIt is a very disappointing policy to be honest. It kicks me off git because Rust is\nnot available on my platform, representing tens of thousands of users in North\nAmerican alone. Rust is not available, but may be in a few years, but there is no\nguarantee that the hardware vendor (HPE) will provide support. I previously\ncommented about the problem with Rust and was not taken seriously. This is\ndisappointing and exclusionary.\n\nThe assertion in the policy that Rust is easily interoperable is incorrect.\n\nNot Thanks,\nRandall\n\n"},{"id":"524783","messageId":"xmqqo6s6uia4.fsf@gitster.g","threadId":"63804","inReplyTo":"db5d22b188740bcb830e4ccf7f19dcc4e6b557bd.1755921357.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v3 06/15] ivec: create a vector type that is interoperable between C and Rust","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-08-23T16:14:43Z","receivedAt":"2025-08-23T16:14:48Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Ezekiel Newren via GitGitGadget\" writes:\n\n> diff --git a/rust/xdiff/src/lib.rs b/rust/xdiff/src/lib.rs\n> index e69de29bb2d1..8b137891791f 100644\n> --- a/rust/xdiff/src/lib.rs\n> +++ b/rust/xdiff/src/lib.rs\n> @@ -0,0 +1 @@\n> +\n\nThis triggers an \"new blank line at EOF\" whitespace error while\napplying. Intended?\n\n"},{"id":"524787","messageId":"CAH=ZcbDuE9AJBWRvx65hfJwwbJ4qJoY7cZo0KcVrR+fWavnnFw@mail.gmail.com","threadId":"63804","inReplyTo":"xmqqo6s6uia4.fsf@gitster.g","subject":"Re: [PATCH v3 06/15] ivec: create a vector type that is interoperable between C and Rust","fromName":"Ezekiel Newren","fromEmail":"ezekielnewren@gmail.com","sentAt":"2025-08-23T16:37:08Z","receivedAt":"2025-08-23T16:37:21Z","isPatch":true,"sender":{"key":"ezekielnewren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/18313324?v=4"},"body":"On Sat, Aug 23, 2025 at 10:14 AM Junio C Hamano wrote:\n>\n> \"Ezekiel Newren via GitGitGadget\" writes:\n>\n> > diff --git a/rust/xdiff/src/lib.rs b/rust/xdiff/src/lib.rs\n> > index e69de29bb2d1..8b137891791f 100644\n> > --- a/rust/xdiff/src/lib.rs\n> > +++ b/rust/xdiff/src/lib.rs\n> > @@ -0,0 +1 @@\n> > +\n>\n> This triggers an \"new blank line at EOF\" whitespace error while\n> applying. Intended?\n\n\"new blank line at EOF\" is intentional, but it is showing up in the\nwrong place in this patch series. Cargo format automatically creates a\nblank line for empty files. These warnings should have shown up on the\n\"xdiff: introduce rust\" commit. I will fix this.\n"},{"id":"524789","messageId":"xmqq8qj9vrpf.fsf@gitster.g","threadId":"63804","inReplyTo":"db5d22b188740bcb830e4ccf7f19dcc4e6b557bd.1755921357.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v3 06/15] ivec: create a vector type that is interoperable between C and Rust","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-08-23T18:05:48Z","receivedAt":"2025-08-23T18:05:53Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Ezekiel Newren via GitGitGadget\" writes:\n\n> From: Ezekiel Newren \n>\n> Trying to use Rust's Vec in C, or git's ALLOC_GROW() macros (via\n> wrapper functions) in Rust is painful because:\n>\n> * C doing vector things the Rust way would require wrapper functions,\n> and Rust doing vector things the C way would require wrapper\n> ...\n> * Currently, Rust defines its own 'Vec' type that is generic, but its\n> memory allocator and struct layout weren't designed for\n> interoperability with C (or any language for that matter), meaning\n> ...\n> * Similarly, git defines ALLOC_GROW() and related macros in\n> git-compat-util.h. While we could add functions allowing Rust to\n> ...\n\nAll the good reasons any C (or any non-Rust language for that\nmatter) projects would want to have an interoperability Shim\nfor their dynamically allocated and grown array-like things.\n\n> To address these issue, introduce a new type, ivec -- short for\n> interoperable vector. (We refer to it as 'ivec' generally, though on\n> the Rust side the struct is called IVec to match Rust style.) \n\nI however was hoping by now Rust getting used more widely, somebody\nhas already created a generic \"this is how you make C-array and Rust\nvectors interoperate\" wrapper that latecomer projects like us can\nuse without inventing our own.\n\n> +INTEROP_OBJS += interop/ivec.o\n> +.PHONY: interop-objs\n> +interop-objs: $(INTEROP_OBJS)\n\nWhat is this phony target used for? No other targets seem to depend\non this one (I am wondering if we need the latter two lines).\n\n> diff --git a/interop/ivec.c b/interop/ivec.c\n> new file mode 100644\n> index 000000000000..9bc2258c04ad\n> --- /dev/null\n> +++ b/interop/ivec.c\n\nI am wondering if this needs a new hierarchy \"interop\"; shouldn't\nthe existing \"compat\" be a good fit enough? I dunno.\n\nEven though this is a shim to somebody else's code, it still is a\npart of our codebase, so our CodingGuidelines for C programs should\napply. \n\n> @@ -0,0 +1,151 @@\n> +#include \"ivec.h\"\n> +\n> +static void ivec_set_capacity(void* self, usize new_capacity) {\n> +\tstruct rawivec *this = self;\n\n - Asterisk sticks to the variable, not type.\n\n - The opening and closing {braces} for the function body are\n written at the leftmost column on its own line.\n\n - There should be a blank line between the declarations and the\n first statement.\n\n> +\tif (new_capacity == 0)\n> +\t\tFREE_AND_NULL(this->ptr);\n> +\telse\n> +\t\tthis->ptr = xrealloc(this->ptr, new_capacity * this->element_size);\n> +\tthis->capacity = new_capacity;\n> +}\n> +\n> +void ivec_init(void* self, usize element_size) {\n> +\tstruct rawivec *this = self;\n> +\tthis->ptr = NULL;\n> +\tthis->length = 0;\n> +\tthis->capacity = 0;\n> +\tthis->element_size = element_size;\n> +}\n\nI notice that this reintroduces a variable named \"this\", which was\neradicated in 585c0e2e (diff: rename 'this' variables, 2018-02-14).\n\nI do not think those who want to use C++ compilers on our C code\nwould not mind \"self\", so how about doing something like...\n\n\n\tvoid ivec_init(void *self_, usize element_size)\n\t{\n\t\tstruct rawivec *self = self;\n\n\t\tself->ptr = NULL;\n\t\tself->len = 0;\n\t\tself->capacity = 0;\n\t\tself->element_size = element_size;\n\t}\n\n... perhaps?\n\n> diff --git a/interop/ivec.h b/interop/ivec.h\n> new file mode 100644\n> index 000000000000..98be4bbeb54a\n> --- /dev/null\n> +++ b/interop/ivec.h\n> @@ -0,0 +1,52 @@\n> +#ifndef IVEC_H\n> +#define IVEC_H\n> +\n> +#include \"../git-compat-util.h\"\n\nAs we use -I. on the command line, there is no need to add \"../\"\nhere; just writing\n\n\t#include \n\nshould be enough. Also, if this file does not depend on the\nservices compat-util header provides (and I do not think it does\nfrom a brief look at its contents), it is better not to include it.\n\nInstead, the sources (like ivec.c next door we just saw) should\nbegin themselves with #include of git-compat-util.h header before\nincluding ivec.h.\n\n> diff --git a/rust/xdiff/src/lib.rs b/rust/xdiff/src/lib.rs\n> index e69de29bb2d1..8b137891791f 100644\n> --- a/rust/xdiff/src/lib.rs\n> +++ b/rust/xdiff/src/lib.rs\n> @@ -0,0 +1 @@\n> +\n\nIf this empty line in an otherwise empty file is absolutely\nnecessary to make Rust work, then please arrange .gitattributes to\ntell git that this file is excempt from the usual blank-at-eof\nwhitespace rule we use. If not, remove that unnecessary empty line.\n\nOr perhaps remove the file altogether if nobody looks at it???\n\nIn any case, given that our top-level .gitattributes file starts\nwith\n\n * whitespace=!indent,trail,space\n *.[ch] whitespace=indent,trail,space diff=cpp\n *.sh whitespace=indent,trail,space text eol=lf\n ...\n *.bat text eol=crlf\n CODE_OF_CONDUCT.md -whitespace\n ...\n\nI think a new rule to cover \"*.rs\" and perhaps *.toml files right\nbefore rules for each specific file begin would be in order.\n\nThanks.\n\n"},{"id":"524790","messageId":"CABPp-BHdHQFv74GDbe=pJBFBALAMZoGsJDhSGqPbT3Daadnd4A@mail.gmail.com","threadId":"63804","inReplyTo":"031601dc143f$7a9a25d0$6fce7170$@nexbridge.com","subject":"Re: [PATCH v3 02/15] xdiff: introduce rust","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2025-08-23T18:30:26Z","receivedAt":"2025-08-23T18:30:38Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"Hi Randall,\n\nOn Sat, Aug 23, 2025 at 8:06 AM wrote:\n>\n> On August 23, 2025 10:26 AM, Kristoffer Haugsbakk wrote:\n> >On Sat, Aug 23, 2025, at 15:43, rsbecker@nexbridge.com wrote:\n[...]\n> >> Does this introduce Rust as a mandatory dependency for git? If so, it\n> >> cuts out numerous platforms.\n> >\n> >The proposed platform support policy is in patch 1.\n> >\n> >https://lore.kernel.org/git/6d065f550fe871cf010409f7bd2a63438cf52723.1755\n> >921357.git.gitgitgadget@gmail.com/\n>\n> It is a very disappointing policy to be honest. It kicks me off git because Rust is\n> not available on my platform, representing tens of thousands of users in North\n> American alone. Rust is not available, but may be in a few years, but there is no\n> guarantee that the hardware vendor (HPE) will provide support. I previously\n> commented about the problem with Rust and was not taken seriously. This is\n> disappointing and exclusionary.\n\nI don't think that's fair. A quick reminder on the history: There was\nlots of excitement about potentially introducing Rust two years ago at\nour virtual Git contributors conference. Taylor formally proposed\nadopting it on the mailing list a year and a half ago. And at Git\nMerge last year, among those in attendance, there was broad\nsignificant interest in adopting Rust with unanimous support for\nletting it move forward among those that were present (which, yes, we\nknow wasn't everyone). And there's the three rounds so far of this\npatch series. At every discussion where you weren't present, someone\nelse would always bring up you and NonStop, and point out how you've\nbeen a very positive long-term member of the Git community and how\nRust adoption would likely negatively affect you, which would be\nregrettable. We waited years to adopt Rust precisely (and I believe\nsolely) because of your objections. Josh and Calvin even went the\nroute of making optional not-even-built-by-default Rust libraries\n(libgit-rs and libgit-sys) when they wanted to add some Rust bindings.\nIf years of deference by other community members isn't considered\ntaking you seriously, I don't know what is.\n\nI agree that it is disappointing that there isn't a clear way to both\ngain the compelling advantages of Rust while also retaining the full\ncurrent extent of our widespread platform support. It's doubly\nunfortunate since you're such a positive contributing member of the\ncommunity. But not allowing us to ever gain the advantages of Rust is\nproblematic too. So, a decision has to be made, one way or the other.\n\nIf it helps, here's the statements I've seen from long term community\nmembers on Ezekiel's proposal for a hard dependency so far, most of\nwhich call out the reduced platform support (whether in favor of the\nproposal or not):\n * Randall: https://lore.kernel.org/git/031601dc143f$7a9a25d0$6fce7170$@nexbridge.com/\n * brian: https://lore.kernel.org/git/aHlwZPbiKnakMN75@fruit.crustytoothpaste.net/\n * Taylor: https://lore.kernel.org/git/aHl4U98BBvpA5eKF@nand.local/\n * Patrick: https://lore.kernel.org/git/aH-CN0RYFmpm7fMt@pks.im/\n * Phillip: https://lore.kernel.org/git/f439958d-64ce-417f-8175-720f69387d48@gmail.com/\n\nThere's also been some emails that can be read as implicitly making a\nposition statement on the topic from long term community members:\n * Junio: https://lore.kernel.org/git/xmqqzfd12ujv.fsf@gitster.g/\n * Johannes: https://lore.kernel.org/git/ac871bc4-df93-31f4-55f2-d6fc538a422d@gmx.de/\n * Elijah: https://lore.kernel.org/git/pull.1980.git.git.1752784344.gitgitgadget@gmail.com/\n(I figured my noted assistance of this series meant I didn't need to\nexplicitly call out my support for it.)\n\n> The assertion in the policy that Rust is easily interoperable is incorrect.\n\nAre you mixing up interoperability with portability? Without further\ncontext than your email provides, it appears so to me. Rust code can\ncall C code and vice-versa within the same process without huge\namounts of serializing and deserializing of data structures, and\nwithout what amounts to something close to an operating system context\nswitch in order to ensure call stacks are as expected for the language\nin question. To me, that means we can call the two languages easily\ninteroperable. On the other hand, portability of those languages is\nabout whether those languages have compilers supported on various\nhardware platforms. The document explicitly calls out that fewer\nsystems have a Rust compiler than have a C compiler, and that Rust\nadoption would thus reduce how portable Git is. Are you referring to\nthis lower portability that the document itself also calls out, or are\nyou pointing out additional issues with interoperation between the\nlanguages on a platform where compilers for both languages exist? If\nthe latter, could you provide more details?\n\n\nI know my email is probably disappointing to you, at a minimum. I'm\nsorry about that. I hope it's helpful, at least in having links to\nwhere various folks in the community stand if nothing else.\n"},{"id":"524791","messageId":"aKoVcVexWi212pAl@fruit.crustytoothpaste.net","threadId":"63804","inReplyTo":"CABPp-BHdHQFv74GDbe=pJBFBALAMZoGsJDhSGqPbT3Daadnd4A@mail.gmail.com","subject":"Re: [PATCH v3 02/15] xdiff: introduce rust","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2025-08-23T19:24:33Z","receivedAt":"2025-08-23T19:24:35Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On 2025-08-23 at 18:30:26, Elijah Newren wrote:\n> I don't think that's fair. A quick reminder on the history: There was\n> lots of excitement about potentially introducing Rust two years ago at\n> our virtual Git contributors conference. Taylor formally proposed\n> adopting it on the mailing list a year and a half ago. And at Git\n> Merge last year, among those in attendance, there was broad\n> significant interest in adopting Rust with unanimous support for\n> letting it move forward among those that were present (which, yes, we\n> know wasn't everyone). And there's the three rounds so far of this\n> patch series. At every discussion where you weren't present, someone\n> else would always bring up you and NonStop, and point out how you've\n> been a very positive long-term member of the Git community and how\n> Rust adoption would likely negatively affect you, which would be\n> regrettable. We waited years to adopt Rust precisely (and I believe\n> solely) because of your objections. Josh and Calvin even went the\n> route of making optional not-even-built-by-default Rust libraries\n> (libgit-rs and libgit-sys) when they wanted to add some Rust bindings.\n> If years of deference by other community members isn't considered\n> taking you seriously, I don't know what is.\n> \n> I agree that it is disappointing that there isn't a clear way to both\n> gain the compelling advantages of Rust while also retaining the full\n> current extent of our widespread platform support. It's doubly\n> unfortunate since you're such a positive contributing member of the\n> community. But not allowing us to ever gain the advantages of Rust is\n> problematic too. So, a decision has to be made, one way or the other.\n\nI think it's worth saying that I do appreciate your (Randall's) positive\ncontributions as well and I would love some way to continue to support\nNonStop as we adopt Rust. To be clear, I care deeply about portability:\nI have owned PowerPC, UltraSPARC, MIPS, and ARM hardware, and I test\nmany of my personal projects on at least Linux, FreeBSD, and NetBSD.\n\nThere is an alternative Rust compiler, mrustc[0], which is written in\nC++ and that I have played around with to see if it could meet our\nneeds. I've been very busy lately and haven't had the time to test it\nout fully, and although it will likely require some upstream changes for\nstatic libraries and a compatibility wrapper because its minicargo is\nvery limited in functionality, it might be an option that we could\nleverage. There will necessarily be work on Rust upstream as well, but\nI'm hoping that mrustc will at least open doors for us.\n\nI also think that Rust is becoming a more and more common language in\ntechnology because of its interoperability with C and its memory safety.\nThe support policy I wrote up explains why there is an increasing push\nfrom governments, security professionals, and the technology industry\nfor memory-safe languages. If Git is to continue its success and broad\nadoption, we don't want it to be labelled software that is using\nsecurity anti-patterns, and we also don't want it to be a CVE factory\nlike libxml2 or ImageMagick. This is the reason I ultimately started\nwork on the SHA-256 project many years ago: I knew we'd need to do it\nfor security reasons and that without a more secure hash algorithm, Git\nwould eventually be dropped.\n\nMy hope is that NonStop can find some way to support Rust because I\nthink it's a compelling language and NonStop would greatly benefit from\nthe wider variety of software available. My sense of previous\ndiscussions was that we do very much want NonStop to continue to come\nalong as we support Rust in Git and that if there are ways we make it\neasier for both, we'd want to do that. That's certainly my view, at\nleast.\n\n[0] https://github.com/thepowersgang/mrustc\n-- \nbrian m. carlson (they/them)\nToronto, Ontario, CA\n"},{"id":"524794","messageId":"033d01dc1469$30088aa0$90199fe0$@nexbridge.com","threadId":"63804","inReplyTo":"aKoVcVexWi212pAl@fruit.crustytoothpaste.net","subject":"RE: [PATCH v3 02/15] xdiff: introduce rust","fromName":"","fromEmail":"rsbecker@nexbridge.com","sentAt":"2025-08-23T20:04:45Z","receivedAt":"2025-08-23T20:05:27Z","isPatch":true,"sender":{"key":"randall.becker@nexbridge.ca","avatar":"https://avatars.githubusercontent.com/u/28956764?v=4"},"body":"On August 23, 2025 3:25 PM, brian m. carlson wrote:\n>On 2025-08-23 at 18:30:26, Elijah Newren wrote:\n>> I don't think that's fair. A quick reminder on the history: There was\n>> lots of excitement about potentially introducing Rust two years ago at\n>> our virtual Git contributors conference. Taylor formally proposed\n>> adopting it on the mailing list a year and a half ago. And at Git\n>> Merge last year, among those in attendance, there was broad\n>> significant interest in adopting Rust with unanimous support for\n>> letting it move forward among those that were present (which, yes, we\n>> know wasn't everyone). And there's the three rounds so far of this\n>> patch series. At every discussion where you weren't present, someone\n>> else would always bring up you and NonStop, and point out how you've\n>> been a very positive long-term member of the Git community and how\n>> Rust adoption would likely negatively affect you, which would be\n>> regrettable. We waited years to adopt Rust precisely (and I believe\n>> solely) because of your objections. Josh and Calvin even went the\n>> route of making optional not-even-built-by-default Rust libraries\n>> (libgit-rs and libgit-sys) when they wanted to add some Rust bindings.\n>> If years of deference by other community members isn't considered\n>> taking you seriously, I don't know what is.\n>>\n>> I agree that it is disappointing that there isn't a clear way to both\n>> gain the compelling advantages of Rust while also retaining the full\n>> current extent of our widespread platform support. It's doubly\n>> unfortunate since you're such a positive contributing member of the\n>> community. But not allowing us to ever gain the advantages of Rust is\n>> problematic too. So, a decision has to be made, one way or the other.\n>\n>I think it's worth saying that I do appreciate your (Randall's) positive contributions\n>as well and I would love some way to continue to support NonStop as we adopt\n>Rust. To be clear, I care deeply about portability:\n>I have owned PowerPC, UltraSPARC, MIPS, and ARM hardware, and I test many of\n>my personal projects on at least Linux, FreeBSD, and NetBSD.\n>\n>There is an alternative Rust compiler, mrustc[0], which is written in\n>C++ and that I have played around with to see if it could meet our\n>needs. I've been very busy lately and haven't had the time to test it out fully, and\n>although it will likely require some upstream changes for static libraries and a\n>compatibility wrapper because its minicargo is very limited in functionality, it might\n>be an option that we could leverage. There will necessarily be work on Rust\n>upstream as well, but I'm hoping that mrustc will at least open doors for us.\n>\n>I also think that Rust is becoming a more and more common language in technology\n>because of its interoperability with C and its memory safety.\n>The support policy I wrote up explains why there is an increasing push from\n>governments, security professionals, and the technology industry for memory-safe\n>languages. If Git is to continue its success and broad adoption, we don't want it to\n>be labelled software that is using security anti-patterns, and we also don't want it to\n>be a CVE factory like libxml2 or ImageMagick. This is the reason I ultimately started\n>work on the SHA-256 project many years ago: I knew we'd need to do it for security\n>reasons and that without a more secure hash algorithm, Git would eventually be\n>dropped.\n>\n>My hope is that NonStop can find some way to support Rust because I think it's a\n>compelling language and NonStop would greatly benefit from the wider variety of\n>software available. My sense of previous discussions was that we do very much\n>want NonStop to continue to come along as we support Rust in Git and that if there\n>are ways we make it easier for both, we'd want to do that. That's certainly my view,\n>at least.\n>\n>[0] https://github.com/thepowersgang/mrustc\n\nI appreciate the encouragement, Brian. I have been trying to port Rust (and GO)\nfor years, without success on the platform. It is only POSIX, but not Linux, which\nseems to be the requirement to do almost anything anymore.\n\nI gave mrustc a try before. It appears to require GCC, which does not port to NonStop.\nIf you have any build recipes that work with c11, that would be helpful. We are\nExpecting c17 soon. I can run c11 in c++ mode, but the makefiles seem to require\nG++, which is part of GCC.\n\n"},{"id":"524795","messageId":"CAH=ZcbAvhxVdg_LBPj5XQMYCYi+cqxMavJ8uUi-DoGs85Biu3g@mail.gmail.com","threadId":"63804","inReplyTo":"xmqq8qj9vrpf.fsf@gitster.g","subject":"Re: [PATCH v3 06/15] ivec: create a vector type that is interoperable between C and Rust","fromName":"Ezekiel Newren","fromEmail":"ezekielnewren@gmail.com","sentAt":"2025-08-23T20:29:19Z","receivedAt":"2025-08-23T20:29:32Z","isPatch":true,"sender":{"key":"ezekielnewren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/18313324?v=4"},"body":"On Sat, Aug 23, 2025 at 12:05 PM Junio C Hamano wrote:\n> > To address these issue, introduce a new type, ivec -- short for\n> > interoperable vector. (We refer to it as 'ivec' generally, though on\n> > the Rust side the struct is called IVec to match Rust style.)\n>\n> I however was hoping by now Rust getting used more widely, somebody\n> has already created a generic \"this is how you make C-array and Rust\n> vectors interoperate\" wrapper that latecomer projects like us can\n> use without inventing our own.\n\nI've looked for code that will do what Git needs, but as far as I know\nnothing can do everything that my ivec can.\n\n> > +INTEROP_OBJS += interop/ivec.o\n> > +.PHONY: interop-objs\n> > +interop-objs: $(INTEROP_OBJS)\n>\n> What is this phony target used for? No other targets seem to depend\n> on this one (I am wondering if we need the latter two lines).\n\nYou are correct. I will remove those lines.\n\n> > diff --git a/interop/ivec.c b/interop/ivec.c\n> > new file mode 100644\n> > index 000000000000..9bc2258c04ad\n> > --- /dev/null\n> > +++ b/interop/ivec.c\n>\n> I am wondering if this needs a new hierarchy \"interop\"; shouldn't\n> the existing \"compat\" be a good fit enough? I dunno.\n\nI had considered compat/, but I thought it didn’t fit. I thought it\nmeant that the same API would exist everywhere, as opposed to “We\nspeak different languages, but we’ve agreed on a translator or common\nprotocol”. In particular, an example from ivec, a function on both\nthe C and Rust sides:\n\n void ivec_extend_from_slice(void *_self, void const *ptr, usize size);\n pub fn extend_from_slice(&mut self, slice: &[T]) where T: Clone,\n\nThe Rust side uses a slice or “fat” pointer, where the C side uses two\narguments (a pointer and a size) in its place. The API is different,\neven if semantically they are the same and they are interoperable.\n\nWas I reading too much into the meaning of compat/? Do folks object\nto using interop/?\n\n> Even though this is a shim to somebody else's code, it still is a\n> part of our codebase, so our CodingGuidelines for C programs should\n> apply.\n\nSorry I missed those. I will fix them up.\n\n> > @@ -0,0 +1,151 @@\n> > +#include \"ivec.h\"\n> > +\n> > +static void ivec_set_capacity(void* self, usize new_capacity) {\n> > + struct rawivec *this = self;\n>\n> - Asterisk sticks to the variable, not type.\n>\n> - The opening and closing {braces} for the function body are\n> written at the leftmost column on its own line.\n>\n> - There should be a blank line between the declarations and the\n> first statement.\n\nI will make those changes.\n\n> > + if (new_capacity == 0)\n> > + FREE_AND_NULL(this->ptr);\n> > + else\n> > + this->ptr = xrealloc(this->ptr, new_capacity * this->element_size);\n> > + this->capacity = new_capacity;\n> > +}\n> > +\n> > +void ivec_init(void* self, usize element_size) {\n> > + struct rawivec *this = self;\n> > + this->ptr = NULL;\n> > + this->length = 0;\n> > + this->capacity = 0;\n> > + this->element_size = element_size;\n> > +}\n>\n> I notice that this reintroduces a variable named \"this\", which was\n> eradicated in 585c0e2e (diff: rename 'this' variables, 2018-02-14).\n>\n> I do not think those who want to use C++ compilers on our C code\n> would not mind \"self\", so how about doing something like...\n>\n>\n> void ivec_init(void *self_, usize element_size)\n> {\n> struct rawivec *self = self;\n>\n> self->ptr = NULL;\n> self->len = 0;\n> self->capacity = 0;\n> self->element_size = element_size;\n> }\n>\n> ... perhaps?\n\nThat sounds good. I will make those changes.\n\n> > diff --git a/interop/ivec.h b/interop/ivec.h\n> > new file mode 100644\n> > index 000000000000..98be4bbeb54a\n> > --- /dev/null\n> > +++ b/interop/ivec.h\n> > @@ -0,0 +1,52 @@\n> > +#ifndef IVEC_H\n> > +#define IVEC_H\n> > +\n> > +#include \"../git-compat-util.h\"\n>\n> As we use -I. on the command line, there is no need to add \"../\"\n> here; just writing\n>\n> #include \n>\n> should be enough. Also, if this file does not depend on the\n> services compat-util header provides (and I do not think it does\n> from a brief look at its contents), it is better not to include it.\n\nThis file actually does depend on git-compat-util.h, particularly the\nRust primitive definitions (e.g. usize, u64, etc...).\n\nI'll use the include style you mentioned.\n\n> > diff --git a/rust/xdiff/src/lib.rs b/rust/xdiff/src/lib.rs\n> > index e69de29bb2d1..8b137891791f 100644\n> > --- a/rust/xdiff/src/lib.rs\n> > +++ b/rust/xdiff/src/lib.rs\n> > @@ -0,0 +1 @@\n> > +\n>\n> If this empty line in an otherwise empty file is absolutely\n> necessary to make Rust work, then please arrange .gitattributes to\n> tell git that this file is excempt from the usual blank-at-eof\n> whitespace rule we use. If not, remove that unnecessary empty line.\n>\n> Or perhaps remove the file altogether if nobody looks at it???\n>\n> In any case, given that our top-level .gitattributes file starts\n> with\n>\n> * whitespace=!indent,trail,space\n> *.[ch] whitespace=indent,trail,space diff=cpp\n> *.sh whitespace=indent,trail,space text eol=lf\n> ...\n> *.bat text eol=crlf\n> CODE_OF_CONDUCT.md -whitespace\n> ...\n>\n> I think a new rule to cover \"*.rs\" and perhaps *.toml files right\n> before rules for each specific file begin would be in order.\n\nThis is only a problem for empty files, and we only have empty .rs\nfiles to setup the basic Rust build in this commit. It's not a problem\nlater in this series and shouldn't be a problem in the future. I will\nremove those blank lines from those commits.\n"},{"id":"524796","messageId":"878qj9bws5.fsf@gentoo.org","threadId":"63804","inReplyTo":"aKoVcVexWi212pAl@fruit.crustytoothpaste.net","subject":"Re: [PATCH v3 02/15] xdiff: introduce rust","fromName":"Sam James","fromEmail":"sam@gentoo.org","sentAt":"2025-08-23T20:36:26Z","receivedAt":"2025-08-23T20:36:34Z","isPatch":true,"sender":{"key":"sam@gentoo.org","avatar":"https://avatars.githubusercontent.com/u/11667869?v=4"},"body":"\"brian m. carlson\" writes:\n\n> On 2025-08-23 at 18:30:26, Elijah Newren wrote:\n>> I don't think that's fair. A quick reminder on the history: There was\n>> lots of excitement about potentially introducing Rust two years ago at\n>> our virtual Git contributors conference. Taylor formally proposed\n>> adopting it on the mailing list a year and a half ago. And at Git\n>> Merge last year, among those in attendance, there was broad\n>> significant interest in adopting Rust with unanimous support for\n>> letting it move forward among those that were present (which, yes, we\n>> know wasn't everyone). And there's the three rounds so far of this\n>> patch series. At every discussion where you weren't present, someone\n>> else would always bring up you and NonStop, and point out how you've\n>> been a very positive long-term member of the Git community and how\n>> Rust adoption would likely negatively affect you, which would be\n>> regrettable. We waited years to adopt Rust precisely (and I believe\n>> solely) because of your objections. Josh and Calvin even went the\n>> route of making optional not-even-built-by-default Rust libraries\n>> (libgit-rs and libgit-sys) when they wanted to add some Rust bindings.\n>> If years of deference by other community members isn't considered\n>> taking you seriously, I don't know what is.\n>> \n>> I agree that it is disappointing that there isn't a clear way to both\n>> gain the compelling advantages of Rust while also retaining the full\n>> current extent of our widespread platform support. It's doubly\n>> unfortunate since you're such a positive contributing member of the\n>> community. But not allowing us to ever gain the advantages of Rust is\n>> problematic too. So, a decision has to be made, one way or the other.\n>\n> I think it's worth saying that I do appreciate your (Randall's) positive\n> contributions as well and I would love some way to continue to support\n> NonStop as we adopt Rust. To be clear, I care deeply about portability:\n> I have owned PowerPC, UltraSPARC, MIPS, and ARM hardware, and I test\n> many of my personal projects on at least Linux, FreeBSD, and NetBSD.\n>\n> There is an alternative Rust compiler, mrustc[0], which is written in\n> C++ and that I have played around with to see if it could meet our\n> needs.\n\nAs far as I'm aware, mrustc is intended purely for having a bootstrap\npath to rustc, not to be a full blown Rust implementation.\n\nWe discussed the other options in this area in\nhttps://lore.kernel.org/git/874iv4gqxv.fsf@gentoo.org/ and Patrick's\nreply.\n\nI still think it's dubious to move to something where there's only one\nimplementation (and an implementation that moves very fast) when\ncurrently we go to pains to support even incomplete C compilers! See the\n\"test balloon\" for 'bool'.\n\n> [...]\n\nsam\n"},{"id":"524797","messageId":"aKov6qwXzrn7TCH6@cloudsdale.lanodan.eu","threadId":"63804","inReplyTo":"aKoVcVexWi212pAl@fruit.crustytoothpaste.net","subject":"Re: [PATCH v3 02/15] xdiff: introduce rust","fromName":"Haelwenn (lanodan) Monnier","fromEmail":"contact@hacktivis.me","sentAt":"2025-08-23T21:17:30Z","receivedAt":"2025-08-23T21:24:11Z","isPatch":true,"sender":{"key":"contact@hacktivis.me","avatar":null},"body":"[2025-08-23 19:24:33+0000] brian m. carlson:\n>On 2025-08-23 at 18:30:26, Elijah Newren wrote:\n>> I don't think that's fair. A quick reminder on the history: There was\n>> lots of excitement about potentially introducing Rust two years ago at\n>> our virtual Git contributors conference. Taylor formally proposed\n>> adopting it on the mailing list a year and a half ago. And at Git\n>> Merge last year, among those in attendance, there was broad\n>> significant interest in adopting Rust with unanimous support for\n>> letting it move forward among those that were present (which, yes, we\n>> know wasn't everyone). And there's the three rounds so far of this\n>> patch series. At every discussion where you weren't present, someone\n>> else would always bring up you and NonStop, and point out how you've\n>> been a very positive long-term member of the Git community and how\n>> Rust adoption would likely negatively affect you, which would be\n>> regrettable. We waited years to adopt Rust precisely (and I believe\n>> solely) because of your objections. Josh and Calvin even went the\n>> route of making optional not-even-built-by-default Rust libraries\n>> (libgit-rs and libgit-sys) when they wanted to add some Rust bindings.\n>> If years of deference by other community members isn't considered\n>> taking you seriously, I don't know what is.\n>>\n>> I agree that it is disappointing that there isn't a clear way to both\n>> gain the compelling advantages of Rust while also retaining the full\n>> current extent of our widespread platform support. It's doubly\n>> unfortunate since you're such a positive contributing member of the\n>> community. But not allowing us to ever gain the advantages of Rust is\n>> problematic too. So, a decision has to be made, one way or the other.\n>\n>I think it's worth saying that I do appreciate your (Randall's) positive\n>contributions as well and I would love some way to continue to support\n>NonStop as we adopt Rust. To be clear, I care deeply about portability:\n>I have owned PowerPC, UltraSPARC, MIPS, and ARM hardware, and I test\n>many of my personal projects on at least Linux, FreeBSD, and NetBSD.\n>\n>There is an alternative Rust compiler, mrustc[0], which is written in\n>C++ and that I have played around with to see if it could meet our\n>needs. I've been very busy lately and haven't had the time to test it\n>out fully, and although it will likely require some upstream changes for\n>static libraries and a compatibility wrapper because its minicargo is\n>very limited in functionality, it might be an option that we could\n>leverage. There will necessarily be work on Rust upstream as well, but\n>I'm hoping that mrustc will at least open doors for us.\n>\n>I also think that Rust is becoming a more and more common language in\n>technology because of its interoperability with C and its memory safety.\n>The support policy I wrote up explains why there is an increasing push\n>from governments, security professionals, and the technology industry\n>for memory-safe languages. If Git is to continue its success and broad\n>adoption, we don't want it to be labelled software that is using\n>security anti-patterns, and we also don't want it to be a CVE factory\n>like libxml2 or ImageMagick. This is the reason I ultimately started\n>work on the SHA-256 project many years ago: I knew we'd need to do it\n>for security reasons and that without a more secure hash algorithm, Git\n>would eventually be dropped.\n>\n>My hope is that NonStop can find some way to support Rust because I\n>think it's a compelling language and NonStop would greatly benefit from\n>the wider variety of software available. My sense of previous\n>discussions was that we do very much want NonStop to continue to come\n>along as we support Rust in Git and that if there are ways we make it\n>easier for both, we'd want to do that. That's certainly my view, at\n>least.\n>\n>[0] https://github.com/thepowersgang/mrustc\n>-- \n>brian m. carlson (they/them)\n>Toronto, Ontario, CA\n\nHello,\n\nmrustc isn't really a alternative compiler, it only serves\nto bootstrap rustc+cargo from source code rather than binaries,\nyou can't really use it to compile arbitrary Rust code.\n\nYou'd still need to port LLVM and rustc.\n\ngccrs would more be the alternative compiler but it still seems\nto have a long road ahead of it: https://rust-gcc.github.io/\n\nBest regards\n"},{"id":"524807","messageId":"71B2DFE6-77E5-47FE-9FAC-AFC1B85DA0E2@gmail.com","threadId":"63804","inReplyTo":"db5d22b188740bcb830e4ccf7f19dcc4e6b557bd.1755921357.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v3 06/15] ivec: create a vector type that is interoperable between C and Rust","fromName":"Ben Knoble","fromEmail":"ben.knoble@gmail.com","sentAt":"2025-08-24T13:31:22Z","receivedAt":"2025-08-24T13:31:35Z","isPatch":true,"sender":{"key":"ben.knoble@gmail.com","avatar":"https://avatars.githubusercontent.com/u/22802209?v=4"},"body":"\n> Le 22 août 2025 à 23:56, Ezekiel Newren via GitGitGadget a écrit :\n> \n> From: Ezekiel Newren \n> \n> Trying to use Rust's Vec in C, or git's ALLOC_GROW() macros (via\n> wrapper functions) in Rust is painful because:\n> \n> * C doing vector things the Rust way would require wrapper functions,\n> and Rust doing vector things the C way would require wrapper\n> functions, so ivec was created to ensure a consistent contract\n> between the 2 languages for how to manipulate a vector.\n> * Currently, Rust defines its own 'Vec' type that is generic, but its\n> memory allocator and struct layout weren't designed for\n> interoperability with C (or any language for that matter), meaning\n> that the C side cannot push to or expand a 'Vec' without defining\n> wrapper functions in Rust that C can call. Without special care,\n> the two languages might use different allocators (malloc/free on\n> the C side, and possibly something else in Rust), which would make\n> it difficult for a function in one language to free elements\n> allocated by a call from a function in the other language.\n> * Similarly, git defines ALLOC_GROW() and related macros in\n> git-compat-util.h. While we could add functions allowing Rust to\n> invoke something similar to those macros, passing three variables\n> (pointer, length, allocated_size) instead of a single variable\n> (vector) across the language boundary requires more cognitive\n> overhead for readers to keep track of and makes it easier to make\n> mistakes. Further, for low-level components that we want to\n> eventually convert to pure Rust, such triplets would feel very out\n> of place.\n\nI’m mildly surprised Vec isn’t a good fit: isn’t it a pointer, length, capacity triple? But it sounds like the main issue is allocator interop… which I would also have thought was supported? At least the current version is documented as being generic against an Allocator, too.\n\n> \n> To address these issue, introduce a new type, ivec -- short for\n> interoperable vector. (We refer to it as 'ivec' generally, though on\n> the Rust side the struct is called IVec to match Rust style.) This new\n> type is specifically designed for FFI purposes, so that both languages\n> handle the vector in the same way, though it could be used on either\n> side independently. This type is designed such that it can easily be\n> replaced by a standard Rust 'Vec' once interoperability is no longer a\n> concern.\n\nAm I reading the patch correctly that the ivec implementation is primarily C? I’m not familiar with too many FFI projects in Rust, but I might have hoped we could write parts in Rust to gain any benefits from that, too. Is that a fool’s errand I’m thinking of?"},{"id":"524815","messageId":"aKtDWW-tXOjlJgSh@pks.im","threadId":"63804","inReplyTo":"CABPp-BEOVBUa7_sTJybgFsgcwAUMeFFhNJEDVnYp_6TYnqu2rg@mail.gmail.com","subject":"Re: [-SPAM-] [PATCH v2 00/17] RFC: Accelerate xdiff and begin its rustification","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-08-24T16:52:41Z","receivedAt":"2025-08-24T16:52:57Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Mon, Aug 18, 2025 at 07:00:16PM -0700, Elijah Newren wrote:\n> On Fri, Aug 15, 2025 at 8:10 AM Ramsay Jones\n> wrote:\n> >\n> > On 15/08/2025 02:22, Ezekiel Newren via GitGitGadget wrote:\n> > > Changes in this second round of this RFC:\n> > >\n> > > * Now builds and passes tests on all platforms (example run:\n> > > https://github.com/ezekielnewren/git/actions/runs/16974821401). Special\n> > > thanks to Johannes Schindelin for patches to things for Windows and\n> > > linux32.\n> >\n> > Hmm, builds on *all* platforms may be a bit optimistic (it doesn't on\n> > cygwin, for instance), so I'm guessing you mean all platforms which\n> > have CI defined. Perhaps you could mention the platforms which you\n> > have tested on. :)\n> \n> Ezekiel says this email didn't show up in his inbox (no idea why), but\n> yes what was meant was all platforms where gitgitgadget CI runs. If\n> you follow the github.com link in the text that you quoted, you can\n> see all those platforms (various windows flavors, various osx builds,\n> musl, sparse, static analysis, etc.).\n\nI do have some patches sitting around for a long while already that\nimplements CI via MSYS2 in different environments. It works with both\nMSYS and MinGW, where I think they are somewhat related to Cygwin? If it\nwould prove useful I could maybe polish this patch series and send it\nupstream.\n\nPatrick\n"},{"id":"524888","messageId":"CABPp-BF44xgh5uJhCKXE8aSN5otyHOAJYNqB_bfLj1Z7_FANCw@mail.gmail.com","threadId":"63804","inReplyTo":"xmqq8qj9vrpf.fsf@gitster.g","subject":"Re: [PATCH v3 06/15] ivec: create a vector type that is interoperable between C and Rust","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2025-08-25T19:16:24Z","receivedAt":"2025-08-25T19:16:36Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"On Sat, Aug 23, 2025 at 11:05 AM Junio C Hamano wrote:\n> > diff --git a/interop/ivec.c b/interop/ivec.c\n> > new file mode 100644\n> > index 000000000000..9bc2258c04ad\n> > --- /dev/null\n> > +++ b/interop/ivec.c\n>\n> Even though this is a shim to somebody else's code, it still is a\n> part of our codebase, so our CodingGuidelines for C programs should\n> apply.\n\nSorry, I should have caught these in my preliminary review before he\nsent this off to the list. One question, though...\n\n> > diff --git a/interop/ivec.h b/interop/ivec.h\n> > new file mode 100644\n> > index 000000000000..98be4bbeb54a\n> > --- /dev/null\n> > +++ b/interop/ivec.h\n> > @@ -0,0 +1,52 @@\n> > +#ifndef IVEC_H\n> > +#define IVEC_H\n> > +\n> > +#include \"../git-compat-util.h\"\n>\n> As we use -I. on the command line, there is no need to add \"../\"\n> here; just writing\n>\n> #include \n>\n> should be enough. Also, if this file does not depend on the\n> services compat-util header provides (and I do not think it does\n> from a brief look at its contents), it is better not to include it.\n\nShould this rather be\n\n #include \"git-compat-util.h\"\n\nwith quotes rather than angle brackets? In particular:\n\n$ git grep include.*git-compat-util -- '*.[ch]' | wc -l\n362\n$ git grep include.*git-compat-util -- '*/*.[ch]' | wc -l\n125\n\nSo, we have 362 includes of git-compat-util.h in our codebase, 125\nfrom subdirectories. Of those:\n\n$ git grep include.*git-compat-util -- '*.[ch]' | grep '\"' | wc -l\n361\n$ git grep include.*git-compat-util -- '*.[ch]' | grep '<' | wc -l\n1\n\nOnly one of these include statements uses angle brackets -- the\ncompiler-tricks/not-constant.c file (which appears to be a temporary\nhack that we'll eventually delete). I had always assumed <> were for\nsystem includes and \"\" for project includes, but a quick Google search\nshows the actual situation is quite a bit murkier than I'd realized.\nStill, our current project practice appears to be double quotes; is\nthat fine here or are you suggesting you'd like the current project\npractice to be changed?\n"},{"id":"524892","messageId":"CAH=ZcbA=-iEFnJ-TecAZL_EX-f3pAShDhdq=S2XWkQHYgRZV7Q@mail.gmail.com","threadId":"63804","inReplyTo":"71B2DFE6-77E5-47FE-9FAC-AFC1B85DA0E2@gmail.com","subject":"Re: [PATCH v3 06/15] ivec: create a vector type that is interoperable between C and Rust","fromName":"Ezekiel Newren","fromEmail":"ezekielnewren@gmail.com","sentAt":"2025-08-25T20:40:19Z","receivedAt":"2025-08-25T20:40:33Z","isPatch":true,"sender":{"key":"ezekielnewren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/18313324?v=4"},"body":"On Sun, Aug 24, 2025 at 7:31 AM Ben Knoble wrote:\n> I’m mildly surprised Vec isn’t a good fit: isn’t it a pointer, length, capacity triple? But it sounds like the main issue is allocator interop… which I would also have thought was supported? At least the current version is documented as being generic against an Allocator, too.\n\nConceptually yes, semantically and syntactically no. On top of Vec\nnot being defined with #[repr(C)] (which ensures field order, C ABI\nlayout, padding, etc...) the struct definition for Vec isn't constant\nbetween Rust versions. I'd be open to suggestions for an alternative\nto my ivec type.\n\n=== Rust version 1.61.0 ===\nfrom: https://doc.rust-lang.org/1.61.0/src/alloc/vec/mod.rs.html#400\n#[stable(feature = \"rust1\", since = \"1.0.0\")]\n#[cfg_attr(not(test), rustc_diagnostic_item = \"Vec\")]\n#[rustc_insignificant_dtor]\npub struct Vec {\n buf: RawVec,\n len: usize,\n}\n\nfrom: https://doc.rust-lang.org/1.61.0/src/alloc/raw_vec.rs.html#52\n#[allow(missing_debug_implementations)]\npub(crate) struct RawVec {\n ptr: Unique,\n cap: usize,\n alloc: A,\n}\n\n=== Rust version 1.89.0 ===\nfrom: https://doc.rust-lang.org/1.89.0/src/alloc/vec/mod.rs.html#414\n#[stable(feature = \"rust1\", since = \"1.0.0\")]\n#[rustc_diagnostic_item = \"Vec\"]\n#[rustc_insignificant_dtor]\npub struct Vec {\n buf: RawVec,\n len: usize,\n}\n\nfrom: https://doc.rust-lang.org/1.89.0/src/alloc/raw_vec/mod.rs.html#74\n#[allow(missing_debug_implementations)]\npub(crate) struct RawVec {\n inner: RawVecInner,\n _marker: PhantomData,\n}\n\nfrom: https://doc.rust-lang.org/1.89.0/src/alloc/raw_vec/mod.rs.html#86\n#[allow(missing_debug_implementations)]\nstruct RawVecInner {\n ptr: Unique,\n /// Never used for ZSTs; it's `capacity()`'s responsibility to\nreturn usize::MAX in that case.\n ///\n /// # Safety\n ///\n /// `cap` must be in the `0..=isize::MAX` range.\n cap: Cap,\n alloc: A,\n}\n\n> Am I reading the patch correctly that the ivec implementation is primarily C? I’m not familiar with too many FFI projects in Rust, but I might have hoped we could write parts in Rust to gain any benefits from that, too. Is that a fool’s errand I’m thinking of?\n\nThe ivec type is defined and implemented in C (interop/ivec.[ch]) and\nRust (rust/interop/src/ivec.rs). When I started writing the ivec type\nI didn't know if the Git community would accept a hard dependency on\nRust, so I made ivec usable in C without needing Rust.\n"},{"id":"524919","messageId":"xmqqo6s2myil.fsf@gitster.g","threadId":"63804","inReplyTo":"CABPp-BF44xgh5uJhCKXE8aSN5otyHOAJYNqB_bfLj1Z7_FANCw@mail.gmail.com","subject":"Re: [PATCH v3 06/15] ivec: create a vector type that is interoperable between C and Rust","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-08-26T05:40:18Z","receivedAt":"2025-08-26T05:40:21Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Elijah Newren writes:\n\n>> > +#include \"../git-compat-util.h\"\n>>\n>> As we use -I. on the command line, there is no need to add \"../\"\n>> here; just writing\n>>\n>> #include \n>>\n>> should be enough. Also, if this file does not depend on the\n>> services compat-util header provides (and I do not think it does\n>> from a brief look at its contents), it is better not to include it.\n>\n> Should this rather be\n>\n> #include \"git-compat-util.h\"\n\nI meant <>; when \"\" included header is not found, it falls back as\nif it were <> included, IIRC, so writing <> when you specify exactly\nwhere your headers are with -I. avoids such unnecessary fallback in\ntheory, but as both <> and \"\" search for implementation-defined\nplaces, the distinction does not make much practical difference.\n\n> Still, our current project practice appears to be double quotes; is\n> that fine here or are you suggesting you'd like the current project\n> practice to be changed?\n\nIt would be nice if we could do so, but I do not think it is worth\nthe patch churn.\n\n"},{"id":"524946","messageId":"CALnO6CASgMQ=cbQ_ijWXV0RMMSZgvS47r8ucTro7Wc4pgZ9_jQ@mail.gmail.com","threadId":"63804","inReplyTo":"CAH=ZcbA=-iEFnJ-TecAZL_EX-f3pAShDhdq=S2XWkQHYgRZV7Q@mail.gmail.com","subject":"Re: [PATCH v3 06/15] ivec: create a vector type that is interoperable between C and Rust","fromName":"D. Ben Knoble","fromEmail":"ben.knoble@gmail.com","sentAt":"2025-08-26T13:30:30Z","receivedAt":"2025-08-26T13:30:46Z","isPatch":true,"sender":{"key":"ben.knoble@gmail.com","avatar":"https://avatars.githubusercontent.com/u/22802209?v=4"},"body":"On Mon, Aug 25, 2025 at 4:40 PM Ezekiel Newren wrote:\n>\n> On Sun, Aug 24, 2025 at 7:31 AM Ben Knoble wrote:\n> > I’m mildly surprised Vec isn’t a good fit: isn’t it a pointer, length, capacity triple? But it sounds like the main issue is allocator interop… which I would also have thought was supported? At least the current version is documented as being generic against an Allocator, too.\n>\n> Conceptually yes, semantically and syntactically no. On top of Vec\n> not being defined with #[repr(C)] (which ensures field order, C ABI\n> layout, padding, etc...) the struct definition for Vec isn't constant\n> between Rust versions. I'd be open to suggestions for an alternative\n> to my ivec type.\n\nAh, thanks—I had forgotten about the #[repr(C)] needs and changes. Makes sense.\n\n> > Am I reading the patch correctly that the ivec implementation is primarily C? I’m not familiar with too many FFI projects in Rust, but I might have hoped we could write parts in Rust to gain any benefits from that, too. Is that a fool’s errand I’m thinking of?\n>\n> The ivec type is defined and implemented in C (interop/ivec.[ch]) and\n> Rust (rust/interop/src/ivec.rs). When I started writing the ivec type\n> I didn't know if the Git community would accept a hard dependency on\n> Rust, so I made ivec usable in C without needing Rust.\n\nRight—I saw both implementations, but it looked like C did most of the\nwork, which was my main question. Re-reading, it looks like Rust does\nmore work than I thought (with implementations of insert/push/etc.)\n\nThat said, I think it's sensible to leave the type useable from just C\nunless/until Rust becomes required (and then we can move things over).\n\nThanks!\n\n-- \nD. Ben Knoble\n"},{"id":"524992","messageId":"CAH=ZcbD_pX1YdZbt9b-xMmcu2806twhjECez4HhCyE9iBf-9=Q@mail.gmail.com","threadId":"63804","inReplyTo":"CALnO6CASgMQ=cbQ_ijWXV0RMMSZgvS47r8ucTro7Wc4pgZ9_jQ@mail.gmail.com","subject":"Re: [PATCH v3 06/15] ivec: create a vector type that is interoperable between C and Rust","fromName":"Ezekiel Newren","fromEmail":"ezekielnewren@gmail.com","sentAt":"2025-08-26T18:47:06Z","receivedAt":"2025-08-26T18:47:19Z","isPatch":true,"sender":{"key":"ezekielnewren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/18313324?v=4"},"body":"On Tue, Aug 26, 2025 at 7:30 AM D. Ben Knoble wrote:\n> > > Am I reading the patch correctly that the ivec implementation is primarily C? I’m not familiar with too many FFI projects in Rust, but I might have hoped we could write parts in Rust to gain any benefits from that, too. Is that a fool’s errand I’m thinking of?\n> >\n> > The ivec type is defined and implemented in C (interop/ivec.[ch]) and\n> > Rust (rust/interop/src/ivec.rs). When I started writing the ivec type\n> > I didn't know if the Git community would accept a hard dependency on\n> > Rust, so I made ivec usable in C without needing Rust.\n>\n> Right—I saw both implementations, but it looked like C did most of the\n> work, which was my main question. Re-reading, it looks like Rust does\n> more work than I thought (with implementations of insert/push/etc.)\n>\n> That said, I think it's sensible to leave the type useable from just C\n> unless/until Rust becomes required (and then we can move things over).\n\nI like your idea of implementation consolidation. I just don't know\nwhat that would look like yet.\n\nIt's not straightforward because C doesn't have generics. I'll use\nIVec as an example, but this applies to any generic type in Rust. For\na function like push() in IVec it will have N definitions if there\nare N IVec types. e.g. If your code uses IVec, IVec,\nIVec that would mean that pub fn push(&mut self) {} would compile\nto 3 functions. If you don't use #[no_mangle] you'd have to figure out\nthe Rust compiler's exact behavior for function names when calling it\nfrom C, which isn't stable or easily predictable. If you do use\n#[no_mangle] then the Rust compiler can't generate a generic function\nfor each type.\n\nAnother problem is that the functions in ivec mostly deal with\nresizing the memory rather than controlling access to memory for the C\nside. Even if the C side used Rust defined functions, that wouldn't\nsolve memory access issues to the pointer on the C side. We could\nenforce access to each element by requiring C to call a Rust defined\nfunction for each element, but that sounds very painful and slow. ivec\nis meant to be used as a scaffolding type to help transition C to\nRust.\n\nOther projects that do use Rust's builtin Vec (or some other\ncollection type) often Box it and write wrapper functions. This means\nthat the C side sees an opaque void* instead of a transparent struct\nlike ivec with ptr, length, capacity, and element_size.\n\nI'm curious if the community has more design feedback, or suggestions\nfor an alternative to my ivec type.\n"},{"id":"525005","messageId":"aK4u1RktHxkQtlWR@fruit.crustytoothpaste.net","threadId":"63804","inReplyTo":"CAH=ZcbD_pX1YdZbt9b-xMmcu2806twhjECez4HhCyE9iBf-9=Q@mail.gmail.com","subject":"Re: [PATCH v3 06/15] ivec: create a vector type that is interoperable between C and Rust","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2025-08-26T22:01:57Z","receivedAt":"2025-08-26T22:02:05Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On 2025-08-26 at 18:47:06, Ezekiel Newren wrote:\n> I'm curious if the community has more design feedback, or suggestions\n> for an alternative to my ivec type.\n\nI think this is a fine approach. We used a similar Vec-like\nstructure when porting a service from C to Rust at $DAYJOB, but we've\nalso used the boxing approach to good success.\n\nUltimately, I think boxing is the right choice when we don't need access\nfrom C. For instance, if I were to convert the loose object mapping\ncode to Rust, I would store the data as a pair of Box> and\nthen attach them to struct repository as `void *`.\n\nBut it's less definitive when you need access from C. Because of the\nway we intimately mess with the internals of our data structures in this\ncodebase, the ivec is probably the right choice here, so I'd keep that.\n-- \nbrian m. carlson (they/them)\nToronto, Ontario, CA\n"},{"id":"525020","messageId":"aK5mJI1NfVQDmDXN@nand.local","threadId":"63804","inReplyTo":"CABPp-BHdHQFv74GDbe=pJBFBALAMZoGsJDhSGqPbT3Daadnd4A@mail.gmail.com","subject":"Re: [PATCH v3 02/15] xdiff: introduce rust","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2025-08-27T01:57:56Z","receivedAt":"2025-08-27T01:57:58Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"On Sat, Aug 23, 2025 at 11:30:26AM -0700, Elijah Newren wrote:\n> > The assertion in the policy that Rust is easily interoperable is incorrect.\n>\n> Are you mixing up interoperability with portability? Without further\n> context than your email provides, it appears so to me. Rust code can\n> call C code and vice-versa within the same process without huge\n> amounts of serializing and deserializing of data structures, and\n> without what amounts to something close to an operating system context\n> switch in order to ensure call stacks are as expected for the language\n> in question. To me, that means we can call the two languages easily\n> interoperable. On the other hand, portability of those languages is\n> about whether those languages have compilers supported on various\n> hardware platforms. The document explicitly calls out that fewer\n> systems have a Rust compiler than have a C compiler, and that Rust\n> adoption would thus reduce how portable Git is. Are you referring to\n> this lower portability that the document itself also calls out, or are\n> you pointing out additional issues with interoperation between the\n> languages on a platform where compilers for both languages exist? If\n> the latter, could you provide more details?\n\nI think that this is the main point from my point of view. Yes, we are\nstrictly worsening the project's portability by adding Rust as a\nnon-optional build component. But it is *not* the case that two Git\nclients (one hypothetical one built with Rust components, one existing\none without) can't work on the same Git repository, even including one\non the same machine.\n\nForgetting Rust for a moment, I don't think it is a realistic goal to\nhave support for all platforms that could possibly want to run Git. I\nwould imagine that there are platforms today that cannot run the latest\nand greatest version of Git for just that reason. My hope is that\nwhatever version(s) *are* compatible with those platforms are good\nenough to support the workflows that those users need.\n\nSo my personal feeling is that that (not having a 100% portable version\nof Git across all possible platforms) is OK. But of course that does\nraise the concern that security fixes will be more difficult to backport\nacross a hypothetical version boundary where Rust is introduced.\n\nTo that end, I would note a couple of things:\n\n - This assumes that the Rust code has the same security vulnerabilities\n as the C code that it replaces. I don't think that is a given\n whatsoever, and I would bet that emperically there are fewer such\n vulnerabilities on the Rust side than on the C one (in fact, that is\n one of the reasons that we are considering Rust in the first place;\n brian m. carlson explains this point quite well IMHO).\n\n - If there *is* a security vulnerability in the Rust code that also\n presents a vulnerability on the corresponding C side, I would hope\n that the project's track record of generously backporting security\n fixes would suggest that we would do so in this case as well, despite\n crossing a language boundary.\n\n On the other side of that coin, if there is a security vulnerability\n in an older version of Git that isn't present in a newer one\n (regardless of whether or not Rust is involved), I would imagine that\n that we would write security patches against an even earlier maint-\n branch and forward-port them up to the most recent vulnerable\n version.\n\nSo my impression is that the main contention here is a concern that\nworsening the portability will make it harder to push out security fixes\nin either direction. But I don't think that's necessarily the case. Even\nif it is, I would again hope that the track record of the folks on the\ngit-security list would suggest that we'd do the right thing and not\nabandon users on older platforms the moment Rust is introduced into the\ncodebase.\n\nThanks,\nTaylor\n"},{"id":"525038","messageId":"01f101dc1760$5eef42b0$1ccdc810$@nexbridge.com","threadId":"63804","inReplyTo":"aK5mJI1NfVQDmDXN@nand.local","subject":"RE: [PATCH v3 02/15] xdiff: introduce rust","fromName":"","fromEmail":"rsbecker@nexbridge.com","sentAt":"2025-08-27T14:39:10Z","receivedAt":"2025-08-27T14:39:59Z","isPatch":true,"sender":{"key":"randall.becker@nexbridge.ca","avatar":"https://avatars.githubusercontent.com/u/28956764?v=4"},"body":"On August 26, 2025 9:58 PM, Taylor Blau wrote:\n>On Sat, Aug 23, 2025 at 11:30:26AM -0700, Elijah Newren wrote:\n>> > The assertion in the policy that Rust is easily interoperable is incorrect.\n>>\n>> Are you mixing up interoperability with portability? Without further\n>> context than your email provides, it appears so to me. Rust code can\n>> call C code and vice-versa within the same process without huge\n>> amounts of serializing and deserializing of data structures, and\n>> without what amounts to something close to an operating system context\n>> switch in order to ensure call stacks are as expected for the language\n>> in question. To me, that means we can call the two languages easily\n>> interoperable. On the other hand, portability of those languages is\n>> about whether those languages have compilers supported on various\n>> hardware platforms. The document explicitly calls out that fewer\n>> systems have a Rust compiler than have a C compiler, and that Rust\n>> adoption would thus reduce how portable Git is. Are you referring to\n>> this lower portability that the document itself also calls out, or are\n>> you pointing out additional issues with interoperation between the\n>> languages on a platform where compilers for both languages exist? If\n>> the latter, could you provide more details?\n>\n>I think that this is the main point from my point of view. Yes, we are strictly\n>worsening the project's portability by adding Rust as a non-optional build\n>component. But it is *not* the case that two Git clients (one hypothetical one built\n>with Rust components, one existing one without) can't work on the same Git\n>repository, even including one on the same machine.\n>\n>Forgetting Rust for a moment, I don't think it is a realistic goal to have support for all\n>platforms that could possibly want to run Git. I would imagine that there are\n>platforms today that cannot run the latest and greatest version of Git for just that\n>reason. My hope is that whatever version(s) *are* compatible with those platforms\n>are good enough to support the workflows that those users need.\n>\n>So my personal feeling is that that (not having a 100% portable version of Git across\n>all possible platforms) is OK. But of course that does raise the concern that security\n>fixes will be more difficult to backport across a hypothetical version boundary\n>where Rust is introduced.\n>\n>To that end, I would note a couple of things:\n>\n> - This assumes that the Rust code has the same security vulnerabilities\n> as the C code that it replaces. I don't think that is a given\n> whatsoever, and I would bet that emperically there are fewer such\n> vulnerabilities on the Rust side than on the C one (in fact, that is\n> one of the reasons that we are considering Rust in the first place;\n> brian m. carlson explains this point quite well IMHO).\n>\n> - If there *is* a security vulnerability in the Rust code that also\n> presents a vulnerability on the corresponding C side, I would hope\n> that the project's track record of generously backporting security\n> fixes would suggest that we would do so in this case as well, despite\n> crossing a language boundary.\n>\n> On the other side of that coin, if there is a security vulnerability\n> in an older version of Git that isn't present in a newer one\n> (regardless of whether or not Rust is involved), I would imagine that\n> that we would write security patches against an even earlier maint-\n> branch and forward-port them up to the most recent vulnerable\n> version.\n>\n>So my impression is that the main contention here is a concern that worsening the\n>portability will make it harder to push out security fixes in either direction. But I\n>don't think that's necessarily the case. Even if it is, I would again hope that the track\n>record of the folks on the git-security list would suggest that we'd do the right thing\n>and not abandon users on older platforms the moment Rust is introduced into the\n>codebase.\n\nThis is indeed my concern and hope, Taylor, as the maintainer for a platform that is\nfeeling abandoned. Please note that HPE NonStop is an actively maintained and\nvendor supported commercial platform based on x86_64 POSIX, just not a\nLinux/Windows machine.\nThank you.\n\n"},{"id":"525062","messageId":"xmqqsehc1ypi.fsf@gitster.g","threadId":"63804","inReplyTo":"01f101dc1760$5eef42b0$1ccdc810$@nexbridge.com","subject":"Re: [PATCH v3 02/15] xdiff: introduce rust","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-08-27T17:06:17Z","receivedAt":"2025-08-27T17:06:21Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":" writes:\n\n>>So my impression is that the main contention here is a concern that worsening the\n>>portability will make it harder to push out security fixes in either direction. But I\n>>don't think that's necessarily the case. Even if it is, I would again hope that the track\n>>record of the folks on the git-security list would suggest that we'd do the right thing\n>>and not abandon users on older platforms the moment Rust is introduced into the\n>>codebase.\n>\n> This is indeed my concern and hope, Taylor, as the maintainer for a platform that is\n> feeling abandoned. Please note that HPE NonStop is an actively maintained and\n> vendor supported commercial platform based on x86_64 POSIX, just not a\n> Linux/Windows machine.\n\nThanks for a friendly conversation, but I would have to say that\nTaylor's \"we know we end up having to support both, and we will do\nso\" is way underestimates the cost to do so. And I hope that an\nactively maintained and vendor supported commercial platform would\nbear the burden of the major part of that cost themselves, when it\nbecomes necessary to do such a dual support.\n\nThanks.\n"},{"id":"525064","messageId":"023c01dc1776$440a55a0$cc1f00e0$@nexbridge.com","threadId":"63804","inReplyTo":"xmqqsehc1ypi.fsf@gitster.g","subject":"RE: [PATCH v3 02/15] xdiff: introduce rust","fromName":"","fromEmail":"rsbecker@nexbridge.com","sentAt":"2025-08-27T17:15:56Z","receivedAt":"2025-08-27T17:16:35Z","isPatch":true,"sender":{"key":"randall.becker@nexbridge.ca","avatar":"https://avatars.githubusercontent.com/u/28956764?v=4"},"body":"On August 27, 2025 1:06 PM, Junio C Hamano wrote:\n> writes:\n>\n>>>So my impression is that the main contention here is a concern that\n>>>worsening the portability will make it harder to push out security\n>>>fixes in either direction. But I don't think that's necessarily the\n>>>case. Even if it is, I would again hope that the track record of the\n>>>folks on the git-security list would suggest that we'd do the right\n>>>thing and not abandon users on older platforms the moment Rust is introduced\n>into the codebase.\n>>\n>> This is indeed my concern and hope, Taylor, as the maintainer for a\n>> platform that is feeling abandoned. Please note that HPE NonStop is an\n>> actively maintained and vendor supported commercial platform based on\n>> x86_64 POSIX, just not a Linux/Windows machine.\n>\n>Thanks for a friendly conversation, but I would have to say that Taylor's \"we know\n>we end up having to support both, and we will do so\" is way underestimates the\n>cost to do so. And I hope that an actively maintained and vendor supported\n>commercial platform would bear the burden of the major part of that cost\n>themselves, when it becomes necessary to do such a dual support.\n\nIf the platform provider had taken on git, that might be possible, but it is not\nthe case. This will come down to my small team to try to cope with this\nsituation - generally without any help from anyone else. I can do much,\nbut this will likely come down to my doing all of the work after hours\nand on weekends with no other support, as has happened for the past\ndecade.\n\n"},{"id":"525071","messageId":"aK9mx2XemppIaKVI@nand.local","threadId":"63804","inReplyTo":"xmqqsehc1ypi.fsf@gitster.g","subject":"Re: [PATCH v3 02/15] xdiff: introduce rust","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2025-08-27T20:12:55Z","receivedAt":"2025-08-27T20:13:03Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"On Wed, Aug 27, 2025 at 10:06:17AM -0700, Junio C Hamano wrote:\n> writes:\n>\n> >>So my impression is that the main contention here is a concern that worsening the\n> >>portability will make it harder to push out security fixes in either direction. But I\n> >>don't think that's necessarily the case. Even if it is, I would again hope that the track\n> >>record of the folks on the git-security list would suggest that we'd do the right thing\n> >>and not abandon users on older platforms the moment Rust is introduced into the\n> >>codebase.\n> >\n> > This is indeed my concern and hope, Taylor, as the maintainer for a platform that is\n> > feeling abandoned. Please note that HPE NonStop is an actively maintained and\n> > vendor supported commercial platform based on x86_64 POSIX, just not a\n> > Linux/Windows machine.\n>\n> Thanks for a friendly conversation, but I would have to say that\n> Taylor's \"we know we end up having to support both, and we will do\n> so\" is way underestimates the cost to do so.\n\nI don't mean to imply that doing so would not be costly or require\nadditional effort. I was trying to highlight that I believe we on\nthe git-security list have demonstrated a track record of supporting\nquite old release tracks when new security releases are cut.\n\nI don't mean to suggest whatsoever that adding Rust into the mix would\nsomehow not have an effect on the costliness of maintaining support for\nolder versions, just that I believe we have show ourselves to be up to\nthe challenge.\n\n(As an aside, I mentioned in my earlier email to Randall that I have a\nsuspicion that Rust code will have fewer security issues than C code,\nand so the likelihood of needing to backport a security fix from Rust to\nC seems lower to me than having to simply patch old C code. Time will\ntell, I guess.)\n\nThanks,\nTaylor\n"},{"id":"525072","messageId":"xmqqh5xszf91.fsf@gitster.g","threadId":"63804","inReplyTo":"aK9mx2XemppIaKVI@nand.local","subject":"Re: [PATCH v3 02/15] xdiff: introduce rust","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-08-27T20:22:34Z","receivedAt":"2025-08-27T20:22:38Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Taylor Blau writes:\n\n> (As an aside, I mentioned in my earlier email to Randall that I have a\n> suspicion that Rust code will have fewer security issues than C code,\n> and so the likelihood of needing to backport a security fix from Rust to\n> C seems lower to me than having to simply patch old C code. Time will\n> tell, I guess.)\n\nJust like back when scripted Porcelains were rewritten in C, in 5\nyears, when a lot of the existing C code is rewritten, who among us\nwould care to backport or \"simply patch\" old C code?\n\nThis of course assumes that these platforms that lack Rust still\nlack Rust after 5 years, yet still matters to the users, and the\nvendor does not care to support Git themselves. Maybe one of these\nthree conditions would change and make the problem go away ;-)\n\n\n"},{"id":"525332","messageId":"aLbSA5KsBdD4wW_B@pks.im","threadId":"63804","inReplyTo":"xmqqh5xszf91.fsf@gitster.g","subject":"Re: [PATCH v3 02/15] xdiff: introduce rust","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-09-02T11:16:19Z","receivedAt":"2025-09-02T11:16:31Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Wed, Aug 27, 2025 at 01:22:34PM -0700, Junio C Hamano wrote:\n> Taylor Blau writes:\n> \n> > (As an aside, I mentioned in my earlier email to Randall that I have a\n> > suspicion that Rust code will have fewer security issues than C code,\n> > and so the likelihood of needing to backport a security fix from Rust to\n> > C seems lower to me than having to simply patch old C code. Time will\n> > tell, I guess.)\n> \n> Just like back when scripted Porcelains were rewritten in C, in 5\n> years, when a lot of the existing C code is rewritten, who among us\n> would care to backport or \"simply patch\" old C code?\n> \n> This of course assumes that these platforms that lack Rust still\n> lack Rust after 5 years, yet still matters to the users, and the\n> vendor does not care to support Git themselves. Maybe one of these\n> three conditions would change and make the problem go away ;-)\n\nIt will definitely require additional maintenance by us. I think it's\nreasonable to say that platforms without Rust won't get new features\nanymore. But when it comes to security fixes or significant bugs I think\nit's less sensible to say that they're left on their own.\n\nI proposed this in a separate branch of these threads, but we could\ncounteract this by declaring the last major version before we introduce\nRust as an LTS version that will receive both security and severe bug\nfixes going forward. Ideally, that LTS release would continue to be\nmaintained until the gcc-rs backend is ready for prime time, which\nshould alleviate a lot of the portability concerns.\n\nAs Pierre-Emmanuel menitoned in [1], the backend is likely to stabilize\nnext year. One or two years of backports for that particular LTS version\ndoesn't feel too bad. And if it does become more involved we can maybe\nalso distribute the load and rely on maintainers of impacted platforms\nwithout Rust to help out with the backporting.\n\nAlso, all of this feels like a significant shift. I'm strongly in favor\nof adopting Rust in our codebase, but I think we should do so carefully.\nSo we might take it extra carefully and say that Rust will become a\nmandatory dependency in Git 3.0, where the last release before Git 3.0\nwill become an LTS release.\n\nPatrick\n\n[1]: <7bf054a1-0196-4ad8-aaa4-a432cd2c93a5@embecosm.com>\n"},{"id":"525334","messageId":"87qzwpm6r1.fsf@gentoo.org","threadId":"63804","inReplyTo":"aLbSA5KsBdD4wW_B@pks.im","subject":"Re: [PATCH v3 02/15] xdiff: introduce rust","fromName":"Sam James","fromEmail":"sam@gentoo.org","sentAt":"2025-09-02T11:30:26Z","receivedAt":"2025-09-02T11:30:33Z","isPatch":true,"sender":{"key":"sam@gentoo.org","avatar":"https://avatars.githubusercontent.com/u/11667869?v=4"},"body":"Patrick Steinhardt writes:\n\n> On Wed, Aug 27, 2025 at 01:22:34PM -0700, Junio C Hamano wrote:\n>> Taylor Blau writes:\n>> \n>> > (As an aside, I mentioned in my earlier email to Randall that I have a\n>> > suspicion that Rust code will have fewer security issues than C code,\n>> > and so the likelihood of needing to backport a security fix from Rust to\n>> > C seems lower to me than having to simply patch old C code. Time will\n>> > tell, I guess.)\n>> \n>> Just like back when scripted Porcelains were rewritten in C, in 5\n>> years, when a lot of the existing C code is rewritten, who among us\n>> would care to backport or \"simply patch\" old C code?\n>> \n>> This of course assumes that these platforms that lack Rust still\n>> lack Rust after 5 years, yet still matters to the users, and the\n>> vendor does not care to support Git themselves. Maybe one of these\n>> three conditions would change and make the problem go away ;-)\n>\n> It will definitely require additional maintenance by us. I think it's\n> reasonable to say that platforms without Rust won't get new features\n> anymore. But when it comes to security fixes or significant bugs I think\n> it's less sensible to say that they're left on their own.\n>\n> I proposed this in a separate branch of these threads, but we could\n> counteract this by declaring the last major version before we introduce\n> Rust as an LTS version that will receive both security and severe bug\n> fixes going forward. Ideally, that LTS release would continue to be\n> maintained until the gcc-rs backend is ready for prime time, which\n> should alleviate a lot of the portability concerns.\n>\n\nThat would be enormously appreciated and make me happy if it is possible.\n\n> As Pierre-Emmanuel menitoned in [1], the backend is likely to stabilize\n> next year. One or two years of backports for that particular LTS version\n> doesn't feel too bad. And if it does become more involved we can maybe\n> also distribute the load and rely on maintainers of impacted platforms\n> without Rust to help out with the backporting.\n\nI'd be open to that if we're still in the position of needing it by then.\n\n>\n> Also, all of this feels like a significant shift. I'm strongly in favor\n> of adopting Rust in our codebase, but I think we should do so carefully.\n> So we might take it extra carefully and say that Rust will become a\n> mandatory dependency in Git 3.0, where the last release before Git 3.0\n> will become an LTS release.\n\nThis is what I was hoping for :)\n\nIt should be considered a \"breaking change\" in a sense (the\n\"compatibility profile\" of git changed) and 3.0 would be fitting.\n\nIt would perhaps liberate git developers in feeling free to make other\nchanges while adopting Rust as well if they see fit.\n\n>\n> Patrick\n>\n> [1]: <7bf054a1-0196-4ad8-aaa4-a432cd2c93a5@embecosm.com>\n\nsam\n"},{"id":"525359","messageId":"aLco7uHFZaHnfxBa@fruit.crustytoothpaste.net","threadId":"63804","inReplyTo":"aLbSA5KsBdD4wW_B@pks.im","subject":"Re: [PATCH v3 02/15] xdiff: introduce rust","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2025-09-02T17:27:10Z","receivedAt":"2025-09-02T17:27:12Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On 2025-09-02 at 11:16:19, Patrick Steinhardt wrote:\n> As Pierre-Emmanuel menitoned in [1], the backend is likely to stabilize\n> next year. One or two years of backports for that particular LTS version\n> doesn't feel too bad. And if it does become more involved we can maybe\n> also distribute the load and rely on maintainers of impacted platforms\n> without Rust to help out with the backporting.\n\nI'm very much in favour of supporting gccrs when it's available, but I\nalso want to say that it currently is targeting 1.49, which is much\nolder than we want. It's also not necessarily going to be fully usable\nor bug free in that amount of time.\n\nI also want to point out that it's important that the maintainers of\naffected platforms build the tooling necessary for their platforms to be\nsupported. I'm not seeing ports of LLVM to those architectures or\ncontributions to gccrs that would make those platforms easier to\nsupport.\n\n> Also, all of this feels like a significant shift. I'm strongly in favor\n> of adopting Rust in our codebase, but I think we should do so carefully.\n> So we might take it extra carefully and say that Rust will become a\n> mandatory dependency in Git 3.0, where the last release before Git 3.0\n> will become an LTS release.\n\nI'd prefer we not wait that long. I'm doing some work in building the\nnew loose object mapping using Rust and it's much more efficient than\nwriting it in C because we don't have to sort the data when we use a\nBTreeMap. The code is much simpler, shorter, and easier to write.\n\nNobody else is currently working on the interoperability code and we\nexpressed that we ideally wanted it for Git 3.0. Being able to use Rust\nmeans I can write that code faster, with fewer errors (and hence less\ndebugging time), and better tests. Otherwise, I'm afraid that it will\ntake longer and we might not have it fully upstream for Git 3.0.\n\nWe also have this series right now, which we'd have to abandon if we're\nnot going to support Rust right away. I'd like to retain Ezekiel as a\ncontributor and incorporate Rust, and I think the best time to adopt\nRust is now, not at Git 3.0.\n-- \nbrian m. carlson (they/them)\nToronto, Ontario, CA\n"},{"id":"525367","messageId":"87plc8lmjf.fsf@gentoo.org","threadId":"63804","inReplyTo":"aLco7uHFZaHnfxBa@fruit.crustytoothpaste.net","subject":"Re: [PATCH v3 02/15] xdiff: introduce rust","fromName":"Sam James","fromEmail":"sam@gentoo.org","sentAt":"2025-09-02T18:47:00Z","receivedAt":"2025-09-02T18:47:06Z","isPatch":true,"sender":{"key":"sam@gentoo.org","avatar":"https://avatars.githubusercontent.com/u/11667869?v=4"},"body":"\"brian m. carlson\" writes:\n\n> On 2025-09-02 at 11:16:19, Patrick Steinhardt wrote:\n>> As Pierre-Emmanuel menitoned in [1], the backend is likely to stabilize\n>> next year. One or two years of backports for that particular LTS version\n>> doesn't feel too bad. And if it does become more involved we can maybe\n>> also distribute the load and rely on maintainers of impacted platforms\n>> without Rust to help out with the backporting.\n>\n> I'm very much in favour of supporting gccrs when it's available, but I\n> also want to say that it currently is targeting 1.49, which is much\n> older than we want. It's also not necessarily going to be fully usable\n> or bug free in that amount of time.\n>\n> I also want to point out that it's important that the maintainers of\n> affected platforms build the tooling necessary for their platforms to be\n> supported. I'm not seeing ports of LLVM to those architectures or\n> contributions to gccrs that would make those platforms easier to\n> support.\n\nThis isn't accurate. gccrs doesn't need particular porting to arches: at\nleast not yet, and if it does, it'll be very minor; any changes of this\nsort will be in crates themselves which would go upstream.\n\nAs for the libgccjit-based backend for rustc, see\nhttps://github.com/rust-lang/rustc_codegen_gcc/issues/49,\nhttps://github.com/rust-lang/rustc_codegen_gcc/issues/744, and\nhttps://github.com/rust-lang/rustc_codegen_gcc/issues/742 for discussion\nand complications. But to say that nobody is doing it or working towards\nit is inaccurate.\n\n>\n>> Also, all of this feels like a significant shift. I'm strongly in favor\n>> of adopting Rust in our codebase, but I think we should do so carefully.\n>> So we might take it extra carefully and say that Rust will become a\n>> mandatory dependency in Git 3.0, where the last release before Git 3.0\n>> will become an LTS release.\n>\n> I'd prefer we not wait that long. I'm doing some work in building the\n> new loose object mapping using Rust and it's much more efficient than\n> writing it in C because we don't have to sort the data when we use a\n> BTreeMap. The code is much simpler, shorter, and easier to write.\n>\n\nI still think adopting Rust is a compatibility break and a \"breaking\nchange\". Again, keeping in mind that for adopting C99 features (!), the\nGit project used \"test balloons\" very very recently.\n\n> Nobody else is currently working on the interoperability code and we\n> expressed that we ideally wanted it for Git 3.0. Being able to use Rust\n> means I can write that code faster, with fewer errors (and hence less\n> debugging time), and better tests. Otherwise, I'm afraid that it will\n> take longer and we might not have it fully upstream for Git 3.0.\n>\n> We also have this series right now, which we'd have to abandon if we're\n> not going to support Rust right away. I'd like to retain Ezekiel as a\n> contributor and incorporate Rust, and I think the best time to adopt\n> Rust is now, not at Git 3.0.\n\nI think there's going to be various issues that arise even on platforms\nthat support Rust that would make it fitting for Git 3.0, at least for\nthe first few releases that incorporate Rust. I'll note that the series\nisn't currently using Meson's Rust integration as QEMU is doing.\n\nsam\n"},{"id":"525395","messageId":"aLfU5sEa-RE3X4G2@pks.im","threadId":"63804","inReplyTo":"aLco7uHFZaHnfxBa@fruit.crustytoothpaste.net","subject":"Re: [PATCH v3 02/15] xdiff: introduce rust","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-09-03T05:40:54Z","receivedAt":"2025-09-03T05:41:12Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Tue, Sep 02, 2025 at 05:27:10PM +0000, brian m. carlson wrote:\n> On 2025-09-02 at 11:16:19, Patrick Steinhardt wrote:\n> > As Pierre-Emmanuel menitoned in [1], the backend is likely to stabilize\n> > next year. One or two years of backports for that particular LTS version\n> > doesn't feel too bad. And if it does become more involved we can maybe\n> > also distribute the load and rely on maintainers of impacted platforms\n> > without Rust to help out with the backporting.\n> \n> I'm very much in favour of supporting gccrs when it's available, but I\n> also want to say that it currently is targeting 1.49, which is much\n> older than we want. It's also not necessarily going to be fully usable\n> or bug free in that amount of time.\n\nI cannot really say much about this. Overall I think that the rapid\nrelease cycles and rapid adoption by projects that one typically sees in\nRust are an indicator to me that the whole ecosystem is not yet stable.\n\nIf I had the choice, I'd much rather adopt an ancient version of Rust if\nit means that more platforms can support it.\n\n> I also want to point out that it's important that the maintainers of\n> affected platforms build the tooling necessary for their platforms to be\n> supported. I'm not seeing ports of LLVM to those architectures or\n> contributions to gccrs that would make those platforms easier to\n> support.\n\nThe gccrs maintainers are actively working on that backend, and as far\nas I understand the main difference between LLVM and gccrs is that the\nlatter doesn't have to be ported over to every single platform\nindividually.\n\n> > Also, all of this feels like a significant shift. I'm strongly in favor\n> > of adopting Rust in our codebase, but I think we should do so carefully.\n> > So we might take it extra carefully and say that Rust will become a\n> > mandatory dependency in Git 3.0, where the last release before Git 3.0\n> > will become an LTS release.\n> \n> I'd prefer we not wait that long. I'm doing some work in building the\n> new loose object mapping using Rust and it's much more efficient than\n> writing it in C because we don't have to sort the data when we use a\n> BTreeMap. The code is much simpler, shorter, and easier to write.\n\nI still think we need to be mindful around the community though. I\nunderstand that we want to have Rust in the codebase, and as I said I'm\nin favor of adopting it. But we also have a certain responsibility with\nGit given that it's used by almost every single developer out there.\n\nA compromise could be to ease into Rust: we adopt Rust, but before Git\n3.0 it is entirely optional. So Git will continue to work alright even\nif there is no Rust compiler available. On the one hand this plays\nnicely with platforms that do not have Rust. On the other hand it also\nallows us to slowly iterate on the build infra for Rust, because I'm\nvery sure that there's going to be issues there initially.\n\nWith that we can:\n\n - Build confidence in our Rust tooling.\n\n - Figure out things as we go.\n\n - Give distributions and other platforms enough time to prepare for\n Rust becoming mandatory.\n\nI think adopting Rust as a mandatory dependency out of nowhere would not\nbe playing nice. It may require significant effort from distros to adapt\nto the new reality, so we should give them time to do so.\n\nNote that I'm not saying that we need to have both a C and Rust\nimplementation for everything written in Rust. I don't think that's\nsustainable in any way. But any feature written in Rust should be a\n_new_ feature that can be disabled and that users can live without for\nthe time being.\n\n> Nobody else is currently working on the interoperability code and we\n> expressed that we ideally wanted it for Git 3.0. Being able to use Rust\n> means I can write that code faster, with fewer errors (and hence less\n> debugging time), and better tests. Otherwise, I'm afraid that it will\n> take longer and we might not have it fully upstream for Git 3.0.\n> \n> We also have this series right now, which we'd have to abandon if we're\n> not going to support Rust right away. I'd like to retain Ezekiel as a\n> contributor and incorporate Rust, and I think the best time to adopt\n> Rust is now, not at Git 3.0.\n\nIt would be a shame, but right now it's a risky bet to build anything on\ntop of Rust given that we don't officially accept it in Git yet. We need\nto first make the decision whether or not we want to have it right now,\nand if so how that's supposed to look like.\n\nPatrick\n"},{"id":"525439","messageId":"febfce4f-a609-4802-be64-e72d95837c10@ramsayjones.plus.com","threadId":"63804","inReplyTo":"aLfU5sEa-RE3X4G2@pks.im","subject":"Re: [PATCH v3 02/15] xdiff: introduce rust","fromName":"Ramsay Jones","fromEmail":"ramsay@ramsayjones.plus.com","sentAt":"2025-09-03T16:22:09Z","receivedAt":"2025-09-03T16:36:45Z","isPatch":true,"sender":{"key":"ramsay@ramsayjones.plus.com","avatar":"https://avatars.githubusercontent.com/u/33702710?v=4"},"body":"\n\nOn 03/09/2025 06:40, Patrick Steinhardt wrote:\n> On Tue, Sep 02, 2025 at 05:27:10PM +0000, brian m. carlson wrote:\n>> On 2025-09-02 at 11:16:19, Patrick Steinhardt wrote:\n[snip]\n\n>> I'd prefer we not wait that long. I'm doing some work in building the\n>> new loose object mapping using Rust and it's much more efficient than\n>> writing it in C because we don't have to sort the data when we use a\n>> BTreeMap. The code is much simpler, shorter, and easier to write.\n> \n> I still think we need to be mindful around the community though. I\n> understand that we want to have Rust in the codebase, and as I said I'm\n> in favor of adopting it. But we also have a certain responsibility with\n> Git given that it's used by almost every single developer out there.\n> \n> A compromise could be to ease into Rust: we adopt Rust, but before Git\n> 3.0 it is entirely optional. So Git will continue to work alright even\n> if there is no Rust compiler available. On the one hand this plays\n> nicely with platforms that do not have Rust. On the other hand it also\n> allows us to slowly iterate on the build infra for Rust, because I'm\n> very sure that there's going to be issues there initially.\n> \n> With that we can:\n> \n> - Build confidence in our Rust tooling.\n> \n> - Figure out things as we go.\n> \n> - Give distributions and other platforms enough time to prepare for\n> Rust becoming mandatory.\n> \n> I think adopting Rust as a mandatory dependency out of nowhere would not\n> be playing nice. It may require significant effort from distros to adapt\n> to the new reality, so we should give them time to do so.\n\nI agree with everything you say above. Thank you for saying it. ;)\n\nIt is somewhat unfortunate that 'xdiff' was chosen as an initial\nproject for this, since a git without the ability to produce a diff\nis, well, practically useless!\n\n[I don't have any objection to making the rust code optional in this\ncase - it should be easily possible to have both C and rust xdiff code].\n\nI have already stopped building git on cygwin, since rust is not\ncurrently available on cygwin (and I'm not aware of any effort to\nport it there - although LLVM v20+ was recently made available).\n\nATB,\nRamsay Jones\n\n\n\n"},{"id":"525454","messageId":"87bjnr5rcm.fsf@gmail.com","threadId":"63804","inReplyTo":"87plc8lmjf.fsf@gentoo.org","subject":"Re: [PATCH v3 02/15] xdiff: introduce rust","fromName":"Collin Funk","fromEmail":"collin.funk1@gmail.com","sentAt":"2025-09-03T18:22:01Z","receivedAt":"2025-09-03T18:22:05Z","isPatch":true,"sender":{"key":"collin.funk1@gmail.com","avatar":"https://avatars.githubusercontent.com/u/65689063?v=4"},"body":"Sam James writes:\n\n> I still think adopting Rust is a compatibility break and a \"breaking\n> change\". Again, keeping in mind that for adopting C99 features (!), the\n> Git project used \"test balloons\" very very recently.\n>\n>> Nobody else is currently working on the interoperability code and we\n>> expressed that we ideally wanted it for Git 3.0. Being able to use Rust\n>> means I can write that code faster, with fewer errors (and hence less\n>> debugging time), and better tests. Otherwise, I'm afraid that it will\n>> take longer and we might not have it fully upstream for Git 3.0.\n>>\n>> We also have this series right now, which we'd have to abandon if we're\n>> not going to support Rust right away. I'd like to retain Ezekiel as a\n>> contributor and incorporate Rust, and I think the best time to adopt\n>> Rust is now, not at Git 3.0.\n>\n> I think there's going to be various issues that arise even on platforms\n> that support Rust that would make it fitting for Git 3.0, at least for\n> the first few releases that incorporate Rust. I'll note that the series\n> isn't currently using Meson's Rust integration as QEMU is doing.\n\nJust want to voice my agreement with Sam here.\n\nIt seems strange that we have a test balloon for compound literals,\nsomething that GCC has supported since before 2001 [1]. But at the same\ntime require a platform to support Rust. If a platform has Rust support,\nit certainly has a compiler supporting compound literals.\n\nCollin\n\n[1] https://github.com/gcc-mirror/gcc/commit/cedd825f0f18088f7235f02136021bd63a2e12df\n"},{"id":"525470","messageId":"xmqqms7bchln.fsf@gitster.g","threadId":"63804","inReplyTo":"aLfU5sEa-RE3X4G2@pks.im","subject":"Re: [PATCH v3 02/15] xdiff: introduce rust","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-09-03T22:10:44Z","receivedAt":"2025-09-03T22:10:48Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Patrick Steinhardt writes:\n\n> I still think we need to be mindful around the community though. I\n> understand that we want to have Rust in the codebase, and as I said I'm\n> in favor of adopting it. But we also have a certain responsibility with\n> Git given that it's used by almost every single developer out there.\n>\n> A compromise could be to ease into Rust: we adopt Rust, but before Git\n> 3.0 it is entirely optional. So Git will continue to work alright even\n> if there is no Rust compiler available. On the one hand this plays\n> nicely with platforms that do not have Rust. On the other hand it also\n> allows us to slowly iterate on the build infra for Rust, because I'm\n> very sure that there's going to be issues there initially.\n>\n> With that we can:\n>\n> - Build confidence in our Rust tooling.\n>\n> - Figure out things as we go.\n>\n> - Give distributions and other platforms enough time to prepare for\n> Rust becoming mandatory.\n>\n> I think adopting Rust as a mandatory dependency out of nowhere would not\n> be playing nice. It may require significant effort from distros to adapt\n> to the new reality, so we should give them time to do so.\n\nNot just distros on exotic platforms, but also for users who cannot\nafford to see regressions. \n\n> Note that I'm not saying that we need to have both a C and Rust\n> implementation for everything written in Rust. I don't think that's\n> sustainable in any way. But any feature written in Rust should be a\n> _new_ feature that can be disabled and that users can live without for\n> the time being.\n\nYes, if we can find such modular niche, it would be ideal. But how\nmany areas that we can cleanly plug an optional thing in without\ndisrupting existing codebase are there? Offhand, all I'd think of\nare a new merge backend, a new rebase backend, a transport helper,\nor perhaps a new diff-algorithm?\n\n> It would be a shame, but right now it's a risky bet to build anything on\n> top of Rust given that we don't officially accept it in Git yet. We need\n> to first make the decision whether or not we want to have it right now,\n> and if so how that's supposed to look like.\n\nOne more thing that I noticed. What are our plans for the two\ndirectories in contrib/libgit-{sys,rs}/? IIRC, the new stuff from\nEzekiel did not interact with them at all, but it did not remove\nthem either, so I am a bit lost.\n\nThanks.\n\n"},{"id":"525472","messageId":"shv56ip5fs5ij653xyp2blun7e4in3gccjxl7k6qani5lwgich@ih5o7qu2fk4v","threadId":"63804","inReplyTo":"xmqqms7bchln.fsf@gitster.g","subject":"Re: [PATCH v3 02/15] xdiff: introduce rust","fromName":"Josh Steadmon","fromEmail":"steadmon@google.com","sentAt":"2025-09-03T22:48:51Z","receivedAt":"2025-09-03T22:49:00Z","isPatch":true,"sender":{"key":"steadmon@google.com","avatar":"https://avatars.githubusercontent.com/u/2654920?v=4"},"body":"On 2025.09.03 15:10, Junio C Hamano wrote:\n> One more thing that I noticed. What are our plans for the two\n> directories in contrib/libgit-{sys,rs}/? IIRC, the new stuff from\n> Ezekiel did not interact with them at all, but it did not remove\n> them either, so I am a bit lost.\n> \n> Thanks.\n\nI haven't followed this series closely, but I wouldn't expect it to\ninteract with contrib/libgit-*, since those libraries are intended use\nby external projects.\n\nThat said, we at Google don't currently have plans on expanding\nlibgit-*, since JJ has been able to meet its needs by shelling out to\nthe Git CLI for cases where gitoxide is not sufficient. It's possible\n(probable??) that we might return to libgit-rs in the future, but\nnothing is on the radar right now.\n\nWhen libgit-* was still under review, brian said[0] they were interested\nin building on it, but I don't know if that is still accurate. brian,\nany update on that?\n\n[0] https://lore.kernel.org/git/Z47kr0_fYYdaMWyA@tapette.crustytoothpaste.net/\n"},{"id":"525476","messageId":"aLjj9cG9_K6YLfeA@fruit.crustytoothpaste.net","threadId":"63804","inReplyTo":"aLfU5sEa-RE3X4G2@pks.im","subject":"Re: [PATCH v3 02/15] xdiff: introduce rust","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2025-09-04T00:57:25Z","receivedAt":"2025-09-04T00:57:34Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On 2025-09-03 at 05:40:54, Patrick Steinhardt wrote:\n> If I had the choice, I'd much rather adopt an ancient version of Rust if\n> it means that more platforms can support it.\n\nI think you may be assuming that gccrs targeting Rust 1.49 will\nmagically make it work on more platforms than upstream Rust will.\nThat's not the case.\n\ngccrs targeting Rust 1.49 will use libstd (the standard library) and\nlibcore (the library for freestanding implementations) from Rust 1.49\nand that means it will only support those platforms that Rust 1.49 did.\nFor instance, Rust added support for Apache NuttX relatively recently.\nEven if it has stellar support in GCC, it won't work with that version\nof gccrs because the underlying libraries don't support any of those\nplatforms. The only thing you can target that you couldn't before are\nsystems that use neither libstd nor libcore—which essentially means the\nLinux kernel. It's like using a glibc from 2009 and expecting to work\non RISC-V—it simply won't[0].\n\nIf you need support for new platforms, that requires a much _newer_\nversion of Rust. Thus, to be able to use gccrs, porters need to use the\nexisting gcc codegen backend and get that code in immediately so that\nwhen gccrs is out and supports Rust 1.91, the standard library will work\nwith those platforms. The fastest way to getting platforms supported is\nto port LLVM and then add them to upstream Rust that way.\n\nI know there has been much complaint about the six-week lifespan of Rust\nreleases. I myself dislike that. But the situation is that LTS\nreleases require extensive amounts of work and nobody has stepped up to\ndo that or pay for it to be done. Without dedicated staffing, it's not\ngoing to happen. That also means that individual projects decide what\nversions of Rust they do and don't want to support.\n\nWe're already supporting the version in Debian stable for a year after\nthe new release comes out, so we're already far behind what everyone\nelse is doing. For comparison, Rust 1.48 is in Debian 11, so we'd be\nsupporting an effectively five-year-old compiler instead of a\nthree-year-old compiler.\n\nRequiring Rust 1.49 instead of Rust 1.63 makes it harder to use tools\nlike bindgen and cbindgen, which exist to automatically create types and\nfunctions in one language in the other. That, in turn, will hinder our\nability to effectively write code that crosses the boundary and\nintroduce hard-to-find bugs, since we'll have to do that work manually.\nMy experience is that these kinds of bugs tend to actually show up more\nfrequently on less common platforms, like big-endian systems, so we'll\nbe worsening the platform experience for those systems.\n\nFor context, when we ported a core service from C to Rust at work, we\nused bindgen to generate C struct definitions, which made the process\nmuch easier and avoided random crashes. As a result, nobody noticed the\nfact that we ported it incrementally over a couple of years. If we\nhadn't used bindgen, we probably would have had lots of random segfaults\ndue to failing to maintain compatibility between Rust and C definitions\nof the same structures, which users would not have appreciated and would\nnot have helped our goal of making our software more reliable and easier\nto maintain.\n\n> The gccrs maintainers are actively working on that backend, and as far\n> as I understand the main difference between LLVM and gccrs is that the\n> latter doesn't have to be ported over to every single platform\n> individually.\n\nI don't think that's the case. gccrs has to be compiled for every\nplatform just like LLVM does. LLVM is actually easier to support\nbecause it can cross-compile from any platform to any platform without\nrecompilation. For instance, I can target riscv64gc-unknown-openbsd on\nmy Debian amd64 laptop assuming I can provide the necessary libraries\nfor OpenBSD when compiling, but GCC requires me to specifically compile\na compiler for that platform.\n\nIn any event, any portability changes will also likely need to go into\nlibstd and libcore, which is used identically with both compilers.\n\nIt is, however, the case that GCC supports more architectures (and\npossibly more architecture/OS combinations) than LLVM. For instance,\nDEC Alpha and IA64 are only supported by GCC at the moment.\n\n> I think adopting Rust as a mandatory dependency out of nowhere would not\n> be playing nice. It may require significant effort from distros to adapt\n> to the new reality, so we should give them time to do so.\n\nWe've actually had this discussion on the list several times where we've\nproposed the inclusion of Rust. This is not the first time it's come\nup, or the second. It was explicitly mentioned a year ago on the list\nthat we wanted to adopt Rust in the notes from the Contributor Summit.\n\nThere has been plenty of notice that this is coming down the line. It's\nnot accurate to claim it's \"out of nowhere\" nor to claim that people\nhave not had plenty of time to port their systems.\n\nDistros and porters should not be insensible to the increasing use of\nRust or the need for them to get their systems working. For instance,\nyou cannot run a GNOME or MATE desktop environment without librsvg2,\nwhich is written in Rust. Python's cryptography package adopted Rust\nover four years ago and there was the same gnashing of teeth[1], yet\nlittle progress has been made by porters on the same affected\narchitectures since that time. In that time, Debian has bootstrapped\nand released an entire RISC-V port, complete with Rust.\n\nI want to be clear I'm not opposed to supporting less common operating\nsystems or architectures. For many years, my laptop was a PowerPC Mac,\nand I've owned UltraSPARC, MIPS, and ARM hardware. For personal code, I\ntry to test it in CI on at least Linux, macOS, FreeBSD, and NetBSD. But\nalso, when a Debian package has not worked properly on PowerPC or\nUltraSPARC, I've stepped up and fixed it. My requests to other projects\nwhen porting have been things like asking to write valid C or C++ (by\nnot making unaligned accesses or avoiding endianness assumptions, for\ninstance) and not to refrain from adding new languages or features.\n\nIt should be stated that there is a very easy way to get Rust working,\nand that's to port LLVM to the platform in question. IA-64 was removed\nin 2009, but it might be possible to resurrect that out of tree if\nthere's interest and maybe even get it re-accepted upstream. I'll point\nout that AIX, Solaris, and QNX have done the necessary porting work to\nget LLVM and Rust working over the past couple years, so it's not out of\nthe question for other platforms to do so as well. And, for the\navoidance of doubt, I would be absolutely delighted if we were able to\nsupport additional platforms with Rust as well.\n\nAlso, the approach of making it an optional component directly\ncontradicts the proposed policy I wrote up. That's a recipe for\nadditional burdensome work maintaining two implementations, when we\nactually want to make it easier for people to contribute functionality.\nIt also doesn't provide any of the memory safety benefits or address any\nof the concerns from governments, security professionals, and other\nparties about the real and substantial risks of continuing to develop in\nC.\n\nFor example, there is zero chance I will implement any of the\nSHA-1/SHA-256 compatibility code twice. I'm already doing that in my\nfree time without any compensation at all and it's unreasonable to\nexpect me to do it twice or even to #ifdef out all the places it would\nneed to go. I am happy to let someone else take responsibility for the\nproject instead, however, if they would like to do those things.\n\n> It would be a shame, but right now it's a risky bet to build anything on\n> top of Rust given that we don't officially accept it in Git yet. We need\n> to first make the decision whether or not we want to have it right now,\n> and if so how that's supposed to look like.\n\nI think we had made the decision at the 2024 Contributor's Summit that\nwe wanted to adopt Rust in Git, so it was more of a matter of sending\nthe patches than actually making that decision. As I recall, the\ndecision was unanimous.\n\n[0] RISC-V was developed in 2010.\n[1] https://www.reddit.com/r/rust/comments/lfysy9/pythons_cryptography_package_introduced_build/\n-- \nbrian m. carlson (they/them)\nToronto, Ontario, CA\n"},{"id":"525488","messageId":"aLlzj-FxXCmBXTQz@pks.im","threadId":"63804","inReplyTo":"xmqqms7bchln.fsf@gitster.g","subject":"Re: [PATCH v3 02/15] xdiff: introduce rust","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-09-04T11:10:07Z","receivedAt":"2025-09-04T11:10:23Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Wed, Sep 03, 2025 at 03:10:44PM -0700, Junio C Hamano wrote:\n> Patrick Steinhardt writes:\n> > Note that I'm not saying that we need to have both a C and Rust\n> > implementation for everything written in Rust. I don't think that's\n> > sustainable in any way. But any feature written in Rust should be a\n> > _new_ feature that can be disabled and that users can live without for\n> > the time being.\n> \n> Yes, if we can find such modular niche, it would be ideal. But how\n> many areas that we can cleanly plug an optional thing in without\n> disrupting existing codebase are there? Offhand, all I'd think of\n> are a new merge backend, a new rebase backend, a transport helper,\n> or perhaps a new diff-algorithm?\n\nNot too many, I guess.\n\nIf we cannot find anything, an alternative could also be to take a very\nsimple subsystem that doesn't see a lot of changes and convert that to\nRust. We'd retain both implementations in that case, which I mentioned\nis painful because we now have to keep both in sync. But if we say that\nthis is a testballoon, only, and that we don't continue to convert other\ncode until Git 3.0, then that might be fine.\n\n\"varint.c\" could be a good match. It's trivial, only 30 lines of code,\nand completely standalone.\n\nWe could still build new and optional functionality via Rust, but I\nguess it also doesn't hurt to have a test balloon that is part of\nlibgit.a to test interoperability.\n\nI'll send patches later today.\n\nPatrick\n"},{"id":"525491","messageId":"aLl6iFXeAvL_hvqR@pks.im","threadId":"63804","inReplyTo":"aLjj9cG9_K6YLfeA@fruit.crustytoothpaste.net","subject":"Re: [PATCH v3 02/15] xdiff: introduce rust","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-09-04T11:39:52Z","receivedAt":"2025-09-04T11:40:02Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Thu, Sep 04, 2025 at 12:57:25AM +0000, brian m. carlson wrote:\n> On 2025-09-03 at 05:40:54, Patrick Steinhardt wrote:\n> > If I had the choice, I'd much rather adopt an ancient version of Rust if\n> > it means that more platforms can support it.\n> \n> I think you may be assuming that gccrs targeting Rust 1.49 will\n> magically make it work on more platforms than upstream Rust will.\n> That's not the case.\n\nI don't have enough context to be able to tell. I'm mostly going by what\nthe gccrs maintainers themselves are saying. But if I'm misunderstanding\nwhat gccrs will bring to the table I'm happy to be corrected.\n\n[snip]\n> > I think adopting Rust as a mandatory dependency out of nowhere would not\n> > be playing nice. It may require significant effort from distros to adapt\n> > to the new reality, so we should give them time to do so.\n> \n> We've actually had this discussion on the list several times where we've\n> proposed the inclusion of Rust. This is not the first time it's come\n> up, or the second. It was explicitly mentioned a year ago on the list\n> that we wanted to adopt Rust in the notes from the Contributor Summit.\n> \n> There has been plenty of notice that this is coming down the line. It's\n> not accurate to claim it's \"out of nowhere\" nor to claim that people\n> have not had plenty of time to port their systems.\n> \n> Distros and porters should not be insensible to the increasing use of\n> Rust or the need for them to get their systems working. For instance,\n> you cannot run a GNOME or MATE desktop environment without librsvg2,\n> which is written in Rust. Python's cryptography package adopted Rust\n> over four years ago and there was the same gnashing of teeth[1], yet\n> little progress has been made by porters on the same affected\n> architectures since that time. In that time, Debian has bootstrapped\n> and released an entire RISC-V port, complete with Rust.\n\nDiscussions of theoretical nature are one thing though. The transition\nthat is actually happening is a different thing, and distributions will\nneed to prepare for this. We already had multiple distro maintainers\ncoming into these discussions saying that this will require a bunch of\nwork, which should be an indicator to us that we need to take it slow.\nWe should accommodate for that.\n\n[snip]\n> It should be stated that there is a very easy way to get Rust working,\n> and that's to port LLVM to the platform in question. IA-64 was removed\n> in 2009, but it might be possible to resurrect that out of tree if\n> there's interest and maybe even get it re-accepted upstream. I'll point\n> out that AIX, Solaris, and QNX have done the necessary porting work to\n> get LLVM and Rust working over the past couple years, so it's not out of\n> the question for other platforms to do so as well. And, for the\n> avoidance of doubt, I would be absolutely delighted if we were able to\n> support additional platforms with Rust as well.\n\nI cannot really say how hard or easy it is to port LLVM to a different\nplatform. I'd be surprised though if that work really was that easy.\n\n> Also, the approach of making it an optional component directly\n> contradicts the proposed policy I wrote up. That's a recipe for\n> additional burdensome work maintaining two implementations, when we\n> actually want to make it easier for people to contribute functionality.\n> It also doesn't provide any of the memory safety benefits or address any\n> of the concerns from governments, security professionals, and other\n> parties about the real and substantial risks of continuing to develop in\n> C.\n\nThe only reason why we want to have it as an optional component is to\nmake the transitioning period easier for downstream distributors. And\nthe intent is not to convert major components -- it should be trivial\ncomponents that we can use as test balloons, similar to how we did it\nfor all of our C99 test balloons.\n\nWe cannot just pull the rug away under their feet without advance notice\nthat this is going to happen.\n\n> For example, there is zero chance I will implement any of the\n> SHA-1/SHA-256 compatibility code twice. I'm already doing that in my\n> free time without any compensation at all and it's unreasonable to\n> expect me to do it twice or even to #ifdef out all the places it would\n> need to go. I am happy to let someone else take responsibility for the\n> project instead, however, if they would like to do those things.\n\nAnd that's totally fair. From my point of view, this compatibility code\nis a _new_ feature that we are adding to Git. And as I mentioned, I\nthink it is reasonable to say that new features may be implemented in\nRust now already, as platforms that aren't yet ready wouldn't lose any\nexisting functionality.\n\n> > It would be a shame, but right now it's a risky bet to build anything on\n> > top of Rust given that we don't officially accept it in Git yet. We need\n> > to first make the decision whether or not we want to have it right now,\n> > and if so how that's supposed to look like.\n> \n> I think we had made the decision at the 2024 Contributor's Summit that\n> we wanted to adopt Rust in Git, so it was more of a matter of sending\n> the patches than actually making that decision. As I recall, the\n> decision was unanimous.\n\nI think most or even all of the contributors are on board. But we never\nreally talked about timelines, or how we want to introduce Rust, so\nthat's a discussion we need to have now.\n\nPatrick\n"},{"id":"525514","messageId":"87v7lymiik.fsf@gentoo.org","threadId":"63804","inReplyTo":"aLl6iFXeAvL_hvqR@pks.im","subject":"Re: [PATCH v3 02/15] xdiff: introduce rust","fromName":"Sam James","fromEmail":"sam@gentoo.org","sentAt":"2025-09-04T13:53:07Z","receivedAt":"2025-09-04T13:53:13Z","isPatch":true,"sender":{"key":"sam@gentoo.org","avatar":"https://avatars.githubusercontent.com/u/11667869?v=4"},"body":"Patrick Steinhardt writes:\n\n> On Thu, Sep 04, 2025 at 12:57:25AM +0000, brian m. carlson wrote:\n>> On 2025-09-03 at 05:40:54, Patrick Steinhardt wrote:\n>> > If I had the choice, I'd much rather adopt an ancient version of Rust if\n>> > it means that more platforms can support it.\n>> \n>> I think you may be assuming that gccrs targeting Rust 1.49 will\n>> magically make it work on more platforms than upstream Rust will.\n>> That's not the case.\n>\n> I don't have enough context to be able to tell. I'm mostly going by what\n> the gccrs maintainers themselves are saying. But if I'm misunderstanding\n> what gccrs will bring to the table I'm happy to be corrected.\n>\n\n(I also think it's obvious that once gccrs can handle 1.49, we will have\nto put effort into making things build with it. Not sure who wanted or\nclaimed magic. I just think relyling on a single implementation isn't a\ngood idea.)\n\n> [snip]\n>> > I think adopting Rust as a mandatory dependency out of nowhere would not\n>> > be playing nice. It may require significant effort from distros to adapt\n>> > to the new reality, so we should give them time to do so.\n>> \n>> We've actually had this discussion on the list several times where we've\n>> proposed the inclusion of Rust. This is not the first time it's come\n>> up, or the second. It was explicitly mentioned a year ago on the list\n>> that we wanted to adopt Rust in the notes from the Contributor Summit.\n>> \n>> There has been plenty of notice that this is coming down the line. It's\n>> not accurate to claim it's \"out of nowhere\" nor to claim that people\n>> have not had plenty of time to port their systems.\n>> \n>> Distros and porters should not be insensible to the increasing use of\n>> Rust or the need for them to get their systems working. For instance,\n>> you cannot run a GNOME or MATE desktop environment without librsvg2,\n>> which is written in Rust. Python's cryptography package adopted Rust\n>> over four years ago and there was the same gnashing of teeth[1], yet\n>> little progress has been made by porters on the same affected\n>> architectures since that time. In that time, Debian has bootstrapped\n>> and released an entire RISC-V port, complete with Rust.\n>\n> Discussions of theoretical nature are one thing though. The transition\n> that is actually happening is a different thing, and distributions will\n> need to prepare for this. We already had multiple distro maintainers\n> coming into these discussions saying that this will require a bunch of\n> work, which should be an indicator to us that we need to take it slow.\n> We should accommodate for that.\n\nI imagine most distributions have absolutely zero awareness of this\nthread or plans for git. See below.\n\n>\n> [snip]\n>> It should be stated that there is a very easy way to get Rust working,\n>> and that's to port LLVM to the platform in question. IA-64 was removed\n>> in 2009, but it might be possible to resurrect that out of tree if\n>> there's interest and maybe even get it re-accepted upstream. I'll point\n>> out that AIX, Solaris, and QNX have done the necessary porting work to\n>> get LLVM and Rust working over the past couple years, so it's not out of\n>> the question for other platforms to do so as well. And, for the\n>> avoidance of doubt, I would be absolutely delighted if we were able to\n>> support additional platforms with Rust as well.\n>\n> I cannot really say how hard or easy it is to port LLVM to a different\n> platform. I'd be surprised though if that work really was that easy.\n\nI think it's an interesting characterisation indeed.\n\n>\n>> Also, the approach of making it an optional component directly\n>> contradicts the proposed policy I wrote up. That's a recipe for\n>> additional burdensome work maintaining two implementations, when we\n>> actually want to make it easier for people to contribute functionality.\n>> It also doesn't provide any of the memory safety benefits or address any\n>> of the concerns from governments, security professionals, and other\n>> parties about the real and substantial risks of continuing to develop in\n>> C.\n>\n> The only reason why we want to have it as an optional component is to\n> make the transitioning period easier for downstream distributors. And\n> the intent is not to convert major components -- it should be trivial\n> components that we can use as test balloons, similar to how we did it\n> for all of our C99 test balloons.\n\nYes, even if it were just for one release, having it optional for\nsomething would mean we can adjust packaging without some huge pressure\nwhere git had 0 Rust in one release and then mandatory Rust in another.\n\n(I would of course prefer far more than one release, but I've tried\nthroughout this thread to give options even if the one I'd prefer isn't\npursued, not \"teeth gnash\").\n\nsam\n"},{"id":"525543","messageId":"xmqqbjnqb4qz.fsf@gitster.g","threadId":"63804","inReplyTo":"aLlzj-FxXCmBXTQz@pks.im","subject":"Re: [PATCH v3 02/15] xdiff: introduce rust","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-09-04T15:45:56Z","receivedAt":"2025-09-04T15:46:00Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Patrick Steinhardt writes:\n\n> If we cannot find anything, an alternative could also be to take a very\n> simple subsystem that doesn't see a lot of changes and convert that to\n> Rust. We'd retain both implementations in that case, which I mentioned\n> is painful because we now have to keep both in sync. But if we say that\n> this is a testballoon, only, and that we don't continue to convert other\n> code until Git 3.0, then that might be fine.\n>\n> \"varint.c\" could be a good match. It's trivial, only 30 lines of code,\n> and completely standalone.\n\nI am afraid that it is a bit too trivial. I didn't mention this\npossibility of maintaining parallel implementations, but the\nquiescent area I had in mind was patch-delta.c (no, I am not that\nambitious to suggest diff-delta.c as the first example).\n\n> We could still build new and optional functionality via Rust, but I\n> guess it also doesn't hurt to have a test balloon that is part of\n> libgit.a to test interoperability.\n\nOK.\n"},{"id":"525573","messageId":"CAH=ZcbBLAKaE733_2_2qbFTYCfwGq37RfF-Z3vaKL1ZR49msAA@mail.gmail.com","threadId":"63804","inReplyTo":"aLl6iFXeAvL_hvqR@pks.im","subject":"Re: [PATCH v3 02/15] xdiff: introduce rust","fromName":"Ezekiel Newren","fromEmail":"ezekielnewren@gmail.com","sentAt":"2025-09-04T23:17:10Z","receivedAt":"2025-09-04T23:17:23Z","isPatch":true,"sender":{"key":"ezekielnewren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/18313324?v=4"},"body":"On Thu, Sep 4, 2025 at 5:40 AM Patrick Steinhardt wrote:\n> The only reason why we want to have it as an optional component is to\n> make the transitioning period easier for downstream distributors. And\n> the intent is not to convert major components -- it should be trivial\n> components that we can use as test balloons, similar to how we did it\n> for all of our C99 test balloons.\n>\n> We cannot just pull the rug away under their feet without advance notice\n> that this is going to happen.\n\nI think making Rust optional for at least 1 version is a viable path.\nI'm not opposed to that idea; it was just easier to develop and talk\nabout Rust as a hard dependency. I needed to know if making Rust\noptional was in demand before spending any significant time on that.\n"},{"id":"525585","messageId":"CABPp-BFNoLC+TdtuEq5Nx+VcFJ-WFga2r0E+eq=fFaaCN_sRGg@mail.gmail.com","threadId":"63804","inReplyTo":"aLl6iFXeAvL_hvqR@pks.im","subject":"Re: [PATCH v3 02/15] xdiff: introduce rust","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2025-09-05T03:54:19Z","receivedAt":"2025-09-05T03:54:32Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"On Thu, Sep 4, 2025 at 4:40 AM Patrick Steinhardt wrote:\n>\n> On Thu, Sep 04, 2025 at 12:57:25AM +0000, brian m. carlson wrote:\n> > On 2025-09-03 at 05:40:54, Patrick Steinhardt wrote:\n\n[snip]\n\n> > Also, the approach of making it an optional component directly\n> > contradicts the proposed policy I wrote up. That's a recipe for\n> > additional burdensome work maintaining two implementations, when we\n> > actually want to make it easier for people to contribute functionality.\n> > It also doesn't provide any of the memory safety benefits or address any\n> > of the concerns from governments, security professionals, and other\n> > parties about the real and substantial risks of continuing to develop in\n> > C.\n>\n> The only reason why we want to have it as an optional component is to\n> make the transitioning period easier for downstream distributors. And\n> the intent is not to convert major components -- it should be trivial\n> components that we can use as test balloons, similar to how we did it\n> for all of our C99 test balloons.\n>\n> We cannot just pull the rug away under their feet without advance notice\n> that this is going to happen.\n\nI find this statement a bit problematic for four reasons:\n\n(1) \"without advance notice\" was already pointed out to be inaccurate\nin this thread, including in the exact email you are responding to;\nyou could argue that there hasn't been _sufficient_ advance notice,\nbut then there should be more details about what is and isn't\nsufficient. Merely repeating this claim which brian just barely\npointed out to you as false almost feels dishonest.\n\n(2) \"pull the rug away\" seems hyperbolic. I would have liked some\nexplanation as to how a transition period is expected to help, and how\nthe existing transition period has been insufficient. You do hint a\nlittle at the former, which I'll discuss more in point 4, but you\nneglect the latter to the point of pretending it didn't exist. In\nshort, why is a further transition period needed, and how will it\ndiffer from the existing one we've already had? It's not clear to me\nwhy distributors must immediately update to the latest git version.\nTaylor discussed this aspect in detail in this thread; you even\nresponded briefly (and tangentially?), but still as far as I can tell\npresume the latest and greatest is mandatory for them to adopt without\nstating why. Maybe they do need to adopt the latest and greatest, but\nI haven't seen folks state why that's the case. Did I miss it?\n\nIt also feels like Rust support is being lumped in with \"breaking\nchanges\", which to me feels misleading. Historically, we have talked\nabout breaking changes and deprecation periods and such so that users\ncould adjust scripts or their command lines such that they would work\nacross multiple versions of Git. The Rust case is somewhat different\nin that we're not discussing behavioral changes of git, merely\nimplementation differences. If someone has both a C-only version of\ngit and a newer version of git that was built with both Rust and C,\nany commands they run should behave the same as far as the C-vs-Rust\ngoes (unless we have our normal discussions about specific behavior\nand any deprecations we want to do related to it, of course).\n\nI do agree that reduced platform support is a negative change (though\nRust brings other advantages that may offset this downside depending\non your viewpoint), but I don't see why it's a breaking change and\nespecially not a \"pull the rug away under their feet\" change.\n\n(3) the use of \"cannot\" presupposes the policy stance which we are\nhaving a discussion about, which, whether intended or not, feels like\nan unfair way to attempt to shut down the conversation.\n\n(4) you suggest that adding Rust as an optional component should avoid\nthe problem, yet we've already had Rust as an optional component for\nthe last three releases, going back to 2.49.0. (libgit-rs and\nlibgit-sys). In this case, you helpfully provided some details\ndistinguishing the type of optional component you want -- the\nreference to a test balloon suggests you want an optional component\nthat is turned on by default (but which users can easily turn off).\nAm I correct that this is your intention? If that's the case, then\nthat's a useful distinction, but I think that distinction needs to be\nmade a bit more clearly (and as a side effect, acknowledge that Rust\nhas already been optionally shipped in some form, and was even\nspecifically highlighted by GitHub's and GitLab's blog posts about the\nv2.49.0 release, among other places)\n\n> > For example, there is zero chance I will implement any of the\n> > SHA-1/SHA-256 compatibility code twice. I'm already doing that in my\n> > free time without any compensation at all and it's unreasonable to\n> > expect me to do it twice or even to #ifdef out all the places it would\n> > need to go. I am happy to let someone else take responsibility for the\n> > project instead, however, if they would like to do those things.\n>\n> And that's totally fair. From my point of view, this compatibility code\n> is a _new_ feature that we are adding to Git. And as I mentioned, I\n> think it is reasonable to say that new features may be implemented in\n> Rust now already, as platforms that aren't yet ready wouldn't lose any\n> existing functionality.\n\nAm I correct to understand that you're suggesting a policy where brian\ncannot modify any existing code to be written in Rust, and can only\nadd new Rust code? Perhaps the SHA-1/SHA-256 compatibility code is\njust new code, or can be done with minimal changes to existing C code\nwhile adding new code. If so, maybe this is a workable solution for\nhim.\n\nBut if it can't be done with minimal changes to existing C code and\nthis policy would impair brian's ability to deliver the compatibility\ncode, then I think this policy would be unworkable. I really don't\nwant to hamstring brian's ability to implement the compatibility code.\nIt has sat dormant for years with no one else stepping up to the\nplate, it's a really important project, and brian has time and energy\nnow. I don't want any chicken-and-egg problems introduced for him\nwith the 3.0 release. Even though I've been working with Ezekiel on\nxdiff, and I'm obviously a bit biased in that area, I find the\nsha1-sha256 compatibility work to be more critical and something we\nshould do everything possible to facilitate.\n\n> I think most or even all of the contributors are on board. But we never\n> really talked about timelines, or how we want to introduce Rust, so\n> that's a discussion we need to have now.\n\nAgreed.\n"},{"id":"525586","messageId":"CABPp-BEgzQg4MOsepFwnfg8AfE5xv2JxKpQa1rGyOpwWW00HqQ@mail.gmail.com","threadId":"63804","inReplyTo":"87v7lymiik.fsf@gentoo.org","subject":"Re: [PATCH v3 02/15] xdiff: introduce rust","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2025-09-05T03:55:02Z","receivedAt":"2025-09-05T03:55:15Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"On Thu, Sep 4, 2025 at 6:53 AM Sam James wrote:\n>\n> Patrick Steinhardt writes:\n\n[...]\n\n> >> Also, the approach of making it an optional component directly\n> >> contradicts the proposed policy I wrote up. That's a recipe for\n> >> additional burdensome work maintaining two implementations, when we\n> >> actually want to make it easier for people to contribute functionality.\n> >> It also doesn't provide any of the memory safety benefits or address any\n> >> of the concerns from governments, security professionals, and other\n> >> parties about the real and substantial risks of continuing to develop in\n> >> C.\n> >\n> > The only reason why we want to have it as an optional component is to\n> > make the transitioning period easier for downstream distributors. And\n> > the intent is not to convert major components -- it should be trivial\n> > components that we can use as test balloons, similar to how we did it\n> > for all of our C99 test balloons.\n>\n> Yes, even if it were just for one release, having it optional for\n> something would mean we can adjust packaging without some huge pressure\n> where git had 0 Rust in one release and then mandatory Rust in another.\n>\n> (I would of course prefer far more than one release, but I've tried\n> throughout this thread to give options even if the one I'd prefer isn't\n> pursued, not \"teeth gnash\").\n\nRust has been an optional component of git for the last three releases\nalready, going back to v2.49.0. See the v2.49.0 release notes, or\ne.g. https://github.blog/open-source/git/highlights-from-git-2-49/\n[*].\n\n[*] A quote: \"This release marks a major milestone in the Git project\nwith the first pieces of Rust code being checked in...\"\n"},{"id":"525593","messageId":"aLqIHCdlbwF5X6Cm@pks.im","threadId":"63804","inReplyTo":"CABPp-BFNoLC+TdtuEq5Nx+VcFJ-WFga2r0E+eq=fFaaCN_sRGg@mail.gmail.com","subject":"Re: [PATCH v3 02/15] xdiff: introduce rust","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-09-05T06:50:04Z","receivedAt":"2025-09-05T06:50:20Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Thu, Sep 04, 2025 at 08:54:19PM -0700, Elijah Newren wrote:\n> On Thu, Sep 4, 2025 at 4:40 AM Patrick Steinhardt wrote:\n> >\n> > On Thu, Sep 04, 2025 at 12:57:25AM +0000, brian m. carlson wrote:\n> > > On 2025-09-03 at 05:40:54, Patrick Steinhardt wrote:\n> > > Also, the approach of making it an optional component directly\n> > > contradicts the proposed policy I wrote up. That's a recipe for\n> > > additional burdensome work maintaining two implementations, when we\n> > > actually want to make it easier for people to contribute functionality.\n> > > It also doesn't provide any of the memory safety benefits or address any\n> > > of the concerns from governments, security professionals, and other\n> > > parties about the real and substantial risks of continuing to develop in\n> > > C.\n> >\n> > The only reason why we want to have it as an optional component is to\n> > make the transitioning period easier for downstream distributors. And\n> > the intent is not to convert major components -- it should be trivial\n> > components that we can use as test balloons, similar to how we did it\n> > for all of our C99 test balloons.\n> >\n> > We cannot just pull the rug away under their feet without advance notice\n> > that this is going to happen.\n> \n> I find this statement a bit problematic for four reasons:\n> \n> (1) \"without advance notice\" was already pointed out to be inaccurate\n> in this thread, including in the exact email you are responding to;\n> you could argue that there hasn't been _sufficient_ advance notice,\n> but then there should be more details about what is and isn't\n> sufficient. Merely repeating this claim which brian just barely\n> pointed out to you as false almost feels dishonest.\n\nI think there is a difference between communication that happens on the\nmailing list/contributors summit and communication that is intended for\nthe broader ecosystem:\n\n - The former is basically us developers discussing potential futures\n and reviewing patches. It would be _nice_ if distro maintainers of\n Git were to read these, but given the large volume of traffic in\n general I think it unlikely that majority of maintainers is keeping\n up with that traffic.\n\n - The latter is in the form of e.g. our release notes as well as our\n BreakingChanges document. These _are_ intended to be reviewed by\n maintainers, and the blame is on them if they don't do so.\n\nWe have never communicated either via release notes or via any kind of\ncommitted document that Rust is going to become mandatory. There have\nbeen lots of large threads discussing it, true. But navigating these\nthreads and estimating consensus isn't easy even for us developers, so\nit's going to be even harder for outsiders to the community.\n\n> (2) \"pull the rug away\" seems hyperbolic. I would have liked some\n> explanation as to how a transition period is expected to help, and how\n> the existing transition period has been insufficient. You do hint a\n> little at the former, which I'll discuss more in point 4, but you\n> neglect the latter to the point of pretending it didn't exist. In\n> short, why is a further transition period needed, and how will it\n> differ from the existing one we've already had? It's not clear to me\n> why distributors must immediately update to the latest git version.\n> Taylor discussed this aspect in detail in this thread; you even\n> responded briefly (and tangentially?), but still as far as I can tell\n> presume the latest and greatest is mandatory for them to adopt without\n> stating why. Maybe they do need to adopt the latest and greatest, but\n> I haven't seen folks state why that's the case. Did I miss it?\n\nThe problem here is that we don't have a story to tell yet. I agree that\nnot everyone always needs the latest and greatest, which is also why I\nmentioned that I think it's fine for _new_ features to be developed in\nRust right away.\n\nBut the story is altogether different for bug and security fixes.\n\n - We of course backport security fixes, but would that also be the\n case if we had ported the subsystem to Rust already and now had to\n implement the security fix twice?\n\n - What happens if only the old C version has a security bug? Do we\n still fix it?\n\n - Likewise, what happens with important bug fixes? We tend to backport\n those that are easy-ish to backport, but if people are potentially\n stuck with an older Git version for years it will become harder for\n us to do so.\n\nI think without us having a proper answer to these questions we _are_\npulling the rug away. Distros may be stuck with an old version of Git\nfor a significant time, and from my point of view we have to do a couple\nof compromises there.\n\n> It also feels like Rust support is being lumped in with \"breaking\n> changes\", which to me feels misleading. Historically, we have talked\n> about breaking changes and deprecation periods and such so that users\n> could adjust scripts or their command lines such that they would work\n> across multiple versions of Git. The Rust case is somewhat different\n> in that we're not discussing behavioral changes of git, merely\n> implementation differences. If someone has both a C-only version of\n> git and a newer version of git that was built with both Rust and C,\n> any commands they run should behave the same as far as the C-vs-Rust\n> goes (unless we have our normal discussions about specific behavior\n> and any deprecations we want to do related to it, of course).\n> \n> I do agree that reduced platform support is a negative change (though\n> Rust brings other advantages that may offset this downside depending\n> on your viewpoint), but I don't see why it's a breaking change and\n> especially not a \"pull the rug away under their feet\" change.\n\nI honestly don't quite understand this perspective. How isn't it\nbreaking that you cannot use that Git version at all anymore?\n\n> (3) the use of \"cannot\" presupposes the policy stance which we are\n> having a discussion about, which, whether intended or not, feels like\n> an unfair way to attempt to shut down the conversation.\n\nSorry, that's not my intent.\n\n> (4) you suggest that adding Rust as an optional component should avoid\n> the problem, yet we've already had Rust as an optional component for\n> the last three releases, going back to 2.49.0. (libgit-rs and\n> libgit-sys).\n\nI don't really think that either libgit-rs or libgit-sys help in any\nway. These are part of \"contrib/\", not built by default, and neither are\nthey consumed by anyone out there. So there is no reason for anyone to\nbuild that library to the best of my knowledge.\n\n> In this case, you helpfully provided some details distinguishing the\n> type of optional component you want -- the reference to a test balloon\n> suggests you want an optional component that is turned on by default\n> (but which users can easily turn off). Am I correct that this is your\n> intention? If that's the case, then that's a useful distinction, but\n> I think that distinction needs to be made a bit more clearly (and as a\n> side effect, acknowledge that Rust has already been optionally shipped\n> in some form, and was even specifically highlighted by GitHub's and\n> GitLab's blog posts about the v2.49.0 release, among other places)\n\nYes. I think we need to have a test balloon that allows us to iterate on\nthe build infrastructure and allows distributors to test with them. I\nthink that test balloon needs to be integrated into core Git so that it\nis part of the normal build process, because otherwise it wouldn't have\nany exposure at all and thus not serve its purpose.\n\n> > > For example, there is zero chance I will implement any of the\n> > > SHA-1/SHA-256 compatibility code twice. I'm already doing that in my\n> > > free time without any compensation at all and it's unreasonable to\n> > > expect me to do it twice or even to #ifdef out all the places it would\n> > > need to go. I am happy to let someone else take responsibility for the\n> > > project instead, however, if they would like to do those things.\n> >\n> > And that's totally fair. From my point of view, this compatibility code\n> > is a _new_ feature that we are adding to Git. And as I mentioned, I\n> > think it is reasonable to say that new features may be implemented in\n> > Rust now already, as platforms that aren't yet ready wouldn't lose any\n> > existing functionality.\n> \n> Am I correct to understand that you're suggesting a policy where brian\n> cannot modify any existing code to be written in Rust, and can only\n> add new Rust code? Perhaps the SHA-1/SHA-256 compatibility code is\n> just new code, or can be done with minimal changes to existing C code\n> while adding new code. If so, maybe this is a workable solution for\n> him.\n\nYeah, that's my hope, as well. There's probably nouances to this though,\nand we'll have to figure it out once the series hits the mailing list.\nSo...\n\n> But if it can't be done with minimal changes to existing C code and\n> this policy would impair brian's ability to deliver the compatibility\n> code, then I think this policy would be unworkable. I really don't\n> want to hamstring brian's ability to implement the compatibility code.\n> It has sat dormant for years with no one else stepping up to the\n> plate, it's a really important project, and brian has time and energy\n> now. I don't want any chicken-and-egg problems introduced for him\n> with the 3.0 release. Even though I've been working with Ezekiel on\n> xdiff, and I'm obviously a bit biased in that area, I find the\n> sha1-sha256 compatibility work to be more critical and something we\n> should do everything possible to facilitate.\n\n... I guess we'll have to see how this looks like in the end. If the\nseries rewrites a bunch of subsystems in Rust I think we should figure\nout whether we can do without that. Or, in the worst case, whether it is\nfeasible to conditionally compile some of the code with either C or\nRust, even though nobody likes that.\n\nPatrick\n"},{"id":"525600","messageId":"aLqeCy9aKgDk6DyT@pks.im","threadId":"63804","inReplyTo":"xmqqbjnqb4qz.fsf@gitster.g","subject":"Re: [PATCH v3 02/15] xdiff: introduce rust","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-09-05T08:23:39Z","receivedAt":"2025-09-05T08:23:49Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Thu, Sep 04, 2025 at 08:45:56AM -0700, Junio C Hamano wrote:\n> Patrick Steinhardt writes:\n> \n> > If we cannot find anything, an alternative could also be to take a very\n> > simple subsystem that doesn't see a lot of changes and convert that to\n> > Rust. We'd retain both implementations in that case, which I mentioned\n> > is painful because we now have to keep both in sync. But if we say that\n> > this is a testballoon, only, and that we don't continue to convert other\n> > code until Git 3.0, then that might be fine.\n> >\n> > \"varint.c\" could be a good match. It's trivial, only 30 lines of code,\n> > and completely standalone.\n> \n> I am afraid that it is a bit too trivial. I didn't mention this\n> possibility of maintaining parallel implementations, but the\n> quiescent area I had in mind was patch-delta.c (no, I am not that\n> ambitious to suggest diff-delta.c as the first example).\n\nOh, it certainly is very trivial. I think for the initial infra this is\na good trait though, as it means that we don't yet have to care about\ninterop between different parts of Git and can rather focus on the\nbigger topic, which is the process for how to introduce Rust in the\nfirst place.\n\nBut I agree, once we have the initial Rust infra landed we should then\nalso gain familiarity with more involved subsystems that _do_ require us\nto hook into other subsystems so that we are forced to extend our build\nsystems as needed.\n\nPatrick\n"},{"id":"525606","messageId":"ada227ec-94aa-4563-800e-05c116a361a8@gmail.com","threadId":"63804","inReplyTo":"CABPp-BFNoLC+TdtuEq5Nx+VcFJ-WFga2r0E+eq=fFaaCN_sRGg@mail.gmail.com","subject":"Re: [PATCH v3 02/15] xdiff: introduce rust","fromName":"Phillip Wood","fromEmail":"phillip.wood123@gmail.com","sentAt":"2025-09-05T10:31:30Z","receivedAt":"2025-09-05T10:31:34Z","isPatch":true,"sender":{"key":"phillip.wood@dunelm.org.uk","avatar":null},"body":"Hi Elijah\n\nOn 05/09/2025 04:54, Elijah Newren wrote:\n> \n> (1) \"without advance notice\" was already pointed out to be inaccurate\n> in this thread, including in the exact email you are responding to;\n> you could argue that there hasn't been _sufficient_ advance notice,\n> but then there should be more details about what is and isn't\n> sufficient. Merely repeating this claim which brian just barely\n> pointed out to you as false almost feels dishonest.\n\nI think there is a difference of understanding of what constitutes \n\"advanced notice\". While it is true that there have been discussions on \nthe list for a couple of years where people were clearly enthusiastic \nabout adopting rust those discussions have always petered out after \nconcerns about portability were raised without us actually adopting \nrust. In those discussions there has been no clear conclusion about \nwhether rust would be mandatory or optional. I think from the point of \nview of an outsider who was following the mailing list it has not been \nclear exactly where the rust discussion was going. For someone not \nfollowing the mailing list but just reading the release notes there has \nbeen no indication that we're thinking of rust mandatory for building \ngit as opposed to offering rust bindings for our C code.\n\n> (2) \"pull the rug away\" seems hyperbolic. I would have liked some\n> explanation as to how a transition period is expected to help, and how\n> the existing transition period has been insufficient.\n\nI'm very unclear what \"the existing transition period\" has been\n\n > [...] > (4) you suggest that adding Rust as an optional component \nshould avoid\n> the problem, yet we've already had Rust as an optional component for\n> the last three releases, going back to 2.49.0. (libgit-rs and\n> libgit-sys).\n\nRight but from the point of view of someone trying to build git on a \nplatform without rust support there is a world of difference between \nhaving some optional bindings for rust external projects to use, and \nmaking rust mandatory to build git.\n\nI would like us to adopt rust but I am concerned about the implications \nfor platforms without rust and think we should give some notice in the \nform a clear announcement in the release notes once we have a concrete \nplan. That plan should include a decision on what commitment we can \nrealistically offer with regard to security updates for platforms \nwithout a rust compiler so maintainers on those platforms have a clear \nidea of how long they will be supported.\n\nThanks\n\nPhillip\n\n"},{"id":"525608","messageId":"87cy85f82r.fsf@gentoo.org","threadId":"63804","inReplyTo":"ada227ec-94aa-4563-800e-05c116a361a8@gmail.com","subject":"Re: [PATCH v3 02/15] xdiff: introduce rust","fromName":"Sam James","fromEmail":"sam@gentoo.org","sentAt":"2025-09-05T11:32:44Z","receivedAt":"2025-09-05T11:32:50Z","isPatch":true,"sender":{"key":"sam@gentoo.org","avatar":"https://avatars.githubusercontent.com/u/11667869?v=4"},"body":"Phillip Wood writes:\n\n> Hi Elijah\n>\n> On 05/09/2025 04:54, Elijah Newren wrote:\n>> (1) \"without advance notice\" was already pointed out to be\n>> inaccurate\n>> in this thread, including in the exact email you are responding to;\n>> you could argue that there hasn't been _sufficient_ advance notice,\n>> but then there should be more details about what is and isn't\n>> sufficient. Merely repeating this claim which brian just barely\n>> pointed out to you as false almost feels dishonest.\n>\n> I think there is a difference of understanding of what constitutes\n> \"advanced notice\". While it is true that there have been discussions\n> on the list for a couple of years where people were clearly\n> enthusiastic about adopting rust those discussions have always petered\n> out after concerns about portability were raised without us actually\n> adopting rust. In those discussions there has been no clear conclusion\n> about whether rust would be mandatory or optional. I think from the\n> point of view of an outsider who was following the mailing list it has\n> not been clear exactly where the rust discussion was going. For\n> someone not following the mailing list but just reading the release\n> notes there has been no indication that we're thinking of rust\n> mandatory for building git as opposed to offering rust bindings for\n> our C code.\n>\n>> (2) \"pull the rug away\" seems hyperbolic. I would have liked some\n>> explanation as to how a transition period is expected to help, and how\n>> the existing transition period has been insufficient.\n>\n> I'm very unclear what \"the existing transition period\" has been\n>\n>> [...] > (4) you suggest that adding Rust as an optional component\n> should avoid\n>> the problem, yet we've already had Rust as an optional component for\n>> the last three releases, going back to 2.49.0. (libgit-rs and\n>> libgit-sys).\n>\n> Right but from the point of view of someone trying to build git on a\n> platform without rust support there is a world of difference between\n> having some optional bindings for rust external projects to use, and\n> making rust mandatory to build git.\n\nEntirely agreed with the whole email.\n\nI'll make some further observations wrt bindings:\n\n1) Distributions often don't enable bindings for languages unless a user\nrequests them, or at the very least they're considered low priority (and\nbindings existing for a language in a project do *not* imply the project\nis going to be rewritten in that language);\n\n2) There would be no value in distributions building Rust bindings\nbecause Rust doesn't really support \"system-wide\" libraries. It doesn't\nmake sense as far as I can tell to install the bindings right now. I\ndon't even know where Rust bindings should be installed to, I've never\nseen a package want them installed before.\n\n3) It's not integrated with the Meson build system we're using so I\nwouldn't have paid any attention to it, at least unless a user requested\nit;\n\n4) It's in contrib/.\n\n>\n> I would like us to adopt rust but I am concerned about the\n> implications for platforms without rust and think we should give some\n> notice in the form a clear announcement in the release notes once we\n> have a concrete plan. That plan should include a decision on what\n> commitment we can realistically offer with regard to security updates\n> for platforms without a rust compiler so maintainers on those\n> platforms have a clear idea of how long they will be supported.\n>\n> Thanks\n>\n> Phillip\n\n\nsam\n"},{"id":"525625","messageId":"ba386547-10e0-45e2-95ad-c47e84919abf@gmail.com","threadId":"63804","inReplyTo":"ada227ec-94aa-4563-800e-05c116a361a8@gmail.com","subject":"Re: [PATCH v3 02/15] xdiff: introduce rust","fromName":"Phillip Wood","fromEmail":"phillip.wood123@gmail.com","sentAt":"2025-09-05T13:14:43Z","receivedAt":"2025-09-05T13:14:17Z","isPatch":true,"sender":{"key":"phillip.wood@dunelm.org.uk","avatar":null},"body":"On 05/09/2025 11:31, Phillip Wood wrote:\n> \n> I would like us to adopt rust but I am concerned about the implications \n> for platforms without rust and think we should give some notice in the \n> form a clear announcement in the release notes once we have a concrete \n> plan. That plan should include a decision on what commitment we can \n> realistically offer with regard to security updates for platforms \n> without a rust compiler so maintainers on those platforms have a clear \n> idea of how long they will be supported.\n\nHere's what such an announcement might look like\n\n This release introduces an optional dependency on rust that is\n enabled by default. Platforms without a rust compiler can continue\n to build git by passing NO_RUST=1. In six months time we plan to\n make rust mandatory for building git. From that point git 2.x.y (the\n last version that can be built without rust) will continue to\n receive security updates for three years.\n\nTo me the important elements are:\n\n1) There is a short period where rust is optional. This allows\n (i) Distributors on platforms without a rust compiler time to notify\n their users that in the future they will only be able to offer\n security updates.\n (ii) Distributors on platforms with a rust compiler time to adjust\n their build procedures to include rust.\n (iii) The git project time to gain experience of using rust and writing\n the necessary bindings while building with it is optional.\n\n2) Rust is enabled by default so platforms without a rust compiler are\n made aware of the problem but have an easy way to continue to build\n git while rust is optional.\n\n3) There is a period of a small number of years where we continue to\n provide security updates for a version of git that can be built\n without rust. This is intended to allow a realistic time for\n distributors on platforms without a rust compiler to port one or make\n other arrangements for providing future security updates without\n placing an undue burden on the project to provide security updates\n for niche platforms indefinitely.\n\nThanks\n\nPhillip\n"},{"id":"525630","messageId":"aLrkRS_EuGN9FfVE@pks.im","threadId":"63804","inReplyTo":"ba386547-10e0-45e2-95ad-c47e84919abf@gmail.com","subject":"Re: [PATCH v3 02/15] xdiff: introduce rust","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-09-05T13:23:17Z","receivedAt":"2025-09-05T13:23:27Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Fri, Sep 05, 2025 at 02:14:43PM +0100, Phillip Wood wrote:\n> On 05/09/2025 11:31, Phillip Wood wrote:\n> > \n> > I would like us to adopt rust but I am concerned about the implications\n> > for platforms without rust and think we should give some notice in the\n> > form a clear announcement in the release notes once we have a concrete\n> > plan. That plan should include a decision on what commitment we can\n> > realistically offer with regard to security updates for platforms\n> > without a rust compiler so maintainers on those platforms have a clear\n> > idea of how long they will be supported.\n> \n> Here's what such an announcement might look like\n> \n> This release introduces an optional dependency on rust that is\n> enabled by default. Platforms without a rust compiler can continue\n> to build git by passing NO_RUST=1. In six months time we plan to\n> make rust mandatory for building git. From that point git 2.x.y (the\n> last version that can be built without rust) will continue to\n> receive security updates for three years.\n> \n> To me the important elements are:\n> \n> 1) There is a short period where rust is optional. This allows\n> (i) Distributors on platforms without a rust compiler time to notify\n> their users that in the future they will only be able to offer\n> security updates.\n> (ii) Distributors on platforms with a rust compiler time to adjust\n> their build procedures to include rust.\n> (iii) The git project time to gain experience of using rust and writing\n> the necessary bindings while building with it is optional.\n> \n> 2) Rust is enabled by default so platforms without a rust compiler are\n> made aware of the problem but have an easy way to continue to build\n> git while rust is optional.\n> \n> 3) There is a period of a small number of years where we continue to\n> provide security updates for a version of git that can be built\n> without rust. This is intended to allow a realistic time for\n> distributors on platforms without a rust compiler to port one or make\n> other arrangements for providing future security updates without\n> placing an undue burden on the project to provide security updates\n> for niche platforms indefinitely.\n\nSomething like this is part of the BreakingChanges document I'm\nproposing in [1]. I think we should also highlight this upcoming change\nin the next release notes, with a pointer to that document.\n\nPatrick\n\n[1]: <20250904-b4-pks-rust-breaking-change-v1-0-3af1d25e0be9@pks.im>\n"},{"id":"525648","messageId":"xmqqplc43o7c.fsf@gitster.g","threadId":"63804","inReplyTo":"ba386547-10e0-45e2-95ad-c47e84919abf@gmail.com","subject":"Re: [PATCH v3 02/15] xdiff: introduce rust","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-09-05T15:37:27Z","receivedAt":"2025-09-05T15:37:30Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Phillip Wood writes:\n\n> This release introduces an optional dependency on rust that is\n> enabled by default. Platforms without a rust compiler can continue\n> to build git by passing NO_RUST=1. In six months time we plan to\n> make rust mandatory for building git. From that point git 2.x.y (the\n> last version that can be built without rust) will continue to\n> receive security updates for three years.\n>\n> To me the important elements are:\n>\n> 1) There is a short period where rust is optional. This allows\n> (i) Distributors on platforms without a rust compiler time to notify\n> their users that in the future they will only be able to offer\n> security updates.\n> (ii) Distributors on platforms with a rust compiler time to adjust\n> their build procedures to include rust.\n> (iii) The git project time to gain experience of using rust and writing\n> the necessary bindings while building with it is optional.\n\nGood. I am not sure \"short\" should be an important element, but\nhaving a known and agreed-upon deadline helps.\n\n> 2) Rust is enabled by default so platforms without a rust compiler are\n> made aware of the problem but have an easy way to continue to build\n> git while rust is optional.\n\nObviously there is nothing to disagree with here, as it is the\ndefinition of the word \"optional\" ;-).\n\n> 3) There is a period of a small number of years where we continue to\n> provide security updates for a version of git that can be built\n> without rust. This is intended to allow a realistic time for\n> distributors on platforms without a rust compiler to port one or make\n> other arrangements for providing future security updates without\n> placing an undue burden on the project to provide security updates\n> for niche platforms indefinitely.\n\nI am not willing to see such a support for multiple years, though.\nIf the first item is 6 months, this backporting stale releases\nshould be on the same order of timeperiod.\n\nIf it were \"3 years of optional period, 18 months of backporting\nsecurity updates\", I would find it more realistic. It would give\nthose platform maintainers enough time to robby, fundraise, or\notherwise campaign to bring Rust on their system. I personally find\nthat 6 months is way too short (if we are _only_ looking for an\nexcuse to say \"we have given them ample time to react, and now it is\ntheir problem\", 6 months may be good enough, though).\n\n"},{"id":"525697","messageId":"CABPp-BG3Zcw63vNziy86MvYNubefn1SmPvXefpqpA=a+42KT8A@mail.gmail.com","threadId":"63804","inReplyTo":"aLqIHCdlbwF5X6Cm@pks.im","subject":"Re: [PATCH v3 02/15] xdiff: introduce rust","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2025-09-07T04:10:28Z","receivedAt":"2025-09-07T04:10:41Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"Sorry for the delay; life outside of work is challenging at the moment...\n\nOn Thu, Sep 4, 2025 at 11:50 PM Patrick Steinhardt wrote:\n>\n> On Thu, Sep 04, 2025 at 08:54:19PM -0700, Elijah Newren wrote:\n> > On Thu, Sep 4, 2025 at 4:40 AM Patrick Steinhardt wrote:\n> > >\n> > > On Thu, Sep 04, 2025 at 12:57:25AM +0000, brian m. carlson wrote:\n> > > > On 2025-09-03 at 05:40:54, Patrick Steinhardt wrote:\n> > > > Also, the approach of making it an optional component directly\n> > > > contradicts the proposed policy I wrote up. That's a recipe for\n> > > > additional burdensome work maintaining two implementations, when we\n> > > > actually want to make it easier for people to contribute functionality.\n> > > > It also doesn't provide any of the memory safety benefits or address any\n> > > > of the concerns from governments, security professionals, and other\n> > > > parties about the real and substantial risks of continuing to develop in\n> > > > C.\n> > >\n> > > The only reason why we want to have it as an optional component is to\n> > > make the transitioning period easier for downstream distributors. And\n> > > the intent is not to convert major components -- it should be trivial\n> > > components that we can use as test balloons, similar to how we did it\n> > > for all of our C99 test balloons.\n> > >\n> > > We cannot just pull the rug away under their feet without advance notice\n> > > that this is going to happen.\n> >\n> > I find this statement a bit problematic for four reasons:\n> >\n> > (1) \"without advance notice\" was already pointed out to be inaccurate\n> > in this thread, including in the exact email you are responding to;\n> > you could argue that there hasn't been _sufficient_ advance notice,\n> > but then there should be more details about what is and isn't\n> > sufficient. Merely repeating this claim which brian just barely\n> > pointed out to you as false almost feels dishonest.\n>\n> I think there is a difference between communication that happens on the\n> mailing list/contributors summit and communication that is intended for\n> the broader ecosystem:\n>\n> - The former is basically us developers discussing potential futures\n> and reviewing patches. It would be _nice_ if distro maintainers of\n> Git were to read these, but given the large volume of traffic in\n> general I think it unlikely that majority of maintainers is keeping\n> up with that traffic.\n>\n> - The latter is in the form of e.g. our release notes as well as our\n> BreakingChanges document. These _are_ intended to be reviewed by\n> maintainers, and the blame is on them if they don't do so.\n>\n> We have never communicated either via release notes or via any kind of\n> committed document that Rust is going to become mandatory. There have\n> been lots of large threads discussing it, true. But navigating these\n> threads and estimating consensus isn't easy even for us developers, so\n> it's going to be even harder for outsiders to the community.\n\nI like this framing; this is useful.\n\nI agree that we haven't communicated that it'll be mandatory, though\nwe have communicated beyond the list that Rust was likely coming:\n * The contributor summit notes on Rust (posted at\nhttps://lore.kernel.org/git/Zu2D%2Fb1ZJbTlC1ml@nand.local/) were\nwidely picked up at other sites (e.g.\nhttps://lwn.net/Articles/998115/,\nhttps://www.reddit.com/r/linux/comments/1hcsvk5/nonstop_discussion_around_adding_rust_to_git/)\n * The release notes mention initial Rust inclusion\n(https://lore.kernel.org/git/xmqqfrjfilc8.fsf@gitster.g/, \"Foreign\nlanguage interface for Rust into our code base has been added.\")\n * The GitHub blog on highlights from 2.49.0 (widely linked at news\nsites even in preference to the release notes) adds more detail: \"This\nrelease marks a major milestone in the Git project with the first\npieces of Rust code being checked in\"\n(https://github.blog/open-source/git/highlights-from-git-2-49/)\n\nNow, I can fully get behind that this may be _inadequate_ notice, and\nI really like the idea of a test balloon. I'm just noting that I very\nmuch disagree with the characterization that there has been no notice\nbeyond the mailing list about Rust likely coming at some point, and\nwant us to make sure that if we delay, we use the time to meaningfully\nprovide more notice than we have already. Another optional Rust\ncomponent that doesn't build by default, for example, fails that test.\n\n> > (2) \"pull the rug away\" seems hyperbolic. I would have liked some\n> > explanation as to how a transition period is expected to help, and how\n> > the existing transition period has been insufficient. You do hint a\n> > little at the former, which I'll discuss more in point 4, but you\n> > neglect the latter to the point of pretending it didn't exist. In\n> > short, why is a further transition period needed, and how will it\n> > differ from the existing one we've already had? It's not clear to me\n> > why distributors must immediately update to the latest git version.\n> > Taylor discussed this aspect in detail in this thread; you even\n> > responded briefly (and tangentially?), but still as far as I can tell\n> > presume the latest and greatest is mandatory for them to adopt without\n> > stating why. Maybe they do need to adopt the latest and greatest, but\n> > I haven't seen folks state why that's the case. Did I miss it?\n>\n> The problem here is that we don't have a story to tell yet. I agree that\n> not everyone always needs the latest and greatest, which is also why I\n> mentioned that I think it's fine for _new_ features to be developed in\n> Rust right away.\n>\n> But the story is altogether different for bug and security fixes.\n>\n> - We of course backport security fixes, but would that also be the\n> case if we had ported the subsystem to Rust already and now had to\n> implement the security fix twice?\n>\n> - What happens if only the old C version has a security bug? Do we\n> still fix it?\n>\n> - Likewise, what happens with important bug fixes? We tend to backport\n> those that are easy-ish to backport, but if people are potentially\n> stuck with an older Git version for years it will become harder for\n> us to do so.\n>\n> I think without us having a proper answer to these questions we _are_\n> pulling the rug away. Distros may be stuck with an old version of Git\n> for a significant time, and from my point of view we have to do a couple\n> of compromises there.\n\nThese are good questions...but they are ones to which I suspect\ndelaying will not provide the answer. In fact, I don't think we'll\n_ever_ have the answer to these questions, no matter how much we delay\nor discuss. Traditionally, if an issue was more severe, it has been\nbackported to more versions, even if the backport wasn't trivial.\nThere's a cost/benefit tradeoff to be had for each vulnerability, and\nchanges to the area making backports either be easy or hard always\nneed to be weighed against the severity of the vulnerability. I don't\nsee that changing, and overpromising hurts in the long run probably\nmore than having no guidance. I just don't see us coming up with\n\"proper answers\" (which I'm guessing means fully spelled out answers?)\nto these questions ahead of time. The answer to all of them is\nprobably \"we'll weigh the severity of the issue and the cost to\nbackport and give the last C-only version significant extra weight in\nour considerations\". I doubt we'll ever be able to promise any more\ndetail than that until we get concrete cases; I'm not even sure that\nthis statement is acceptable to everyone on the list from the\noverpromising angle despite being as incomplete as it is.\n\n> > It also feels like Rust support is being lumped in with \"breaking\n> > changes\", which to me feels misleading. Historically, we have talked\n> > about breaking changes and deprecation periods and such so that users\n> > could adjust scripts or their command lines such that they would work\n> > across multiple versions of Git. The Rust case is somewhat different\n> > in that we're not discussing behavioral changes of git, merely\n> > implementation differences. If someone has both a C-only version of\n> > git and a newer version of git that was built with both Rust and C,\n> > any commands they run should behave the same as far as the C-vs-Rust\n> > goes (unless we have our normal discussions about specific behavior\n> > and any deprecations we want to do related to it, of course).\n> >\n> > I do agree that reduced platform support is a negative change (though\n> > Rust brings other advantages that may offset this downside depending\n> > on your viewpoint), but I don't see why it's a breaking change and\n> > especially not a \"pull the rug away under their feet\" change.\n>\n> I honestly don't quite understand this perspective. How isn't it\n> breaking that you cannot use that Git version at all anymore?\n\nUsers might often face cases where they have to use different versions\nof git -- at home, at work, on different work machines, etc. As such,\nwhen something forces workflow changes, we have to be cognizant of\nthat and provide deprecation periods, release announcement notices,\netc. That's the point of our care around breaking changes.\n\nIf _distributors_ can't build a new version of git, users can still\nuse older versions. They don't have to change their workflows. When\ndistributors eventually figure out how to build a newer version\n(because they work around pthreads not existing on their platform, or\nthey add stdbool to their compiler, or they port Rust to their system\nor whatever), then when the new version becomes available, users can\nuse it without changes to their workflow. The _users_ weren't broken.\n\nI still don't see why distributors _must_ ship the latest version of\nGit and why folks on some platforms are considered broken if they are\nusing a slightly older version. Let me ask again: has anyone answered\nwhy this is considered mandatory? If they have, I've missed it, but\nI've asked multiple times. Even if you want to lump \"distributors\ncannot build a newer version\" under the umbrella of \"breaking\nchanges\", I argue it's a much different kind of break and one which\nmerits different timelines for handling than e.g. lumping it in with\n3.0.\n\n> > (3) the use of \"cannot\" presupposes the policy stance which we are\n> > having a discussion about, which, whether intended or not, feels like\n> > an unfair way to attempt to shut down the conversation.\n>\n> Sorry, that's not my intent.\n\nThanks, and I appreciate you patiently explaining your point of view\nin more detail.\n\n> > (4) you suggest that adding Rust as an optional component should avoid\n> > the problem, yet we've already had Rust as an optional component for\n> > the last three releases, going back to 2.49.0. (libgit-rs and\n> > libgit-sys).\n>\n> I don't really think that either libgit-rs or libgit-sys help in any\n> way. These are part of \"contrib/\", not built by default, and neither are\n> they consumed by anyone out there. So there is no reason for anyone to\n> build that library to the best of my knowledge.\n\nI'm fully willing to accept they are inadequate notice (and perhaps\neven barely helpful), but disagree with the characterization that they\ndon't help at all:\n * they were consumed in the past by Google\n * they recently received patches from someone outside Google\n(https://lore.kernel.org/git/20250826233525.2635432-1-davvid@gmail.com/)\n * they were mentioned in the release notes highlighting at a minimum\nthat Rust is being added to Git.\n * they were highlighted in blog posts from both GitLab and GitHub as\nbeing noteworthy new things in the v2.49.0 release\n\nI agree with you there is certainly more we can do, and I like your\nidea of a test ballon. Let's just avoid repeating the problem of\nadding an optional component that no one will try to build except for\nthose for whom we know can build it; doing that would provide no more\nnotice and thus provide no incremental benefit over libgit-rs and\nlibgit-sys.\n"},{"id":"525710","messageId":"042f01dc2011$da9dcda0$8fd968e0$@nexbridge.com","threadId":"63804","inReplyTo":"CABPp-BG3Zcw63vNziy86MvYNubefn1SmPvXefpqpA=a+42KT8A@mail.gmail.com","subject":"RE: [PATCH v3 02/15] xdiff: introduce rust","fromName":"","fromEmail":"rsbecker@nexbridge.com","sentAt":"2025-09-07T16:09:46Z","receivedAt":"2025-09-07T16:10:58Z","isPatch":true,"sender":{"key":"randall.becker@nexbridge.ca","avatar":"https://avatars.githubusercontent.com/u/28956764?v=4"},"body":"On September 7, 2025 12:10 AM, Elijah Newren wrote:\n>Sorry for the delay; life outside of work is challenging at the moment...\n>\n\nI am going to address the critical point mentioned below and snip the rest for brevity.\n\n>I still don't see why distributors _must_ ship the latest version of Git and why folks\n>on some platforms are considered broken if they are using a slightly older version.\n>Let me ask again: has anyone answered why this is considered mandatory? If they\n>have, I've missed it, but I've asked multiple times. Even if you want to lump\n>\"distributors cannot build a newer version\" under the umbrella of \"breaking\n>changes\", I argue it's a much different kind of break and one which merits different\n>timelines for handling than e.g. lumping it in with 3.0.\n\nI do not see that distributors _must_ ship the latest version. Suppose we are on\n2.51.0 and a CVE comes out that prohibits its use in an organization that does\nnot allow any medium-high to high CVEs. This represents hundreds of thousands\nof impacted users in my community alone. How does the CVE get applied if the\nlatest cannot be built and the git team does not apply the CVE fixes to old\nversions. Personally, I do not care if git versions are different between work\nand home, or even between CI/CD and other platforms. I don't even care\nif I have to use JGit instead of git in some situations (which I see is a likely\noutcome of this discussion). Is there an official statement of what an LTS\nmeans? In other projects LTS is typically, and formally by policy 5 years.\nFrom what others have said here, positions of 6 months, 3 years, and\n\"apply it yourself if you want to continue to use git\" have been made.\n\nThe core problem of adding a breaking dependency is when a CVE comes\nout that prohibits git from being used at all. If the git team is not going\nto provide a clear statement, one way or another, if how CVEs (at\nwhatever severity level) will not have a commitment of any kind,\nthen distributors are essentially cast adrift and on our own. It would\nbe helpful of those of us who donate our time, for no compensation,\nare able to plan for this in a meaningful way. Please remember that\nwe have to justify our participation to our management teams to be\nallowed to continue to participate. Nothing is free from this end\nand if fixing (not just applying fixes) CVEs are now 100% our\nresponsibility, if would be critical to know that when we build our\nbusiness cases to our bosses, who I am fairly certain will say an\nemphatic no.\n\nAlso remember that without support from the git team, the\ncode base is no longer the same, meaning the auditors will not\nnecessarily accept fixes from third-party sources. This particular\npoint enabled adoption on some platforms, particularly NonStop.\nAdoption was at 1-2 customers when we had a divergent code\nbased because some platform fixes being different from the\nstandard code-base and could not be certified as valid. Once the\ncode-base because common, adoption was rapid and enthusiastic.\nIf this goes away, I suspect that adoption rates will go negative.\nI am aware that that particular discussion is actually happening\nin some organizations in my community right now, with companies\nlooking for alternatives to git based on this discussion thread.\n\nWith over a decade of respect and participation,\nRandall\n\n"},{"id":"525766","messageId":"aL56TGLvqpDzmrSn@pks.im","threadId":"63804","inReplyTo":"CABPp-BG3Zcw63vNziy86MvYNubefn1SmPvXefpqpA=a+42KT8A@mail.gmail.com","subject":"Re: [PATCH v3 02/15] xdiff: introduce rust","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-09-08T06:40:12Z","receivedAt":"2025-09-08T06:40:29Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Sat, Sep 06, 2025 at 09:10:28PM -0700, Elijah Newren wrote:\n> On Thu, Sep 4, 2025 at 11:50 PM Patrick Steinhardt wrote:\n> > The problem here is that we don't have a story to tell yet. I agree that\n> > not everyone always needs the latest and greatest, which is also why I\n> > mentioned that I think it's fine for _new_ features to be developed in\n> > Rust right away.\n> >\n> > But the story is altogether different for bug and security fixes.\n> >\n> > - We of course backport security fixes, but would that also be the\n> > case if we had ported the subsystem to Rust already and now had to\n> > implement the security fix twice?\n> >\n> > - What happens if only the old C version has a security bug? Do we\n> > still fix it?\n> >\n> > - Likewise, what happens with important bug fixes? We tend to backport\n> > those that are easy-ish to backport, but if people are potentially\n> > stuck with an older Git version for years it will become harder for\n> > us to do so.\n> >\n> > I think without us having a proper answer to these questions we _are_\n> > pulling the rug away. Distros may be stuck with an old version of Git\n> > for a significant time, and from my point of view we have to do a couple\n> > of compromises there.\n> \n> These are good questions...but they are ones to which I suspect\n> delaying will not provide the answer. In fact, I don't think we'll\n> _ever_ have the answer to these questions, no matter how much we delay\n> or discuss. Traditionally, if an issue was more severe, it has been\n> backported to more versions, even if the backport wasn't trivial.\n> There's a cost/benefit tradeoff to be had for each vulnerability, and\n> changes to the area making backports either be easy or hard always\n> need to be weighed against the severity of the vulnerability. I don't\n> see that changing, and overpromising hurts in the long run probably\n> more than having no guidance. I just don't see us coming up with\n> \"proper answers\" (which I'm guessing means fully spelled out answers?)\n> to these questions ahead of time.\n\nThat's fair, I guess. I don't think we need to fully spell out the\nanswer to each of these questions. But I think we should have some\ngeneral alignment on how we'll handle the last non-Rust release, and\nwhat some guarantees are that we can and want to provide.\n\n> The answer to all of them is\n> probably \"we'll weigh the severity of the issue and the cost to\n> backport and give the last C-only version significant extra weight in\n> our considerations\". I doubt we'll ever be able to promise any more\n> detail than that until we get concrete cases; I'm not even sure that\n> this statement is acceptable to everyone on the list from the\n> overpromising angle despite being as incomplete as it is.\n\nTrue, we don't want to overburden us, either. This is mostly why I\nproposed the compromise of saying \"We provide you with updates for the\nLTS version for X amount of time. If you still depend on it after that\ntime, we will be happy to pass over maintainership of that branch to the\ncommunity.\"\n\n[snip]\n> I still don't see why distributors _must_ ship the latest version of\n> Git and why folks on some platforms are considered broken if they are\n> using a slightly older version. Let me ask again: has anyone answered\n> why this is considered mandatory? If they have, I've missed it, but\n> I've asked multiple times. Even if you want to lump \"distributors\n> cannot build a newer version\" under the umbrella of \"breaking\n> changes\", I argue it's a much different kind of break and one which\n> merits different timelines for handling than e.g. lumping it in with\n> 3.0.\n\nTo me it's not necessarily about the _latest_ version, rather about\n_any_ version. Some distributions will not be able to build Git at all\nanymore, so they are stuck at the last non-Rust version for the time\nbeing. And seeing that the timeline is years for them to get Rust\nsupport they may not be on a slightly older version, but on an ancient\nversion eventually.\n\nSo the question to me is less whether users of that distro will miss out\non new features, which I think is acceptable. Hence my statement that it\nis fine from my point of view for new features to be written in Rust\nimmediately.\n\n NB: There's some nuance here. If newer features mean that users\n cannot interact with modern upstreams anymore the picture would\n change quite significantly. But the only work that really comes to\n my mind is SHA256, which already exists. I don't know whether the\n interop code may fall into this category, I hope it doesn't.\n\nBut the bigger question is whether that old version still gets security\nupdates and important bug fixes. If the only available non-Rust version\nis riddled with security holes then these distros won't be able to\nprovide it at all anymore.\n\n> > > (4) you suggest that adding Rust as an optional component should avoid\n> > > the problem, yet we've already had Rust as an optional component for\n> > > the last three releases, going back to 2.49.0. (libgit-rs and\n> > > libgit-sys).\n> >\n> > I don't really think that either libgit-rs or libgit-sys help in any\n> > way. These are part of \"contrib/\", not built by default, and neither are\n> > they consumed by anyone out there. So there is no reason for anyone to\n> > build that library to the best of my knowledge.\n> \n> I'm fully willing to accept they are inadequate notice (and perhaps\n> even barely helpful), but disagree with the characterization that they\n> don't help at all:\n> * they were consumed in the past by Google\n> * they recently received patches from someone outside Google\n> (https://lore.kernel.org/git/20250826233525.2635432-1-davvid@gmail.com/)\n> * they were mentioned in the release notes highlighting at a minimum\n> that Rust is being added to Git.\n> * they were highlighted in blog posts from both GitLab and GitHub as\n> being noteworthy new things in the v2.49.0 release\n\nOkay, fair.\n\n> I agree with you there is certainly more we can do, and I like your\n> idea of a test ballon. Let's just avoid repeating the problem of\n> adding an optional component that no one will try to build except for\n> those for whom we know can build it; doing that would provide no more\n> notice and thus provide no incremental benefit over libgit-rs and\n> libgit-sys.\n\nFully agreed. My current proposal includes several steps of how we\ntighten the screws here, where we gradually start to require Rust by\ndefault on more platforms. Before Git 3.0 it's still possible to opt\nout, but eventually distributors need to opt out explicitly. So that\nshould hopefully alert them that something is cooking.\n\nPatrick\n"},{"id":"525767","messageId":"aL56XQFIRRjg88YD@pks.im","threadId":"63804","inReplyTo":"xmqqplc43o7c.fsf@gitster.g","subject":"Re: [PATCH v3 02/15] xdiff: introduce rust","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-09-08T06:40:29Z","receivedAt":"2025-09-08T06:40:38Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Fri, Sep 05, 2025 at 08:37:27AM -0700, Junio C Hamano wrote:\n> Phillip Wood writes:\n> > 3) There is a period of a small number of years where we continue to\n> > provide security updates for a version of git that can be built\n> > without rust. This is intended to allow a realistic time for\n> > distributors on platforms without a rust compiler to port one or make\n> > other arrangements for providing future security updates without\n> > placing an undue burden on the project to provide security updates\n> > for niche platforms indefinitely.\n> \n> I am not willing to see such a support for multiple years, though.\n> If the first item is 6 months, this backporting stale releases\n> should be on the same order of timeperiod.\n> \n> If it were \"3 years of optional period, 18 months of backporting\n> security updates\", I would find it more realistic. It would give\n> those platform maintainers enough time to robby, fundraise, or\n> otherwise campaign to bring Rust on their system. I personally find\n> that 6 months is way too short (if we are _only_ looking for an\n> excuse to say \"we have given them ample time to react, and now it is\n> their problem\", 6 months may be good enough, though).\n\nYeah, I also think that six months is a bit short, but three years on\nthe other hand feels like it will cause quite some pain on our side. My\nplan is shooting for roughly one year of optional support, which is\nstill way shorter than the three years you mention.\n\nHow would you feel about:\n\n - Pinning a date for Git 3.0 at the end of next year and tying\n mandatory Rust to it.\n\n - Guaranteeing at least one year of security backports for 2.99 (or\n whatever the last release before Git 3.0 is).\n\n - Explicitly stating that if anybody requires to maintain that version\n afterwards, we are happy to let the community maintain that branch\n going forward.\n\nThat means that we stop supporting 2.99 in a bit more than ~two years\nfrom now, but we don't fully close the door on it if people still rely\non it.\n\nPatrick\n"},{"id":"525789","messageId":"ad54bb8f-04e4-4bf7-a13f-6ae7b967b718@gmail.com","threadId":"63804","inReplyTo":"042f01dc2011$da9dcda0$8fd968e0$@nexbridge.com","subject":"Re: [PATCH v3 02/15] xdiff: introduce rust","fromName":"Phillip Wood","fromEmail":"phillip.wood123@gmail.com","sentAt":"2025-09-08T10:12:20Z","receivedAt":"2025-09-08T10:11:41Z","isPatch":true,"sender":{"key":"phillip.wood@dunelm.org.uk","avatar":null},"body":"Hi Randall\n\nOn 07/09/2025 17:09, rsbecker@nexbridge.com wrote:\n> On September 7, 2025 12:10 AM, Elijah Newren wrote:\n>> Sorry for the delay; life outside of work is challenging at the moment...\n>>\n> \n> I am going to address the critical point mentioned below and snip the rest for brevity.\n> \n>> I still don't see why distributors _must_ ship the latest version of Git and why folks\n>> on some platforms are considered broken if they are using a slightly older version.\n>> Let me ask again: has anyone answered why this is considered mandatory? If they\n>> have, I've missed it, but I've asked multiple times. Even if you want to lump\n>> \"distributors cannot build a newer version\" under the umbrella of \"breaking\n>> changes\", I argue it's a much different kind of break and one which merits different\n>> timelines for handling than e.g. lumping it in with 3.0.\n> \n> I do not see that distributors _must_ ship the latest version. Suppose we are on\n> 2.51.0 and a CVE comes out that prohibits its use in an organization that does\n> not allow any medium-high to high CVEs. This represents hundreds of thousands\n> of impacted users in my community alone. How does the CVE get applied if the\n> latest cannot be built and the git team does not apply the CVE fixes to old\n> versions. Personally, I do not care if git versions are different between work\n> and home, or even between CI/CD and other platforms. I don't even care\n> if I have to use JGit instead of git in some situations (which I see is a likely\n> outcome of this discussion). Is there an official statement of what an LTS\n> means?\n\nWe're currently discussing what promises we can make about supporting a \nnon-rust version of git.\n\n> In other projects LTS is typically, and formally by policy 5 years.\n\nI know commercial linux distributions offer that kind of support but are \nthere really open source projects that guarantee 5 years of security \nupdates without any kind of support contract?\n\n> From what others have said here, positions of 6 months, 3 years, and\n> \"apply it yourself if you want to continue to use git\" have been made.\n\nYes it is still being discussed, and no one is volunteering to offer \nfive years of support.\n\n> The core problem of adding a breaking dependency is when a CVE comes\n> out that prohibits git from being used at all. If the git team is not going\n> to provide a clear statement, one way or another, if how CVEs (at\n> whatever severity level) will not have a commitment of any kind,\n> then distributors are essentially cast adrift and on our own. It would\n> be helpful of those of us who donate our time, for no compensation,\n> are able to plan for this in a meaningful way.\n\nDoesn't your company make a front end to git? Are you saying that the \nmanagement does not allocate any staff time to work on git itself and \nexpects the community to provide it with free security updates?\n\n> Please remember that\n> we have to justify our participation to our management teams to be\n> allowed to continue to participate. \nI'm confused by this, as the sentence before say's you're donating your \ntime for no compensation.\n\n> Nothing is free from this end\n> and if fixing (not just applying fixes) CVEs are now 100% our\n> responsibility, if would be critical to know that when we build our\n> business cases to our bosses, who I am fairly certain will say an\n> emphatic no.\n\nIn the long term, unless your platform gains a rust compiler I'm afraid \nI think that is most likely outcome.\n\n> Also remember that without support from the git team, the\n> code base is no longer the same, meaning the auditors will not\n> necessarily accept fixes from third-party sources.\n\nI think I saw a suggestion/question about the possibility of hosting any \nlong term support branch that is maintained by interested parties within \nthe main repository. Would that help?\n\nI appreciate that any move to rust would be very disappointing and \ndisruptive to you but the community has to weigh up the benefits rust \nhas to offer against that.\n\nPhillip\n\n> This particular\n> point enabled adoption on some platforms, particularly NonStop.\n> Adoption was at 1-2 customers when we had a divergent code\n> based because some platform fixes being different from the\n> standard code-base and could not be certified as valid. Once the\n> code-base because common, adoption was rapid and enthusiastic.\n> If this goes away, I suspect that adoption rates will go negative.\n> I am aware that that particular discussion is actually happening\n> in some organizations in my community right now, with companies\n> looking for alternatives to git based on this discussion thread.\n> \n> With over a decade of respect and participation,\n> Randall\n> \n\n"},{"id":"525831","messageId":"CAH=ZcbAjpgAVjVK6iYEr2150a+WgFfxrWuJUoR1pa08JqM4BDw@mail.gmail.com","threadId":"63804","inReplyTo":"042f01dc2011$da9dcda0$8fd968e0$@nexbridge.com","subject":"Re: [PATCH v3 02/15] xdiff: introduce rust","fromName":"Ezekiel Newren","fromEmail":"ezekielnewren@gmail.com","sentAt":"2025-09-08T15:10:07Z","receivedAt":"2025-09-08T15:10:21Z","isPatch":true,"sender":{"key":"ezekielnewren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/18313324?v=4"},"body":"On Sun, Sep 7, 2025 at 10:10 AM wrote:\n>\n> On September 7, 2025 12:10 AM, Elijah Newren wrote:\n> >Sorry for the delay; life outside of work is challenging at the moment...\n> >\n>\n> I am going to address the critical point mentioned below and snip the rest for brevity.\n>\n> >I still don't see why distributors _must_ ship the latest version of Git and why folks\n> >on some platforms are considered broken if they are using a slightly older version.\n> >Let me ask again: has anyone answered why this is considered mandatory? If they\n> >have, I've missed it, but I've asked multiple times. Even if you want to lump\n> >\"distributors cannot build a newer version\" under the umbrella of \"breaking\n> >changes\", I argue it's a much different kind of break and one which merits different\n> >timelines for handling than e.g. lumping it in with 3.0.\n>\n> I do not see that distributors _must_ ship the latest version. Suppose we are on\n> 2.51.0 and a CVE comes out that prohibits its use in an organization that does\n> not allow any medium-high to high CVEs. This represents hundreds of thousands\n> of impacted users in my community alone. How does the CVE get applied if the\n> latest cannot be built and the git team does not apply the CVE fixes to old\n> versions. Personally, I do not care if git versions are different between work\n> and home, or even between CI/CD and other platforms. I don't even care\n> ...\n\nOk, that answers the question for NonStop, but that doesn't answer the\nquestion for the plethora of other distributions. Most distributions\ndon't ship the latest version of Git in their package manager, and if\nan organization deems it critical to have the latest they can build it\nthemselves and ignore the Git version in the package manager. So why\ndoes Windows, Mac, Linux, etc... _need_ the latest version of Git in\nthe package manager?\n\nIf security updates are backported to NonStop, until that platform\nsupports Rust, then I don't see why using an older version of Git in\nWindows, Mac, Linux, etc... is a catastrophe. Most existing\ndistributions _can_ package the latest version of Git, but they\n_don't_.\n\nI reiterate Elijah's question \"Why _must_ distributors ship the latest\nversion of Git?\".\n"},{"id":"525834","messageId":"CABPp-BEEU0yhurwewuRjrceU+AeHy9vYzXaOFmK5u0nnoSbp6w@mail.gmail.com","threadId":"63804","inReplyTo":"042f01dc2011$da9dcda0$8fd968e0$@nexbridge.com","subject":"Re: [PATCH v3 02/15] xdiff: introduce rust","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2025-09-08T15:31:26Z","receivedAt":"2025-09-08T15:31:38Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"On Sun, Sep 7, 2025 at 9:10 AM wrote:\n>\n> On September 7, 2025 12:10 AM, Elijah Newren wrote:\n> >Sorry for the delay; life outside of work is challenging at the moment...\n> >\n>\n> I am going to address the critical point mentioned below and snip the rest for brevity.\n>\n> >I still don't see why distributors _must_ ship the latest version of Git and why folks\n> >on some platforms are considered broken if they are using a slightly older version.\n> >Let me ask again: has anyone answered why this is considered mandatory? If they\n> >have, I've missed it, but I've asked multiple times. Even if you want to lump\n> >\"distributors cannot build a newer version\" under the umbrella of \"breaking\n> >changes\", I argue it's a much different kind of break and one which merits different\n> >timelines for handling than e.g. lumping it in with 3.0.\n>\n> I do not see that distributors _must_ ship the latest version. Suppose we are on\n> 2.51.0 and a CVE comes out that prohibits its use in an organization that does\n> not allow any medium-high to high CVEs. This represents hundreds of thousands\n> of impacted users in my community alone. How does the CVE get applied if the\n> latest cannot be built and the git team does not apply the CVE fixes to old\n> versions. Personally, I do not care if git versions are different between work\n> and home, or even between CI/CD and other platforms. I don't even care\n> if I have to use JGit instead of git in some situations (which I see is a likely\n> outcome of this discussion). Is there an official statement of what an LTS\n> means? In other projects LTS is typically, and formally by policy 5 years.\n> From what others have said here, positions of 6 months, 3 years, and\n> \"apply it yourself if you want to continue to use git\" have been made.\n>\n> The core problem of adding a breaking dependency is when a CVE comes\n> out that prohibits git from being used at all. If the git team is not going\n> to provide a clear statement, one way or another, if how CVEs (at\n> whatever severity level) will not have a commitment of any kind,\n> then distributors are essentially cast adrift and on our own. It would\n> be helpful of those of us who donate our time, for no compensation,\n> are able to plan for this in a meaningful way. Please remember that\n> we have to justify our participation to our management teams to be\n> allowed to continue to participate. Nothing is free from this end\n> and if fixing (not just applying fixes) CVEs are now 100% our\n> responsibility, if would be critical to know that when we build our\n> business cases to our bosses, who I am fairly certain will say an\n> emphatic no.\n>\n> Also remember that without support from the git team, the\n> code base is no longer the same, meaning the auditors will not\n> necessarily accept fixes from third-party sources. This particular\n> point enabled adoption on some platforms, particularly NonStop.\n> Adoption was at 1-2 customers when we had a divergent code\n> based because some platform fixes being different from the\n> standard code-base and could not be certified as valid. Once the\n> code-base because common, adoption was rapid and enthusiastic.\n> If this goes away, I suspect that adoption rates will go negative.\n> I am aware that that particular discussion is actually happening\n> in some organizations in my community right now, with companies\n> looking for alternatives to git based on this discussion thread.\n>\n> With over a decade of respect and participation,\n> Randall\n\nThanks, Randall, this is useful information. In regards to one point\nnot fully covered by Phillip:\n\n> Also remember that without support from the git team, the\n> code base is no longer the same, meaning the auditors will not\n> necessarily accept fixes from third-party sources.\n\nWhy does it need to be \"third-party\" sources? Linus years ago blessed\nhaving someone else be in charge of providing updates for stable\nreleases of Linux. Junio could do the same with Git and similarly\nmark an individual or group of people as the maintainers for the last\nRust-optional version of Git, and those individuals could make\nofficial releases of Git with extended security fix support. Then\nit's not every platform repeating the backporting work that needs to\nbe done, but rather individuals from the affected platform(s)\ncollaborating on that work and then making official first-party\nreleases.\n"},{"id":"525835","messageId":"049301dc20d5$d1643340$742c99c0$@nexbridge.com","threadId":"63804","inReplyTo":"ad54bb8f-04e4-4bf7-a13f-6ae7b967b718@gmail.com","subject":"RE: [PATCH v3 02/15] xdiff: introduce rust","fromName":"","fromEmail":"rsbecker@nexbridge.com","sentAt":"2025-09-08T15:32:36Z","receivedAt":"2025-09-08T15:33:27Z","isPatch":true,"sender":{"key":"randall.becker@nexbridge.ca","avatar":"https://avatars.githubusercontent.com/u/28956764?v=4"},"body":"On September 8, 2025 6:12 AM, Phillip Wood wrote:\n>On 07/09/2025 17:09, rsbecker@nexbridge.com wrote:\n>> On September 7, 2025 12:10 AM, Elijah Newren wrote:\n>>> Sorry for the delay; life outside of work is challenging at the moment...\n>>>\n>>\n>> I am going to address the critical point mentioned below and snip the rest for\n>brevity.\n>>\n>>> I still don't see why distributors _must_ ship the latest version of\n>>> Git and why folks on some platforms are considered broken if they are using a\n>slightly older version.\n>>> Let me ask again: has anyone answered why this is considered\n>>> mandatory? If they have, I've missed it, but I've asked multiple\n>>> times. Even if you want to lump \"distributors cannot build a newer\n>>> version\" under the umbrella of \"breaking changes\", I argue it's a\n>>> much different kind of break and one which merits different timelines for\n>handling than e.g. lumping it in with 3.0.\n>>\n>> I do not see that distributors _must_ ship the latest version. Suppose\n>> we are on\n>> 2.51.0 and a CVE comes out that prohibits its use in an organization\n>> that does not allow any medium-high to high CVEs. This represents\n>> hundreds of thousands of impacted users in my community alone. How\n>> does the CVE get applied if the latest cannot be built and the git\n>> team does not apply the CVE fixes to old versions. Personally, I do\n>> not care if git versions are different between work and home, or even\n>> between CI/CD and other platforms. I don't even care if I have to use\n>> JGit instead of git in some situations (which I see is a likely\n>> outcome of this discussion). Is there an official statement of what an LTS means?\n>\n>We're currently discussing what promises we can make about supporting a non-rust\n>version of git.\n>\n>> In other projects LTS is typically, and formally by policy 5 years.\n>\n>I know commercial linux distributions offer that kind of support but are there really\n>open source projects that guarantee 5 years of security updates without any kind\n>of support contract?\n\nOpenSSL provides 5 years of security fix support (at no cost) for LTS designated\nreleases. Currently 3.0 ending Sept 2026 and 3.5 ending around October 2030.\nAfter those dates, there is a fee-based support arrangement available.\n\n>> From what others have said here, positions of 6 months, 3 years, and\n>> \"apply it yourself if you want to continue to use git\" have been made.\n>\n>Yes it is still being discussed, and no one is volunteering to offer five years of\n>support.\n>\n>> The core problem of adding a breaking dependency is when a CVE comes\n>> out that prohibits git from being used at all. If the git team is not\n>> going to provide a clear statement, one way or another, if how CVEs\n>> (at whatever severity level) will not have a commitment of any kind,\n>> then distributors are essentially cast adrift and on our own. It would\n>> be helpful of those of us who donate our time, for no compensation,\n>> are able to plan for this in a meaningful way.\n>\n>Doesn't your company make a front end to git? Are you saying that the\n>management does not allocate any staff time to work on git itself and expects the\n>community to provide it with free security updates?\n\nThe REAL PROBLEM that is not being addressed in this thread is that large\ncompanies (the ones who process your credit cards, build your cars,\nmanufacture your drugs, and tool your factories (a.k.a. NonStop customers),\nare generally unwilling to accept CVE fixes from third parties. The fixes have\nto be part of the official code base or the fixes will not accepted. That means\nthat either:\n\n1. the git team has to officially sanction the fixes; or\n\n2. do the fixes themselves.\n\nA compromise may be possible to keep a support branch around in the official\ngit repo, for those of us who do not have rust available to contribute to,\nspecifically for post C-deprecation CVE fixes, but I am not sure this is practical.\nIt would also require occasional assistance from the git team, to make sense of\nsome of the fixes, If they apply, as none of us are rust experts.\n\nI am already allocated to spending between 10 and 20 hours a month to git,\nwhich usually involves running and verifying build/test cycles. Since git tests\nare flakey, in some cases, I have to manually examine each failure\nsituation and decide whether the failures are sufficient to pass the releases.\nThese have been reported previously without resolution and do not bear\ndiscussion here.\n\nIt is important to understand that many git customers in high audit situations\nbuild git on their own because they do not trust third party builds, so this\nneeds to remain an option.\n\n>> Please remember that\n>> we have to justify our participation to our management teams to be\n>> allowed to continue to participate.\n>I'm confused by this, as the sentence before say's you're donating your time for no\n>compensation.\n\nMy company pretends to donates my time with very little direct benefit to\nthem. My participation is because I feel it is important for my community.\nNo, I do not get a salary for my git time. It is evenings and weekends. Any time\nI spend during working hours has to be made up during off hours.\n\n>\n>> Nothing is free from this end\n>> and if fixing (not just applying fixes) CVEs are now 100% our\n>> responsibility, if would be critical to know that when we build our\n>> business cases to our bosses, who I am fairly certain will say an\n>> emphatic no.\n>\n>In the long term, unless your platform gains a rust compiler I'm afraid I think that is\n>most likely outcome.\n>\n>> Also remember that without support from the git team, the code base is\n>> no longer the same, meaning the auditors will not necessarily accept\n>> fixes from third-party sources.\n>\n>I think I saw a suggestion/question about the possibility of hosting any long term\n>support branch that is maintained by interested parties within the main repository.\n>Would that help?\n\nDEFINITELY. With assistance as above. With some help, we may be able to make\nthis work. It might require a deeper participation on my part and those on my\nteam to approve changes, which we would consider.\n\n>I appreciate that any move to rust would be very disappointing and disruptive to\n>you but the community has to weigh up the benefits rust has to offer against that.\n\nThe community has plans for Rust but they have not taken shape fully as of yet. I\nhave personally been badgering product management to make this happen, and I\nmight know more in a month or so. However, this takes time, and 6 months is not\nenough.\n\n--Randall\n\n"},{"id":"525836","messageId":"049401dc20d6$69086400$3b192c00$@nexbridge.com","threadId":"63804","inReplyTo":"CABPp-BEEU0yhurwewuRjrceU+AeHy9vYzXaOFmK5u0nnoSbp6w@mail.gmail.com","subject":"RE: [PATCH v3 02/15] xdiff: introduce rust","fromName":"","fromEmail":"rsbecker@nexbridge.com","sentAt":"2025-09-08T15:36:50Z","receivedAt":"2025-09-08T15:37:28Z","isPatch":true,"sender":{"key":"randall.becker@nexbridge.ca","avatar":"https://avatars.githubusercontent.com/u/28956764?v=4"},"body":"On September 8, 2025 11:31 AM, Elijah Newren wrote:\n>On Sun, Sep 7, 2025 at 9:10 AM wrote:\n>>\n>> On September 7, 2025 12:10 AM, Elijah Newren wrote:\n>> >Sorry for the delay; life outside of work is challenging at the moment...\n>> >\n>>\n>> I am going to address the critical point mentioned below and snip the rest for\n>brevity.\n>>\n>> >I still don't see why distributors _must_ ship the latest version of\n>> >Git and why folks on some platforms are considered broken if they are using a\n>slightly older version.\n>> >Let me ask again: has anyone answered why this is considered\n>> >mandatory? If they have, I've missed it, but I've asked multiple\n>> >times. Even if you want to lump \"distributors cannot build a newer\n>> >version\" under the umbrella of \"breaking changes\", I argue it's a\n>> >much different kind of break and one which merits different timelines for\n>handling than e.g. lumping it in with 3.0.\n>>\n>> I do not see that distributors _must_ ship the latest version. Suppose\n>> we are on\n>> 2.51.0 and a CVE comes out that prohibits its use in an organization\n>> that does not allow any medium-high to high CVEs. This represents\n>> hundreds of thousands of impacted users in my community alone. How\n>> does the CVE get applied if the latest cannot be built and the git\n>> team does not apply the CVE fixes to old versions. Personally, I do\n>> not care if git versions are different between work and home, or even\n>> between CI/CD and other platforms. I don't even care if I have to use\n>> JGit instead of git in some situations (which I see is a likely\n>> outcome of this discussion). Is there an official statement of what an LTS means?\n>In other projects LTS is typically, and formally by policy 5 years.\n>> From what others have said here, positions of 6 months, 3 years, and\n>> \"apply it yourself if you want to continue to use git\" have been made.\n>>\n>> The core problem of adding a breaking dependency is when a CVE comes\n>> out that prohibits git from being used at all. If the git team is not\n>> going to provide a clear statement, one way or another, if how CVEs\n>> (at whatever severity level) will not have a commitment of any kind,\n>> then distributors are essentially cast adrift and on our own. It would\n>> be helpful of those of us who donate our time, for no compensation,\n>> are able to plan for this in a meaningful way. Please remember that we\n>> have to justify our participation to our management teams to be\n>> allowed to continue to participate. Nothing is free from this end and\n>> if fixing (not just applying fixes) CVEs are now 100% our\n>> responsibility, if would be critical to know that when we build our\n>> business cases to our bosses, who I am fairly certain will say an\n>> emphatic no.\n>>\n>> Also remember that without support from the git team, the code base is\n>> no longer the same, meaning the auditors will not necessarily accept\n>> fixes from third-party sources. This particular point enabled adoption\n>> on some platforms, particularly NonStop.\n>> Adoption was at 1-2 customers when we had a divergent code based\n>> because some platform fixes being different from the standard\n>> code-base and could not be certified as valid. Once the code-base\n>> because common, adoption was rapid and enthusiastic.\n>> If this goes away, I suspect that adoption rates will go negative.\n>> I am aware that that particular discussion is actually happening in\n>> some organizations in my community right now, with companies looking\n>> for alternatives to git based on this discussion thread.\n>>\n>> With over a decade of respect and participation, Randall\n>\n>Thanks, Randall, this is useful information. In regards to one point not fully covered\n>by Phillip:\n>\n>> Also remember that without support from the git team, the code base is\n>> no longer the same, meaning the auditors will not necessarily accept\n>> fixes from third-party sources.\n>\n>Why does it need to be \"third-party\" sources? Linus years ago blessed having\n>someone else be in charge of providing updates for stable releases of Linux. Junio\n>could do the same with Git and similarly mark an individual or group of people as\n>the maintainers for the last Rust-optional version of Git, and those individuals could\n>make official releases of Git with extended security fix support. Then it's not every\n>platform repeating the backporting work that needs to be done, but rather\n>individuals from the affected platform(s) collaborating on that work and then\n>making official first-party releases.\n\nLinux has one set of rules, and other platforms have others. I do not define the\naudit requirements for PCI, SWIFT, or HIPPA compliance (and other rules outside\nof North America), which apply one way or another to most of my community.\nThe audit teams, which are both internal to the companies and at\ngovernmental regulatory levels, do this. It is 100% out of my control but is a\nreality. Fixes to any code involved in managing financial and health instruments\nmust be done by authorized and recognized sources. I am not one of them.\n\n"},{"id":"525846","messageId":"049501dc20d7$0bb08ed0$2311ac70$@nexbridge.com","threadId":"63804","inReplyTo":"CAH=ZcbAjpgAVjVK6iYEr2150a+WgFfxrWuJUoR1pa08JqM4BDw@mail.gmail.com","subject":"RE: [PATCH v3 02/15] xdiff: introduce rust","fromName":"","fromEmail":"rsbecker@nexbridge.com","sentAt":"2025-09-08T15:41:23Z","receivedAt":"2025-09-08T15:41:59Z","isPatch":true,"sender":{"key":"randall.becker@nexbridge.ca","avatar":"https://avatars.githubusercontent.com/u/28956764?v=4"},"body":"On September 8, 2025 11:10 AM, Ezekiel Newren wrote:\n>On Sun, Sep 7, 2025 at 10:10 AM wrote:\n>>\n>> On September 7, 2025 12:10 AM, Elijah Newren wrote:\n>> >Sorry for the delay; life outside of work is challenging at the moment...\n>> >\n>>\n>> I am going to address the critical point mentioned below and snip the rest for\n>brevity.\n>>\n>> >I still don't see why distributors _must_ ship the latest version of\n>> >Git and why folks on some platforms are considered broken if they are using a\n>slightly older version.\n>> >Let me ask again: has anyone answered why this is considered\n>> >mandatory? If they have, I've missed it, but I've asked multiple\n>> >times. Even if you want to lump \"distributors cannot build a newer\n>> >version\" under the umbrella of \"breaking changes\", I argue it's a\n>> >much different kind of break and one which merits different timelines for\n>handling than e.g. lumping it in with 3.0.\n>>\n>> I do not see that distributors _must_ ship the latest version. Suppose\n>> we are on\n>> 2.51.0 and a CVE comes out that prohibits its use in an organization\n>> that does not allow any medium-high to high CVEs. This represents\n>> hundreds of thousands of impacted users in my community alone. How\n>> does the CVE get applied if the latest cannot be built and the git\n>> team does not apply the CVE fixes to old versions. Personally, I do\n>> not care if git versions are different between work and home, or even\n>> between CI/CD and other platforms. I don't even care ...\n>\n>Ok, that answers the question for NonStop, but that doesn't answer the question\n>for the plethora of other distributions. Most distributions don't ship the latest\n>version of Git in their package manager, and if an organization deems it critical to\n>have the latest they can build it themselves and ignore the Git version in the\n>package manager. So why does Windows, Mac, Linux, etc... _need_ the latest\n>version of Git in the package manager?\n>\n>If security updates are backported to NonStop, until that platform supports Rust,\n>then I don't see why using an older version of Git in Windows, Mac, Linux, etc... is a\n>catastrophe. Most existing distributions _can_ package the latest version of Git, but\n>they _don't_.\n>\n>I reiterate Elijah's question \"Why _must_ distributors ship the latest version of\n>Git?\".\n\nMy emphatic answer is that they do not. There is no requirement from me or anyone\nI know to ship the latest version. What is crucial is that there be fixes for medium-high\nand above CVEs that are delivered in 30 days from initial fix availability (that would be\nin Rust, for this conversation and applied to C). If that were supported, I could live with\nas would my customers and their auditors for LTS releases. Please see my expectation\nof LTS defined elsewhere in this thread - essentially 5 years. Perhaps 3 at a bare minimum.\n\n"},{"id":"525850","messageId":"CABPp-BHQyQk+Vkzm3RhVXxvNrpLB--bCLYwSCfZBQhB6PGBQQQ@mail.gmail.com","threadId":"63804","inReplyTo":"049401dc20d6$69086400$3b192c00$@nexbridge.com","subject":"Re: [PATCH v3 02/15] xdiff: introduce rust","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2025-09-08T16:13:25Z","receivedAt":"2025-09-08T16:13:37Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"On Mon, Sep 8, 2025 at 8:37 AM wrote:\n>\n> On September 8, 2025 11:31 AM, Elijah Newren wrote:\n> >On Sun, Sep 7, 2025 at 9:10 AM wrote:\n> >>\n> >> On September 7, 2025 12:10 AM, Elijah Newren wrote:\n\n> >Thanks, Randall, this is useful information. In regards to one point not fully covered\n> >by Phillip:\n> >\n> >> Also remember that without support from the git team, the code base is\n> >> no longer the same, meaning the auditors will not necessarily accept\n> >> fixes from third-party sources.\n> >\n> >Why does it need to be \"third-party\" sources? Linus years ago blessed having\n> >someone else be in charge of providing updates for stable releases of Linux. Junio\n> >could do the same with Git and similarly mark an individual or group of people as\n> >the maintainers for the last Rust-optional version of Git, and those individuals could\n> >make official releases of Git with extended security fix support. Then it's not every\n> >platform repeating the backporting work that needs to be done, but rather\n> >individuals from the affected platform(s) collaborating on that work and then\n> >making official first-party releases.\n>\n> Linux has one set of rules, and other platforms have others. I do not define the\n> audit requirements for PCI, SWIFT, or HIPPA compliance (and other rules outside\n> of North America), which apply one way or another to most of my community.\n> The audit teams, which are both internal to the companies and at\n> governmental regulatory levels, do this. It is 100% out of my control but is a\n> reality. Fixes to any code involved in managing financial and health instruments\n> must be done by authorized and recognized sources. I am not one of them.\n\nPerhaps I wasn't clear? Let me try to summarize what I've understood\nof the conversation:\n\nRandall: We need to have official git releases for the last\nRust-optional release.\nElijah: Great! Let's enable interested folks to make official git\nreleases for the last Rust-optional release.\nRandall: We need to have official git releases for the last\nRust-optional release.\n\nWhich makes me just want to repeat what I said last time -- let's\nenable some folks to do that.\n"},{"id":"525864","messageId":"049f01dc20e2$42c75650$c85602f0$@nexbridge.com","threadId":"63804","inReplyTo":"CABPp-BHQyQk+Vkzm3RhVXxvNrpLB--bCLYwSCfZBQhB6PGBQQQ@mail.gmail.com","subject":"RE: [PATCH v3 02/15] xdiff: introduce rust","fromName":"","fromEmail":"rsbecker@nexbridge.com","sentAt":"2025-09-08T17:01:38Z","receivedAt":"2025-09-08T17:02:34Z","isPatch":true,"sender":{"key":"randall.becker@nexbridge.ca","avatar":"https://avatars.githubusercontent.com/u/28956764?v=4"},"body":"On September 8, 2025 12:13 PM, Elijah Newren wrote:\n>On Mon, Sep 8, 2025 at 8:37 AM wrote:\n>>\n>> On September 8, 2025 11:31 AM, Elijah Newren wrote:\n>> >On Sun, Sep 7, 2025 at 9:10 AM wrote:\n>> >>\n>> >> On September 7, 2025 12:10 AM, Elijah Newren wrote:\n>\n>> >Thanks, Randall, this is useful information. In regards to one point not fully\n>covered\n>> >by Phillip:\n>> >\n>> >> Also remember that without support from the git team, the code base is\n>> >> no longer the same, meaning the auditors will not necessarily accept\n>> >> fixes from third-party sources.\n>> >\n>> >Why does it need to be \"third-party\" sources? Linus years ago blessed having\n>> >someone else be in charge of providing updates for stable releases of Linux.\n>Junio\n>> >could do the same with Git and similarly mark an individual or group of people as\n>> >the maintainers for the last Rust-optional version of Git, and those individuals\n>could\n>> >make official releases of Git with extended security fix support. Then it's not\n>every\n>> >platform repeating the backporting work that needs to be done, but rather\n>> >individuals from the affected platform(s) collaborating on that work and then\n>> >making official first-party releases.\n>>\n>> Linux has one set of rules, and other platforms have others. I do not define the\n>> audit requirements for PCI, SWIFT, or HIPPA compliance (and other rules outside\n>> of North America), which apply one way or another to most of my community.\n>> The audit teams, which are both internal to the companies and at\n>> governmental regulatory levels, do this. It is 100% out of my control but is a\n>> reality. Fixes to any code involved in managing financial and health instruments\n>> must be done by authorized and recognized sources. I am not one of them.\n>\n>Perhaps I wasn't clear? Let me try to summarize what I've understood\n>of the conversation:\n>\n>Randall: We need to have official git releases for the last\n>Rust-optional release.\n>Elijah: Great! Let's enable interested folks to make official git\n>releases for the last Rust-optional release.\n>Randall: We need to have official git releases for the last\n>Rust-optional release.\n>\n>Which makes me just want to repeat what I said last time -- let's\n>enable some folks to do that.\n\nOk, what does that look like. When and how? Will I get added to the list of\ncommitters for this? How many people? Starting when, ending when? It would\nbe very nice to have some kind of project plan for this with dependencies\nand milestones - I have been asking. If someone sends me a list, I can start\nofficially tracking it.\n\n"},{"id":"527050","messageId":"9818dc92-3569-3e6f-0252-245c2bf0bf84@gmx.de","threadId":"63804","inReplyTo":"dd3a7ab0-947b-4592-a086-8c7028f02ffd@gmail.com","subject":"gitoxide-compatible licensing of Git's Rust code, was Re: [PATCH 6/7] xdiff: conditionally use Rust's implementation of xxhash","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2025-09-23T09:57:18Z","receivedAt":"2025-09-23T09:57:28Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Phillip,\n\nOn Sun, 20 Jul 2025, Phillip Wood wrote:\n\n> Hi Johannes\n> \n> On 19/07/2025 22:53, Johannes Schindelin wrote:\n> > Hi Ezekiel,\n> > \n> > On Thu, 17 Jul 2025, Ezekiel Newren via GitGitGadget wrote:\n> > \n> > > diff --git a/rust/xdiff/src/lib.rs b/rust/xdiff/src/lib.rs\n> > > index e69de29bb2d1..96975975a1ba 100644\n> > > --- a/rust/xdiff/src/lib.rs\n> > > +++ b/rust/xdiff/src/lib.rs\n> > > @@ -0,0 +1,7 @@\n> > > +\n> > > +\n> > > +#[no_mangle]\n> > > +unsafe extern \"C\" fn xxh3_64(ptr: *const u8, size: usize) -> u64 {\n> > > + let slice = std::slice::from_raw_parts(ptr, size);\n> > > + xxhash_rust::xxh3::xxh3_64(slice)\n> > > +}\n> > \n> > I know that this is a pretty small file, but I do notice that it does not\n> > have a license header.\n> > \n> > This reminds me of the unfortunate oversight to be careful about making\n> > (and keeping) libgit.a's source files compatible with libgit2's license to\n> > nurture a fruitful exchange between those two projects.\n> \n> I'm not sure I follow your reasoning here. libgit2 was started after git and\n> chose to use an incompatible license. I wasn't around at the time but isn't\n> there a list of git contributors who are happy to re-license their\n> contributions with the linking exception used by libgit2?\n\nLet me provide some historical context that might clarify the licensing\nconcern.\n\nWhen libgit2 was created, the goal was to demonstrate that Git\nfunctionality could be packaged as a reusable library, inviting innovation\nvia 3rd-party products. The project took existing Git code (with\npermission) and wrapped it with a proper API. The hope was that this would\neventually become the foundation for Git itself.\n\nWhat we learned from that experience is instructive: license\nincompatibility became an insurmountable barrier to code sharing between\nthe projects.\n\nLack of functionality prevented commercial products using libgit2 to\nprovide powerful user interfaces that outshine Git's own user experience,\nunless they accepted the limited functionality.\n\nEven when volunteers wanted to port new Git features to libgit2, the\nlicensing prevented it.\n\nThis fragmentation weakened both projects - libgit2 couldn't benefit from\nGit's innovations, and Git couldn't leverage libgit2's API improvements\nnor corporate contributions that would have been more likely to target\nlibgit2 than Git.\n\nYou can see this in full action: merge ORT, partial clone, sparse index,\netc. All of those features are missing from libgit2, with little hope to\nend up otherwise.\n\nInnovations such as geographically-distributed, redundant data stores, or\nXet-like big-file storage to replace e.g. Git LFS with a fully native\nsolution, haven't happened, despite libgit2's architecture offering the\nextensibility and proper delineation to make such improvements cleaner\nand much more straight-forward than Git's own source code would allow.\n\n> > With Rust, we still have a really good chance to learn from history and\n> > avoid that mistake: Gitoxide is a very exciting project with clear overlap\n> > in its mission to implement Git functionality in Rust. Gitoxide is\n> > dual-licensed under the Apache License v2 and the MIT license (see\n> > https://github.com/GitoxideLabs/gitoxide?tab=readme-ov-file#license).\n> > \n> > Would you mind adding a license header to that file that explicitly allows\n> > the contents of the file to be used in Gitoxide, to get the Rust effort\n> > started on a good foot?\n> \n> I wary of that for two reasons. Firstly over time it is de-facto re-licensing\n> git as the amount of rust code grows and the amount of C code shrinks which\n> deserves a wider discussion. Secondly it makes it harder to convert our C code\n> which is licensed under GPL2 (or in the case of xdiff LGPL) to rust if the\n> rust code uses a different license.\n\nThe industry adopted libgit2 widely precisely because it provided what Git\ndidn't: a clean API for building tools. But the licensing barrier meant\nthat innovation had to happen in isolation.\n\nDue to the feature disparity, we saw \"libgit2 evacuation\" efforts,\nstarting with Visual Studio, later GitHub and GitLab followed, where work\nwas duplicated by moving away from libgit2 towards a distinctly non-API\nway to invoke Git functionality: by spawning full-blown `git` processes\nand communicating by parsing `stdout`, risking regressions due to typo\nfixes such as the infamous \"up-to-date -> up to date\" patches. Such a lot\nof extra work, away from proper API calls, just because of that\nfragmentation!\n\nWith Rust and Gitoxide, we have a rare opportunity to avoid this\nfragmentation from the start. Gitoxide's permissive dual licensing means\ncode can flow both ways. This isn't about \"slipping in\" a license change -\nit's about learning from what happened before.\n\nBy the way, you made it sound as if I asked to re-license existing code,\nwhich is not the case. I specifically asked for new code to be licensed in\na way that avoids to straight up prevent collaboration with the Gitoxide\nproject from the get-go.\n\nIt would not even take more than something as simple as GPLv2+exception.\nWe do have prior art for that: The Git project itself suggests in its very\nown `COPYING` file to use the following license in new files:\n\n This file is licensed under the GPL v2, or a later version\n at the discretion of Linus.\n\nNote the exception? For new Rust code (and of course excluding code that\nhas been ported verbatim from GPLv2-licensed code), GPL v2 could be used\nwith an exception along these lines: This file is licensed under the GPL\nv2, with the exception that it can be freely used in the Gitoxide project.\n\nI am not a lawyer (which everybody but laywers are nowadays required to\nsay), therefore this likely needs some tweaking.\n\n> If someone wants to start a discussion about re-licensing git (and is\n> prepared to do all of the associated admin in the event that it happens)\n> then by all means do so but I don't think it we want to slip such a\n> change into this series.\n\nThe \"wider discussion\" you mention is exactly what we need; Starting with\ncompatible licensing makes that discussion possible rather than\npurely theoretical and moot.\n\nCiao,\nJohannes\n"},{"id":"527118","messageId":"20250923174825.GB1136654@coredump.intra.peff.net","threadId":"63804","inReplyTo":"9818dc92-3569-3e6f-0252-245c2bf0bf84@gmx.de","subject":"Re: gitoxide-compatible licensing of Git's Rust code, was Re: [PATCH 6/7] xdiff: conditionally use Rust's implementation of xxhash","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2025-09-23T17:48:25Z","receivedAt":"2025-09-23T17:48:27Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Tue, Sep 23, 2025 at 11:57:18AM +0200, Johannes Schindelin wrote:\n\n> It would not even take more than something as simple as GPLv2+exception.\n> We do have prior art for that: The Git project itself suggests in its very\n> own `COPYING` file to use the following license in new files:\n> \n> This file is licensed under the GPL v2, or a later version\n> at the discretion of Linus.\n> \n> Note the exception? For new Rust code (and of course excluding code that\n> has been ported verbatim from GPLv2-licensed code), GPL v2 could be used\n> with an exception along these lines: This file is licensed under the GPL\n> v2, with the exception that it can be freely used in the Gitoxide project.\n\nI think this \"and of course\" parenthetical might be a sticking point.\nObviously taking the code verbatim and re-licensing it is not allowed.\nBut I think even reading the C code and then writing substantially\nsimilar Rust code may be legally questionable. The Rust code under the\nmore permissive license has to either be clean-room, or have permission\nfor re-licensing from the original authors (which is getting to be all\nbut impossible over time as code ends up being touched by many people).\n\nI think this is the same issue that libgit2 ran into. If it were just a\nmatter of porting over and re-writing new features, more of it would\nhave been done. But for code to come under the new license it can't just\nbe a port, but has to be an independent work.\n\nSo I wonder if this just creates the same awkward silo between Git's\nRust code and its C code (that we already have between Git and libgit2).\n\n> I am not a lawyer (which everybody but laywers are nowadays required to\n> say), therefore this likely needs some tweaking.\n\nMe either. I do like the goal you're trying to accomplish, but I worry\nthat it will end up causing headaches down the line. Even if we, the\ndevelopers, are a bit permissive about what constitutes \"porting\" and\ndon't require a clean-room implementation, this kind of thing scares off\nthe legal teams that approve using the Rust modules in bigger projects.\nIIRC Microsoft put in a big effort into vetting libgit2's provenance\nbefore agreeing to use it in Visual Studio.\n\n-Peff\n"},{"id":"527214","messageId":"bfaaf26f-5759-4812-9057-b3e0bf7c7949@gmail.com","threadId":"63804","inReplyTo":"20250923174825.GB1136654@coredump.intra.peff.net","subject":"Re: gitoxide-compatible licensing of Git's Rust code, was Re: [PATCH 6/7] xdiff: conditionally use Rust's implementation of xxhash","fromName":"Phillip Wood","fromEmail":"phillip.wood123@gmail.com","sentAt":"2025-09-24T13:48:26Z","receivedAt":"2025-09-24T13:48:29Z","isPatch":true,"sender":{"key":"phillip.wood@dunelm.org.uk","avatar":null},"body":"On 23/09/2025 18:48, Jeff King wrote:\n> On Tue, Sep 23, 2025 at 11:57:18AM +0200, Johannes Schindelin wrote:\n> \n>> It would not even take more than something as simple as GPLv2+exception.\n>> We do have prior art for that: The Git project itself suggests in its very\n>> own `COPYING` file to use the following license in new files:\n>>\n>> This file is licensed under the GPL v2, or a later version\n>> at the discretion of Linus.\n>>\n>> Note the exception? For new Rust code (and of course excluding code that\n>> has been ported verbatim from GPLv2-licensed code), GPL v2 could be used\n>> with an exception along these lines: This file is licensed under the GPL\n>> v2, with the exception that it can be freely used in the Gitoxide project.\n> \n> I think this \"and of course\" parenthetical might be a sticking point.\n> Obviously taking the code verbatim and re-licensing it is not allowed.\n> But I think even reading the C code and then writing substantially\n> similar Rust code may be legally questionable. The Rust code under the\n> more permissive license has to either be clean-room, or have permission\n> for re-licensing from the original authors (which is getting to be all\n> but impossible over time as code ends up being touched by many people).\n> \n> I think this is the same issue that libgit2 ran into. If it were just a\n> matter of porting over and re-writing new features, more of it would\n> have been done. But for code to come under the new license it can't just\n> be a port, but has to be an independent work.\n\nThanks for putting this so clearly, I agree with everything that you've \nwritten here. Another thing I'm concerned/confused about is how the \nexception for a single project works in practice. Does it mean that a \nthird party that wants to re-use some code from GitOxide has to check if \nthe code originally came from Git to determine which license it is \nunder? Or does it mean that anyone who wants to use Git's code without \nthe copyleft restrictions can do so if they launder it through GitOxide \nfirst? Neither of those seems like a great outcome.\n\nThanks\n\nPhillip\n\n"},{"id":"527278","messageId":"20250925022555.GA3202669@coredump.intra.peff.net","threadId":"63804","inReplyTo":"bfaaf26f-5759-4812-9057-b3e0bf7c7949@gmail.com","subject":"Re: gitoxide-compatible licensing of Git's Rust code, was Re: [PATCH 6/7] xdiff: conditionally use Rust's implementation of xxhash","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2025-09-25T02:25:55Z","receivedAt":"2025-09-25T02:26:03Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Wed, Sep 24, 2025 at 02:48:26PM +0100, Phillip Wood wrote:\n\n> Thanks for putting this so clearly, I agree with everything that you've\n> written here. Another thing I'm concerned/confused about is how the\n> exception for a single project works in practice. Does it mean that a third\n> party that wants to re-use some code from GitOxide has to check if the code\n> originally came from Git to determine which license it is under? Or does it\n> mean that anyone who wants to use Git's code without the copyleft\n> restrictions can do so if they launder it through GitOxide first? Neither of\n> those seems like a great outcome.\n\nIf I understand the suggestion correctly, it's not to license it\nspecifically to GitOxide. It's to use a permissive license (like GPL\nwith linking exception) that would make it compatible with other\nprojects with similar licenses (like GitOxide).\n\n-Peff\n"},{"id":"527283","messageId":"aNTWPSWTaHnW3wKt@pks.im","threadId":"63804","inReplyTo":"20250925022555.GA3202669@coredump.intra.peff.net","subject":"Re: gitoxide-compatible licensing of Git's Rust code, was Re: [PATCH 6/7] xdiff: conditionally use Rust's implementation of xxhash","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-09-25T05:42:21Z","receivedAt":"2025-09-25T05:42:28Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Wed, Sep 24, 2025 at 10:25:55PM -0400, Jeff King wrote:\n> On Wed, Sep 24, 2025 at 02:48:26PM +0100, Phillip Wood wrote:\n> \n> > Thanks for putting this so clearly, I agree with everything that you've\n> > written here. Another thing I'm concerned/confused about is how the\n> > exception for a single project works in practice. Does it mean that a third\n> > party that wants to re-use some code from GitOxide has to check if the code\n> > originally came from Git to determine which license it is under? Or does it\n> > mean that anyone who wants to use Git's code without the copyleft\n> > restrictions can do so if they launder it through GitOxide first? Neither of\n> > those seems like a great outcome.\n> \n> If I understand the suggestion correctly, it's not to license it\n> specifically to GitOxide. It's to use a permissive license (like GPL\n> with linking exception) that would make it compatible with other\n> projects with similar licenses (like GitOxide).\n\nYeah, we certainly shouldn't single out a specific project from my point\nof view. But going with something like LGPL or GPL with linking\nexception would be quite a sensible choice from my point of view.\n\nI cannot really say much about the concerns. Should we maybe ask the SFC\nfor some guidance here?\n\nPatrick\n"},{"id":"527410","messageId":"20140030-6bf1-4393-a941-bfdbc69c79fb@gmail.com","threadId":"63804","inReplyTo":"20250925022555.GA3202669@coredump.intra.peff.net","subject":"Re: gitoxide-compatible licensing of Git's Rust code, was Re: [PATCH 6/7] xdiff: conditionally use Rust's implementation of xxhash","fromName":"Phillip Wood","fromEmail":"phillip.wood123@gmail.com","sentAt":"2025-09-26T10:06:38Z","receivedAt":"2025-09-26T10:07:07Z","isPatch":true,"sender":{"key":"phillip.wood@dunelm.org.uk","avatar":null},"body":"On 25/09/2025 03:25, Jeff King wrote:\n> On Wed, Sep 24, 2025 at 02:48:26PM +0100, Phillip Wood wrote:\n> \n>> Thanks for putting this so clearly, I agree with everything that you've\n>> written here. Another thing I'm concerned/confused about is how the\n>> exception for a single project works in practice. Does it mean that a third\n>> party that wants to re-use some code from GitOxide has to check if the code\n>> originally came from Git to determine which license it is under? Or does it\n>> mean that anyone who wants to use Git's code without the copyleft\n>> restrictions can do so if they launder it through GitOxide first? Neither of\n>> those seems like a great outcome.\n> \n> If I understand the suggestion correctly, it's not to license it\n> specifically to GitOxide. It's to use a permissive license (like GPL\n> with linking exception) that would make it compatible with other\n> projects with similar licenses (like GitOxide).\n\nI was responding to this paragraph in Johannes' message\n\n Note the exception? For new Rust code (and of course excluding code\n that has been ported verbatim from GPLv2-licensed code), GPL v2\n could be used with an exception along these lines: This file is\n licensed under the GPL v2, with the exception that it can be freely\n used in the Gitoxide project.\n\nThat suggestion is pretty close to what libgit2 has in its \ngit.git-authors file[1]. I'm not sure how practical it is to special \ncase just one project though.\n\nThanks\n\nPhillip\n\n[1] https://github.com/libgit2/libgit2/blob/main/git.git-authors\n\n"},{"id":"527864","messageId":"20251003031805.GB6381@coredump.intra.peff.net","threadId":"63804","inReplyTo":"20140030-6bf1-4393-a941-bfdbc69c79fb@gmail.com","subject":"Re: gitoxide-compatible licensing of Git's Rust code, was Re: [PATCH 6/7] xdiff: conditionally use Rust's implementation of xxhash","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2025-10-03T03:18:05Z","receivedAt":"2025-10-03T03:18:07Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Fri, Sep 26, 2025 at 11:06:38AM +0100, Phillip Wood wrote:\n\n> > If I understand the suggestion correctly, it's not to license it\n> > specifically to GitOxide. It's to use a permissive license (like GPL\n> > with linking exception) that would make it compatible with other\n> > projects with similar licenses (like GitOxide).\n> \n> I was responding to this paragraph in Johannes' message\n> \n> Note the exception? For new Rust code (and of course excluding code\n> that has been ported verbatim from GPLv2-licensed code), GPL v2\n> could be used with an exception along these lines: This file is\n> licensed under the GPL v2, with the exception that it can be freely\n> used in the Gitoxide project.\n> \n> That suggestion is pretty close to what libgit2 has in its git.git-authors\n> file[1]. I'm not sure how practical it is to special case just one project\n> though.\n\nAh, yeah, I may have misunderstood the proposal then.\n\nI don't think that really changes much with respect to my concern, which\nis for existing code. That probably needs to be explicitly granted\npermission to relicense, whether to a specific project or not, and it is\nhard to pinpoint a definitive author for a lot of it (even if many\nauthors have agreed to relicensing) because it has been touched by so\nmany people over the years.\n\nSo there are probably two separate legal issues to consider:\n\n - how the explicit re-licensing grant is worded\n\n - what problems may came up with porting existing code that has been\n touched by many people\n\n-Peff\n"},{"id":"527872","messageId":"ea27273a-378e-4f75-90f2-6615ce297a43@gmail.com","threadId":"63804","inReplyTo":"20251003031805.GB6381@coredump.intra.peff.net","subject":"Re: gitoxide-compatible licensing of Git's Rust code, was Re: [PATCH 6/7] xdiff: conditionally use Rust's implementation of xxhash","fromName":"Phillip Wood","fromEmail":"phillip.wood123@gmail.com","sentAt":"2025-10-03T09:51:47Z","receivedAt":"2025-10-03T09:51:52Z","isPatch":true,"sender":{"key":"phillip.wood@dunelm.org.uk","avatar":null},"body":"On 03/10/2025 04:18, Jeff King wrote:\n> On Fri, Sep 26, 2025 at 11:06:38AM +0100, Phillip Wood wrote:\n> \n>>> If I understand the suggestion correctly, it's not to license it\n>>> specifically to GitOxide. It's to use a permissive license (like GPL\n>>> with linking exception) that would make it compatible with other\n>>> projects with similar licenses (like GitOxide).\n>>\n>> I was responding to this paragraph in Johannes' message\n>>\n>> Note the exception? For new Rust code (and of course excluding code\n>> that has been ported verbatim from GPLv2-licensed code), GPL v2\n>> could be used with an exception along these lines: This file is\n>> licensed under the GPL v2, with the exception that it can be freely\n>> used in the Gitoxide project.\n>>\n>> That suggestion is pretty close to what libgit2 has in its git.git-authors\n>> file[1]. I'm not sure how practical it is to special case just one project\n>> though.\n> \n> Ah, yeah, I may have misunderstood the proposal then.\n> \n> I don't think that really changes much with respect to my concern, which\n> is for existing code.\n\nI agree it doesn't change anything with regard to that, I think it just \nadds more potential problems.\n> That probably needs to be explicitly granted\n> permission to relicense, whether to a specific project or not, and it is\n> hard to pinpoint a definitive author for a lot of it (even if many\n> authors have agreed to relicensing) because it has been touched by so\n> many people over the years.\n> \n> So there are probably two separate legal issues to consider:\n> \n> - how the explicit re-licensing grant is worded\n> \n> - what problems may came up with porting existing code that has been\n> touched by many people\n\nIf we want to seriously consider this we should probably reach out to \nConservancy for some advice. Did this end up being discussed at the \nContributor's Summit?\n\nThanks\n\nPhillip\n"},{"id":"527925","messageId":"CAHTeOx84BaAS1tkGdvoj1c6z+We+NobsJsTpfshw-xHn-NXGLw@mail.gmail.com","threadId":"63804","inReplyTo":"dd3a7ab0-947b-4592-a086-8c7028f02ffd@gmail.com","subject":"Re: [PATCH 6/7] xdiff: conditionally use Rust's implementation of xxhash","fromName":"Yee Cheng Chin","fromEmail":"ychin.macvim@gmail.com","sentAt":"2025-10-05T05:32:04Z","receivedAt":"2025-10-05T05:32:41Z","isPatch":true,"sender":{"key":"ychin.macvim@gmail.com","avatar":null},"body":"Hi, I have a different but related question. Xdiff is currently\nlicensed under LGPL, not GPL. With the new Rust code not having any\nlicense header in their files, what is the intention for the licensing\nfor them? Given that these are derived from the original Xdiff code, I\nwould have imagined they would use the existing LGPL license for\nXdiff, but the lack of licensing header makes it seem like they are\njust going to inherit the GPL license from Git. Is this a conscious\nrelicensing effort? Or just something that hasn't come up yet?\nOtherwise if xdiff stops being a standalone codebase (due to it\nrelying on the Rust components), it would essentially mean it ceases\nto be a library that could be used by other parties.\n\nThis is critical for downstream projects that use Xdiff. For example,\nlibgit2 mtaintains an xdiff fork, and Vim (which I contribute to) /\nNeovim currently use Xdiff as the internal diff engine, which per my\nunderstanding is only possible due to the license being LGPL (Vim is\nlicensed under the Vim license and Neovim is under Vim / Apache).\n\nOn Sun, Jul 20, 2025 at 3:15 AM Phillip Wood wrote:\n>\n> Hi Johannes\n>\n> On 19/07/2025 22:53, Johannes Schindelin wrote:\n> > Hi Ezekiel,\n> >\n> > On Thu, 17 Jul 2025, Ezekiel Newren via GitGitGadget wrote:\n> >\n> >> diff --git a/rust/xdiff/src/lib.rs b/rust/xdiff/src/lib.rs\n> >> index e69de29bb2d1..96975975a1ba 100644\n> >> --- a/rust/xdiff/src/lib.rs\n> >> +++ b/rust/xdiff/src/lib.rs\n> >> @@ -0,0 +1,7 @@\n> >> +\n> >> +\n> >> +#[no_mangle]\n> >> +unsafe extern \"C\" fn xxh3_64(ptr: *const u8, size: usize) -> u64 {\n> >> + let slice = std::slice::from_raw_parts(ptr, size);\n> >> + xxhash_rust::xxh3::xxh3_64(slice)\n> >> +}\n> >\n> > I know that this is a pretty small file, but I do notice that it does not\n> > have a license header.\n> >\n> > This reminds me of the unfortunate oversight to be careful about making\n> > (and keeping) libgit.a's source files compatible with libgit2's license to\n> > nurture a fruitful exchange between those two projects.\n>\n> I'm not sure I follow your reasoning here. libgit2 was started after git\n> and chose to use an incompatible license. I wasn't around at the time\n> but isn't there a list of git contributors who are happy to re-license\n> their contributions with the linking exception used by libgit2?\n>\n> > With Rust, we still have a really good chance to learn from history and\n> > avoid that mistake: Gitoxide is a very exciting project with clear overlap\n> > in its mission to implement Git functionality in Rust. Gitoxide is\n> > dual-licensed under the Apache License v2 and the MIT license (see\n> > https://github.com/GitoxideLabs/gitoxide?tab=readme-ov-file#license).\n> >\n> > Would you mind adding a license header to that file that explicitly allows\n> > the contents of the file to be used in Gitoxide, to get the Rust effort\n> > started on a good foot?\n>\n> I wary of that for two reasons. Firstly over time it is de-facto\n> re-licensing git as the amount of rust code grows and the amount of C\n> code shrinks which deserves a wider discussion. Secondly it makes it\n> harder to convert our C code which is licensed under GPL2 (or in the\n> case of xdiff LGPL) to rust if the rust code uses a different license.\n>\n> If someone wants to start a discussion about re-licensing git (and is\n> prepared to do all of the associated admin in the event that it happens)\n> then by all means do so but I don't think it we want to slip such a\n> change into this series.\n>\n> Thanks\n>\n> Phillip\n>\n>\n"},{"id":"528056","messageId":"aOTZUpVzEjWNQIND@pks.im","threadId":"63804","inReplyTo":"ea27273a-378e-4f75-90f2-6615ce297a43@gmail.com","subject":"Re: gitoxide-compatible licensing of Git's Rust code, was Re: [PATCH 6/7] xdiff: conditionally use Rust's implementation of xxhash","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-10-07T09:11:46Z","receivedAt":"2025-10-07T09:12:01Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Fri, Oct 03, 2025 at 10:51:47AM +0100, Phillip Wood wrote:\n> On 03/10/2025 04:18, Jeff King wrote:\n> > That probably needs to be explicitly granted\n> > permission to relicense, whether to a specific project or not, and it is\n> > hard to pinpoint a definitive author for a lot of it (even if many\n> > authors have agreed to relicensing) because it has been touched by so\n> > many people over the years.\n> > \n> > So there are probably two separate legal issues to consider:\n> > \n> > - how the explicit re-licensing grant is worded\n> > \n> > - what problems may came up with porting existing code that has been\n> > touched by many people\n> \n> If we want to seriously consider this we should probably reach out to\n> Conservancy for some advice. Did this end up being discussed at the\n> Contributor's Summit?\n\nNot really, no, as we didn't have enough time anymore towards the end.\n\nBut I agree, seeking advice from the SFC might be helpful in this\ncontext. At least until now I haven't heard from anybody who would be\nagainst the idea itself, excluding the potential re-licensing woes.\n\nWould this be something that the PLC can do? I don't think it makes\nsense for random Git folks from the mailing list to approach them.\n\nCc'ing the PLC.\n\nPatrick\n"},{"id":"530803","messageId":"51c8c538-4baa-02e3-c8c2-4004626efd59@gmx.de","threadId":"63804","inReplyTo":"ea27273a-378e-4f75-90f2-6615ce297a43@gmail.com","subject":"Re: gitoxide-compatible licensing of Git's Rust code, was Re: [PATCH 6/7] xdiff: conditionally use Rust's implementation of xxhash","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2025-11-17T13:37:02Z","receivedAt":"2025-11-17T13:37:12Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Phillip,\n\nOn Fri, 3 Oct 2025, Phillip Wood wrote:\n\n> On 03/10/2025 04:18, Jeff King wrote:\n> > On Fri, Sep 26, 2025 at 11:06:38AM +0100, Phillip Wood wrote:\n> > \n> > > > If I understand the suggestion correctly, it's not to license it\n> > > > specifically to GitOxide. It's to use a permissive license (like GPL\n> > > > with linking exception) that would make it compatible with other\n> > > > projects with similar licenses (like GitOxide).\n> > >\n> > > I was responding to this paragraph in Johannes' message\n> > >\n> > > Note the exception? For new Rust code (and of course excluding code\n> > > that has been ported verbatim from GPLv2-licensed code), GPL v2\n> > > could be used with an exception along these lines: This file is\n> > > licensed under the GPL v2, with the exception that it can be freely\n> > > used in the Gitoxide project.\n> > >\n> > > That suggestion is pretty close to what libgit2 has in its git.git-authors\n> > > file[1]. I'm not sure how practical it is to special case just one project\n> > > though.\n> > \n> > Ah, yeah, I may have misunderstood the proposal then.\n> > \n> > I don't think that really changes much with respect to my concern, which\n> > is for existing code.\n> \n> I agree it doesn't change anything with regard to that, I think it just adds\n> more potential problems.\n\nWhile we were discussing this, the decision kind of has been made already:\nhttps://lore.kernel.org/git/20251027004404.2152927-1-sandals@crustytoothpaste.net/\n\nThis saves us the effort to discuss the benefits and risks of opening up\nto collaborating with e.g. Gitoxide (or with Jujutsu, which most likely\nalso would have preferred a more open license), at the expense of closing\nthat door gently, quietly and firmly.\n\nCiao,\nJohannes\n"}]}