git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v2 14/15] rust: add a small wrapper around the hashfile code

From
brian m. carlson <sandals@crustytoothpaste.net>
Date
Nov 17, 2025, 22:16 UTC
Message-ID
<20251117221621.2863243-15-sandals@crustytoothpaste.net>
In-Reply-To
<20251117221621.2863243-1-sandals@crustytoothpaste.net>

Our new binary object map code avoids needing to be intimately involved with file handling by simply writing data to an object implement Write. This makes it very easy to test by writing to a Cursor wrapping a Vec for tests, and thus decouples it from intimate knowledge about how we handle files.

However, we will actually want to write our data to an actual file, since that's the most practical way to persist data. Implement a wrapper around the hashfile code that implements the Write trait so that we can write our object map into a file.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 Makefile         |  1 +
 src/csum_file.rs | 81 ++++++++++++++++++++++++++++++++++++++++++++++++
 src/lib.rs       |  1 +
 src/meson.build  |  1 +
 4 files changed, 84 insertions(+)
 create mode 100644 src/csum_file.rs
diff --git a/Makefile b/Makefile
index 40785c14fd..b05709c5e9 100644
--- a/Makefile
+++ b/Makefile
@@ -1534,6 +1534,7 @@ CLAR_TEST_OBJS += $(UNIT_TEST_DIR)/unit-test.o
 
 UNIT_TEST_OBJS += $(UNIT_TEST_DIR)/test-lib.o
 
+RUST_SOURCES += src/csum_file.rs
 RUST_SOURCES += src/hash.rs
 RUST_SOURCES += src/lib.rs
 RUST_SOURCES += src/loose.rs
diff --git a/src/csum_file.rs b/src/csum_file.rs
new file mode 100644
index 0000000000..7f2c6c4fcb
--- /dev/null
+++ b/src/csum_file.rs
@@ -0,0 +1,81 @@
+// This program is free software; you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation: version 2 of the License, dated June 1991.
+//
+// This program is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License along
+// with this program; if not, see <https://www.gnu.org/licenses/>.
+
+use crate::hash::{HashAlgorithm, GIT_MAX_RAWSZ};
+use std::ffi::CStr;
+use std::io::{self, Write};
+use std::os::raw::c_void;
+
+/// A writer that can write files identified by their hash or containing a trailing hash.
+pub struct HashFile {
+    ptr: *mut c_void,
+    algo: HashAlgorithm,
+}
+
+impl HashFile {
+    /// Create a new HashFile.
+    ///
+    /// The hash used will be `algo`, its name should be in `name`, and an open file descriptor
+    /// pointing to that file should be in `fd`.
+    pub fn new(algo: HashAlgorithm, fd: i32, name: &CStr) -> HashFile {
+        HashFile {
+            ptr: unsafe { c::hashfd(algo.hash_algo_ptr(), fd, name.as_ptr()) },
+            algo,
+        }
+    }
+
+    /// Finalize this HashFile instance.
+    ///
+    /// Returns the hash computed over the data.
+    pub fn finalize(self, component: u32, flags: u32) -> Vec<u8> {
+        let mut result = vec![0u8; GIT_MAX_RAWSZ];
+        unsafe { c::finalize_hashfile(self.ptr, result.as_mut_ptr(), component, flags) };
+        result.truncate(self.algo.raw_len());
+        result
+    }
+}
+
+impl Write for HashFile {
+    fn write(&mut self, data: &[u8]) -> io::Result<usize> {
+        for chunk in data.chunks(u32::MAX as usize) {
+            unsafe {
+                c::hashwrite(
+                    self.ptr,
+                    chunk.as_ptr() as *const c_void,
+                    chunk.len() as u32,
+                )
+            };
+        }
+        Ok(data.len())
+    }
+
+    fn flush(&mut self) -> io::Result<()> {
+        unsafe { c::hashflush(self.ptr) };
+        Ok(())
+    }
+}
+
+pub mod c {
+    use std::os::raw::{c_char, c_int, c_void};
+
+    extern "C" {
+        pub fn hashfd(algop: *const c_void, fd: i32, name: *const c_char) -> *mut c_void;
+        pub fn hashwrite(f: *mut c_void, data: *const c_void, len: u32);
+        pub fn hashflush(f: *mut c_void);
+        pub fn finalize_hashfile(
+            f: *mut c_void,
+            data: *mut u8,
+            component: u32,
+            flags: u32,
+        ) -> c_int;
+    }
+}
diff --git a/src/lib.rs b/src/lib.rs
index 442f9433dc..0c598298b1 100644
--- a/src/lib.rs
+++ b/src/lib.rs
@@ -1,3 +1,4 @@
+pub mod csum_file;
 pub mod hash;
 pub mod loose;
 pub mod varint;
diff --git a/src/meson.build b/src/meson.build
index 1eea068519..45739957b4 100644
--- a/src/meson.build
+++ b/src/meson.build
@@ -1,4 +1,5 @@
 libgit_rs_sources = [
+  'csum_file.rs',
   'hash.rs',
   'lib.rs',
   'loose.rs',
Previous: brian m. carlsonNext: brian m. carlson
Message 99 of 118 in “SHA-1/SHA-256 interoperability, part 2”
  1. 00/14 SHA-1/SHA-256 interoperability, part 2brian m. carlson, Oct 27, 2025
  2. 14/14 object-file-convert: always make sure object ID algo is validbrian m. carlson, Oct 27, 2025
  3. 05/14 rust: add a hash algorithm abstractionbrian m. carlson, Oct 27, 2025
  4. Patrick SteinhardtOct 28, 2025
  5. Ezekiel NewrenOct 28, 2025
  6. Junio C HamanoOct 28, 2025
  7. Ezekiel NewrenOct 28, 2025
  8. Junio C HamanoOct 29, 2025
  9. Junio C HamanoOct 29, 2025
  10. 11/14 rust: add functionality to hash an objectbrian m. carlson, Oct 27, 2025
  11. Patrick SteinhardtOct 28, 2025
  12. brian m. carlsonOct 29, 2025
  13. Patrick SteinhardtOct 29, 2025
  14. Ezekiel NewrenOct 28, 2025
  15. brian m. carlsonOct 29, 2025
  16. Ben KnobleOct 29, 2025
  17. 07/14 csum-file: define hashwrite's count as a uint32_tbrian m. carlson, Oct 27, 2025
  18. Ezekiel NewrenOct 28, 2025
  19. 09/14 hash: expose hash context functions to Rustbrian m. carlson, Oct 27, 2025
  20. Junio C HamanoOct 29, 2025
  21. brian m. carlsonOct 30, 2025
  22. Junio C HamanoOct 30, 2025
  23. 13/14 rust: add a small wrapper around the hashfile codebrian m. carlson, Oct 27, 2025
  24. Ezekiel NewrenOct 28, 2025
  25. brian m. carlsonOct 29, 2025
  26. 06/14 hash: add a function to look up hash algo structsbrian m. carlson, Oct 27, 2025
  27. Patrick SteinhardtOct 28, 2025
  28. Junio C HamanoOct 28, 2025
  29. brian m. carlsonNov 4, 2025
  30. Junio C HamanoNov 4, 2025
  31. 10/14 rust: add a build.rs script for testsbrian m. carlson, Oct 27, 2025
  32. Patrick SteinhardtOct 28, 2025
  33. Ezekiel NewrenOct 28, 2025
  34. Junio C HamanoOct 29, 2025
  35. Ezekiel NewrenOct 29, 2025
  36. Junio C HamanoOct 29, 2025
  37. Patrick SteinhardtOct 30, 2025
  38. Junio C HamanoOct 30, 2025
  39. Ezekiel NewrenOct 31, 2025
  40. Junio C HamanoNov 1, 2025
  41. 12/14 rust: add a new binary loose object map formatbrian m. carlson, Oct 27, 2025
  42. Patrick SteinhardtOct 28, 2025
  43. brian m. carlsonOct 29, 2025
  44. Patrick SteinhardtOct 29, 2025
  45. Junio C HamanoOct 29, 2025
  46. Junio C HamanoOct 29, 2025
  47. 08/14 write-or-die: add an fsync component for the loose object mapbrian m. carlson, Oct 27, 2025
  48. 02/14 conversion: don't crash when no destination algobrian m. carlson, Oct 27, 2025
  49. 03/14 hash: use uint32_t for object_id algorithmbrian m. carlson, Oct 27, 2025
  50. Patrick SteinhardtOct 28, 2025
  51. Ezekiel NewrenOct 28, 2025
  52. Junio C HamanoOct 28, 2025
  53. Ezekiel NewrenOct 28, 2025
  54. Junio C HamanoOct 28, 2025
  55. brian m. carlsonOct 30, 2025
  56. Collin FunkOct 30, 2025
  57. brian m. carlsonNov 3, 2025
  58. brian m. carlsonOct 29, 2025
  59. Patrick SteinhardtOct 29, 2025
  60. 04/14 rust: add a ObjectID structbrian m. carlson, Oct 27, 2025
  61. Patrick SteinhardtOct 28, 2025
  62. Ezekiel NewrenOct 28, 2025
  63. brian m. carlsonOct 29, 2025
  64. Junio C HamanoOct 28, 2025
  65. brian m. carlsonOct 29, 2025
  66. brian m. carlsonOct 29, 2025
  67. Patrick SteinhardtOct 29, 2025
  68. brian m. carlsonOct 30, 2025
  69. 01/14 repository: require Rust support for interoperabilitybrian m. carlson, Oct 27, 2025
  70. Patrick SteinhardtOct 28, 2025
  71. Junio C HamanoOct 29, 2025
  72. Junio C HamanoOct 29, 2025
  73. Ezekiel NewrenNov 11, 2025
  74. Junio C HamanoNov 14, 2025
  75. Junio C HamanoNov 14, 2025
  76. Junio C HamanoNov 17, 2025
  77. brian m. carlsonNov 17, 2025
  78. Junio C HamanoNov 18, 2025
  79. brian m. carlsonNov 19, 2025
  80. Junio C HamanoNov 19, 2025
  81. Ezekiel NewrenNov 19, 2025
  82. Ezekiel NewrenNov 20, 2025
  83. brian m. carlsonNov 20, 2025
  84. Ezekiel NewrenNov 20, 2025
  85. Junio C HamanoNov 20, 2025
  86. 00/15 SHA-1/SHA-256 interoperability, part 2brian m. carlson, Nov 17, 2025
  87. 02/15 conversion: don't crash when no destination algobrian m. carlson, Nov 17, 2025
  88. 03/15 hash: use uint32_t for object_id algorithmbrian m. carlson, Nov 17, 2025
  89. 01/15 repository: require Rust support for interoperabilitybrian m. carlson, Nov 17, 2025
  90. 04/15 rust: add a ObjectID structbrian m. carlson, Nov 17, 2025
  91. 06/15 hash: add a function to look up hash algo structsbrian m. carlson, Nov 17, 2025
  92. 08/15 csum-file: define hashwrite's count as a uint32_tbrian m. carlson, Nov 17, 2025
  93. 05/15 rust: add a hash algorithm abstractionbrian m. carlson, Nov 17, 2025
  94. 09/15 write-or-die: add an fsync component for the object mapbrian m. carlson, Nov 17, 2025
  95. 10/15 hash: expose hash context functions to Rustbrian m. carlson, Nov 17, 2025
  96. 07/15 rust: add additional helpers for ObjectIDbrian m. carlson, Nov 17, 2025
  97. 12/15 rust: add functionality to hash an objectbrian m. carlson, Nov 17, 2025
  98. 11/15 rust: add a build.rs script for testsbrian m. carlson, Nov 17, 2025
  99. 14/15 rust: add a small wrapper around the hashfile codebrian m. carlson, Nov 17, 2025
  100. 15/15 object-file-convert: always make sure object ID algo is validbrian m. carlson, Nov 17, 2025
  101. 13/15 rust: add a new binary object map formatbrian m. carlson, Nov 17, 2025
  102. 00/16 SHA-1/SHA-256 interoperability, part 2brian m. carlson, Feb 7, 2026
  103. 04/16 rust: add a ObjectID structbrian m. carlson, Feb 7, 2026
  104. 02/16 conversion: don't crash when no destination algobrian m. carlson, Feb 7, 2026
  105. 01/16 repository: require Rust support for interoperabilitybrian m. carlson, Feb 7, 2026
  106. 03/16 hash: use uint32_t for object_id algorithmbrian m. carlson, Feb 7, 2026
  107. 07/16 rust: add additional helpers for ObjectIDbrian m. carlson, Feb 7, 2026
  108. 14/16 rust: add a new binary object map formatbrian m. carlson, Feb 7, 2026
  109. 08/16 csum-file: define hashwrite's count as a uint32_tbrian m. carlson, Feb 7, 2026
  110. 06/16 hash: add a function to look up hash algo structsbrian m. carlson, Feb 7, 2026
  111. 11/16 rust: fix linking binaries with cargobrian m. carlson, Feb 7, 2026
  112. 12/16 rust: add a build.rs script for testsbrian m. carlson, Feb 7, 2026
  113. 10/16 hash: expose hash context functions to Rustbrian m. carlson, Feb 7, 2026
  114. 05/16 rust: add a hash algorithm abstractionbrian m. carlson, Feb 7, 2026
  115. 09/16 write-or-die: add an fsync component for the object mapbrian m. carlson, Feb 7, 2026
  116. 13/16 rust: add functionality to hash an objectbrian m. carlson, Feb 7, 2026
  117. 15/16 rust: add a small wrapper around the hashfile codebrian m. carlson, Feb 7, 2026
  118. 16/16 object-file-convert: always make sure object ID algo is validbrian m. carlson, Feb 7, 2026

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.