git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v2 08/13] object-file: handle compatibility algorithm

From
Christian Couder <christian.couder@gmail.com>
Date
Oct 23, 2026, 07:25 UTC
Message-ID
<20261023072559.3204273-9-christian.couder@gmail.com>
In-Reply-To
<20261023072559.3204273-1-christian.couder@gmail.com>
From: "brian m. carlson" <sandals@crustytoothpaste.net>

When we check in a large blob, we stream it directly to a pack and then write an index. However, in compatibility mode, we also need to make sure to compute the object ID for the compatibility algorithm as well as the main algorithm or we will incorrectly store the null OID in the index we create. Do so here to avoid corrupting the repository.

Add a test that adds a large file in a repository with a compatibility algorithm.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Christian Couder <christian.couder@gmail.com>
---
 object-file.c                 | 18 ++++++++++++++++--
 t/t1016-compatObjectFormat.sh | 14 ++++++++++++++
 2 files changed, 30 insertions(+), 2 deletions(-)
diff --git a/object-file.c b/object-file.c
index c049bf13f7..b4c4a2ea0f 100644
--- a/object-file.c
+++ b/object-file.c
@@ -747,6 +747,7 @@ static int hash_stream(struct odb_stream *stream,
  */
 static void stream_to_pack(struct transaction_packfile *state,
 			   struct git_hash_ctx *ctx,
+			   struct git_hash_ctx *compat_ctx,
 			   struct odb_stream *stream)
 {
 	git_zstream s;
@@ -774,6 +775,9 @@ static void stream_to_pack(struct transaction_packfile *state,
 				is_finished = true;
 
 			git_hash_update(ctx, ibuf, rsize);
+			if (compat_ctx)
+				git_hash_update(compat_ctx,
+						ibuf, rsize);
 
 			s.next_in = ibuf;
 			s.avail_in = rsize;
@@ -882,16 +886,22 @@ static int odb_transaction_files_write_object_stream(struct odb_transaction *bas
 								 struct odb_transaction_files,
 								 base);
 	struct transaction_packfile *state = &transaction->packfile;
-	struct git_hash_ctx ctx;
+	struct git_hash_ctx ctx, compat_ctx;
+	struct object_id compat_oid;
 	unsigned char obuf[16384];
 	unsigned header_len;
 	struct hashfile_checkpoint checkpoint;
 	struct pack_idx_entry *idx;
+	const struct git_hash_algo *compat = transaction->base.source->odb->repo->compat_hash_algo;
 
 	header_len = format_object_header((char *)obuf, sizeof(obuf),
 					  stream->type, stream->size);
 	git_hash_init(&ctx, transaction->base.source->odb->repo->hash_algo);
 	git_hash_update(&ctx, obuf, header_len);
+	if (compat) {
+		git_hash_init(&compat_ctx, compat);
+		git_hash_update(&compat_ctx, obuf, header_len);
+	}
 
 	/*
 	 * If writing another object to the packfile could result in it
@@ -916,8 +926,10 @@ static int odb_transaction_files_write_object_stream(struct odb_transaction *bas
 	hashfile_checkpoint(state->f, &checkpoint);
 	idx->offset = state->offset;
 	crc32_begin(state->f);
-	stream_to_pack(state, &ctx, stream);
+	stream_to_pack(state, &ctx, compat ? &compat_ctx : NULL, stream);
 	git_hash_final_oid(result_oid, &ctx);
+	if (compat)
+		git_hash_final_oid(&compat_oid, &compat_ctx);
 
 	idx->crc32 = crc32_end(state->f);
 	if (already_written(transaction, result_oid)) {
@@ -926,6 +938,8 @@ static int odb_transaction_files_write_object_stream(struct odb_transaction *bas
 		free(idx);
 	} else {
 		oidcpy(&idx->oid, result_oid);
+		if (compat)
+			oidcpy(&idx->compat_oid, &compat_oid);
 		ALLOC_GROW(state->written,
 			   state->nr_written + 1,
 			   state->alloc_written);
diff --git a/t/t1016-compatObjectFormat.sh b/t/t1016-compatObjectFormat.sh
index 1560b6a2e2..af9532aa1f 100755
--- a/t/t1016-compatObjectFormat.sh
+++ b/t/t1016-compatObjectFormat.sh
@@ -228,6 +228,20 @@ do
 		test_copy_bytes 8 <v3.idx | test-tool hexdump >actual &&
 		test_cmp expect actual
 	'
+
+	test_expect_success 'add a large file streamed into a pack' '
+		test_when_finished "rm -rf large-repo" &&
+		create_compat_repo $hash large-repo &&
+		git -C large-repo config set core.bigFileThreshold 1k &&
+		test-tool genrandom large 5000 >large-repo/large &&
+		git -C large-repo add large &&
+		ls large-repo/.git/objects/pack/pack-*.pack &&
+		git -C large-repo hash-object large >expect &&
+		git -C large-repo rev-parse :large >actual &&
+		test_cmp expect actual &&
+		git -C large-repo cat-file blob :large >actual &&
+		test_cmp large-repo/large actual
+	'
 done
 cd "$base"
 
-- 
2.56.0.215.g47fdf2208c
Previous: Christian CouderNext: Christian Couder
Message 9 of 17 in “SHA-1/SHA-256 interoperability, part 3: pack index v3”
  1. 00/13 SHA-1/SHA-256 interoperability, part 3: pack index v3Christian Couder, Oct 23, 2026
  2. 01/13 loose: allow looking up in object map with sourceChristian Couder, Oct 23, 2026
  3. 02/13 loose: rename repo_add_loose_object_mapChristian Couder, Oct 23, 2026
  4. 03/13 loose: add a flag to control whether we write mapped OIDsChristian Couder, Oct 23, 2026
  5. 04/13 odb: implement interoperability functionsChristian Couder, Oct 23, 2026
  6. 05/13 object-file-convert: implement mapping in terms of object DBChristian Couder, Oct 23, 2026
  7. 06/13 pack-write: add support for pack index v3Christian Couder, Oct 23, 2026
  8. 07/13 builtin: populate pack index v3 fields when indexingChristian Couder, Oct 23, 2026
  9. 08/13 object-file: handle compatibility algorithmChristian Couder, Oct 23, 2026
  10. 09/13 packfile: add support for reading pack index v3Christian Couder, Oct 23, 2026
  11. 10/13 packfile: validate pack index v3 tablesChristian Couder, Oct 23, 2026
  12. 11/13 loose: map object IDs using pack index v3Christian Couder, Oct 23, 2026
  13. 12/13 packfile: allow looking up compat object IDs in packsChristian Couder, Oct 23, 2026
  14. 13/13 doc: move pack index v3 documentationChristian Couder, Oct 23, 2026
  15. Ben KnobleOct 23, 2026
  16. Junio C HamanoOct 23, 2026
  17. Christian CouderOct 27, 2026

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.