git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v2 2/3] pack-bitmap.c: avoid uninitialized `pack_int_id` during reuse

From
Taylor Blau <me@ttaylorr.com>
Date
Jun 10, 2024, 20:10 UTC
Message-ID
<a3c28f12020712818060bd50d1483507c9b11556.1718050244.git.me@ttaylorr.com>
In-Reply-To
<cover.1718050244.git.me@ttaylorr.com>

When performing multi-pack reuse, reuse_partial_packfile_from_bitmap() is responsible for generating an array of bitmapped_pack structs from which to perform reuse.

In the multi-pack case, we loop over the MIDXs packs and copy the result of calling `nth_bitmapped_pack()` to construct the list of reusable paths.

But we may also want to do pack-reuse over a single pack, either because we only had one pack to perform reuse over (in the case of single-pack bitmaps), or because we explicitly asked to do single pack reuse even with a MIDX[^1].

When this is the case, the array we generate of reusable packs contains only a single element, which is either (a) the pack attached to the single-pack bitmap, or (b) the MIDX's preferred pack.

In 795006fff4 (pack-bitmap: gracefully handle missing BTMP chunks, 2024-04-15), we refactored the reuse_partial_packfile_from_bitmap() function and stopped assigning the pack_int_id field when reusing only the MIDX's preferred pack. This results in an uninitialized read down in try_partial_reuse() like so:

    ==7474==WARNING: MemorySanitizer: use-of-uninitialized-value
    #0 0x55c5cd191dde in try_partial_reuse pack-bitmap.c:1887:8
    #1 0x55c5cd191dde in reuse_partial_packfile_from_bitmap_1 pack-bitmap.c:2001:8
    #2 0x55c5cd191dde in reuse_partial_packfile_from_bitmap pack-bitmap.c:2105:3
    #3 0x55c5cce0bd0e in get_object_list_from_bitmap builtin/pack-objects.c:4043:3
    #4 0x55c5cce0bd0e in get_object_list builtin/pack-objects.c:4156:27
    #5 0x55c5cce0bd0e in cmd_pack_objects builtin/pack-objects.c:4596:3
    #6 0x55c5ccc8fac8 in run_builtin git.c:474:11

which happens when try_partial_reuse() tries to call midx_pair_to_pack_pos() when it tries to reject cross-pack deltas.

Avoid the uninitialized read by ensuring that the pack_int_id field is set in the single-pack reuse case by setting it to either the MIDX preferred pack's pack_int_id, or '0', in the case of single-pack bitmaps. In the latter case, we never read the pack_int_id field, so the choice of '0' is arbitrary.

Guard against further regressions in this area by adding a test which ensures that we do not throw out deltas from the preferred pack as "cross-pack" due to an uninitialized pack_int_id.

[^1]: This can happen for a couple of reasons, either because the
  repository is configured with 'pack.allowPackReuse=(true|single)', or
  because the MIDX was generated prior to the introduction of the BTMP
  chunk, which contains information necessary to perform multi-pack
  reuse.
Reported-by: Kyle Lippincott <spectral@google.com>
Signed-off-by: Taylor Blau <me@ttaylorr.com>
---
 pack-bitmap.c               | 10 ++++++++++
 t/t5332-multi-pack-reuse.sh | 26 ++++++++++++++++++++++++++
 2 files changed, 36 insertions(+)
diff --git a/pack-bitmap.c b/pack-bitmap.c
index fe8e8a51d3..8b9a2c698f 100644
--- a/pack-bitmap.c
+++ b/pack-bitmap.c
@@ -2073,6 +2073,7 @@ void reuse_partial_packfile_from_bitmap(struct bitmap_index *bitmap_git,
 		QSORT(packs, packs_nr, bitmapped_pack_cmp);
 	} else {
 		struct packed_git *pack;
+		uint32_t pack_int_id;
 
 		if (bitmap_is_midx(bitmap_git)) {
 			uint32_t preferred_pack_pos;
@@ -2083,12 +2084,21 @@ void reuse_partial_packfile_from_bitmap(struct bitmap_index *bitmap_git,
 			}
 
 			pack = bitmap_git->midx->packs[preferred_pack_pos];
+			pack_int_id = preferred_pack_pos;
 		} else {
 			pack = bitmap_git->pack;
+			/*
+			 * Any value for 'pack_int_id' will do here. When we
+			 * process the pack via try_partial_reuse(), we won't
+			 * use the `pack_int_id` field since we have a non-MIDX
+			 * bitmap.
+			 */
+			pack_int_id = 0;
 		}
 
 		ALLOC_GROW(packs, packs_nr + 1, packs_alloc);
 		packs[packs_nr].p = pack;
+		packs[packs_nr].pack_int_id = pack_int_id;
 		packs[packs_nr].bitmap_nr = pack->num_objects;
 		packs[packs_nr].bitmap_pos = 0;
 
diff --git a/t/t5332-multi-pack-reuse.sh b/t/t5332-multi-pack-reuse.sh
index 3c20738bce..ed823f37bc 100755
--- a/t/t5332-multi-pack-reuse.sh
+++ b/t/t5332-multi-pack-reuse.sh
@@ -204,4 +204,30 @@ test_expect_success 'omit delta from uninteresting base (cross pack)' '
 	test_pack_objects_reused_all $(($objects_nr - 1)) $packs_nr
 '
 
+test_expect_success 'non-omitted delta in MIDX preferred pack' '
+	test_config pack.allowPackReuse single &&
+
+	cat >p1.objects <<-EOF &&
+	$(git rev-parse $base)
+	^$(git rev-parse $delta^)
+	EOF
+	cat >p2.objects <<-EOF &&
+	$(git rev-parse F)
+	EOF
+
+	p1="$(git pack-objects --revs $packdir/pack <p1.objects)" &&
+	p2="$(git pack-objects --revs $packdir/pack <p2.objects)" &&
+
+	cat >in <<-EOF &&
+	pack-$p1.idx
+	pack-$p2.idx
+	EOF
+	git multi-pack-index write --bitmap --stdin-packs \
+		--preferred-pack=pack-$p1.pack <in &&
+
+	git show-index <$packdir/pack-$p1.idx >expect &&
+
+	test_pack_objects_reused_all $(wc -l <expect) 1
+'
+
 test_done
-- 
2.45.2.447.g6b4ffca7ec.dirty
Previous: Taylor BlauNext: Jeff King
Message 7 of 15 in “pack-bitmap.c: avoid uninitialized `pack_int_id` during reuse”
  1. pack-bitmap.c: avoid uninitialized `pack_int_id` during reuseTaylor Blau, Jun 9, 2024
  2. Patrick SteinhardtJun 10, 2024
  3. Taylor BlauJun 10, 2024
  4. Patrick SteinhardtJun 11, 2024
  5. 0/3 midx: various brown paper bag fixesTaylor Blau, Jun 10, 2024
  6. 1/3 midx-write.c: do not read existing MIDX with `packs_to_include`Taylor Blau, Jun 10, 2024
  7. 2/3 pack-bitmap.c: avoid uninitialized `pack_int_id` during reuseTaylor Blau, Jun 10, 2024
  8. Jeff KingJun 11, 2024
  9. Junio C HamanoJun 11, 2024
  10. 3/3 pack-revindex.c: guard against out-of-bounds pack lookupsTaylor Blau, Jun 10, 2024
  11. 0/3 midx: various brown paper bag fixesTaylor Blau, Jun 11, 2024
  12. 1/3 midx-write.c: do not read existing MIDX with `packs_to_include`Taylor Blau, Jun 11, 2024
  13. 2/3 pack-bitmap.c: avoid uninitialized `pack_int_id` during reuseTaylor Blau, Jun 11, 2024
  14. 3/3 pack-revindex.c: guard against out-of-bounds pack lookupsTaylor Blau, Jun 11, 2024
  15. Taylor BlauJun 11, 2024

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.