git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 6/6] default pack.indexDuplicates to false

From
Jeff King <peff@peff.net>
Date
Aug 24, 2013, 00:02 UTC
Message-ID
<20130824000245.GF25833@sigill.intra.peff.net>
In-Reply-To
<20130824000111.GA20255@sigill.intra.peff.net>

We should never see duplicate objects in packs, and it is unknown what kind of complications such packs could create during the repacking process. The previous commit introduced a safety valve for checking packs coming into the repository and being indexed by index-pack.

Let's turn the safety valve on by default. This helps protect sites receiving packfiles from potentially malicious strangers, and shouldn't affect normal use (and if it does, it will have helped us identify a buggy packfile writer). In a pinch, users can recover by turning off the option, or by resorting to unpack-objects to explode the pack.

Note that this also turns the option on for packs we write ourselves (e.g., via pack-objects, fast-import, or streaming large blobs). We do not expect maliciously constructed packfiles in these code paths, of course, but it can serve as an extra check that we have no accidentally created a buggy pack (and the check itself is cheap to perform).

Signed-off-by: Jeff King <peff@peff.net>
Acked-by: Nicolas Pitre <nico@fluxnic.net>
---
 pack-write.c                      | 1 -
 t/t5308-pack-detect-duplicates.sh | 9 ++++-----
 2 files changed, 4 insertions(+), 6 deletions(-)
diff --git a/pack-write.c b/pack-write.c
index da946a7..1e3c459 100644
--- a/pack-write.c
+++ b/pack-write.c
@@ -7,7 +7,6 @@ void reset_pack_idx_option(struct pack_idx_option *opts)
 	memset(opts, 0, sizeof(*opts));
 	opts->version = 2;
 	opts->off32_limit = 0x7fffffff;
-	opts->allow_duplicates = 1;
 }
 
 static int sha1_compare(const void *_a, const void *_b)
diff --git a/t/t5308-pack-detect-duplicates.sh b/t/t5308-pack-detect-duplicates.sh
index 97ce2e0..e0ba5e1 100755
--- a/t/t5308-pack-detect-duplicates.sh
+++ b/t/t5308-pack-detect-duplicates.sh
@@ -37,10 +37,10 @@ test_expect_success 'index-pack will allow duplicate objects by default' '
 	git index-pack --stdin <no-dups.pack
 '
 
-test_expect_success 'index-pack will allow duplicate objects by default' '
+test_expect_success 'index-pack will allow duplicate objects if asked' '
 	clear_packs &&
 	create_pack dups.pack 100 &&
-	git index-pack --stdin <dups.pack
+	git -c pack.indexDuplicates=true index-pack --stdin <dups.pack
 '
 
 test_expect_success 'create batch-check test vectors' '
@@ -70,11 +70,10 @@ test_expect_success 'index-pack can reject packs with duplicates' '
 	test_cmp expect actual
 '
 
-test_expect_success 'index-pack can reject packs with duplicates' '
+test_expect_success 'index-pack rejects packs with duplicates by default' '
 	clear_packs &&
 	create_pack dups.pack 2 &&
-	test_must_fail \
-		git -c pack.indexDuplicates=0 index-pack --stdin <dups.pack &&
+	test_must_fail git index-pack --stdin <dups.pack &&
 	test_expect_code 1 git cat-file -e $LO_SHA1
 '
 
-- 
1.8.4.rc2.28.g6bb5f3f
Previous: Jeff KingNext: Johannes Sixt
Message 24 of 37 in “duplicate objects in packfile”
  1. Jeff KingAug 14, 2013
  2. Junio C HamanoAug 14, 2013
  3. Junio C HamanoAug 14, 2013
  4. Nicolas PitreAug 14, 2013
  5. Jeff KingAug 16, 2013
  6. 0/4 duplicate objects in packfilesJeff King, Aug 21, 2013
  7. 1/4 sha1-lookup: handle duplicate keys with GIT_USE_LOOKUPJeff King, Aug 21, 2013
  8. 2/4 index-pack: optionally reject packs with duplicate objectsJeff King, Aug 21, 2013
  9. Duy NguyenAug 22, 2013
  10. Jeff KingAug 22, 2013
  11. 0/6 duplicate objects and delta cycles, oh my!Jeff King, Aug 22, 2013
  12. 1/6 test-sha1: add a binary output modeJeff King, Aug 22, 2013
  13. 2/6 sha1-lookup: handle duplicate keys with GIT_USE_LOOKUPJeff King, Aug 22, 2013
  14. Junio C HamanoAug 23, 2013
  15. Jeff KingAug 23, 2013
  16. Nicolas PitreAug 23, 2013
  17. Jeff KingAug 23, 2013
  18. 0/6 duplicate objects and delta cyclesJeff King, Aug 24, 2013
  19. 1/6 test-sha1: add a binary output modeJeff King, Aug 24, 2013
  20. 2/6 sha1-lookup: handle duplicate keys with GIT_USE_LOOKUPJeff King, Aug 24, 2013
  21. 3/6 add tests for indexing packs with delta cyclesJeff King, Aug 24, 2013
  22. 4/6 test index-pack on packs with recoverable delta cyclesJeff King, Aug 24, 2013
  23. 5/6 index-pack: optionally reject packs with duplicate objectsJeff King, Aug 24, 2013
  24. 6/6 default pack.indexDuplicates to falseJeff King, Aug 24, 2013
  25. Johannes SixtAug 23, 2013
  26. Jeff KingAug 23, 2013
  27. 3/6 add tests for indexing packs with delta cyclesJeff King, Aug 22, 2013
  28. 4/6 test index-pack on packs with recoverable delta cyclesJeff King, Aug 22, 2013
  29. 5/6 index-pack: optionally reject packs with duplicate objectsJeff King, Aug 22, 2013
  30. 6/6 default pack.indexDuplicates to falseJeff King, Aug 22, 2013
  31. Nicolas PitreAug 22, 2013
  32. 3/4 reject duplicates when indexing a packfile we createdJeff King, Aug 21, 2013
  33. 4/4 index-pack: optionally skip duplicate packfile entriesJeff King, Aug 21, 2013
  34. Junio C HamanoAug 21, 2013
  35. Jeff KingAug 22, 2013
  36. Junio C HamanoAug 21, 2013
  37. Nicolas PitreAug 14, 2013

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.