git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 3/6] add tests for indexing packs with delta cycles

From
Jeff King <peff@peff.net>
Date
Aug 22, 2013, 23:14 UTC
Message-ID
<20130822231440.GC17060@sigill.intra.peff.net>
In-Reply-To
<20130822231215.GA16978@sigill.intra.peff.net>

If we receive a broken or malicious pack from a remote, we will feed it to index-pack. As index-pack processes the objects as a stream, reconstructing and hashing each object to get its name, it is not very susceptible to doing the wrong with bad data (it simply notices that the data is bogus and aborts).

However, one question raised on the list is whether it could be susceptible to problems during the delta-resolution phase. In particular, can a cycle in the packfile deltas cause us to go into an infinite loop or cause any other problem?

The answer is no.

We cannot have a cycle of delta-base offsets, because they go only in one direction (the OFS_DELTA object mentions its base by an offset towards the beginning of the file, and we explicitly reject negative offsets).

We can have a cycle of REF_DELTA objects, which refer to base objects by sha1 name. However, index-pack does not know these sha1 names ahead of time; it has to reconstruct the objects to get their names, and it cannot do so if there is a delta cycle (in other words, it does not even realize there is a cycle, but only that there are items that cannot be resolved).

Even though we can reason out that index-pack should handle this fine, let's add a few tests to make sure it behaves correctly.

Signed-off-by: Jeff King <peff@peff.net>
---
 t/lib-pack.sh                | 22 +++++++++++++++++
 t/t5309-pack-delta-cycles.sh | 59 ++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 81 insertions(+)
 create mode 100755 t/t5309-pack-delta-cycles.sh
diff --git a/t/lib-pack.sh b/t/lib-pack.sh
index 61c5d19..e028c40 100644
--- a/t/lib-pack.sh
+++ b/t/lib-pack.sh
@@ -55,6 +55,28 @@ pack_obj() {
 			printf '\062\170\234\143\267\3\0\0\116\0\106'
 			return
 			;;
+		01d7713666f4de822776c7622c10f1b07de280dc)
+			printf '\165\1\327\161\66\146\364\336\202\47\166' &&
+			printf '\307\142\54\20\361\260\175\342\200\334\170' &&
+			printf '\234\143\142\142\142\267\003\0\0\151\0\114'
+			return
+			;;
+		esac
+		;;
+
+	# blob containing "\7\0"
+	01d7713666f4de822776c7622c10f1b07de280dc)
+		case "$2" in
+		'')
+			printf '\062\170\234\143\147\0\0\0\20\0\10'
+			return
+			;;
+		e68fe8129b546b101aee9510c5328e7f21ca1d18)
+			printf '\165\346\217\350\22\233\124\153\20\32\356' &&
+			printf '\225\20\305\62\216\177\41\312\35\30\170\234' &&
+			printf '\143\142\142\142\147\0\0\0\53\0\16'
+			return
+			;;
 		esac
 		;;
 	esac
diff --git a/t/t5309-pack-delta-cycles.sh b/t/t5309-pack-delta-cycles.sh
new file mode 100755
index 0000000..9b3f2c3
--- /dev/null
+++ b/t/t5309-pack-delta-cycles.sh
@@ -0,0 +1,59 @@
+#!/bin/sh
+
+test_description='test index-pack handling of delta cycles in packfiles'
+. ./test-lib.sh
+. ../lib-pack.sh
+
+# Two similar-ish objects that we have computed deltas between.
+A=01d7713666f4de822776c7622c10f1b07de280dc
+B=e68fe8129b546b101aee9510c5328e7f21ca1d18
+
+# double-check our hand-constucted packs
+test_expect_success 'index-pack works with a single delta (A->B)' '
+	clear_packs &&
+	{
+		pack_header 2 &&
+		pack_obj $A $B &&
+		pack_obj $B
+	} >ab.pack &&
+	pack_trailer ab.pack &&
+	git index-pack --stdin <ab.pack &&
+	git cat-file -t $A &&
+	git cat-file -t $B
+'
+
+test_expect_success 'index-pack works with a single delta (B->A)' '
+	clear_packs &&
+	{
+		pack_header 2 &&
+		pack_obj $A &&
+		pack_obj $B $A
+	} >ba.pack &&
+	pack_trailer ba.pack &&
+	git index-pack --stdin <ba.pack &&
+	git cat-file -t $A &&
+	git cat-file -t $B
+'
+
+test_expect_success 'index-pack detects missing base objects' '
+	clear_packs &&
+	{
+		pack_header 1 &&
+		pack_obj $A $B
+	} >missing.pack &&
+	pack_trailer missing.pack &&
+	test_must_fail git index-pack --fix-thin --stdin <missing.pack
+'
+
+test_expect_success 'index-pack detects REF_DELTA cycles' '
+	clear_packs &&
+	{
+		pack_header 2 &&
+		pack_obj $A $B &&
+		pack_obj $B $A
+	} >cycle.pack &&
+	pack_trailer cycle.pack &&
+	test_must_fail git index-pack --fix-thin --stdin <cycle.pack
+'
+
+test_done
-- 
1.8.4.rc2.28.g6bb5f3f
Previous: Jeff KingNext: Jeff King
Message 27 of 37 in “duplicate objects in packfile”
  1. Jeff KingAug 14, 2013
  2. Junio C HamanoAug 14, 2013
  3. Junio C HamanoAug 14, 2013
  4. Nicolas PitreAug 14, 2013
  5. Jeff KingAug 16, 2013
  6. 0/4 duplicate objects in packfilesJeff King, Aug 21, 2013
  7. 1/4 sha1-lookup: handle duplicate keys with GIT_USE_LOOKUPJeff King, Aug 21, 2013
  8. 2/4 index-pack: optionally reject packs with duplicate objectsJeff King, Aug 21, 2013
  9. Duy NguyenAug 22, 2013
  10. Jeff KingAug 22, 2013
  11. 0/6 duplicate objects and delta cycles, oh my!Jeff King, Aug 22, 2013
  12. 1/6 test-sha1: add a binary output modeJeff King, Aug 22, 2013
  13. 2/6 sha1-lookup: handle duplicate keys with GIT_USE_LOOKUPJeff King, Aug 22, 2013
  14. Junio C HamanoAug 23, 2013
  15. Jeff KingAug 23, 2013
  16. Nicolas PitreAug 23, 2013
  17. Jeff KingAug 23, 2013
  18. 0/6 duplicate objects and delta cyclesJeff King, Aug 24, 2013
  19. 1/6 test-sha1: add a binary output modeJeff King, Aug 24, 2013
  20. 2/6 sha1-lookup: handle duplicate keys with GIT_USE_LOOKUPJeff King, Aug 24, 2013
  21. 3/6 add tests for indexing packs with delta cyclesJeff King, Aug 24, 2013
  22. 4/6 test index-pack on packs with recoverable delta cyclesJeff King, Aug 24, 2013
  23. 5/6 index-pack: optionally reject packs with duplicate objectsJeff King, Aug 24, 2013
  24. 6/6 default pack.indexDuplicates to falseJeff King, Aug 24, 2013
  25. Johannes SixtAug 23, 2013
  26. Jeff KingAug 23, 2013
  27. 3/6 add tests for indexing packs with delta cyclesJeff King, Aug 22, 2013
  28. 4/6 test index-pack on packs with recoverable delta cyclesJeff King, Aug 22, 2013
  29. 5/6 index-pack: optionally reject packs with duplicate objectsJeff King, Aug 22, 2013
  30. 6/6 default pack.indexDuplicates to falseJeff King, Aug 22, 2013
  31. Nicolas PitreAug 22, 2013
  32. 3/4 reject duplicates when indexing a packfile we createdJeff King, Aug 21, 2013
  33. 4/4 index-pack: optionally skip duplicate packfile entriesJeff King, Aug 21, 2013
  34. Junio C HamanoAug 21, 2013
  35. Jeff KingAug 22, 2013
  36. Junio C HamanoAug 21, 2013
  37. Nicolas PitreAug 14, 2013

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.