git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 2/4] t5310: test delta reuse with bitmaps

From
Jeff King <peff@peff.net>
Date
Sep 1, 2018, 07:48 UTC
Message-ID
<20180901074813.GB25461@sigill.intra.peff.net>
In-Reply-To
<20180901074145.GA24023@sigill.intra.peff.net>

Commit 6a1e32d532 (pack-objects: reuse on-disk deltas for thin "have" objects, 2018-08-21) taught pack-objects a new optimization trick. Since this wasn't meant to change user-visible behavior, but only produce smaller packs more quickly, testing focused on t/perf/p5311.

However, since people don't run perf tests very often, we should make sure that the feature is exercised in the regular test suite. This patch does so.

Signed-off-by: Jeff King <peff@peff.net>
---
Side note: If we do squash patch 1 into the eariler series, that will
invalidate the commit id mentioned in the commit message here.
 t/t5310-pack-bitmaps.sh | 93 +++++++++++++++++++++++++++++++++++++++++
 1 file changed, 93 insertions(+)
diff --git a/t/t5310-pack-bitmaps.sh b/t/t5310-pack-bitmaps.sh
index 557bd0d0c0..af38776054 100755
--- a/t/t5310-pack-bitmaps.sh
+++ b/t/t5310-pack-bitmaps.sh
@@ -342,4 +342,97 @@ test_expect_success 'truncated bitmap fails gracefully' '
 	test_i18ngrep corrupt stderr
 '
 
+# have_delta <obj> <expected_base>
+#
+# Note that because this relies on cat-file, it might find _any_ copy of an
+# object in the repository. The caller is responsible for making sure
+# there's only one (e.g., via "repack -ad", or having just fetched a copy).
+have_delta () {
+	echo $2 >expect &&
+	echo $1 | git cat-file --batch-check="%(deltabase)" >actual &&
+	test_cmp expect actual
+}
+
+# Create a state of history with these properties:
+#
+#  - refs that allow a client to fetch some new history, while sharing some old
+#    history with the server; we use branches delta-reuse-old and
+#    delta-reuse-new here
+#
+#  - the new history contains an object that is stored on the server as a delta
+#    against a base that is in the old history
+#
+#  - the base object is not immediately reachable from the tip of the old
+#    history; finding it would involve digging down through history we know the
+#    other side has
+#
+# This should result in a state where fetching from old->new would not
+# traditionally reuse the on-disk delta (because we'd have to dig to realize
+# that the client has it), but we will do so if bitmaps can tell us cheaply
+# that the other side has it.
+test_expect_success 'set up thin delta-reuse parent' '
+	# This first commit contains the buried base object.
+	test-tool genrandom delta 16384 >file &&
+	git add file &&
+	git commit -m "delta base" &&
+	base=$(git rev-parse --verify HEAD:file) &&
+
+	# These intermediate commits bury the base back in history.
+	# This becomes the "old" state.
+	for i in 1 2 3 4 5
+	do
+		echo $i >file &&
+		git commit -am "intermediate $i" || return 1
+	done &&
+	git branch delta-reuse-old &&
+
+	# And now our new history has a delta against the buried base. Note
+	# that this must be smaller than the original file, since pack-objects
+	# prefers to create deltas from smaller objects to larger.
+	test-tool genrandom delta 16300 >file &&
+	git commit -am "delta result" &&
+	delta=$(git rev-parse --verify HEAD:file) &&
+	git branch delta-reuse-new &&
+
+	# Repack with bitmaps and double check that we have the expected delta
+	# relationship.
+	git repack -adb &&
+	have_delta $delta $base
+'
+
+# Now we can sanity-check the non-bitmap behavior (that the server is not able
+# to reuse the delta). This isn't strictly something we care about, so this
+# test could be scrapped in the future. But it makes sure that the next test is
+# actually triggering the feature we want.
+#
+# Note that our tools for working with on-the-wire "thin" packs are limited. So
+# we actually perform the fetch, retain the resulting pack, and inspect the
+# result.
+test_expect_success 'fetch without bitmaps ignores delta against old base' '
+	test_config pack.usebitmaps false &&
+	test_when_finished "rm -rf client.git" &&
+	git init --bare client.git &&
+	(
+		cd client.git &&
+		git config transfer.unpackLimit 1 &&
+		git fetch .. delta-reuse-old:delta-reuse-old &&
+		git fetch .. delta-reuse-new:delta-reuse-new &&
+		have_delta $delta $ZERO_OID
+	)
+'
+
+# And do the same for the bitmap case, where we do expect to find the delta.
+test_expect_success 'fetch with bitmaps can reuse old base' '
+	test_config pack.usebitmaps true &&
+	test_when_finished "rm -rf client.git" &&
+	git init --bare client.git &&
+	(
+		cd client.git &&
+		git config transfer.unpackLimit 1 &&
+		git fetch .. delta-reuse-old:delta-reuse-old &&
+		git fetch .. delta-reuse-new:delta-reuse-new &&
+		have_delta $delta $base
+	)
+'
+
 test_done
-- 
2.19.0.rc1.549.g6ce4dc0f08
Previous: Jeff KingNext: Jeff King
Message 16 of 53 in “test-tool.h: include git-compat-util.h”
  1. test-tool.h: include git-compat-util.hJeff King, Aug 21, 2018
  2. Junio C HamanoAug 21, 2018
  3. 1/6 t/perf: factor boilerplate out of test_perfJeff King, Aug 21, 2018
  4. 2/6 t/perf: factor out percent calculationsJeff King, Aug 21, 2018
  5. 3/6 t/perf: add infrastructure for measuring sizesJeff King, Aug 21, 2018
  6. Derrick StoleeAug 22, 2018
  7. Jeff KingAug 22, 2018
  8. 4/6 t/perf: add perf tests for fetches from a bitmapped serverJeff King, Aug 21, 2018
  9. 5/6 pack-bitmap: save "have" bitmap from walkJeff King, Aug 21, 2018
  10. Derrick StoleeAug 21, 2018
  11. Jeff KingAug 21, 2018
  12. Ævar Arnfjörð BjarmasonAug 31, 2018
  13. Jeff KingAug 31, 2018
  14. 0/4 un-breaking pack-objects with bitmapsJeff King, Sep 1, 2018
  15. 1/4 bitmap_has_sha1_in_uninteresting(): drop BUG checkJeff King, Sep 1, 2018
  16. 2/4 t5310: test delta reuse with bitmapsJeff King, Sep 1, 2018
  17. Jeff KingSep 1, 2018
  18. Ævar Arnfjörð BjarmasonSep 1, 2018
  19. Ben PeartSep 1, 2018
  20. Jeff KingSep 2, 2018
  21. Stefan BellerSep 4, 2018
  22. Junio C HamanoSep 4, 2018
  23. Jeff KingSep 4, 2018
  24. 3/4 traverse_bitmap_commit_list(): don't free resultJeff King, Sep 1, 2018
  25. 4/4 pack-bitmap: drop "loaded" flagJeff King, Sep 1, 2018
  26. Stefan BellerSep 4, 2018
  27. Jeff KingSep 4, 2018
  28. Ævar Arnfjörð BjarmasonSep 8, 2018
  29. Junio C HamanoSep 10, 2018
  30. Jeff KingSep 10, 2018
  31. Junio C HamanoSep 10, 2018
  32. 6/6 pack-objects: reuse on-disk deltas for thin "have" objectsJeff King, Aug 21, 2018
  33. Junio C HamanoAug 21, 2018
  34. Junio C HamanoAug 21, 2018
  35. Jeff KingAug 21, 2018
  36. Jeff KingAug 21, 2018
  37. Junio C HamanoAug 21, 2018
  38. Jeff KingAug 21, 2018
  39. Junio C HamanoAug 21, 2018
  40. Jeff KingAug 21, 2018
  41. 0/9 trailer-parsing false positivesJeff King, Aug 23, 2018
  42. 1/9 trailer: use size_t for string offsetsJeff King, Aug 23, 2018
  43. 2/9 trailer: use size_t for iterating trailer listJeff King, Aug 23, 2018
  44. 3/9 trailer: pass process_trailer_opts to trailer_info_get()Jeff King, Aug 23, 2018
  45. 4/9 interpret-trailers: tighten check for "---" patch boundaryJeff King, Aug 23, 2018
  46. 5/9 interpret-trailers: allow suppressing "---" dividerJeff King, Aug 23, 2018
  47. 6/9 pretty, ref-filter: format %(trailers) with no_divider optionJeff King, Aug 23, 2018
  48. 7/9 sequencer: ignore "---" divider when parsing trailersJeff King, Aug 23, 2018
  49. 8/9 append_signoff: use size_t for string offsetsJeff King, Aug 23, 2018
  50. 9/9 sequencer: handle ignore_footer when parsing trailersJeff King, Aug 23, 2018
  51. Junio C HamanoAug 23, 2018
  52. Jeff KingAug 24, 2018
  53. Jeff KingAug 21, 2018

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.