git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v3 0/7] pack-bitmap: permute existing namehash values

From
Taylor Blau <me@ttaylorr.com>
Date
Sep 17, 2021, 21:21 UTC
Message-ID
<cover.1631913631.git.me@ttaylorr.com>
In-Reply-To
<cover.1631049462.git.me@ttaylorr.com>

Here is a very small reroll of my series to permute values from an existing hash-cache when generating multi-pack reachability bitmaps.

The only changes since last time are a slightly modified version of the performance tests which don't time generating the pack-bitmap. And the new test in t5326 no longer has a redundant `git config pack.writeBitmapHashCache` since it is true by default.

It is based on the `tb/multi-pack-bitmaps` topic, which graduated to master. Thanks in advance for your review on this hopefully-final iteration :-).

Taylor Blau (7):
  t/helper/test-bitmap.c: add 'dump-hashes' mode
  pack-bitmap.c: propagate namehash values from existing bitmaps
  midx.c: respect 'pack.writeBitmapHashcache' when writing bitmaps
  p5326: create missing 'perf-tag' tag
  p5326: don't set core.multiPackIndex unnecessarily
  p5326: generate pack bitmaps before writing the MIDX bitmap
  t5326: test propagating hashcache values
 Documentation/config/pack.txt      |  4 +++
 builtin/multi-pack-index.c         | 21 +++++++++++++++
 midx.c                             |  6 ++++-
 midx.h                             |  1 +
 pack-bitmap.c                      | 41 +++++++++++++++++++++++++-----
 pack-bitmap.h                      |  1 +
 t/helper/test-bitmap.c             | 10 +++++++-
 t/perf/p5326-multi-pack-bitmaps.sh | 15 ++++++++---
 t/t5326-multi-pack-bitmaps.sh      | 30 ++++++++++++++++++++++
 9 files changed, 118 insertions(+), 11 deletions(-)
Range-diff against v2:
1:  4f2b8d9530 = 1:  4f2b8d9530 t/helper/test-bitmap.c: add 'dump-hashes' mode
2:  2cd2f3aa5e = 2:  2cd2f3aa5e pack-bitmap.c: propagate namehash values from existing bitmaps
3:  f0d8f106c2 = 3:  f0d8f106c2 midx.c: respect 'pack.writeBitmapHashcache' when writing bitmaps
4:  a8c6e845ad = 4:  a8c6e845ad p5326: create missing 'perf-tag' tag
5:  191922c8f2 = 5:  191922c8f2 p5326: don't set core.multiPackIndex unnecessarily
6:  040bb40548 ! 6:  59b6914ef8 p5326: generate pack bitmaps before writing the MIDX bitmap
    @@ Commit message
         we have to rely on a single-pack bitmap to generate those values for us.
         Therefore, generate a pack bitmap before the MIDX one in order to ensure
    -    that the MIDX bitmap has entries in its hash-cache.
    +    that the MIDX bitmap has entries in its hash-cache. Since we don't want
    +    to time generating the pack bitmap, move that to a non-perf test run
    +    before we try to generate the MIDX bitmap.
    +
    +    Likewise, get rid of the pack bitmap afterwords, to make certain that we
    +    are not accidentally using it in the performance tests run later on.
         Signed-off-by: Taylor Blau <me@ttaylorr.com>
      ## t/perf/p5326-multi-pack-bitmaps.sh ##
     @@ t/perf/p5326-multi-pack-bitmaps.sh: test_expect_success 'create tags' '
    + 	git tag --message="tag pointing to HEAD" perf-tag HEAD
      '
    ++test_expect_success 'start with bitmapped pack' '
    ++	git repack -adb
    ++'
    ++
      test_perf 'setup multi-pack index' '
     -	git repack -ad &&
    -+	git repack -adb &&
      	git multi-pack-index write --bitmap
      '
    ++test_expect_success 'drop pack bitmap' '
    ++	rm -f .git/objects/pack/pack-*.bitmap
    ++'
    ++
    + test_full_bitmap
    +
    + test_expect_success 'create partial bitmap state' '
7:  fdf71432b3 ! 7:  bb16125915 t5326: test propagating hashcache values
    @@ t/t5326-multi-pack-bitmaps.sh: test_expect_success 'pack.preferBitmapTips' '
     +	(
     +		cd repo &&
     +
    -+		git config pack.writeBitmapHashCache true &&
    -+
     +		test_commit base &&
     +		test_commit base2 &&
     +		git repack -adb &&
--
2.33.0.96.g73915697e6
Previous: Jeff KingNext: Taylor Blau
Message 47 of 55 in “pack-bitmap: permute existing namehash values”
  1. 0/4 pack-bitmap: permute existing namehash valuesTaylor Blau, Sep 7, 2021
  2. 1/4 t/helper/test-bitmap.c: add 'dump-hashes' modeTaylor Blau, Sep 7, 2021
  3. Ævar Arnfjörð BjarmasonSep 8, 2021
  4. Taylor BlauSep 8, 2021
  5. 2/4 pack-bitmap.c: propagate namehash values from existing bitmapsTaylor Blau, Sep 7, 2021
  6. 3/4 midx.c: respect 'pack.writeBitmapHashcache' when writing bitmapsTaylor Blau, Sep 7, 2021
  7. Ævar Arnfjörð BjarmasonSep 8, 2021
  8. Taylor BlauSep 8, 2021
  9. Ævar Arnfjörð BjarmasonSep 9, 2021
  10. Ævar Arnfjörð BjarmasonSep 9, 2021
  11. Taylor BlauSep 9, 2021
  12. Ævar Arnfjörð BjarmasonSep 9, 2021
  13. Taylor BlauSep 9, 2021
  14. Taylor BlauSep 9, 2021
  15. Junio C HamanoSep 13, 2021
  16. Taylor BlauSep 14, 2021
  17. 4/4 t5326: test propagating hashcache valuesTaylor Blau, Sep 7, 2021
  18. Ævar Arnfjörð BjarmasonSep 8, 2021
  19. Taylor BlauSep 8, 2021
  20. Ævar Arnfjörð BjarmasonSep 17, 2021
  21. Taylor BlauSep 17, 2021
  22. Ævar Arnfjörð BjarmasonSep 17, 2021
  23. Junio C HamanoSep 13, 2021
  24. Taylor BlauSep 14, 2021
  25. Junio C HamanoSep 14, 2021
  26. Taylor BlauSep 14, 2021
  27. Taylor BlauSep 14, 2021
  28. Jeff KingSep 14, 2021
  29. Taylor BlauSep 14, 2021
  30. Jeff KingSep 14, 2021
  31. Junio C HamanoSep 14, 2021
  32. 0/7 pack-bitmap: permute existing namehash valuesTaylor Blau, Sep 14, 2021
  33. 1/7 t/helper/test-bitmap.c: add 'dump-hashes' modeTaylor Blau, Sep 14, 2021
  34. 2/7 pack-bitmap.c: propagate namehash values from existing bitmapsTaylor Blau, Sep 14, 2021
  35. 3/7 midx.c: respect 'pack.writeBitmapHashcache' when writing bitmapsTaylor Blau, Sep 14, 2021
  36. 4/7 p5326: create missing 'perf-tag' tagTaylor Blau, Sep 14, 2021
  37. Jeff KingSep 16, 2021
  38. Taylor BlauSep 17, 2021
  39. 5/7 p5326: don't set core.multiPackIndex unnecessarilyTaylor Blau, Sep 14, 2021
  40. Jeff KingSep 16, 2021
  41. 6/7 p5326: generate pack bitmaps before writing the MIDX bitmapTaylor Blau, Sep 14, 2021
  42. Jeff KingSep 16, 2021
  43. Taylor BlauSep 17, 2021
  44. 7/7 t5326: test propagating hashcache valuesTaylor Blau, Sep 14, 2021
  45. Jeff KingSep 16, 2021
  46. Jeff KingSep 16, 2021
  47. 0/7 pack-bitmap: permute existing namehash valuesTaylor Blau, Sep 17, 2021
  48. 1/7 t/helper/test-bitmap.c: add 'dump-hashes' modeTaylor Blau, Sep 17, 2021
  49. 2/7 pack-bitmap.c: propagate namehash values from existing bitmapsTaylor Blau, Sep 17, 2021
  50. 3/7 midx.c: respect 'pack.writeBitmapHashcache' when writing bitmapsTaylor Blau, Sep 17, 2021
  51. 4/7 p5326: create missing 'perf-tag' tagTaylor Blau, Sep 17, 2021
  52. 5/7 p5326: don't set core.multiPackIndex unnecessarilyTaylor Blau, Sep 17, 2021
  53. 6/7 p5326: generate pack bitmaps before writing the MIDX bitmapTaylor Blau, Sep 17, 2021
  54. 7/7 t5326: test propagating hashcache valuesTaylor Blau, Sep 17, 2021
  55. Jeff KingSep 17, 2021

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.