git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v2 5/8] t5326: extract `test_rev_exists`

From
Taylor Blau <me@ttaylorr.com>
Date
Jan 4, 2022, 15:33 UTC
Message-ID
<YdRo1B0Y+ZBvjspa@nand.local>
In-Reply-To
<dbc75e29-aceb-051a-8235-e5c912172aed@gmail.com>
On Mon, Dec 20, 2021 at 01:33:32PM -0500, Derrick Stolee wrote:
Show 26 quoted lines
> On 12/13/2021 8:55 PM, Taylor Blau wrote:
> > To determine which source of data is used for the MIDX's reverse index
> > cache, introduce a helper which forces loading the reverse index, and
> > then looks for the special trace2 event introduced in a previous commit.
> >
> > For now, this helper just looks for when the legacy MIDX .rev file was
> > loaded, but in a subsequent commit will become parameterized over the
> > the reverse index's source.
> >
> > This function replaces checking for the existence of the .rev file. We
> > could write a similar helper to ensure that the .rev file is cleaned up
> > after repacking, but it will make subsequent tests more difficult to
> > write, and provides marginal value since we already check that the MIDX
> > .bitmap file is removed.
>
> ...
>
> > +test_rev_exists () {
> > +	commit="$1"
> > +
> > +	test_expect_success 'reverse index exists' '
> > +		GIT_TRACE2_EVENT_NESTING=10 \
>
> Very recently, b8de3d6 (test-lib.sh: set GIT_TRACE2_EVENT_NESTING,
> 2021-11-29) made it to master and sets this to 100 across the test
> suite, so you don't need this line.

Thanks; I had written it with this in place in case this topic was going to reach master before the topic containing b8de3d6. But it didn't, and you're right that setting GIT_TRACE2_EVENT_NESTING explicitly here is no longer necessary.

Show 10 quoted lines
> > +		GIT_TRACE2_EVENT=$(pwd)/event.trace \
> > +			git rev-list --test-bitmap "$commit" &&
>
> This use of $commit has me worried. Do the tests become too flaky
> to changes in how we choose commits for the bitmaps? Does that
> require callers to be too aware of the refstate when creating the
> bitmaps?
>
> Perhaps just `git rev-list --use-bitmap-index [--objects] HEAD`
> would suffice to generate the trace event?

It's necessary for the group of tests which exercise partial bitmap coverage (see the "setup partial bitmaps" test and below for more). In those tests, we don't have bitmap coverage of HEAD, only HEAD~, so we need to be able to say things like:

    test_rev_exists HEAD~
since just asking for the tip isn't guaranteed to work always.

We could make the argument optional (i.e., `git rev-list --test-bitmap "${1:-HEAD}"`), but I generally find that it makes the callers more difficult to read rather than easier.

On the "--test-bitmap" vs "--objects" thing, I don't think it matters either way. They are both doing basically the same traversal, but "--test-bitmap" does some additional integrity checks on top.

Show 16 quoted lines
> >  test_expect_success 'create single-pack midx with bitmaps' '
> >  	git repack -ad &&
> >  	git multi-pack-index write --bitmap &&
> >  	test_path_is_file $midx &&
> > -	test_path_is_file $midx-$(midx_checksum $objdir).bitmap &&
> > -	test_path_is_file $midx-$(midx_checksum $objdir).rev
> > +	test_path_is_file $midx-$(midx_checksum $objdir).bitmap
> >  '
> >
> > +test_rev_exists HEAD
> > +
>
> Perhaps this helper would be more appropriate as a helper method
> within a test, rather than creating a test of its own? I think
> it looks better to include it next to the setup lines, something
> like

Eh, sure. I won't plan on picking this up, but I'm happy to if you feel strongly about it.

Thanks, Taylor

Previous: Derrick StoleeNext: Taylor Blau
Message 18 of 87 in “midx: prevent bitmap corruption when permuting pack order”
  1. 0/2 midx: prevent bitmap corruption when permuting pack orderTaylor Blau, Dec 8, 2021
  2. 1/2 t5326: demonstrate bitmap corruption after permutationTaylor Blau, Dec 8, 2021
  3. 2/2 midx.c: make changing the preferred pack safeTaylor Blau, Dec 8, 2021
  4. Derrick StoleeDec 8, 2021
  5. Jeff KingDec 8, 2021
  6. Taylor BlauDec 10, 2021
  7. Taylor BlauDec 10, 2021
  8. Taylor BlauDec 11, 2021
  9. Derrick StoleeDec 13, 2021
  10. Taylor BlauDec 13, 2021
  11. 0/8 midx: prevent bitmap corruption when permuting pack orderTaylor Blau, Dec 14, 2021
  12. 1/8 t5326: demonstrate bitmap corruption after permutationTaylor Blau, Dec 14, 2021
  13. 2/8 midx.c: make changing the preferred pack safeTaylor Blau, Dec 14, 2021
  14. 3/8 pack-revindex.c: instrument loading on-disk reverse indexTaylor Blau, Dec 14, 2021
  15. 4/8 t5326: drop unnecessary setupTaylor Blau, Dec 14, 2021
  16. 5/8 t5326: extract `test_rev_exists`Taylor Blau, Dec 14, 2021
  17. Derrick StoleeDec 20, 2021
  18. Taylor BlauJan 4, 2022
  19. 6/8 t5326: move tests to t/lib-bitmap.shTaylor Blau, Dec 14, 2021
  20. 7/8 t/lib-bitmap.sh: parameterize tests over reverse index sourceTaylor Blau, Dec 14, 2021
  21. 8/8 midx: read `RIDX` chunk when presentTaylor Blau, Dec 14, 2021
  22. Derrick StoleeDec 20, 2021
  23. Taylor BlauJan 4, 2022
  24. Junio C HamanoDec 15, 2021
  25. Taylor BlauDec 15, 2021
  26. Junio C HamanoDec 15, 2021
  27. Junio C HamanoDec 15, 2021
  28. Derrick StoleeDec 20, 2021
  29. Taylor BlauDec 20, 2021
  30. Derrick StoleeDec 20, 2021
  31. Junio C HamanoDec 15, 2021
  32. Taylor BlauDec 15, 2021
  33. 0/9 midx: prevent bitmap corruption when permuting pack orderTaylor Blau, Jan 4, 2022
  34. 1/9 t5326: demonstrate bitmap corruption after permutationTaylor Blau, Jan 4, 2022
  35. Jonathan TanJan 20, 2022
  36. Taylor BlauJan 20, 2022
  37. Junio C HamanoJan 20, 2022
  38. Taylor BlauJan 20, 2022
  39. Jonathan TanJan 24, 2022
  40. 2/9 midx.c: make changing the preferred pack safeTaylor Blau, Jan 4, 2022
  41. Junio C HamanoJan 14, 2022
  42. Junio C HamanoJan 14, 2022
  43. Taylor BlauJan 15, 2022
  44. Junio C HamanoJan 15, 2022
  45. Jonathan TanJan 20, 2022
  46. Taylor BlauJan 20, 2022
  47. 3/9 pack-revindex.c: instrument loading on-disk reverse indexTaylor Blau, Jan 4, 2022
  48. Jonathan TanJan 20, 2022
  49. Taylor BlauJan 20, 2022
  50. Jonathan TanJan 24, 2022
  51. 4/9 t5326: drop unnecessary setupTaylor Blau, Jan 4, 2022
  52. 5/9 t5326: extract `test_rev_exists`Taylor Blau, Jan 4, 2022
  53. 6/9 t5326: move tests to t/lib-bitmap.shTaylor Blau, Jan 4, 2022
  54. 7/9 t/lib-bitmap.sh: parameterize tests over reverse index sourceTaylor Blau, Jan 4, 2022
  55. Jonathan TanJan 24, 2022
  56. Taylor BlauJan 25, 2022
  57. Jonathan TanJan 26, 2022
  58. 8/9 midx: read `RIDX` chunk when presentTaylor Blau, Jan 4, 2022
  59. Jonathan TanJan 24, 2022
  60. Taylor BlauJan 25, 2022
  61. Jonathan TanJan 26, 2022
  62. 9/9 pack-bitmap.c: gracefully fallback after opening pack/MIDXTaylor Blau, Jan 4, 2022
  63. Jonathan TanJan 24, 2022
  64. Taylor BlauJan 25, 2022
  65. 0/9 midx: prevent bitmap corruption when permuting pack orderTaylor Blau, Jan 25, 2022
  66. 1/9 t5326: demonstrate bitmap corruption after permutationTaylor Blau, Jan 25, 2022
  67. Ævar Arnfjörð BjarmasonJan 26, 2022
  68. Taylor BlauJan 26, 2022
  69. 2/9 midx.c: make changing the preferred pack safeTaylor Blau, Jan 25, 2022
  70. 3/9 pack-revindex.c: instrument loading on-disk reverse indexTaylor Blau, Jan 25, 2022
  71. Ævar Arnfjörð BjarmasonJan 26, 2022
  72. 4/9 t5326: drop unnecessary setupTaylor Blau, Jan 25, 2022
  73. 5/9 t5326: extract `test_rev_exists`Taylor Blau, Jan 25, 2022
  74. Ævar Arnfjörð BjarmasonJan 26, 2022
  75. Taylor BlauJan 26, 2022
  76. 6/9 t5326: move tests to t/lib-bitmap.shTaylor Blau, Jan 25, 2022
  77. 7/9 t/lib-bitmap.sh: parameterize tests over reverse index sourceTaylor Blau, Jan 25, 2022
  78. 8/9 midx: read `RIDX` chunk when presentTaylor Blau, Jan 25, 2022
  79. Ævar Arnfjörð BjarmasonJan 26, 2022
  80. Taylor BlauJan 26, 2022
  81. 9/9 pack-bitmap.c: gracefully fallback after opening pack/MIDXTaylor Blau, Jan 25, 2022
  82. Ævar Arnfjörð BjarmasonJan 26, 2022
  83. Ævar Arnfjörð BjarmasonJan 26, 2022
  84. Taylor BlauJan 26, 2022
  85. Jonathan TanJan 27, 2022
  86. Taylor BlauFeb 24, 2022
  87. Derrick StoleeJan 27, 2022

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.