Re: [PATCH v2 2/9] t5526: use grep to assert on fetches
- From
Ævar Arnfjörð Bjarmason <avarab@gmail.com>
- Date
- Feb 15, 2022, 21:53 UTC
- Message-ID
- <220215.86r183x043.gmgdl@evledraar.gmail.com>
- In-Reply-To
- <20220215172318.73533-3-chooglen@google.com>
On Wed, Feb 16 2022, Glen Choo wrote:
Show 16 quoted lines
> In a previous commit, we replaced test_cmp invocations with > verify_fetch_result(). Finish the process of removing test_cmp by using > grep in verify_fetch_result() instead. > > This makes the tests less sensitive to changes because, instead of > checking the whole stderr, we only grep for the lines of the form > > * "<old-head>..<new-head>\s+branch\s+-> origin/branch" > * "Fetching submodule <submodule-path>" (if fetching a submodule) > > when we expect the repo to have fetched. If we expect the repo to not > have fetched, grep to make sure the lines are absent. Also, simplify the > assertions by using grep patterns that match only the relevant pieces of > information, e.g. <old-head> is irrelevant because we only want to know > if the fetch was performed, so we don't need to know where the branch > was before the fetch.
I tried ejecting 1/9 and 2/9 out of this series locally, and it passes all tests until the new tests you add in 7/9.
As ugly as some of the pre-image is, I wonder if dropping these first two and biting the bullet and just continuing with the test_cmp is better.
The test_cmp is going to catch issues that even the cleverest grep combinations won't, e.g. in the submodule-in-C series I discovered a bug where all of our testing & the existing series hadn't spotted that we were dropping a \n at the end in one of the messages.
Particularly as:
Show 44 quoted lines
> # Verifies that the expected repositories were fetched. This is done by
> -# concatenating the files expect.err.[super|sub|deep] in the correct
> -# order and comparing it to the actual stderr.
> +# checking that the branches of [super|sub|deep] were updated to
> +# [super|sub|deep]head if the corresponding file exists.
> #
> -# If a repo should not be fetched in the test, its corresponding
> -# expect.err file should be rm-ed.
> +# If the [super|sub|deep] head file does not exist, this verifies that
> +# the corresponding repo was not fetched. Thus, if a repo should not be
> +# fetched in the test, its corresponding head file should be
> +# rm-ed.
> verify_fetch_result() {
> ACTUAL_ERR=$1 &&
> - rm -f expect.err.combined &&
> - if [ -f expect.err.super ]; then
> - cat expect.err.super >>expect.err.combined
> + # Each grep pattern is guaranteed to match the correct repo
> + # because each repo uses a different name for their branch i.e.
> + # "super", "sub" and "deep".
> + if [ -f superhead ]; then
> + grep -E "\.\.$(cat superhead)\s+super\s+-> origin/super" $ACTUAL_ERR
> + else
> + ! grep "super" $ACTUAL_ERR
> fi &&
> - if [ -f expect.err.sub ]; then
> - cat expect.err.sub >>expect.err.combined
> + if [ -f subhead ]; then
> + grep "Fetching submodule submodule" $ACTUAL_ERR &&
> + grep -E "\.\.$(cat subhead)\s+sub\s+-> origin/sub" $ACTUAL_ERR
> + else
> + ! grep "Fetching submodule submodule" $ACTUAL_ERR
> fi &&
> - if [ -f expect.err.deep ]; then
> - cat expect.err.deep >>expect.err.combined
> - fi &&
> - test_cmp expect.err.combined $ACTUAL_ERR
> + if [ -f deephead ]; then
> + grep "Fetching submodule submodule/subdir/deepsubmodule" $ACTUAL_ERR &&
> + grep -E "\.\.$(cat deephead)\s+deep\s+-> origin/deep" $ACTUAL_ERR
> + else
> + ! grep "Fetching submodule submodule/subdir/deepsubmodule" $ACTUAL_ERR
> + fi
> }This sort of thing is really hard to understand and review...
Show 11 quoted lines
> test_expect_success setup ' > @@ -274,13 +277,10 @@ test_expect_success "Recursion doesn't happen when no new commits are fetched in > ' > > test_expect_success "Recursion stops when no new submodule commits are fetched" ' > - head1=$(git rev-parse --short HEAD) && > git add submodule && > git commit -m "new submodule" && > - head2=$(git rev-parse --short HEAD) && > - echo "From $pwd/." > expect.err.super && > - echo " $head1..$head2 super -> origin/super" >>expect.err.super &&
...as opposed to if we just rolled the generation of this into some utility printf function.