threads / discuss / 51147

[Breakage] 2.22.0-rc1 t5401-update-hooks.sh

Subject: [Breakage] 2.22.0-rc1 t5401-update-hooks.sh

## tl;dr

8 messages between May 21, 2019 and May 23, 2019.

replies: 7people: 3as markdown or json

Randall S. Becker· May 21, 2019, 21:47 UTC · lore
Another situation came up during regression testing on NonStop:
t5401 broke on the first run through (most tests), with the failures below.

*** t5401-update-hooks.sh *** not ok 1 - setup # # echo This is a test. >a && # git update-index --add a && # tree0=$(git write-tree) && # commit0=$(echo setup | git commit-tree $tree0) && # echo We hope it works. >a && # git update-index a && # tree1=$(git write-tree) && # commit1=$(echo modify | git commit-tree $tree1 -p $commit0) && # git update-ref refs/heads/master $commit0 && # git update-ref refs/heads/tofail $commit1 && # git clone --bare ./. victim.git && # GIT_DIR=victim.git git update-ref refs/heads/tofail $commit1 && # git update-ref refs/heads/master $commit1 && # git update-ref refs/heads/tofail $commit0 # t5401-update-hooks.sh: line 26: victim.git/hooks/pre-receive: No such file or di rectory chmod: cannot access 'victim.git/hooks/pre-receive': No such file or directory t5401-update-hooks.sh: line 35: victim.git/hooks/update: No such file or directo ry chmod: cannot access 'victim.git/hooks/update': No such file or directory t5401-update-hooks.sh: line 45: victim.git/hooks/post-receive: No such file or d irectory chmod: cannot access 'victim.git/hooks/post-receive': No such file or directory t5401-update-hooks.sh: line 54: victim.git/hooks/post-update: No such file or di rectory chmod: cannot access 'victim.git/hooks/post-update': No such file or directory ok 2 - push etc.

When running the test in isolation, it passes without incident whether or not --verbose is used. So far, this only occurs on the first run through. I wanted to report it, based on the inconsistency of results. This is not the first time tests have acted in this fashion, and I realize it is difficult to do anything about it without being able to recreate the situation.

Such as it is, Randall

brian m. carlson· May 22, 2019, 00:48 UTC · re: Randall S. Becker · lore

Re: [Breakage] 2.22.0-rc1 t5401-update-hooks.sh

On 2019-05-21 at 21:47:54, Randall S. Becker wrote:
Show 5 quoted lines
> When running the test in isolation, it passes without incident whether or
> not --verbose is used. So far, this only occurs on the first run through. I
> wanted to report it, based on the inconsistency of results. This is not the
> first time tests have acted in this fashion, and I realize it is difficult
> to do anything about it without being able to recreate the situation.
Does running git clean -dxf cause it to be reproducible?
-- 
brian m. carlson: Houston, Texas, US
OpenPGP: https://keybase.io/bk2204
Randall S. Becker· May 22, 2019, 13:46 UTC · re: brian m. carlson · lore

RE: [Breakage] 2.22.0-rc1 t5401-update-hooks.sh

On May 21, 2019 20:48, brian m. carlson wrote:
Show 13 quoted lines
> To: Randall S. Becker <rsbecker@nexbridge.com>
> Cc: 'Git Mailing List' <git@vger.kernel.org>
> Subject: Re: [Breakage] 2.22.0-rc1 t5401-update-hooks.sh
> 
> On 2019-05-21 at 21:47:54, Randall S. Becker wrote:
> > When running the test in isolation, it passes without incident whether
> > or not --verbose is used. So far, this only occurs on the first run
> > through. I wanted to report it, based on the inconsistency of results.
> > This is not the first time tests have acted in this fashion, and I
> > realize it is difficult to do anything about it without being able to recreate
> the situation.
> 
> Does running git clean -dxf cause it to be reproducible?
I will give it a go. Having exactly the same behaviour in t7519 subtest 19. I wonder whether there are breadcrumbs not being cleaned up. Will report back when I am able - may take a day or so.

Cheers, Randall

Johannes Schindelin· May 23, 2019, 20:05 UTC · re: Randall S. Becker · lore

Issues with t7519.19, was RE: [Breakage] 2.22.0-rc1 t5401-update-hooks.sh

Hi Randall,
On Wed, 22 May 2019, Randall S. Becker wrote:
Show 19 quoted lines
> On May 21, 2019 20:48, brian m. carlson wrote:
> > To: Randall S. Becker <rsbecker@nexbridge.com>
> > Cc: 'Git Mailing List' <git@vger.kernel.org>
> > Subject: Re: [Breakage] 2.22.0-rc1 t5401-update-hooks.sh
> >
> > On 2019-05-21 at 21:47:54, Randall S. Becker wrote:
> > > When running the test in isolation, it passes without incident
> > > whether or not --verbose is used. So far, this only occurs on the
> > > first run through. I wanted to report it, based on the inconsistency
> > > of results. This is not the first time tests have acted in this
> > > fashion, and I realize it is difficult to do anything about it
> > > without being able to recreate
> > the situation.
> >
> > Does running git clean -dxf cause it to be reproducible?
>
> I will give it a go. Having exactly the same behaviour in t7519 subtest
> 19. I wonder whether there are breadcrumbs not being cleaned up. Will
> report back when I am able - may take a day or so.

I fear that t7519's problems are *completely* unrelated to the t5401 issue you reported earlier. I hunted the t7519 problems down today, and I could imagine that these patches fix your t7519, too:

	https://github.com/gitgitgadget/git/pull/223

Ciao, Johannes

Randall S. Becker· May 23, 2019, 20:18 UTC · re: Johannes Schindelin · lore

RE: Issues with t7519.19, was RE: [Breakage] 2.22.0-rc1 t5401-update-hooks.sh

On May 23, 2019 16:06, Johannes Schindelin wrote:
Show 27 quoted lines
> On Wed, 22 May 2019, Randall S. Becker wrote:
> 
> > On May 21, 2019 20:48, brian m. carlson wrote:
> > > To: Randall S. Becker <rsbecker@nexbridge.com>
> > > Cc: 'Git Mailing List' <git@vger.kernel.org>
> > > Subject: Re: [Breakage] 2.22.0-rc1 t5401-update-hooks.sh
> > >
> > > On 2019-05-21 at 21:47:54, Randall S. Becker wrote:
> > > > When running the test in isolation, it passes without incident
> > > > whether or not --verbose is used. So far, this only occurs on the
> > > > first run through. I wanted to report it, based on the
> > > > inconsistency of results. This is not the first time tests have
> > > > acted in this fashion, and I realize it is difficult to do
> > > > anything about it without being able to recreate
> > > the situation.
> > >
> > > Does running git clean -dxf cause it to be reproducible?
> >
> > I will give it a go. Having exactly the same behaviour in t7519
> > subtest 19. I wonder whether there are breadcrumbs not being cleaned
> > up. Will report back when I am able - may take a day or so.
> 
> I fear that t7519's problems are *completely* unrelated to the t5401 issue
> you reported earlier. I hunted the t7519 problems down today, and I could
> imagine that these patches fix your t7519, too:
> 
> 	https://github.com/gitgitgadget/git/pull/223
From the description, I believe it. Timestamp resolution on NonStop is in microseconds and those are not even slightly simulated. Coupled with this being an MPP not SMP, things can occur within the same microsecond, or in weird situations slightly before or after when comparing the clock on different CPUs. Yes, time-travel is possible at the single microsecond level 😉. Cores are synchronized, but our machine has 4 CPUs and synchronizing the file system across all of them does lead to slightly strange situations.
Johannes Schindelin· May 23, 2019, 20:55 UTC · re: Randall S. Becker · lore

RE: Issues with t7519.19, was RE: [Breakage] 2.22.0-rc1 t5401-update-hooks.sh

Hi Randall,
On Thu, 23 May 2019, Randall S. Becker wrote:
Show 30 quoted lines
> On May 23, 2019 16:06, Johannes Schindelin wrote:
> > On Wed, 22 May 2019, Randall S. Becker wrote:
> >
> > > On May 21, 2019 20:48, brian m. carlson wrote:
> > > > To: Randall S. Becker <rsbecker@nexbridge.com>
> > > > Cc: 'Git Mailing List' <git@vger.kernel.org>
> > > > Subject: Re: [Breakage] 2.22.0-rc1 t5401-update-hooks.sh
> > > >
> > > > On 2019-05-21 at 21:47:54, Randall S. Becker wrote:
> > > > > When running the test in isolation, it passes without incident
> > > > > whether or not --verbose is used. So far, this only occurs on the
> > > > > first run through. I wanted to report it, based on the
> > > > > inconsistency of results. This is not the first time tests have
> > > > > acted in this fashion, and I realize it is difficult to do
> > > > > anything about it without being able to recreate
> > > > the situation.
> > > >
> > > > Does running git clean -dxf cause it to be reproducible?
> > >
> > > I will give it a go. Having exactly the same behaviour in t7519
> > > subtest 19. I wonder whether there are breadcrumbs not being cleaned
> > > up. Will report back when I am able - may take a day or so.
> >
> > I fear that t7519's problems are *completely* unrelated to the t5401 issue
> > you reported earlier. I hunted the t7519 problems down today, and I could
> > imagine that these patches fix your t7519, too:
> >
> > 	https://github.com/gitgitgadget/git/pull/223
>
> From the description, I believe it. Timestamp resolution on NonStop is in microseconds and those are not even slightly simulated. Coupled with this being an MPP not SMP, things can occur within the same microsecond, or in weird situations slightly before or after when comparing the clock on different CPUs. Yes, time-travel is possible at the single microsecond level 😉. Cores are synchronized, but our machine has 4 CPUs and synchronizing the file system across all of them does lead to slightly strange situations.
I tested my patches with
	sh t7519-status-fsmonitor.sh --stress-jobs=8 --stress-limit=10

Why don't you check out my branch, build and test, and then we know whether it fixes your t7519 bug?

Ciao, Johannes

Randall S. Becker· May 23, 2019, 19:57 UTC · re: brian m. carlson · lore

RE: [Breakage] 2.22.0-rc1 t5401-update-hooks.sh

On May 21, 2019 20:48, brian m. carlson wrote:
Show 13 quoted lines
> To: Randall S. Becker <rsbecker@nexbridge.com>
> Cc: 'Git Mailing List' <git@vger.kernel.org>
> Subject: Re: [Breakage] 2.22.0-rc1 t5401-update-hooks.sh
> 
> On 2019-05-21 at 21:47:54, Randall S. Becker wrote:
> > When running the test in isolation, it passes without incident whether
> > or not --verbose is used. So far, this only occurs on the first run
> > through. I wanted to report it, based on the inconsistency of results.
> > This is not the first time tests have acted in this fashion, and I
> > realize it is difficult to do anything about it without being able to recreate
> the situation.
> 
> Does running git clean -dxf cause it to be reproducible?
Made no difference. t5401 passed cleanly after a the clean -dxf. I don't know where the breadcrumbs are. I guess it could be something funky in the file system on platform or with bash.
Johannes Schindelin· May 23, 2019, 21:00 UTC · re: Randall S. Becker · lore

RE: [Breakage] 2.22.0-rc1 t5401-update-hooks.sh

Hi Randall,
On Thu, 23 May 2019, Randall S. Becker wrote:
Show 19 quoted lines
> On May 21, 2019 20:48, brian m. carlson wrote:
> > To: Randall S. Becker <rsbecker@nexbridge.com>
> > Cc: 'Git Mailing List' <git@vger.kernel.org>
> > Subject: Re: [Breakage] 2.22.0-rc1 t5401-update-hooks.sh
> >
> > On 2019-05-21 at 21:47:54, Randall S. Becker wrote:
> > > When running the test in isolation, it passes without incident
> > > whether or not --verbose is used. So far, this only occurs on the
> > > first run through. I wanted to report it, based on the inconsistency
> > > of results. This is not the first time tests have acted in this
> > > fashion, and I realize it is difficult to do anything about it
> > > without being able to recreate
> > the situation.
> >
> > Does running git clean -dxf cause it to be reproducible?
>
> Made no difference. t5401 passed cleanly after a the clean -dxf. I don't
> know where the breadcrumbs are. I guess it could be something funky in
> the file system on platform or with bash.

FWIW I do not see any of the major platforms having any sort of regression in t5401: https://dev.azure.com/gitgitgadget/git/_test/analytics?definitionId=4&contextType=build

So I guess the only way you will find out more about this issue is by instrumenting the code and digging in. You will most likely find that the `--stress` mode will help you recreate the situation rather easily.

Ciao, Johannes

← back to recent threads