Re: Consist timestamps within a checkout/clone

From: Matheus Tavares <matheus.bernardino@usp.br>
Date: Nov 2, 2022, 14:16 UTC
Message-ID: <20221102141609.1603860-1-matheus.bernardino@usp.br>
In-Reply-To: <221101.86bkpq4jan.gmgdl@evledraar.gmail.com>

Show 70 quoted lines

> On Mon, Oct 31 2022, Mark Hills wrote:
>
> > On Mon, 31 Oct 2022, Ævar Arnfjörð Bjarmason wrote:
> >
> >>
> >> On Mon, Oct 31 2022, Mark Hills wrote:
> >>
> >> > Our use case: we commit some compiled objects to the repo, where compiling
> >> > is either slow or requires software which is not always available.
> >> >
> >> > Since upgrading Git 2.26.3 -> 2.32.4 (as part of Alpine Linux OS upgrade)
> >> > we are noticing a change in build behaviour.
> >> >
> >> > Now, after a "git clone" we find the Makefile intermittently attempting
> >> > (and failing) some builds that are not intended.
> >> >
> >> > Indeed, Make is acting reasonably as the source file is sometimes
> >> > marginally newer than the destination (both checked out by Git), example
> >> > below.
> >> >
> >> > I've never had to consider consistency timestamps within a Git checkout
> >> > until now.
> >> >
> >> > It's entirely possible there's _never_ a guarantee of consistency here.
> >> >
> >> > But then something has certainly changed in practice, as this fault has
> >> > gone from never happening to now every couple of days.
> >> >
> >> > Imaginging I can't be the first person to encounter this, I searched for
> >> > existing threads or docs, but overwhemingly the results were question of
> >> > Git tracking the timestamps (as part of the commit) which this is not;
> >> > it's consistency within one checkout.
> >> >
> >> > $ git clone --depth 1 file:///path/to/repo.git
> >> >
> >> > $ stat winner.jpeg
> >> >   File: winner.jpeg
> >> >   Size: 258243          Blocks: 520        IO Block: 4096   regular file
> >> > Device: fd07h/64775d    Inode: 33696       Links: 1
> >> > Access: (0644/-rw-r--r--)  Uid: (  106/ luthier)   Gid: (  106/ luthier)
> >> > Access: 2022-10-31 16:05:17.756858496 +0000
> >> > Modify: 2022-10-31 16:05:17.756858496 +0000
> >> > Change: 2022-10-31 16:05:17.756858496 +0000
> >> >  Birth: -
> >> >
> >> > $ stat winner.svg
> >> >   File: winner.svg
> >> >   Size: 52685           Blocks: 112        IO Block: 4096   regular file
> >> > Device: fd07h/64775d    Inode: 33697       Links: 1
> >> > Access: (0644/-rw-r--r--)  Uid: (  106/ luthier)   Gid: (  106/ luthier)
> >> > Access: 2022-10-31 16:05:17.766859030 +0000
> >> > Modify: 2022-10-31 16:05:17.766859030 +0000
> >> > Change: 2022-10-31 16:05:17.766859030 +0000
> >> >  Birth: -
> >> >
> >> > Elsewhere in the repository, it's clear the timestamps are not consistent:
> >> >
> >> > $ stat Makefile
> >> >   File: Makefile
> >> >   Size: 8369            Blocks: 24         IO Block: 4096   regular file
> >> > Device: fd07h/64775d    Inode: 33655       Links: 1
> >> > Access: (0644/-rw-r--r--)  Uid: (  106/ luthier)   Gid: (  106/ luthier)
> >> > Access: 2022-10-31 16:05:51.628660212 +0000
> >> > Modify: 2022-10-31 16:05:17.746857963 +0000
> >> > Change: 2022-10-31 16:05:17.746857963 +0000
> >> >  Birth: -
> >>
> >> I think you're almost certainly running into the parallel checkout,
> >> which is new in that revision range. Try tweaking checkout.workers and
> >> checkout.thresholdForParallelism (see "man git-config").

This does look like something you would see with parallel checkout, yes. But...

> > Thanks, it will be interesting to try this and I'll report back.
>
> FWIW I was under the impression that we'd made it the default, so unless
> you opted-in it's probably not that.

... it indeed should be disabled by default. It seems Mark didn't manually enable parallel checkout, as the original message only mentions the git upgrade as a changing factor. And Alpine's git installation script for 2.32.4 [1] doesn't seem to change our defaults either.

Perhaps, it just happens that 2.32.4 changed the checkout processing time slightly so that each entry is finished a bit slower (or the system was overloaded at that moment?). Anyways, the creation order (based on the mtimes) looks correct to me from a sequential-checkout point of view: first Makefile, than winner.jpeg, and finally winner.svg. That's the order in which these files would appear in the index, which is the order followed by sequential checkout.

[1]: https://git.alpinelinux.org/aports/tree/main/git/APKBUILD?h=3.14-stable&id=0f3285f2cfcb8362460002c27e219fadbf18c885