Re: Consist timestamps within a checkout/clone

From: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Date: Oct 31, 2022, 20:21 UTC
Message-ID: <221031.86zgdb68p3.gmgdl@evledraar.gmail.com>
In-Reply-To: <2210311614160.25661@stax.localdomain>

On Mon, Oct 31 2022, Mark Hills wrote:

Show 59 quoted lines

> Our use case: we commit some compiled objects to the repo, where compiling 
> is either slow or requires software which is not always available.
>
> Since upgrading Git 2.26.3 -> 2.32.4 (as part of Alpine Linux OS upgrade) 
> we are noticing a change in build behaviour.
>
> Now, after a "git clone" we find the Makefile intermittently attempting 
> (and failing) some builds that are not intended.
>
> Indeed, Make is acting reasonably as the source file is sometimes 
> marginally newer than the destination (both checked out by Git), example 
> below.
>
> I've never had to consider consistency timestamps within a Git checkout 
> until now.
>
> It's entirely possible there's _never_ a guarantee of consistency here.
>
> But then something has certainly changed in practice, as this fault has 
> gone from never happening to now every couple of days.
>
> Imaginging I can't be the first person to encounter this, I searched for 
> existing threads or docs, but overwhemingly the results were question of 
> Git tracking the timestamps (as part of the commit) which this is not; 
> it's consistency within one checkout.
>
> $ git clone --depth 1 file:///path/to/repo.git
>
> $ stat winner.jpeg
>   File: winner.jpeg
>   Size: 258243          Blocks: 520        IO Block: 4096   regular file
> Device: fd07h/64775d    Inode: 33696       Links: 1
> Access: (0644/-rw-r--r--)  Uid: (  106/ luthier)   Gid: (  106/ luthier)
> Access: 2022-10-31 16:05:17.756858496 +0000
> Modify: 2022-10-31 16:05:17.756858496 +0000
> Change: 2022-10-31 16:05:17.756858496 +0000
>  Birth: -
>
> $ stat winner.svg
>   File: winner.svg
>   Size: 52685           Blocks: 112        IO Block: 4096   regular file
> Device: fd07h/64775d    Inode: 33697       Links: 1
> Access: (0644/-rw-r--r--)  Uid: (  106/ luthier)   Gid: (  106/ luthier)
> Access: 2022-10-31 16:05:17.766859030 +0000
> Modify: 2022-10-31 16:05:17.766859030 +0000
> Change: 2022-10-31 16:05:17.766859030 +0000
>  Birth: -
>
> Elsewhere in the repository, it's clear the timestamps are not consistent:
>
> $ stat Makefile
>   File: Makefile
>   Size: 8369            Blocks: 24         IO Block: 4096   regular file
> Device: fd07h/64775d    Inode: 33655       Links: 1
> Access: (0644/-rw-r--r--)  Uid: (  106/ luthier)   Gid: (  106/ luthier)
> Access: 2022-10-31 16:05:51.628660212 +0000
> Modify: 2022-10-31 16:05:17.746857963 +0000
> Change: 2022-10-31 16:05:17.746857963 +0000
>  Birth: -

I think you're almost certainly running into the parallel checkout, which is new in that revision range. Try tweaking checkout.workers and checkout.thresholdForParallelism (see "man git-config").

I can't say without looking at the code/Makefile (and even then, I don't have time to dig here:), but if I had to bet I'd say that your dependencies have probably always been broken with these checked-in files, but they happend to work out if they were checked out in sorted order.

And now with the parallel checkout they're not guaranteed to do that, as some workers will "race ahead" and finish in an unpredictable order.

But that's all just a guess, perhaps it has nothing to do with parallel checkout, such dependency issues are sensitive to all sorts of other things, e.g. maybe git got slightly faster (or slower), so now files that were always on different seconds (or the same) aren't in the state they were in before...