Consist timestamps within a checkout/clone

> Our use case: we commit some compiled objects to the repo, where compiling 
> is either slow or requires software which is not always available.
>
> Since upgrading Git 2.26.3 -> 2.32.4 (as part of Alpine Linux OS upgrade) 
> we are noticing a change in build behaviour.
>
> Now, after a "git clone" we find the Makefile intermittently attempting 
> (and failing) some builds that are not intended.
>
> Indeed, Make is acting reasonably as the source file is sometimes 
> marginally newer than the destination (both checked out by Git), example 
> below.
>
> I've never had to consider consistency timestamps within a Git checkout 
> until now.
>
> It's entirely possible there's _never_ a guarantee of consistency here.
>
> But then something has certainly changed in practice, as this fault has 
> gone from never happening to now every couple of days.
>
> Imaginging I can't be the first person to encounter this, I searched for 
> existing threads or docs, but overwhemingly the results were question of 
> Git tracking the timestamps (as part of the commit) which this is not; 
> it's consistency within one checkout.
>
> $ git clone --depth 1 file:///path/to/repo.git
>
> $ stat winner.jpeg
>   File: winner.jpeg
>   Size: 258243          Blocks: 520        IO Block: 4096   regular file
> Device: fd07h/64775d    Inode: 33696       Links: 1
> Access: (0644/-rw-r--r--)  Uid: (  106/ luthier)   Gid: (  106/ luthier)
> Access: 2022-10-31 16:05:17.756858496 +0000
> Modify: 2022-10-31 16:05:17.756858496 +0000
> Change: 2022-10-31 16:05:17.756858496 +0000
>  Birth: -
>
> $ stat winner.svg
>   File: winner.svg
>   Size: 52685           Blocks: 112        IO Block: 4096   regular file
> Device: fd07h/64775d    Inode: 33697       Links: 1
> Access: (0644/-rw-r--r--)  Uid: (  106/ luthier)   Gid: (  106/ luthier)
> Access: 2022-10-31 16:05:17.766859030 +0000
> Modify: 2022-10-31 16:05:17.766859030 +0000
> Change: 2022-10-31 16:05:17.766859030 +0000
>  Birth: -
>
> Elsewhere in the repository, it's clear the timestamps are not consistent:
>
> $ stat Makefile
>   File: Makefile
>   Size: 8369            Blocks: 24         IO Block: 4096   regular file
> Device: fd07h/64775d    Inode: 33655       Links: 1
> Access: (0644/-rw-r--r--)  Uid: (  106/ luthier)   Gid: (  106/ luthier)
> Access: 2022-10-31 16:05:51.628660212 +0000
> Modify: 2022-10-31 16:05:17.746857963 +0000
> Change: 2022-10-31 16:05:17.746857963 +0000
>  Birth: -

> I think you're almost certainly running into the parallel checkout,
> which is new in that revision range. Try tweaking checkout.workers and
> checkout.thresholdForParallelism (see "man git-config").
>
> I can't say without looking at the code/Makefile (and even then, I don't
> have time to dig here:), but if I had to bet I'd say that your
> dependencies have probably always been broken with these checked-in
> files, but they happend to work out if they were checked out in sorted
> order.
>
> And now with the parallel checkout they're not guaranteed to do that, as
> some workers will "race ahead" and finish in an unpredictable order.

> On Mon, Oct 31, 2022 at 09:21:20PM +0100, Ævar Arnfjörð Bjarmason wrote:
> > I think you're almost certainly running into the parallel checkout,
> > which is new in that revision range. Try tweaking checkout.workers and
> > checkout.thresholdForParallelism (see "man git-config").
> >
> > I can't say without looking at the code/Makefile (and even then, I don't
> > have time to dig here:), but if I had to bet I'd say that your
> > dependencies have probably always been broken with these checked-in
> > files, but they happend to work out if they were checked out in sorted
> > order.
> >
> > And now with the parallel checkout they're not guaranteed to do that, as
> > some workers will "race ahead" and finish in an unpredictable order.
> 
> Doesn't checkout.thresholdForParallelism only matter when
> checkout.workers != 1?
> 
> So what you wrote seems like a reasonable explanation, but only if the
> original reporter set checkout.workers to imply the non-sequential
> behavior in the first place.
> 
> That said...
> 
>   - I also don't know off-hand of a place where we've defined the order
>     where Git will checkout files in the working copy. So depending on
>     that behavior isn't a safe thing to do.
> 
>   - Committing build artifacts into your repository is generally
>     discouraged.

>On Mon, 31 Oct 2022, Taylor Blau wrote:
>> On Mon, Oct 31, 2022 at 09:21:20PM +0100, Ævar Arnfjörð Bjarmason wrote:
>> > I think you're almost certainly running into the parallel checkout,
>> > which is new in that revision range. Try tweaking checkout.workers
>> > and checkout.thresholdForParallelism (see "man git-config").
>> >
>> > I can't say without looking at the code/Makefile (and even then, I
>> > don't have time to dig here:), but if I had to bet I'd say that your
>> > dependencies have probably always been broken with these checked-in
>> > files, but they happend to work out if they were checked out in
>> > sorted order.
>> >
>> > And now with the parallel checkout they're not guaranteed to do
>> > that, as some workers will "race ahead" and finish in an unpredictable order.
>>
>> Doesn't checkout.thresholdForParallelism only matter when
>> checkout.workers != 1?
>>
>> So what you wrote seems like a reasonable explanation, but only if the
>> original reporter set checkout.workers to imply the non-sequential
>> behavior in the first place.
>>
>> That said...
>>
>>   - I also don't know off-hand of a place where we've defined the order
>>     where Git will checkout files in the working copy. So depending on
>>     that behavior isn't a safe thing to do.
>>
>>   - Committing build artifacts into your repository is generally
>>     discouraged.
>
>If it's undefined and never implemented this is reasonable.
>
>But "generally" is a caveat, so while I agree with the statement it also implies
>there's valid cases outside of that. Ones which used to work, too.
>
>Here are some useful cases I have seen for the combination of build rule +
>checked in file:
>
>- part of a build requires licensed software that's not always available
>
>- part of the build requires large memory that other builders generally do
>  not have available
>
>- part of the build process uses a different platform or some other system
>  requirement
>
>- to fetch data eg. from a URL, with a record of the URL/automation but
>  also a copy of the file as a record and for offline use
>
>So it's useful, to retain repeatable automation but not always build from square
>one.
>
>Generally discouraged to check in build results yes, but I've found it very practical.
>
>> So while I'd guess that setting `checkout.workers` back to "1" (if it
>> wasn't already) will probably restore the existing behavior, counting
>> on that behavior in the first place is wrong.
>
>I think perhaps the tail is wagging the dog here, though.
>
>It's 'wrong' because it doesn't work; but I haven't seen anything to make me think
>this is fundamentally or theoretically flawed.
>
>If we had a transactional file system we'd reasonably expect a checkout to be an
>atomic operation -- same timestamp on the files created in that step. A
>discrepancy in timestamps would be considered incorrect; it would imply an 'order'
>to the checkout which, as you say, is order-less.
>
>Sowhat could be the bad outcomes if Git created files stamped with the point in
>time of the "git checkout"?

> On Mon, 31 Oct 2022, Taylor Blau wrote:
>
>> On Mon, Oct 31, 2022 at 09:21:20PM +0100, Ævar Arnfjörð Bjarmason wrote:
>> > I think you're almost certainly running into the parallel checkout,
>> > which is new in that revision range. Try tweaking checkout.workers and
>> > checkout.thresholdForParallelism (see "man git-config").
>> >
>> > I can't say without looking at the code/Makefile (and even then, I don't
>> > have time to dig here:), but if I had to bet I'd say that your
>> > dependencies have probably always been broken with these checked-in
>> > files, but they happend to work out if they were checked out in sorted
>> > order.
>> >
>> > And now with the parallel checkout they're not guaranteed to do that, as
>> > some workers will "race ahead" and finish in an unpredictable order.
>> 
>> Doesn't checkout.thresholdForParallelism only matter when
>> checkout.workers != 1?
>> 
>> So what you wrote seems like a reasonable explanation, but only if the
>> original reporter set checkout.workers to imply the non-sequential
>> behavior in the first place.
>> 
>> That said...
>> 
>>   - I also don't know off-hand of a place where we've defined the order
>>     where Git will checkout files in the working copy. So depending on
>>     that behavior isn't a safe thing to do.
>> 
>>   - Committing build artifacts into your repository is generally
>>     discouraged.
>
> If it's undefined and never implemented this is reasonable.
>
> But "generally" is a caveat, so while I agree with the statement it also 
> implies there's valid cases outside of that. Ones which used to work, too.
>
> Here are some useful cases I have seen for the combination of build rule + 
> checked in file:
>
> - part of a build requires licensed software that's not always available
>
> - part of the build requires large memory that other builders generally do 
>   not have available
>
> - part of the build process uses a different platform or some other system 
>   requirement
>
> - to fetch data eg. from a URL, with a record of the URL/automation but 
>   also a copy of the file as a record and for offline use
>
> So it's useful, to retain repeatable automation but not always build from 
> square one.
>
> Generally discouraged to check in build results yes, but I've found it 
> very practical.
>  
>> So while I'd guess that setting `checkout.workers` back to "1" (if it 
>> wasn't already) will probably restore the existing behavior, counting on 
>> that behavior in the first place is wrong.
>
> I think perhaps the tail is wagging the dog here, though.
>
> It's 'wrong' because it doesn't work; but I haven't seen anything to make 
> me think this is fundamentally or theoretically flawed.
>
> If we had a transactional file system we'd reasonably expect a checkout to 
> be an atomic operation -- same timestamp on the files created in that 
> step. A discrepancy in timestamps would be considered incorrect; it would 
> imply an 'order' to the checkout which, as you say, is order-less.
>
> Sowhat could be the bad outcomes if Git created files stamped with the 
> point in time of the "git checkout"?

> 
> On Mon, Oct 31 2022, Mark Hills wrote:
> 
> > Our use case: we commit some compiled objects to the repo, where compiling 
> > is either slow or requires software which is not always available.
> >
> > Since upgrading Git 2.26.3 -> 2.32.4 (as part of Alpine Linux OS upgrade) 
> > we are noticing a change in build behaviour.
> >
> > Now, after a "git clone" we find the Makefile intermittently attempting 
> > (and failing) some builds that are not intended.
> >
> > Indeed, Make is acting reasonably as the source file is sometimes 
> > marginally newer than the destination (both checked out by Git), example 
> > below.
> >
> > I've never had to consider consistency timestamps within a Git checkout 
> > until now.
> >
> > It's entirely possible there's _never_ a guarantee of consistency here.
> >
> > But then something has certainly changed in practice, as this fault has 
> > gone from never happening to now every couple of days.
> >
> > Imaginging I can't be the first person to encounter this, I searched for 
> > existing threads or docs, but overwhemingly the results were question of 
> > Git tracking the timestamps (as part of the commit) which this is not; 
> > it's consistency within one checkout.
> >
> > $ git clone --depth 1 file:///path/to/repo.git
> >
> > $ stat winner.jpeg
> >   File: winner.jpeg
> >   Size: 258243          Blocks: 520        IO Block: 4096   regular file
> > Device: fd07h/64775d    Inode: 33696       Links: 1
> > Access: (0644/-rw-r--r--)  Uid: (  106/ luthier)   Gid: (  106/ luthier)
> > Access: 2022-10-31 16:05:17.756858496 +0000
> > Modify: 2022-10-31 16:05:17.756858496 +0000
> > Change: 2022-10-31 16:05:17.756858496 +0000
> >  Birth: -
> >
> > $ stat winner.svg
> >   File: winner.svg
> >   Size: 52685           Blocks: 112        IO Block: 4096   regular file
> > Device: fd07h/64775d    Inode: 33697       Links: 1
> > Access: (0644/-rw-r--r--)  Uid: (  106/ luthier)   Gid: (  106/ luthier)
> > Access: 2022-10-31 16:05:17.766859030 +0000
> > Modify: 2022-10-31 16:05:17.766859030 +0000
> > Change: 2022-10-31 16:05:17.766859030 +0000
> >  Birth: -
> >
> > Elsewhere in the repository, it's clear the timestamps are not consistent:
> >
> > $ stat Makefile
> >   File: Makefile
> >   Size: 8369            Blocks: 24         IO Block: 4096   regular file
> > Device: fd07h/64775d    Inode: 33655       Links: 1
> > Access: (0644/-rw-r--r--)  Uid: (  106/ luthier)   Gid: (  106/ luthier)
> > Access: 2022-10-31 16:05:51.628660212 +0000
> > Modify: 2022-10-31 16:05:17.746857963 +0000
> > Change: 2022-10-31 16:05:17.746857963 +0000
> >  Birth: -
> 
> I think you're almost certainly running into the parallel checkout,
> which is new in that revision range. Try tweaking checkout.workers and
> checkout.thresholdForParallelism (see "man git-config").

> I can't say without looking at the code/Makefile (and even then, I don't
> have time to dig here:), but if I had to bet I'd say that your
> dependencies have probably always been broken with these checked-in
> files, but they happend to work out if they were checked out in sorted
> order.
>
> And now with the parallel checkout they're not guaranteed to do that, as
> some workers will "race ahead" and finish in an unpredictable order.

> But that's all just a guess, perhaps it has nothing to do with parallel
> checkout, such dependency issues are sensitive to all sorts of other
> things, e.g. maybe git got slightly faster (or slower), so now files
> that were always on different seconds (or the same) aren't in the state
> they were in before...

> On Mon, 31 Oct 2022, Ævar Arnfjörð Bjarmason wrote:
>
>> 
>> On Mon, Oct 31 2022, Mark Hills wrote:
>> 
>> > Our use case: we commit some compiled objects to the repo, where compiling 
>> > is either slow or requires software which is not always available.
>> >
>> > Since upgrading Git 2.26.3 -> 2.32.4 (as part of Alpine Linux OS upgrade) 
>> > we are noticing a change in build behaviour.
>> >
>> > Now, after a "git clone" we find the Makefile intermittently attempting 
>> > (and failing) some builds that are not intended.
>> >
>> > Indeed, Make is acting reasonably as the source file is sometimes 
>> > marginally newer than the destination (both checked out by Git), example 
>> > below.
>> >
>> > I've never had to consider consistency timestamps within a Git checkout 
>> > until now.
>> >
>> > It's entirely possible there's _never_ a guarantee of consistency here.
>> >
>> > But then something has certainly changed in practice, as this fault has 
>> > gone from never happening to now every couple of days.
>> >
>> > Imaginging I can't be the first person to encounter this, I searched for 
>> > existing threads or docs, but overwhemingly the results were question of 
>> > Git tracking the timestamps (as part of the commit) which this is not; 
>> > it's consistency within one checkout.
>> >
>> > $ git clone --depth 1 file:///path/to/repo.git
>> >
>> > $ stat winner.jpeg
>> >   File: winner.jpeg
>> >   Size: 258243          Blocks: 520        IO Block: 4096   regular file
>> > Device: fd07h/64775d    Inode: 33696       Links: 1
>> > Access: (0644/-rw-r--r--)  Uid: (  106/ luthier)   Gid: (  106/ luthier)
>> > Access: 2022-10-31 16:05:17.756858496 +0000
>> > Modify: 2022-10-31 16:05:17.756858496 +0000
>> > Change: 2022-10-31 16:05:17.756858496 +0000
>> >  Birth: -
>> >
>> > $ stat winner.svg
>> >   File: winner.svg
>> >   Size: 52685           Blocks: 112        IO Block: 4096   regular file
>> > Device: fd07h/64775d    Inode: 33697       Links: 1
>> > Access: (0644/-rw-r--r--)  Uid: (  106/ luthier)   Gid: (  106/ luthier)
>> > Access: 2022-10-31 16:05:17.766859030 +0000
>> > Modify: 2022-10-31 16:05:17.766859030 +0000
>> > Change: 2022-10-31 16:05:17.766859030 +0000
>> >  Birth: -
>> >
>> > Elsewhere in the repository, it's clear the timestamps are not consistent:
>> >
>> > $ stat Makefile
>> >   File: Makefile
>> >   Size: 8369            Blocks: 24         IO Block: 4096   regular file
>> > Device: fd07h/64775d    Inode: 33655       Links: 1
>> > Access: (0644/-rw-r--r--)  Uid: (  106/ luthier)   Gid: (  106/ luthier)
>> > Access: 2022-10-31 16:05:51.628660212 +0000
>> > Modify: 2022-10-31 16:05:17.746857963 +0000
>> > Change: 2022-10-31 16:05:17.746857963 +0000
>> >  Birth: -
>> 
>> I think you're almost certainly running into the parallel checkout,
>> which is new in that revision range. Try tweaking checkout.workers and
>> checkout.thresholdForParallelism (see "man git-config").
>
> Thanks, it will be interesting to try this and I'll report back.

>> I can't say without looking at the code/Makefile (and even then, I don't
>> have time to dig here:), but if I had to bet I'd say that your
>> dependencies have probably always been broken with these checked-in
>> files, but they happend to work out if they were checked out in sorted
>> order.
>>
>> And now with the parallel checkout they're not guaranteed to do that, as
>> some workers will "race ahead" and finish in an unpredictable order.
>
> These are very simple Makefile rules, I don't think these dependencies are 
> broken; but your theory is in good alignment with the observed behaviour.
>
> For example, the rule from the recent case above is:
>
>   %.jpeg:         %.png
>                   convert $< $(IMFLAGS) $@
>
>   %.png:          %.svg
>                   inkscape --export-type=png --export-filename=$@ $<

> On Mon, Oct 31 2022, Mark Hills wrote:
>
> > On Mon, 31 Oct 2022, Ævar Arnfjörð Bjarmason wrote:
> >
> >>
> >> On Mon, Oct 31 2022, Mark Hills wrote:
> >>
> >> > Our use case: we commit some compiled objects to the repo, where compiling
> >> > is either slow or requires software which is not always available.
> >> >
> >> > Since upgrading Git 2.26.3 -> 2.32.4 (as part of Alpine Linux OS upgrade)
> >> > we are noticing a change in build behaviour.
> >> >
> >> > Now, after a "git clone" we find the Makefile intermittently attempting
> >> > (and failing) some builds that are not intended.
> >> >
> >> > Indeed, Make is acting reasonably as the source file is sometimes
> >> > marginally newer than the destination (both checked out by Git), example
> >> > below.
> >> >
> >> > I've never had to consider consistency timestamps within a Git checkout
> >> > until now.
> >> >
> >> > It's entirely possible there's _never_ a guarantee of consistency here.
> >> >
> >> > But then something has certainly changed in practice, as this fault has
> >> > gone from never happening to now every couple of days.
> >> >
> >> > Imaginging I can't be the first person to encounter this, I searched for
> >> > existing threads or docs, but overwhemingly the results were question of
> >> > Git tracking the timestamps (as part of the commit) which this is not;
> >> > it's consistency within one checkout.
> >> >
> >> > $ git clone --depth 1 file:///path/to/repo.git
> >> >
> >> > $ stat winner.jpeg
> >> >   File: winner.jpeg
> >> >   Size: 258243          Blocks: 520        IO Block: 4096   regular file
> >> > Device: fd07h/64775d    Inode: 33696       Links: 1
> >> > Access: (0644/-rw-r--r--)  Uid: (  106/ luthier)   Gid: (  106/ luthier)
> >> > Access: 2022-10-31 16:05:17.756858496 +0000
> >> > Modify: 2022-10-31 16:05:17.756858496 +0000
> >> > Change: 2022-10-31 16:05:17.756858496 +0000
> >> >  Birth: -
> >> >
> >> > $ stat winner.svg
> >> >   File: winner.svg
> >> >   Size: 52685           Blocks: 112        IO Block: 4096   regular file
> >> > Device: fd07h/64775d    Inode: 33697       Links: 1
> >> > Access: (0644/-rw-r--r--)  Uid: (  106/ luthier)   Gid: (  106/ luthier)
> >> > Access: 2022-10-31 16:05:17.766859030 +0000
> >> > Modify: 2022-10-31 16:05:17.766859030 +0000
> >> > Change: 2022-10-31 16:05:17.766859030 +0000
> >> >  Birth: -
> >> >
> >> > Elsewhere in the repository, it's clear the timestamps are not consistent:
> >> >
> >> > $ stat Makefile
> >> >   File: Makefile
> >> >   Size: 8369            Blocks: 24         IO Block: 4096   regular file
> >> > Device: fd07h/64775d    Inode: 33655       Links: 1
> >> > Access: (0644/-rw-r--r--)  Uid: (  106/ luthier)   Gid: (  106/ luthier)
> >> > Access: 2022-10-31 16:05:51.628660212 +0000
> >> > Modify: 2022-10-31 16:05:17.746857963 +0000
> >> > Change: 2022-10-31 16:05:17.746857963 +0000
> >> >  Birth: -
> >>
> >> I think you're almost certainly running into the parallel checkout,
> >> which is new in that revision range. Try tweaking checkout.workers and
> >> checkout.thresholdForParallelism (see "man git-config").

>
> > On Mon, Oct 31 2022, Mark Hills wrote:
> >
> > > On Mon, 31 Oct 2022, Ævar Arnfjörð Bjarmason wrote:
> > >
> > >>
> > >> On Mon, Oct 31 2022, Mark Hills wrote:
> > >>
> > >> > Our use case: we commit some compiled objects to the repo, where compiling
> > >> > is either slow or requires software which is not always available.
> > >> >
> > >> > Since upgrading Git 2.26.3 -> 2.32.4 (as part of Alpine Linux OS upgrade)
> > >> > we are noticing a change in build behaviour.
> > >> >
> > >> > Now, after a "git clone" we find the Makefile intermittently attempting
> > >> > (and failing) some builds that are not intended.
> > >> >
> > >> > Indeed, Make is acting reasonably as the source file is sometimes
> > >> > marginally newer than the destination (both checked out by Git), example
> > >> > below.
> > >> >
> > >> > I've never had to consider consistency timestamps within a Git checkout
> > >> > until now.
> > >> >
> > >> > It's entirely possible there's _never_ a guarantee of consistency here.
> > >> >
> > >> > But then something has certainly changed in practice, as this fault has
> > >> > gone from never happening to now every couple of days.
> > >> >
> > >> > Imaginging I can't be the first person to encounter this, I searched for
> > >> > existing threads or docs, but overwhemingly the results were question of
> > >> > Git tracking the timestamps (as part of the commit) which this is not;
> > >> > it's consistency within one checkout.
> > >> >
> > >> > $ git clone --depth 1 file:///path/to/repo.git
> > >> >
> > >> > $ stat winner.jpeg
> > >> >   File: winner.jpeg
> > >> >   Size: 258243          Blocks: 520        IO Block: 4096   regular file
> > >> > Device: fd07h/64775d    Inode: 33696       Links: 1
> > >> > Access: (0644/-rw-r--r--)  Uid: (  106/ luthier)   Gid: (  106/ luthier)
> > >> > Access: 2022-10-31 16:05:17.756858496 +0000
> > >> > Modify: 2022-10-31 16:05:17.756858496 +0000
> > >> > Change: 2022-10-31 16:05:17.756858496 +0000
> > >> >  Birth: -
> > >> >
> > >> > $ stat winner.svg
> > >> >   File: winner.svg
> > >> >   Size: 52685           Blocks: 112        IO Block: 4096   regular file
> > >> > Device: fd07h/64775d    Inode: 33697       Links: 1
> > >> > Access: (0644/-rw-r--r--)  Uid: (  106/ luthier)   Gid: (  106/ luthier)
> > >> > Access: 2022-10-31 16:05:17.766859030 +0000
> > >> > Modify: 2022-10-31 16:05:17.766859030 +0000
> > >> > Change: 2022-10-31 16:05:17.766859030 +0000
> > >> >  Birth: -
> > >> >
> > >> > Elsewhere in the repository, it's clear the timestamps are not consistent:
> > >> >
> > >> > $ stat Makefile
> > >> >   File: Makefile
> > >> >   Size: 8369            Blocks: 24         IO Block: 4096   regular file
> > >> > Device: fd07h/64775d    Inode: 33655       Links: 1
> > >> > Access: (0644/-rw-r--r--)  Uid: (  106/ luthier)   Gid: (  106/ luthier)
> > >> > Access: 2022-10-31 16:05:51.628660212 +0000
> > >> > Modify: 2022-10-31 16:05:17.746857963 +0000
> > >> > Change: 2022-10-31 16:05:17.746857963 +0000
> > >> >  Birth: -
> > >>
> > >> I think you're almost certainly running into the parallel checkout,
> > >> which is new in that revision range. Try tweaking checkout.workers and
> > >> checkout.thresholdForParallelism (see "man git-config").
>
> This does look like something you would see with parallel checkout, yes.
> But...
>
> > > Thanks, it will be interesting to try this and I'll report back.
> >
> > FWIW I was under the impression that we'd made it the default, so unless
> > you opted-in it's probably not that.
>
> ... it indeed should be disabled by default. It seems Mark didn't
> manually enable parallel checkout, as the original message only mentions
> the git upgrade as a changing factor. And Alpine's git installation
> script for 2.32.4 [1] doesn't seem to change our defaults either.
>
> Perhaps, it just happens that 2.32.4 changed the checkout processing
> time slightly so that each entry is finished a bit slower (or the system
> was overloaded at that moment?). Anyways, the creation order (based on
> the mtimes) looks correct to me from a sequential-checkout point of
> view: first Makefile, than winner.jpeg, and finally winner.svg. That's
> the order in which these files would appear in the index, which is the
> order followed by sequential checkout.
>
> [1]: https://git.alpinelinux.org/aports/tree/main/git/APKBUILD?h=3.14-stable&id=0f3285f2cfcb8362460002c27e219fadbf18c885

> Our use case: we commit some compiled objects to the repo, where compiling
> is either slow or requires software which is not always available.
> 
> Since upgrading Git 2.26.3 -> 2.32.4 (as part of Alpine Linux OS upgrade)
> we are noticing a change in build behaviour.
> 
> Now, after a "git clone" we find the Makefile intermittently attempting
> (and failing) some builds that are not intended.
> 
> Indeed, Make is acting reasonably as the source file is sometimes
> marginally newer than the destination (both checked out by Git), example
> below.

> On 2022-10-31 15:01, Mark Hills wrote:
>> Our use case: we commit some compiled objects to the repo, where compiling
>> is either slow or requires software which is not always available.
>> Since upgrading Git 2.26.3 -> 2.32.4 (as part of Alpine Linux OS
>> upgrade)
>> we are noticing a change in build behaviour.
>> Now, after a "git clone" we find the Makefile intermittently
>> attempting
>> (and failing) some builds that are not intended.
>> Indeed, Make is acting reasonably as the source file is sometimes
>> marginally newer than the destination (both checked out by Git), example
>> below.
>
> A fix for this was proposed in 2018 and dismissed [1].
>
> Back then, the problem was that as Git wrote files into a directory
> sometimes the clock would tick over at a bad time, and we'd end up
> with some files being "newer" than others.  This would sour Make runs
> as you describe.
>
> Nominally this is caused by putting generated files in the repo, but
> many times that is unavoidable (e.g. you're forking an upstream that 
> puts automake-generated stuff in the repo).
>
> IMHO, dismissing the problem back then was a mistake.  At the time I
> advocated teaching Git to give all the files it touches (creates or 
> modifies) in a directory the same mtime (e.g. the time at the start of
> the checkout operation).
>
> Instead the decision was to do nothing in Git, and instead let people
> create their own post-checkout hooks to touch the files.  I (and
> others) argued this was inadequate, to no avail.
>
> 		M.
>
> [1] https://public-inbox.org/git/20180413170129.15310-1-mgorny@gentoo.org/#r

> 
> On Tue, Nov 01 2022, Marc Branchaud wrote:
>>
>> Instead the decision was to do nothing in Git, and instead let people
>> create their own post-checkout hooks to touch the files.  I (and others)
>> argued this was inadequate, to no avail.
>> >> [1] 

> 
> I think that's the wrong take-away from that thread. Maybe a patch for
> this will get rejected in the end, but in that case it wasn't because
> the git project is never going to take a patch like this.
> 
> Maybe it won't, but:
> 
>   * That commit has no tests
>   * It's clearly controversial behavior, so *if* we add it I think it's
>     better to make it opt-in configurable.
>   * Once that's done, you'd need doc changes etc. for that.
> 
> Now, maybe a sufficiently polished version would also be "meh" for
> whatever reason, I just think it's premature to say that a change in
> this direction would never be accepted.

> That being said, I do wonder if software in the wild is being
> monkeypatched to work around issues with make (or make-like tools)
> whether such a change isn't better advocated in e.g. GNU make itself.
> 
> If it added "B" to "MAKEFLAGS" if it detected:
> 
>   * I'm in a git repository
>   * It's the first time I'm running here, or "nothing is built yet"
>   * My dependency graph would be different with "-B"
> 
> Wouldn't that be what people who want this feature are after?
> 
> It's not like it's SCM-agnostic, it already goes to significant trouble
> to cater to RCS and SCCS of all things, so I don't see why they'd
> categorically reject a patch to cater to modern VCS's.
> 
> And, unlike Gike, GNU make wouldn't need to guess that munging
> timestamps would fix it, it can compute both versions of the dependency
> graph, so it would know...

>
> ...
> Indeed, Make is acting reasonably as the source file is sometimes
> marginally newer than the destination (both checked out by Git), example
> below.
>
> I've never had to consider consistency timestamps within a Git checkout
> until now.
>
> It's entirely possible there's _never_ a guarantee of consistency here.

> I have little to add on the underlying issue or non-issue but some
> ideas on how to solve your problem
>
> On Mon, Oct 31, 2022 at 8:39 PM Mark Hills <mark@xwax.org> wrote:
>>
>> ...
>> Indeed, Make is acting reasonably as the source file is sometimes
>> marginally newer than the destination (both checked out by Git), example
>> below.
>>
>> I've never had to consider consistency timestamps within a Git checkout
>> until now.
>>
>> It's entirely possible there's _never_ a guarantee of consistency here.
>
> If your makefile depends on checkout, why not
>   git ls-files | xargs touch
> or if this done in an environment where there's not a fresh clone each
> time, maybe
>   git diff HEAD --name-only --diff-filter=AM | xargs touch
> or something along those lines