git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: Consist timestamps within a checkout/clone

From
Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Date
Nov 1, 2022, 18:34 UTC
Message-ID
<221101.867d0e4ixy.gmgdl@evledraar.gmail.com>
In-Reply-To
<d4db484f-a525-f6db-1bfb-922f788dacd@xwax.org>
On Mon, Oct 31 2022, Mark Hills wrote:
Show 73 quoted lines
> On Mon, 31 Oct 2022, Taylor Blau wrote:
>
>> On Mon, Oct 31, 2022 at 09:21:20PM +0100, Ævar Arnfjörð Bjarmason wrote:
>> > I think you're almost certainly running into the parallel checkout,
>> > which is new in that revision range. Try tweaking checkout.workers and
>> > checkout.thresholdForParallelism (see "man git-config").
>> >
>> > I can't say without looking at the code/Makefile (and even then, I don't
>> > have time to dig here:), but if I had to bet I'd say that your
>> > dependencies have probably always been broken with these checked-in
>> > files, but they happend to work out if they were checked out in sorted
>> > order.
>> >
>> > And now with the parallel checkout they're not guaranteed to do that, as
>> > some workers will "race ahead" and finish in an unpredictable order.
>> 
>> Doesn't checkout.thresholdForParallelism only matter when
>> checkout.workers != 1?
>> 
>> So what you wrote seems like a reasonable explanation, but only if the
>> original reporter set checkout.workers to imply the non-sequential
>> behavior in the first place.
>> 
>> That said...
>> 
>>   - I also don't know off-hand of a place where we've defined the order
>>     where Git will checkout files in the working copy. So depending on
>>     that behavior isn't a safe thing to do.
>> 
>>   - Committing build artifacts into your repository is generally
>>     discouraged.
>
> If it's undefined and never implemented this is reasonable.
>
> But "generally" is a caveat, so while I agree with the statement it also 
> implies there's valid cases outside of that. Ones which used to work, too.
>
> Here are some useful cases I have seen for the combination of build rule + 
> checked in file:
>
> - part of a build requires licensed software that's not always available
>
> - part of the build requires large memory that other builders generally do 
>   not have available
>
> - part of the build process uses a different platform or some other system 
>   requirement
>
> - to fetch data eg. from a URL, with a record of the URL/automation but 
>   also a copy of the file as a record and for offline use
>
> So it's useful, to retain repeatable automation but not always build from 
> square one.
>
> Generally discouraged to check in build results yes, but I've found it 
> very practical.
>  
>> So while I'd guess that setting `checkout.workers` back to "1" (if it 
>> wasn't already) will probably restore the existing behavior, counting on 
>> that behavior in the first place is wrong.
>
> I think perhaps the tail is wagging the dog here, though.
>
> It's 'wrong' because it doesn't work; but I haven't seen anything to make 
> me think this is fundamentally or theoretically flawed.
>
> If we had a transactional file system we'd reasonably expect a checkout to 
> be an atomic operation -- same timestamp on the files created in that 
> step. A discrepancy in timestamps would be considered incorrect; it would 
> imply an 'order' to the checkout which, as you say, is order-less.
>
> Sowhat could be the bad outcomes if Git created files stamped with the 
> point in time of the "git checkout"?

I agree that it's practical in some scenarios, including checking in built assets.

But those that are doing that need to be aware that combining that sort of thing with source control tends to upend your build system's idea of the world.

E.g. until recently in git.git we had a po/git.pot in-tree, which is a "compiled file" (although a plain-text one) that was checked in, and dealing with that in make's dependency graph was a (minor) pain sometimes.

Previous: rsbecker@nexbridge.comNext: Mark Hills
Message 6 of 17 in “Consist timestamps within a checkout/clone”
  1. Mark HillsOct 31, 2022
  2. Ævar Arnfjörð BjarmasonOct 31, 2022
  3. Taylor BlauOct 31, 2022
  4. Mark HillsOct 31, 2022
  5. rsbecker@nexbridge.comOct 31, 2022
  6. Ævar Arnfjörð BjarmasonNov 1, 2022
  7. Mark HillsOct 31, 2022
  8. Ævar Arnfjörð BjarmasonNov 1, 2022
  9. Matheus TavaresNov 2, 2022
  10. Matheus TavaresNov 2, 2022
  11. Andreas SchwabOct 31, 2022
  12. Marc BranchaudNov 1, 2022
  13. Ævar Arnfjörð BjarmasonNov 2, 2022
  14. Marc BranchaudNov 3, 2022
  15. Erik Cervin EdinNov 1, 2022
  16. Ævar Arnfjörð BjarmasonNov 1, 2022
  17. Erik Cervin EdinNov 3, 2022

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.