threads / bug / 32315

(bug?) Inconsistent workdir file timestamps after initial clone.

Subject: (bug?) Inconsistent workdir file timestamps after initial clone.

## tl;dr

7 messages between Dec 11, 2012 and Dec 12, 2012.

replies: 6people: 4as markdown or json

Marc Branchaud· Dec 11, 2012, 20:52 UTC · lore
Hi all,

Occasionally when doing a fresh clone of a repo, if the clock ticks at just the wrong time the checked-out files end up with different timestamps.

The effect of this can be that, when "make" is run in the workdir it'll decide that some files are out of date and try to rebuild them.

(In our particular case, our automated build-bot cloned a submodule of some third-party (i.e. not our) code, where a Makefile.in got an earlier timestamp than its dependent Makefile.am, so "configure && make" then tried to rebuild Makefile.in and the build failed because our build environment has the wrong version of automake.)

I'm completely unfamiliar with the clone-and-checkout parts of git's code, so my first question really is if someone more familiar with the code could look at it (or at least point me to it) to verify whether or not such inconsistent timestamps are possible.

If someone can please confirm that timestamps will always be consistent on the initial checkout of a clone, then I'll have to hunt for a different cause of our build failure.

However, if inconsistent timestamps are possible, I'd like to suggest that this should be fixed. (I'd learn the code and write a patch myself, but as some of you may know I haven't had very much time for git hacking lately.)

Thanks!
		M.
Junio C Hamano· Dec 11, 2012, 21:27 UTC · re: Marc Branchaud · lore

Re: (bug?) Inconsistent workdir file timestamps after initial clone.

Marc Branchaud <marcnarc@xiplink.com> writes:
Show 11 quoted lines
> Occasionally when doing a fresh clone of a repo, if the clock ticks at just
> the wrong time the checked-out files end up with different timestamps.
>
> The effect of this can be that, when "make" is run in the workdir it'll
> decide that some files are out of date and try to rebuild them.
>
> (In our particular case, our automated build-bot cloned a submodule of some
> third-party (i.e. not our) code, where a Makefile.in got an earlier timestamp
> than its dependent Makefile.am, so "configure && make" then tried to rebuild
> Makefile.in and the build failed because our build environment has the wrong
> version of automake.)

Even if you somehow arrange Makefile.in and Makefile.am to have the same timestamp, wouldn't it be up to your "make" to decide which one is newer? Certainly Makefile.in is not newer than Makefile.am, and it is free to try rebuilding it.

Also if you do this after any operation:
    $ rm Makefile.am
    $ git checkout Makefile.am

you will have Makefile.am that is newer than your Makefile.in and you will end up attempting to rebuild it.

The timestamp of a working tree file records the time at which it was created in your working tree. It does not have any relation to the commit or author timestamp of the commit you check it out of. If this command:

    $ git checkout @{1.dacade.ago} Makefile.am

gave your Makefile.am an ancient timestamp, it will break your build.

While not including files that can be rebuilt from the source may be the ideal solution, I've seen projects hide rules to rebuild such a "generated but needs special tools to build" and/or a "generated but normal developers do not have any business rebuilding" file (in your case, Makefile.in) in their Makefiles from the normal targets (like "make all") for this exact reason, when they choose to distribute such files by including in their commits.

Marc Branchaud· Dec 11, 2012, 22:07 UTC · re: Junio C Hamano · lore

Re: (bug?) Inconsistent workdir file timestamps after initial clone.

On 12-12-11 04:27 PM, Junio C Hamano wrote:
Show 18 quoted lines
> Marc Branchaud <marcnarc@xiplink.com> writes:
> 
>> Occasionally when doing a fresh clone of a repo, if the clock ticks at just
>> the wrong time the checked-out files end up with different timestamps.
>>
>> The effect of this can be that, when "make" is run in the workdir it'll
>> decide that some files are out of date and try to rebuild them.
>>
>> (In our particular case, our automated build-bot cloned a submodule of some
>> third-party (i.e. not our) code, where a Makefile.in got an earlier timestamp
>> than its dependent Makefile.am, so "configure && make" then tried to rebuild
>> Makefile.in and the build failed because our build environment has the wrong
>> version of automake.)
> 
> Even if you somehow arrange Makefile.in and Makefile.am to have the
> same timestamp, wouldn't it be up to your "make" to decide which one
> is newer?  Certainly Makefile.in is not newer than Makefile.am, and
> it is free to try rebuilding it.

Well, the makes I've used don't rebuild anything after a "touch *". I think it would surprise a lot of people if their make did rebuild files when their timestamps matched.

Show 7 quoted lines
> Also if you do this after any operation:
> 
>     $ rm Makefile.am
>     $ git checkout Makefile.am
> 
> you will have Makefile.am that is newer than your Makefile.in and
> you will end up attempting to rebuild it.
Yes, of course.  I would never expect otherwise.
Show 9 quoted lines
> The timestamp of a working tree file records the time at which it
> was created in your working tree.  It does not have any relation to
> the commit or author timestamp of the commit you check it out of.
> If this command:
> 
>     $ git checkout @{1.dacade.ago} Makefile.am
> 
> gave your Makefile.am an ancient timestamp, it will break your
> build.
Yes, I agree.

My point is that the initial checkout into an empty working directory should create all files with the same timestamp.

Or, to be a bit more precise, whenever git-checkout *creates* files in the work dir, *all* the created files should have the *same* timestamp (i.e. the current time measured at the start of the checkout's execution, not some bizarro other time specified by some arcane heuristic).

The more I think about it, the more I think it's sloppy for git-checkout to just let the filesystem assign the exact current time to created files. A checkout theoretically should be atomic -- you really shouldn't try to play with any of the files in your workdir while a checkout is underway. It's impractical to really make checkouts atomic, but I think the end result of a checkout should as much as possible look like the checkout happened all at one time.

Show 7 quoted lines
> While not including files that can be rebuilt from the source may be
> the ideal solution, I've seen projects hide rules to rebuild such a
> "generated but needs special tools to build" and/or a "generated but
> normal developers do not have any business rebuilding" file (in your
> case, Makefile.in) in their Makefiles from the normal targets (like
> "make all") for this exact reason, when they choose to distribute
> such files by including in their commits.

I prefer to use the third-party code as-is, without hacking it, to have smooth upgrades in the future.

		M.
Junio C Hamano· Dec 11, 2012, 22:30 UTC · re: Marc Branchaud · lore

Re: (bug?) Inconsistent workdir file timestamps after initial clone.

Marc Branchaud <marcnarc@xiplink.com> writes:
Show 7 quoted lines
> My point is that the initial checkout into an empty working directory should
> create all files with the same timestamp.
>
> Or, to be a bit more precise, whenever git-checkout *creates* files in the
> work dir, *all* the created files should have the *same* timestamp (i.e. the
> current time measured at the start of the checkout's execution, not some
> bizarro other time specified by some arcane heuristic).

My knee-jerk reaction is that it is insane to do so, but what other SCM does such a thing? Even "tar xf" wouldn't do that, I think.

Show 10 quoted lines
>> While not including files that can be rebuilt from the source may be
>> the ideal solution, I've seen projects hide rules to rebuild such a
>> "generated but needs special tools to build" and/or a "generated but
>> normal developers do not have any business rebuilding" file (in your
>> case, Makefile.in) in their Makefiles from the normal targets (like
>> "make all") for this exact reason, when they choose to distribute
>> such files by including in their commits.
>
> I prefer to use the third-party code as-is, without hacking it, to have
> smooth upgrades in the future.

Then perhaps take the complaints to that third-party upstream, not here?

Marc Branchaud· Dec 12, 2012, 15:48 UTC · re: Junio C Hamano · lore

Re: (bug?) Inconsistent workdir file timestamps after initial clone.

On 12-12-11 05:30 PM, Junio C Hamano wrote:
Show 12 quoted lines
> Marc Branchaud <marcnarc@xiplink.com> writes:
> 
>> My point is that the initial checkout into an empty working directory should
>> create all files with the same timestamp.
>>
>> Or, to be a bit more precise, whenever git-checkout *creates* files in the
>> work dir, *all* the created files should have the *same* timestamp (i.e. the
>> current time measured at the start of the checkout's execution, not some
>> bizarro other time specified by some arcane heuristic).
> 
> My knee-jerk reaction is that it is insane to do so, but what other
> SCM does such a thing?
I'm lucky enough to just care about git these days.
> Even "tar xf" wouldn't do that, I think.

"tar xf" uses the timestamps that are stored in the tar file. I see this as an argument against git's exact-current-time-per-file approach: even the tar guys understand that it's insane.

Show 13 quoted lines
>>> While not including files that can be rebuilt from the source may be
>>> the ideal solution, I've seen projects hide rules to rebuild such a
>>> "generated but needs special tools to build" and/or a "generated but
>>> normal developers do not have any business rebuilding" file (in your
>>> case, Makefile.in) in their Makefiles from the normal targets (like
>>> "make all") for this exact reason, when they choose to distribute
>>> such files by including in their commits.
>>
>> I prefer to use the third-party code as-is, without hacking it, to have
>> smooth upgrades in the future.
> 
> Then perhaps take the complaints to that third-party upstream, not
> here?

Well, I thought that while I wait for some dozen-or-so projects to accept changes to their builds, it might be nice for git to solve this problem for me. It is, after all, an effect of the way git operates.

		M.
Torsten Bögershausen· Dec 12, 2012, 17:18 UTC · re: Junio C Hamano · lore

Re: (bug?) Inconsistent workdir file timestamps after initial clone.

 
On 11.12.12 23:30, Junio C Hamano wrote:
Show 13 quoted lines
> Marc Branchaud <marcnarc@xiplink.com> writes:
> 
>> My point is that the initial checkout into an empty working directory should
>> create all files with the same timestamp.
>>
>> Or, to be a bit more precise, whenever git-checkout *creates* files in the
>> work dir, *all* the created files should have the *same* timestamp (i.e. the
>> current time measured at the start of the checkout's execution, not some
>> bizarro other time specified by some arcane heuristic).
> 
> My knee-jerk reaction is that it is insane to do so, but what other
> SCM does such a thing? Even "tar xf" wouldn't do that, I think.
> 
ClearCase is doing such a thing.

You need to check out a file to make it writable: "cleartool checkout main.c" [hack hack] If you after some hacking don't like your changes at all, you run "cleartool unco main.c" (Undo checkout) (In git we just use "git checkout")

While in ClearCase the timestamp of your file jumps back to where it was before the checkout, it gets the current timestamp in git.

One consequence is that ClearCase users may wish to use "ClearMake" rather then make.

A better make (which records all timestamps somewhere) would be helpful.
Show 18 quoted lines
>>> While not including files that can be rebuilt from the source may be
>>> the ideal solution, I've seen projects hide rules to rebuild such a
>>> "generated but needs special tools to build" and/or a "generated but
>>> normal developers do not have any business rebuilding" file (in your
>>> case, Makefile.in) in their Makefiles from the normal targets (like
>>> "make all") for this exact reason, when they choose to distribute
>>> such files by including in their commits.
>>
>> I prefer to use the third-party code as-is, without hacking it, to have
>> smooth upgrades in the future.
> 
> Then perhaps take the complaints to that third-party upstream, not
> here?
> --
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 
Pyeron, Jason J CTR (US)· Dec 12, 2012, 17:26 UTC · re: Torsten Bögershausen · lore

RE: (bug?) Inconsistent workdir file timestamps after initial clone.

Show 38 quoted lines
> -----Original Message-----
> From: Torsten Bögershausen
> Sent: Wednesday, December 12, 2012 12:19 PM
> 
> 
> 
> On 11.12.12 23:30, Junio C Hamano wrote:
> > Marc Branchaud <marcnarc@xiplink.com> writes:
> >
> >> My point is that the initial checkout into an empty working
> directory should
> >> create all files with the same timestamp.
> >>
> >> Or, to be a bit more precise, whenever git-checkout *creates* files
> in the
> >> work dir, *all* the created files should have the *same* timestamp
> (i.e. the
> >> current time measured at the start of the checkout's execution, not
> some
> >> bizarro other time specified by some arcane heuristic).
> >
> > My knee-jerk reaction is that it is insane to do so, but what other
> > SCM does such a thing? Even "tar xf" wouldn't do that, I think.
> >
> 
> 
> ClearCase is doing such a thing.
> 
> You need to check out a file to make it writable:
> "cleartool checkout main.c"
> [hack hack]
> If you after some hacking don't like your changes at all,
> you run
> "cleartool unco main.c" (Undo checkout)
> (In git we just use "git checkout")
> 
> While in ClearCase the timestamp of your file jumps back to where
> it was before the checkout, it gets the current timestamp in git.
I do think that a user preference should decide if git uses metadata timestamps or current timestamp on file operations. I don’t think that the current time is proper as a default operation.
Show 6 quoted lines
> 
> One consequence is that ClearCase users may wish to use "ClearMake"
> rather then make.
> 
> A better make (which records all timestamps somewhere) would be
> helpful.
That is why I always do "make clean" after a rollback. Do you really expect build managers to handle a bi-directional, with regards to time, SDLC? Build managers have worked hard to ensure incremental builds, it is silly to think that they should be re worked.
Show 28 quoted lines
> 
> >>> While not including files that can be rebuilt from the source may
> be
> >>> the ideal solution, I've seen projects hide rules to rebuild such a
> >>> "generated but needs special tools to build" and/or a "generated
> but
> >>> normal developers do not have any business rebuilding" file (in
> your
> >>> case, Makefile.in) in their Makefiles from the normal targets (like
> >>> "make all") for this exact reason, when they choose to distribute
> >>> such files by including in their commits.
> >>
> >> I prefer to use the third-party code as-is, without hacking it, to
> have
> >> smooth upgrades in the future.
> >
> > Then perhaps take the complaints to that third-party upstream, not
> > here?
> > --
> > To unsubscribe from this list: send the line "unsubscribe git" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> >
> 
> --
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

← back to recent threads