Re: [RFC PATCH] checkout: Force matching mtime between files

From: Robin H. Johnson <robbat2@gentoo.org>
Date: Apr 25, 2018, 07:13 UTC
Message-ID: <robbat2-20180425T060717-325652820Z@orbis-terrarum.net>
In-Reply-To: <xmqqtvs18p9o.fsf@gitster-ct.c.googlers.com>

On Tue, Apr 24, 2018 at 08:41:07AM +0900, Junio C Hamano wrote:

Show 15 quoted lines

> "Robin H. Johnson" <robbat2@gentoo.org> writes:
> 
> > On Fri, Apr 13, 2018 at 07:01:29PM +0200, Michał Górny wrote:
> >> Currently git does not control mtimes of files being checked out.  This
> >> means that the only assumption you could make is that all files created
> >> or modified within a single checkout action will have mtime between
> >> start time and end time of this checkout.  The relations between mtimes
> >> of different files depend on the order in which they are checked out,
> >> filesystem speed and timestamp precision.
> >> ...
> > Junio: ping for review or inclusion of this patch?
> 
> I personally did not think this is a good idea and not worth any
> code contamination with calls to utime().  Is there anybody sane who
> thought this was a good idea in the discussion thread?

Nobody responded to the original message until after I pinged about it again.

Since that, one person DID respond, stating that it fixed an issue they had previously reported 6 years ago.

In the thread from 6 years ago, you asked about tar's behavior for mtimes. 'tar xf' restores mtimes from the tar archive, so relative ordering after restore would be the same, and would only rebuild if the original source happened to be dirty.

This behavior is already non-deterministic in Git, and would be improved by the patch.

On a machine with high resolution timestamps or large enough repo that checkout takes a long time, an initial checkout of multiple files does not guarantee the ordering of mtimes of those files. Checking out (A,B) could wind up with them having a different relative mtimes.

For this example, we are doing a checkout of two files A,B being written (either due to initial checkout, or both have changed for some reason).

The example system has this as a property:
- "touch A B" => mtime(A) < mtime(B)
- "touch B A" => mtime(A) > mtime(B)
[touch should not re-order arguments, nor apply the same mtime to all
files. Linux touch at this point makes the syscall of 'utimensat(0,
NULL, NULL, 0)' on each file descriptor]

Existing behavior: mtime(A), mtime(B) are independent of each other, and depend on the exact order of file checkout, along with the resolution of timestamps and how much other work is taking place. If the filesystem has low resolution of timestamps, or the checkout is sufficiently small/fast, the mtimes are likely to be identical already.

New behavior: Strictly mtime(A) == mtime(B)

Example makefile rule:
B: A

Human explanation of makefile rule: file B depends on file A

If the build system triggers on:

* mtime(A) > mtime(B): [strictly greater]
** Old Behavior: It will depend on the exact checkout order. Sometimes
   it will already not rebuild.
** New Behavior: always rebuild, as mtime(A) == mtime(B)

* mtime(A) >= mtime(B): [greater or equal]
** Old Behavior: it will depend on the exact checkout order. Sometimes it
   will, sometimes it won't.
** New Behavior: will not rebuild, as mtime(A) == mtime(B)

-- 
Robin Hugh Johnson
Gentoo Linux: Dev, Infra Lead, Foundation Treasurer
E-Mail   : robbat2@gentoo.org
GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85
GnuPG FP : 7D0B3CEB E9B85B1F 825BCECF EE05E6F6 A48F6136