threads / discuss / 28443

Possible timestamp problems with diff-files?

Subject: Possible timestamp problems with diff-files?

## tl;dr

5 messages between Sep 20, 2011 and Sep 24, 2011.

replies: 4people: 4as markdown or json

Marc Strapetz· Sep 20, 2011, 10:30 UTC · lore

For our Git client, we are invoking

git diff-files--quiet --ignore-submodules

immediately after a commit of *all* changes. Hence, the expected exit code would be 0 (because there are no changes). A user has now reported that for commits with many changes, exit code is sometimes 1. For the last incident, the commit was started at 15:24:11,820 and finished at 15:24:12,329, diff-files was invoked at 15:24:12,455 and failed with exit code 1 at 15:24:21,394. A subsequent diff-files succeeded, so I'm wondering now, if that could be a timestamp problem (maybe related to the Index)?

Note that there are a couple of threads running in the client itself, so I guess actual running time of diff-files was much shorter than 9 seconds and chances are that some parts of diff-files were run in the same second 15:24:12.

System configuration is: Linux splitter 2.6.38-11-server #48-Ubuntu SMP Fri Jul 29 19:20:32 UTC 2011 x86_64 x86_64 x86_64 GNU/Linux

git version 1.7.4.1

-- Best regards, Marc Strapetz ============= syntevo GmbH http://www.syntevo.com http://blog.syntevo.com

Jeff King· Sep 20, 2011, 17:54 UTC · re: Marc Strapetz · lore

Re: Possible timestamp problems with diff-files?

On Tue, Sep 20, 2011 at 12:30:53PM +0200, Marc Strapetz wrote:

Show 12 quoted lines

> For our Git client, we are invoking
> 
> git diff-files--quiet --ignore-submodules
> 
> immediately after a commit of *all* changes. Hence, the expected exit
> code would be 0 (because there are no changes). A user has now reported
> that for commits with many changes, exit code is sometimes 1. For the
> last incident, the commit was started at 15:24:11,820 and finished at
> 15:24:12,329, diff-files was invoked at 15:24:12,455 and failed with
> exit code 1 at 15:24:21,394. A subsequent diff-files succeeded, so I'm
> wondering now, if that could be a timestamp problem (maybe related to
> the Index)?

diff-files is scriptable plumbing, which means it is up to the script writer to decide exactly when the index should be refreshed with respect to the working tree files (because doing so could be kind of expensive, as it needs to stat every file in the working tree). Have you tried running "git update-index --refresh" just before your diff-files?

-Peff

Marc Strapetz· Sep 21, 2011, 12:58 UTC · re: Jeff King · lore

Re: Possible timestamp problems with diff-files?

On 20.09.2011 19:54, Jeff King wrote:

Show 20 quoted lines

> On Tue, Sep 20, 2011 at 12:30:53PM +0200, Marc Strapetz wrote:
> 
>> For our Git client, we are invoking
>>
>> git diff-files--quiet --ignore-submodules
>>
>> immediately after a commit of *all* changes. Hence, the expected exit
>> code would be 0 (because there are no changes). A user has now reported
>> that for commits with many changes, exit code is sometimes 1. For the
>> last incident, the commit was started at 15:24:11,820 and finished at
>> 15:24:12,329, diff-files was invoked at 15:24:12,455 and failed with
>> exit code 1 at 15:24:21,394. A subsequent diff-files succeeded, so I'm
>> wondering now, if that could be a timestamp problem (maybe related to
>> the Index)?
> 
> diff-files is scriptable plumbing, which means it is up to the script
> writer to decide exactly when the index should be refreshed with respect
> to the working tree files (because doing so could be kind of expensive,
> as it needs to stat every file in the working tree). Have you tried
> running "git update-index --refresh" just before your diff-files?

My point is that "git diff-files --quiet" seems to returns 1 and when invoked a short time later (without modifying working tree or invoking any other Git command) it returns 0. This would indicate a bug in Git, I guess. I think we can add more debug logging to our client to track down that problem. However, to do that I'd need some input on what to log?

-- Best regards, Marc Strapetz ============= syntevo GmbH http://www.syntevo.com http://blog.syntevo.com

Junio C Hamano· Sep 21, 2011, 21:33 UTC · re: Marc Strapetz · lore

Re: Possible timestamp problems with diff-files?

Marc Strapetz <marc.strapetz@syntevo.com> writes:

Show 15 quoted lines

> On 20.09.2011 19:54, Jeff King wrote:
>> On Tue, Sep 20, 2011 at 12:30:53PM +0200, Marc Strapetz wrote:
>> 
>>> For our Git client, we are invoking
>>>
>>> git diff-files--quiet --ignore-submodules
>>>
>>> immediately after a commit of *all* changes. Hence, the expected exit
>>> code would be 0 (because there are no changes). A user has now reported
>>> that for commits with many changes, exit code is sometimes 1. For the
>>> last incident, the commit was started at 15:24:11,820 and finished at
>>> 15:24:12,329, diff-files was invoked at 15:24:12,455 and failed with
>>> exit code 1 at 15:24:21,394. A subsequent diff-files succeeded, so I'm
>>> wondering now, if that could be a timestamp problem (maybe related to
>>> the Index)?

What peff said already.

If you do not refresh the cached stat information, diff-files may report "they differ" for a path that is otherwise unchanged without looking at the contents of such a path to notice that the only difference is the cached stat information (the whole and only point of having the cached stat information is to avoid looking at the contents). Also, it may look at the contents of such a path if it has a reason to suspect that the file might have changed if it cannot tell from the cached stat information (look for "racy-git" if you are really interested).

Update the cached stat information before you use plumbing commands in your script.

Robin Rosenberg· Sep 24, 2011, 14:30 UTC · re: Junio C Hamano · lore

Re: Possible timestamp problems with diff-files?

Junio C Hamano skrev 2011-09-21 23.33:

Show 31 quoted lines

> Marc Strapetz<marc.strapetz@syntevo.com>  writes:
>
>> On 20.09.2011 19:54, Jeff King wrote:
>>> On Tue, Sep 20, 2011 at 12:30:53PM +0200, Marc Strapetz wrote:
>>>
>>>> For our Git client, we are invoking
>>>>
>>>> git diff-files--quiet --ignore-submodules
>>>>
>>>> immediately after a commit of *all* changes. Hence, the expected exit
>>>> code would be 0 (because there are no changes). A user has now reported
>>>> that for commits with many changes, exit code is sometimes 1. For the
>>>> last incident, the commit was started at 15:24:11,820 and finished at
>>>> 15:24:12,329, diff-files was invoked at 15:24:12,455 and failed with
>>>> exit code 1 at 15:24:21,394. A subsequent diff-files succeeded, so I'm
>>>> wondering now, if that could be a timestamp problem (maybe related to
>>>> the Index)?
>
> What peff said already.
>
> If you do not refresh the cached stat information, diff-files may report
> "they differ" for a path that is otherwise unchanged without looking at
> the contents of such a path to notice that the only difference is the
> cached stat information (the whole and only point of having the cached
> stat information is to avoid looking at the contents). Also, it may look
> at the contents of such a path if it has a reason to suspect that the file
> might have changed if it cannot tell from the cached stat information
> (look for "racy-git" if you are really interested).
>
> Update the cached stat information before you use plumbing commands in
> your script.

Using git diff instead of git diff-files would do just that, unless I am missing something.

-- robin