Re: Weirdness with git change detection

From: Michael J Gruber <git@grubix.eu>
Date: Aug 18, 2017, 06:56 UTC
Message-ID: <fa4b367d-a984-6f42-a2bd-6bfcabed7c2a@grubix.eu>
In-Reply-To: <20170711082435.a53iuw26oxawcfyq@sigill.intra.peff.net>

Jeff King venit, vidit, dixit 11.07.2017 10:24:

Show 35 quoted lines

> On Tue, Jul 11, 2017 at 10:20:43AM +0200, Torsten Bögershausen wrote:
> 
>>> No problem. I actually think it would be interesting if Git could
>>> somehow detect and warn about this situation. But the obvious way to do
>>> that would be to re-run the clean filter directly after checkout. And
>>> doing that all the time is expensive.
>>
>> Would it be possible to limit the round-trip-check to "git reset --hard" ?
>> If yes, possibly many users are willing to pay the price, if Git tells
>> the "your filters don't round-trip". (Including CRLF conversions)
> 
> Anything's possible, I suppose. But I don't think I'd want that feature
> turned on myself.
> 
>>> Perhaps some kind of "lint" program would be interesting to warn of
>>> possible misconfigurations. Of course people would have to run it for it
>>> to be useful. :)
>>
>> What do you have in mind here ?
>> Don't we need to run some content through the filter(s)?
> 
> I was thinking of a tool that could run a series of checks on the
> repository and nag about potential problems. One of them could be doing
> a round-trip repo->clean->smudge for each file.
> 
> Another one might be warning about files that differ only in case.
> 
> The idea being that users could run "git lint" if they suspect something
> funny is going on. I dunno. It may be a dead-end. Most such
> oddities are better detected and handled during actual git operations if
> we can. So this would really just be for things that are too expensive
> to detect in normal operations.
> 
> -Peff
>

Typically, that problem arises when you turn a filter on or off at some point in your history. Since "attributes" can come from various sources, especially the versioned ".gitattributes" file, unversioned per-repo .git/info/attributes, and global attributes, "git diff" may apply different attributes depending on what you diff (versioned blob, workdir file, out-of-tree file).

This is not made easier by the fact that unversioned config (per repo, per user, global) defines the filter action, and that even upgrades of your filter tools may change the output. So, "filter off/on" is by no means the only possible source of discrepancies.

I've found that when I decide to use a filter like that, the best approach is to either apply it retroactively (filter-branch, unversionsed attributes, that is clean all stored blobs) or make a commit where I specifically note the switch (versioned .gitattributes plus affected blob changes) and what config should go along with it.

All of this is difficult to check or correct automatically, since it depends on user decisions.

About the only thing we could do is checking that "clean(smudge(foo))=clean(foo)" at a specific "point in time" (attributes, config) for specific foo, but that wouldn't catch the case above, even if we iterated over all commits which affect files that the filter (currently) applies to.

Keep in mind that filters are a killer feature, so if you shoot yourself in the foot: it could have come worse ;)

Michael