Re: [PATCH 1/1] Introduce "precious" file concept

From: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Date: Feb 20, 2019, 10:46 UTC
Message-ID: <87ftsi68ke.fsf@evledraar.gmail.com>
In-Reply-To: <CACsJy8B15hORnaOdYW8TNE3Gniv9NBJopyLYmHR5iF0U3beq6g@mail.gmail.com>

On Wed, Feb 20 2019, Duy Nguyen wrote:

Show 50 quoted lines

> On Wed, Feb 20, 2019 at 4:19 PM Ævar Arnfjörð Bjarmason
> <avarab@gmail.com> wrote:
>> > I personally do not believe in "backup log"; if we can screw up and
>> > can fail to stop an operation that must avoid losing info, then we
>> > can screw up the same way and fail to design and implement "backup"
>> > to save info before an operation loses it.
>>
>> Yes, there could be some unforseen interaction between git commands
>> where we should have such a backup log, but did not think to implement
>> it. I'd hope such cases would be reported, and we could fix them.
>>
>> But those sorts of cases aren't why we started discussing this, rather
>> we *know* what the data shredding command interaction is, but there
>> wasn't a consensus for just not shredding data by default by making
>> users use "checkout -f" or "merge -f" to proceed. I.e. taking some
>> variant of my "trashable" patch[1].
>>
>> > If we do a good job in
>> > supporting "precious" in various operations, we can rely less on
>> > "backup log" and still be safe ;-)
>>
>> Is noted in previous discussions[2] I think that's entirely
>> implausible. I think at best the "precious" facility will be used to
>> mark e.g *.o files as "don't check in, but don't clean (Makefile handles
>> it)".
>>
>> Most git users are at the level of only knowing very basic
>> add/commit/pull/push command interaction. I feel strongly that we need
>> to make our tools safe to use by default, and not require some
>> relatively advanced "precious"/attribute facility to be carefully
>> configured in advance so we don't throw away uncommitted work on the
>> likes of merge/checkout.
>
> There is a trade off somewhere. "new user first" should not come at
> the cost for more experienced users.
>
> Making "git checkout/merge" abort while it's working before breaks
> scripts. And requiring to mark trashable files manually duplicates a
> lot of ignore patterns. Have a look at any .gitignore file, the
> majority of them is for discardable files because "ignored" class was
> created with those in mind (*.o and friends). So now you would need to
> add more or less the same set of ignore rules in .gitattributes to
> mark them trashable, and gitignore/gitattributes rules are not exactly
> compatible, you can't just blindly copy them over. Every time you add
> one more .gitignore rule, there's a good chance you need to add a
> similar rule for trashable attribute.
>
> Maybe we just add a new "newbie" config knob and turn on the safety
> nets on. Leave the knob on by default. And I will turn it off in my
> ~/.gitconfig as soon as it's real.

Oh yes, as noted upthread ("My commentary on this whole thing..."[1] ) my position on what we should do at this point is not that we should definitely go one way or the other, but that more investigation is needed.

As my "trashable"[2] patch makes clear we don't even have good tests or documentation for these cases, which would be a good first step.

The one thing that *is* clear from my digging a few months back is that the behavior we have now in git is overzealous when we look at the initial case reported by Shawn way back when it was added.

Specifically, the intention back in 2007 was to fix a case where "git checkout" ("read-tree -m", but whatever) would barf on a branch switch where switching needed to replace a *tracked* "smth" with a *tracked* "smth/file", or the other way around[3].

Does that mean we can just back that behavior out? No, because people might have come to rely on it, but we should start with seeing exactly what it *does* do, whether all those things are important or intended, and maybe we can weight the shredding/backcompat trade-off for some of those differently than others.

So the obvious thing to try would be to see if we can narrowly keep the behavior where we end up shredding a file on disk, *but* are switching between two trees A & B where that have/don't have that file.

Or more generously, try to "git hash-object" arbitrary files we're about to shred, and check if it's already in the object database. That would catch case where e.g. the user switching from A->B and would shred a file, but it (or conflicting dir) is known to neither "A" nor "B", but exists as a checked-in file in unrelated commit "C", which the user recently had checked out (and e.g. their editor auto saved it as-is or something...).

But it's entirely possible that after all that digging we'll come to the conclusion that we can't change this at all, and we're just going to live with all the current caveats.

That doesn't mean that having what amounts to a power user feature to mitigate that damage if you know git well enough that it's going to be a problem is going to help anything but a small minority of users. So "dude, where's my data?" problem will still exist.

Even then there room to maneuver, e.g.:

 X. Perhaps after investigating it's not acceptable to change the
    default for script use, but could we require --force if we detect
    that we're connected to a terminal?

 Y. Or if even that is considered too much, we could have something like
    how help.autoCorrect works, where if we detect we're about to eat
    data we wait for 10 seconds, and invite the user to Ctrl+C now
    because we're about to clobber file "xyz".

 Z. It's for whatever reason still unacceptable to do X or Y (or some
    similar mitigation) for all cases of file shredding, but would be OK
    for some specific sub-cases (e.g. the not known to git-hash-object
    case above), and we have reason to suspect that such a narrow
    mitigation strikes the right trade-off between backwards
    compatibility and preventing the "dude, where's my data?" reports we
    get about this periodically.

1. https://public-inbox.org/git/87wolzo7a1.fsf@evledraar.gmail.com/
2. https://public-inbox.org/git/87zhuf3gs0.fsf@evledraar.gmail.com/
3. https://public-inbox.org/git/87wopj3661.fsf@evledraar.gmail.com/