[suggestion] Fail cherry-pick if it overwrites untracked files with the matching file names

Hello there,

I have filled a bug-report and it is available at https://github.com/rupinderjeet/git-cherry-pick-removes-untracked-files

Since your email only receives plain-text, I assume I can't send the report by uploading it in this email. If there's something I can do from my end, or if you need some more information, please let me know.

From, Rupinderjeet Singh Hans.

Rupinderjeet Singh <rupinderjeet47@gmail.com> writes:

You do not have to assume that.

"git bugreport" gives a template to create a text file, which you can paste/include to the body of your e-mail message without making it an attachment.

If the situation is about cherry-picking a commit that adds a new file F to a checkout of another commit that lacks the file F, I think the command is working exactly as designed.

    $ git init
    $ git commit -m 'initial' --allow-empty
    $ git tag initial
    $ date >file
    $ git add file
    $ git commit -m 'add file'
    $ git tag added
    $ git checkout -b second initial
    ... at this point we are back to the original state
    ... without 'file'
    $ >file
    ... the file is untracked with precious contents
    ... and the presence of it stops a cherry-pick that clobbers it
    $ git cherry-pick added
    error: The following untracked working tree files would be overwritten by merge:
            file
    Please move or remove them before you merge.
    Aborting
    fatal: cherry-pick failed

Now continuing from the above, things get (slightly) interesting

    $ echo file >.gitignore
    $ git cherry-pick added

This will replace "file" with the one from the "added" commit, and that is because the user marked that the "file" in the working tree is expendable.

Files in a working tree controlled by git fall into one of three classes. Tracked files are those that are known to the index and appear in "git ls-files" output. Among the others, ignored files are those that .gitignore mechanism says are expendable. The rest are "untracked", possibly containing valuable contents that should not be lost as the user may choose to 'git add' them later.

Not just cherry-pick but any merge-related operations, including "checkout", follow this semantics. Untracked files are kept, but ignored files are expendable and will be removed if they are in the way to complete the operation the user asks.

    $ rm .gitignore
    $ git checkout master
    error: The following untracked working tree files would be overwritten by checkout:
            file
    Please move or remove them before you switch branches.
    Aborting

    $ echo file >.gitignore
    $ git checkout master
    ... this should succeed, removing "file" whose contents were
    ... marked expendable.

Of course, after switching to 'master' (or cherry-picking 'added'), the project now cares about the 'file'. After all, it bothered to add it to keep track of the changes to its contents. So it is recommended that you would adjust the contents of .gitignore so that it no longer is marked as expendable.

Yes, thank you for being so through with the example. I understand now.

I want to ask if you would suggest that local configuration (or any information required by the project) that is too sensitive to be tracked in git, should either be kept as 'untracked' files or 'outside of git repository'.

Since, any merge or similar action can do this, it should be considered risky to keep files with important information in .gitignore file.

As an example, when I am working on an android project, I usually add firebase-integration info and keystore details in the .gitignore, because I don't want to add them to version control (even by mistake). I have seen it as a good practice suggested by many devs around me, codelabs, online courses, and from experienced developers at google too.

With your explanation, am I correct to think that only the following
kind of information is suitable to be put in .gitignore files?
1. that can be regenerated
2. that doesn't matter when it is lost
3. that isn't used by the files tracked in git repository

Honestly, the name .gitignore implies that git will completely ignore these files. Overwriting an ignored file does seem like a violation of the rule present in .gitignore file. But, if the implementation is intended as you describe, I have definitely learned something new.

Thank you for your answer. My bad for not including the report-file as text, it slipped my mind.

From, Rupinderjeet Singh Hans.

On Sun, 16 Oct 2022 at 00:05, Junio C Hamano <gitster@pobox.com> wrote:

Rupinderjeet Singh <rupinderjeet47@gmail.com> writes:

I think it depends on the project and participants may not have control over it if it is ingrained in the project's build structure, but a separate place may likely be more appropriate, as it would reduce the chance of accidental "git add ." adding everything.

Yes.

Natural consequence of the above

I do not get this, so I decline to comment ;) Is mylib.o that is "ignored", created from tracked mylib.c source, used by mymain.c source that is tracked, when mymain.o would not link without what mylib.o gives it?

Some other SCMs have an extra class "precious" to handle exactly the case you have in mind. You do not accidentally let "git add ." to slurp them into the committed history, but you do want them to be left alone. We don't have it, and that is not because we have any reason to be against having it. It is just it didn't happen.

And nobody bothered to explore the ramification of adding the new class yet. We know about "checkout" and other mergy operations and "add", but there probably are trickier interactions with other parts of the existing system that somebody need to carefully go through and make sure it does not introduce funny inconsistencies.

On Sat, Oct 15, 2022 at 12:23 PM Junio C Hamano <gitster@pobox.com> wrote: [...]

There is a command line option meant to allow tweaking this behavior: --[no-]overwrite-ignore. The default, as Junio explains above, is --overwrite-ignore. The --no-overwrite-ignore option works for `git checkout`. It also can work for `git merge`, but *only* if the merge results in a fast-forward. Although there is code in unpack_trees() to handle this alternate behavior by just setting a simple flag, we don't bother setting that flag in most merge paths even when the user specifies that option. It was just never hooked up. And several other related commands (am, reset, stash, rebase) don't even accept such a flag even though for consistency they probably should.

I've been meaning to get back to it and hook up this command line flag
for the occasional user that wants this alternative behavior.
However, it's not an issue for me or the direct users I support, and
complaints about this behavior from the community are quite rare too,
so I just haven't been too motivated to work on it.  If someone else
wants to take a stab, it should be pretty easy.  Just grep for
"preserve_ignored.*FIXME".  And maybe read up on these two commits for
context:
    04988c8d18 ("unpack-trees: introduce preserve_ignored to
unpack_trees_options", 2021-09-27)
    1b5f37334a ("Remove ignored files by default when they are in the
way", 2021-09-27)

Maybe I should just tag this as #leftoverbits to see if someone else picks it up?

Elijah Newren <newren@gmail.com> writes:

Ah, I totally forgot that relatively recent invention. Thanks for bringing it up.

I personally consider it a bit too blunt an instrument, and in the longer term, it would probably be a better direction to introduce a new class of paths that are not tracked, and call the class "precious", which (1) is similar to "ignored" in that the paths in the class do not get added with "git add ." but (2) unlike "ignored" but like "untracked", they are not considered to be expendable.

Until that happens, though, treating all "ignored" paths as if they are "precious" may be a usable workaround.