threads / bug / 50924

[BUG] rebase --interactive silently overwrites ignored files

Subject: [BUG] rebase --interactive silently overwrites ignored files

## tl;dr

8 messages between Apr 11, 2019 and May 7, 2019.

replies: 7people: 4as markdown or json

wh· Apr 11, 2019, 23:56 UTC · lore

I'm using git 2.20.1 from Debian. Git is usually careful not to overwrite untracked files, including ignored files. But interactive rebase doesn't detect this (non-interactive rebase works fine).

Reproduction: -----

#!/bin/sh mkdir upstream cd upstream git init echo 1 >feature-1 git add feature-1 git commit -m "feature 1"

cd .. git clone upstream local cd local # write some tools for our own convenience echo ours >tools echo /tools >>.git/info/exclude # start working on a feature git checkout -b f2 echo wip >feature-2 git add feature-2 git commit -m "wip"

cd ../upstream # official tools are available echo theirs >tools git add tools git commit -m "tools"

cd ../local git fetch ../upstream master

# this would be okay #git rebase FETCH_HEAD

# problem: overwrites tools silently GIT_EDITOR=true git rebase -i FETCH_HEAD

cat tools
-----
Expected: `git rebase -i` fails because it would have to overwrite the
untracked "tools" file. Contents of tools file remains `ours`.
Actual: Contents of tools file becomes `theirs`.
Phillip Wood· Apr 12, 2019, 16:30 UTC · re: wh · lore

Re: [BUG] rebase --interactive silently overwrites ignored files

Hi
On 12/04/2019 00:56, wh wrote:
> I'm using git 2.20.1 from Debian. Git is usually careful not to
> overwrite untracked files, including ignored files.

Git normally overwrites ignored files, so I think in your example rebase -i is working as expected, I'm surprised that the am based rebase does not overwrite the ignored file. There has been some discussion about introducing 'precious' files that are ignored but protected in the same way as untracked files [1].

Best Wishes
Phillip
[1] https://public-inbox.org/git/20190409102649.22115-1-pclouds@gmail.com/
  But interactive
Show 49 quoted lines
> rebase doesn't detect this (non-interactive rebase works fine).
> 
> Reproduction:
> -----
> 
> #!/bin/sh
> mkdir upstream
> cd upstream
> git init
> echo 1 >feature-1
> git add feature-1
> git commit -m "feature 1"
> 
> cd ..
> git clone upstream local
> cd local
> # write some tools for our own convenience
> echo ours >tools
> echo /tools >>.git/info/exclude
> # start working on a feature
> git checkout -b f2
> echo wip >feature-2
> git add feature-2
> git commit -m "wip"
> 
> cd ../upstream
> # official tools are available
> echo theirs >tools
> git add tools
> git commit -m "tools"
> 
> cd ../local
> git fetch ../upstream master
> 
> # this would be okay
> #git rebase FETCH_HEAD
> 
> # problem: overwrites tools silently
> GIT_EDITOR=true git rebase -i FETCH_HEAD
> 
> cat tools
> 
> -----
> 
> Expected: `git rebase -i` fails because it would have to overwrite the
> untracked "tools" file. Contents of tools file remains `ours`.
> 
> Actual: Contents of tools file becomes `theirs`.
> 
wh· Apr 14, 2019, 01:59 UTC · re: Phillip Wood · lore

Re: [BUG] rebase --interactive silently overwrites ignored files

Thanks for the info about the upcoming "precious" attribute. Looks useful.

I didn't get the impression that Git normally overwrites ignored files. I ran some more experiments:

git rebase FETCH_HEAD # bails git rebase -i FETCH_HEAD # overwrites git merge FETCH_HEAD # bails git reset --keep FETCH_HEAD # bails git reset --merge FETCH_HEAD # bails git checkout FETCH_HEAD # overwrites # without feature 2 locally: git merge FETCH_HEAD # overwrites, fast-forwards git merge --no-ff FETCH_HEAD # bails

On Fri, Apr 12, 2019 at 9:30 AM Phillip Wood <phillip.wood123@gmail.com> wrote:
Show 69 quoted lines
>
> Hi
>
> On 12/04/2019 00:56, wh wrote:
> > I'm using git 2.20.1 from Debian. Git is usually careful not to
> > overwrite untracked files, including ignored files.
>
> Git normally overwrites ignored files, so I think in your example rebase
> -i is working as expected, I'm surprised that the am based rebase does
> not overwrite the ignored file. There has been some discussion about
> introducing 'precious' files that are ignored but protected in the same
> way as untracked files [1].
>
> Best Wishes
>
> Phillip
>
> [1] https://public-inbox.org/git/20190409102649.22115-1-pclouds@gmail.com/
>
>   But interactive
> > rebase doesn't detect this (non-interactive rebase works fine).
> >
> > Reproduction:
> > -----
> >
> > #!/bin/sh
> > mkdir upstream
> > cd upstream
> > git init
> > echo 1 >feature-1
> > git add feature-1
> > git commit -m "feature 1"
> >
> > cd ..
> > git clone upstream local
> > cd local
> > # write some tools for our own convenience
> > echo ours >tools
> > echo /tools >>.git/info/exclude
> > # start working on a feature
> > git checkout -b f2
> > echo wip >feature-2
> > git add feature-2
> > git commit -m "wip"
> >
> > cd ../upstream
> > # official tools are available
> > echo theirs >tools
> > git add tools
> > git commit -m "tools"
> >
> > cd ../local
> > git fetch ../upstream master
> >
> > # this would be okay
> > #git rebase FETCH_HEAD
> >
> > # problem: overwrites tools silently
> > GIT_EDITOR=true git rebase -i FETCH_HEAD
> >
> > cat tools
> >
> > -----
> >
> > Expected: `git rebase -i` fails because it would have to overwrite the
> > untracked "tools" file. Contents of tools file remains `ours`.
> >
> > Actual: Contents of tools file becomes `theirs`.
> >
Phillip Wood· Apr 14, 2019, 13:24 UTC · re: wh · lore

Re: [BUG] rebase --interactive silently overwrites ignored files

On 14/04/2019 02:59, wh wrote:
> Thanks for the info about the upcoming "precious" attribute. Looks useful.
Hmm, unfortunately it's not looking so hopeful now [1]
Show 12 quoted lines
> I didn't get the impression that Git normally overwrites ignored
> files. I ran some more experiments:
> 
> git rebase FETCH_HEAD        # bails
> git rebase -i FETCH_HEAD     # overwrites
> git merge FETCH_HEAD         # bails
> git reset --keep FETCH_HEAD  # bails
> git reset --merge FETCH_HEAD # bails
> git checkout FETCH_HEAD      # overwrites
> # without feature 2 locally:
> git merge FETCH_HEAD         # overwrites, fast-forwards
> git merge --no-ff FETCH_HEAD # bails

Thanks for sharing that, I was assuming the other commands behaved like checkout, I'm intrigued that merge behaves differently to rebase -i as they use the same underlying machinery, I'm a bit snowed under at the moment but I'll try and take a look at what's going sometime in the not too distant future.

Best Wishes
Phillip

[1] https://public-inbox.org/git/CACsJy8AEZ-Lz6zgEsuNukvphB9TTa9FAC1gK05fhnie2xtfc9w@mail.gmail.com/T/#u

Show 70 quoted lines
> On Fri, Apr 12, 2019 at 9:30 AM Phillip Wood <phillip.wood123@gmail.com> wrote:
>>
>> Hi
>>
>> On 12/04/2019 00:56, wh wrote:
>>> I'm using git 2.20.1 from Debian. Git is usually careful not to
>>> overwrite untracked files, including ignored files.
>>
>> Git normally overwrites ignored files, so I think in your example rebase
>> -i is working as expected, I'm surprised that the am based rebase does
>> not overwrite the ignored file. There has been some discussion about
>> introducing 'precious' files that are ignored but protected in the same
>> way as untracked files [1].
>>
>> Best Wishes
>>
>> Phillip
>>
>> [1] https://public-inbox.org/git/20190409102649.22115-1-pclouds@gmail.com/
>>
>>    But interactive
>>> rebase doesn't detect this (non-interactive rebase works fine).
>>>
>>> Reproduction:
>>> -----
>>>
>>> #!/bin/sh
>>> mkdir upstream
>>> cd upstream
>>> git init
>>> echo 1 >feature-1
>>> git add feature-1
>>> git commit -m "feature 1"
>>>
>>> cd ..
>>> git clone upstream local
>>> cd local
>>> # write some tools for our own convenience
>>> echo ours >tools
>>> echo /tools >>.git/info/exclude
>>> # start working on a feature
>>> git checkout -b f2
>>> echo wip >feature-2
>>> git add feature-2
>>> git commit -m "wip"
>>>
>>> cd ../upstream
>>> # official tools are available
>>> echo theirs >tools
>>> git add tools
>>> git commit -m "tools"
>>>
>>> cd ../local
>>> git fetch ../upstream master
>>>
>>> # this would be okay
>>> #git rebase FETCH_HEAD
>>>
>>> # problem: overwrites tools silently
>>> GIT_EDITOR=true git rebase -i FETCH_HEAD
>>>
>>> cat tools
>>>
>>> -----
>>>
>>> Expected: `git rebase -i` fails because it would have to overwrite the
>>> untracked "tools" file. Contents of tools file remains `ours`.
>>>
>>> Actual: Contents of tools file becomes `theirs`.
>>>
Phillip Wood· May 2, 2019, 15:45 UTC · re: wh · lore

Re: [BUG] rebase --interactive silently overwrites ignored files

Hi wh
On 14/04/2019 02:59, wh wrote:
Show 14 quoted lines
> Thanks for the info about the upcoming "precious" attribute. Looks useful.
> 
> I didn't get the impression that Git normally overwrites ignored
> files. I ran some more experiments:
> 
> git rebase FETCH_HEAD        # bails
> git rebase -i FETCH_HEAD     # overwrites
> git merge FETCH_HEAD         # bails
> git reset --keep FETCH_HEAD  # bails
> git reset --merge FETCH_HEAD # bails
> git checkout FETCH_HEAD      # overwrites
> # without feature 2 locally:
> git merge FETCH_HEAD         # overwrites, fast-forwards
> git merge --no-ff FETCH_HEAD # bails

I've had a look at the rebase -i code and I think it only overwrites ignored files when it is fast-forwarding. This matches what merge does when fast-forwarding but I'm not convinced either of them should be doing this by default (I think checkout doing it is probably asking for trouble), as you point out most operations preserve ignored files. When pull fast-forwards it does not overwrite ignored files.

Both checkout and merge have an undocumented --overwrite-ignore option which is on by default. Checkout and fast-forward merges overwriting ignored files dates back a long way to commit 1127148089 ("Loosen "working file will be lost" check in Porcelain-ish", 2006-12-04). There does not seem to have been much discussion of changing the semantics of ignored files around the patches that implemented this [1,2]. There is a brief mention of doing it in this thread [3] but no one seems to comment on the idea as far as I can see. Before this change ignored files were just ignored by status and add but otherwise treated like any other untracked file.

I'll put a patch together to fix rebase -i, I'd like to see the defaults for merge and checkout changed but I'm not sure that would be popular. It does seem like surprising behavior though when most operations try to preserve untracked files.

Best Wishes
Phillip

[1] https://public-inbox.org/git/7vlklnkv39.fsf@assigned-by-dhcp.cox.net/T/#u [2] https://public-inbox.org/git/7vbqmjkuzd.fsf@assigned-by-dhcp.cox.net/T/#u [3]https://public-inbox.org/git/Pine.LNX.4.64.0610081657400.3952@g5.osdl.org/

Show 70 quoted lines
> On Fri, Apr 12, 2019 at 9:30 AM Phillip Wood <phillip.wood123@gmail.com> wrote:
>>
>> Hi
>>
>> On 12/04/2019 00:56, wh wrote:
>>> I'm using git 2.20.1 from Debian. Git is usually careful not to
>>> overwrite untracked files, including ignored files.
>>
>> Git normally overwrites ignored files, so I think in your example rebase
>> -i is working as expected, I'm surprised that the am based rebase does
>> not overwrite the ignored file. There has been some discussion about
>> introducing 'precious' files that are ignored but protected in the same
>> way as untracked files [1].
>>
>> Best Wishes
>>
>> Phillip
>>
>> [1] https://public-inbox.org/git/20190409102649.22115-1-pclouds@gmail.com/
>>
>>    But interactive
>>> rebase doesn't detect this (non-interactive rebase works fine).
>>>
>>> Reproduction:
>>> -----
>>>
>>> #!/bin/sh
>>> mkdir upstream
>>> cd upstream
>>> git init
>>> echo 1 >feature-1
>>> git add feature-1
>>> git commit -m "feature 1"
>>>
>>> cd ..
>>> git clone upstream local
>>> cd local
>>> # write some tools for our own convenience
>>> echo ours >tools
>>> echo /tools >>.git/info/exclude
>>> # start working on a feature
>>> git checkout -b f2
>>> echo wip >feature-2
>>> git add feature-2
>>> git commit -m "wip"
>>>
>>> cd ../upstream
>>> # official tools are available
>>> echo theirs >tools
>>> git add tools
>>> git commit -m "tools"
>>>
>>> cd ../local
>>> git fetch ../upstream master
>>>
>>> # this would be okay
>>> #git rebase FETCH_HEAD
>>>
>>> # problem: overwrites tools silently
>>> GIT_EDITOR=true git rebase -i FETCH_HEAD
>>>
>>> cat tools
>>>
>>> -----
>>>
>>> Expected: `git rebase -i` fails because it would have to overwrite the
>>> untracked "tools" file. Contents of tools file remains `ours`.
>>>
>>> Actual: Contents of tools file becomes `theirs`.
>>>
Duy Nguyen· May 2, 2019, 16:22 UTC · re: Phillip Wood · lore

Re: [BUG] rebase --interactive silently overwrites ignored files

On Thu, May 2, 2019 at 10:45 PM Phillip Wood <phillip.wood123@gmail.com> wrote:
Show 8 quoted lines
>
> Hi wh
>
> On 14/04/2019 02:59, wh wrote:
> > Thanks for the info about the upcoming "precious" attribute. Looks useful.
> >
> > I didn't get the impression that Git normally overwrites ignored
> > files.
Please do not include me in these discussions again. Thank you.
-- 
Duy
Junio C Hamano· May 5, 2019, 04:02 UTC · re: Phillip Wood · lore

Re: [BUG] rebase --interactive silently overwrites ignored files

Phillip Wood <phillip.wood123@gmail.com> writes:
Show 5 quoted lines
> I've had a look at the rebase -i code and I think it only overwrites
> ignored files when it is fast-forwarding. This matches what merge does
> when fast-forwarding but I'm not convinced either of them should be
> doing this by default (I think checkout doing it is probably asking
> for trouble)...
> I'll put a patch together to fix rebase -i, I'd like to see the
> defaults for merge and checkout changed but I'm not sure that would be
> popular.

I suspect that such a change in behaviour for "rebase -i" is not a fix but a regression. Shouldn't it be consistent with checkout and merge?

> It does seem like surprising behavior though when most
> operations try to preserve untracked files.
Are you conflating untracked and ignored?

Because we haven't adopted 'precious' (or whatever the final name would be), which is "ignored but not expendable", ignored files are by definition "ignored and expendable". When checkout (and merge that is its more general form) needs to match the working tree to the index contents and an ingored file is in the way, it should overwrite it.

Until we introduce "ignored but not expendable" class, that is.
Phillip Wood· May 7, 2019, 14:03 UTC · re: Junio C Hamano · lore

Re: [BUG] rebase --interactive silently overwrites ignored files

Hi Junio
On 05/05/2019 05:02, Junio C Hamano wrote:
Show 15 quoted lines
> Phillip Wood <phillip.wood123@gmail.com> writes:
> 
>> I've had a look at the rebase -i code and I think it only overwrites
>> ignored files when it is fast-forwarding. This matches what merge does
>> when fast-forwarding but I'm not convinced either of them should be
>> doing this by default (I think checkout doing it is probably asking
>> for trouble)...
> 
>> I'll put a patch together to fix rebase -i, I'd like to see the
>> defaults for merge and checkout changed but I'm not sure that would be
>> popular.
> 
> I suspect that such a change in behaviour for "rebase -i" is not a
> fix but a regression.  Shouldn't it be consistent with checkout and
> merge?

My problem is that I don't really understand why some operations should overwrite ignored files and others shouldn't. Having nearly all rebase operations preserve ignored files and only overwriting them for a fast-forward seems confusing and probably surprising to users. Is it a bug that pull does not overwrite ignored files when fast-forwarding but merge does? Why is it ok to overwrite ignored files when merge fast-forwards but not otherwise?

Show 8 quoted lines
>> It does seem like surprising behavior though when most
>> operations try to preserve untracked files.
> 
> Are you conflating untracked and ignored?
> 
> Because we haven't adopted 'precious' (or whatever the final name
> would be), which is "ignored but not expendable", ignored files are
> by definition "ignored and expendable".  

My confusion is that they are only "ignored and expendable" in certain circumstances. Many git operations actually fail if they are going to overwrite an ignored file [1] - I don't understand if that is intentional or not.

Best Wishes
Phillip

[1] https://public-inbox.org/git/CAL_tzDFQQtDYMStN+RDVYN_TzJmO+kufMhG9PGHwvsUWREpgWQ@mail.gmail.com/

Show 7 quoted lines
> When checkout (and merge
> that is its more general form) needs to match the working tree to
> the index contents and an ingored file is in the way, it should
> overwrite it.
> 
> Until we introduce "ignored but not expendable" class, that is.
> 

← back to recent threads