[BUG] [git 2.16.1] yeeek ... my files are gone .. by git pull <otherRepositoryUrl>

Hi,

I think we have a problem … or at least I had and I’m not quite sure if this is „working as designed“ but I’m sure it „should not work as it did“.

Because? It pruned a lot of files and even the local repository. by pull by giving another repository URL instead of a known remote

While working in a subpath of my homedir (that is a git repository itself, without any changes in worktree or index: https://bitbucket.org/childnode/marcel/ ) I wanted to clone another repository … but yeah … it’s late for me today and I put in s.th. `git pull git@private.gitlab.instance.example.com:aGroup/repository.git`

next … all committed files are zapped and the repository given has been checked out in my home directory 🤯👻

what? Shouldn’t this just fail? Why can I pass another remote to pull?

🙏 god any untracked / ignored files are still alive

a, yeh, I’m on a mac (for any git configuration … have a look in my repository https://bitbucket.org/childnode/marcel/src/88ff8d0c28bb90dfde3aea9e6c39bb551bea8ca8/.gitconfig?at=master&fileviewer=file-view-default

the console out was:
```
-bash:$ git pull git@private.gitlab.instance.example.com:aGroup/repository.git
Warning: Permanently added the ECDSA host key for IP address '10.1.2.3' to the list of known hosts.
warning: no common commits
remote: Counting objects: 2301, done.
remote: Compressing objects: 100% (710/710), done.
remote: Total 2301 (delta 1040), reused 2239 (delta 1004)
Receiving objects: 100% (2301/2301), 405.41 KiB | 635.00 KiB/s, done.
Resolving deltas: 100% (1040/1040), done.
From private.gitlab.instance.example.com:aGroup/repository
 * branch            HEAD       -> FETCH_HEAD
Fetching submodule .shapps/willgit
Fetching submodule .vim
Fetching submodule .vim/autoload/pathogen
warning: redirecting to https://github.com/tpope/vim-pathogen.git/
Fetching submodule .vim/bundle/ack
warning: redirecting to https://github.com/mileszs/ack.vim.git/
Fetching submodule .vim/bundle/colors-solarized
warning: redirecting to https://github.com/altercation/vim-colors-solarized.git/
Fetching submodule .vim/bundle/flake8
Fetching submodule .vim/bundle/fugitive
warning: redirecting to https://github.com/tpope/vim-fugitive.git/
Fetching submodule .vim/bundle/kwbdi
warning: redirecting to https://github.com/vim-scripts/kwbdi.vim.git/
Fetching submodule .vim/bundle/markdown
warning: redirecting to https://github.com/tpope/vim-markdown.git/
Fetching submodule .vim/bundle/nerdtree
warning: redirecting to https://github.com/scrooloose/nerdtree.git/
Fetching submodule .vim/bundle/unimpaired
warning: redirecting to https://github.com/tpope/vim-unimpaired.git/
Fetching submodule gists/bitbucket/childnode/2015-06-16_G4pLy_prevent-empty-version-comment-in.git
Fetching submodule gists/bitbucket/childnode/2015-06-21_kyAAM_plasticscm-addcurrentworkdir-batch-task
Fetching submodule gists/github/childnode/18de20f4448692257aa3e99c8319b70d
Fetching submodule gists/github/childnode/295dbd6e_hasSize.regex
Fetching submodule gists/github/childnode/4a0de936_gradle_buildSrc_dogfood
Fetching submodule gists/github/childnode/66d4b982_git.rebaseAllBranches
Fetching submodule gists/github/childnode/6df6d14c_ideaGradleProjectSetupForAdditionalSourceSets
Fetching submodule gists/github/childnode/81ae6468_build_jar_manifest.gradle
Fetching submodule gists/github/childnode/85958ff8_extendedHelp.gradle_secret
Fetching submodule gists/github/childnode/88304258_git_deleteAllRemoteBranches.sh
Fetching submodule gists/github/childnode/8f100f90_dockerSaveAllImages.sh
Fetching submodule gists/github/childnode/9741c4d1_idea.warnGenerateWorkspace.gradle_secret
Fetching submodule gists/github/childnode/a175d954_ext.props.gradle_secret
Fetching submodule gists/github/childnode/d15cd5e9_atlassian-confluence-config
Fetching submodule gists/github/childnode/d35cf810dd28775ac5c0e491107215fd
Fetching submodule gists/github/childnode/da08e8a6f989ce0f94077ae1a6b1573b
Fetching submodule gists/github/childnode/e7ef876c_html2ical_secret
Fetching submodule gists/github/childnode/eb3199790f2f82785f62c3150f352ede
Successfully rebased and updated refs/heads/master.

```

trying to fix this up by doing another pull failed: ``` -bash:$ git remote -v origin git@bitbucket.org:childnode/marcel.git (fetch) origin git@bitbucket.org:childnode/marcel.git (push)

-bash:$ git pull fatal: refusing to merge unrelated histories

-bash:$ git pull git@bitbucket.org:childnode/marcel.git
From bitbucket.org:childnode/marcel
 * branch            HEAD       -> FETCH_HEAD
fatal: refusing to merge unrelated histories

```

these messages and the fact that it doesn’t work backward let me think I ran into a collision? really?

revlog looks a bit strange too ``` 04f3066 (HEAD -> master) HEAD@{0}: pull git@private.gitlab.instance.example.com:aGroup/repository.git: checkout 04f3066d03e09323c7341c472be4c45ea5e3a4ff: returning to refs/heads/master 04f3066 (HEAD -> master) HEAD@{1}: pull git@private.gitlab.instance.example.com:aGroup/repository.git: checkout 04f3066d03e09323c7341c472be4c45ea5e3a4ff 88ff8d0 (origin/master, origin/HEAD) HEAD@{2}: pull --rebase=preserve: checkout 88ff8d0c28bb90dfde3aea9e6c39bb551bea8ca8: returning to refs/heads/master ```

where 88ff8d0 was the latest from marcel.git and logs look like the pulled repository is like an orphan aside my original one.

~Marcel

P.S: I reverted by `$ git reset 88ff8d0 && git checkout -- .`

Hi Marcel,

Marcel 'childNo͡.de' Trautwein" wrote:

[...]

Sorry, this is not the most helpful reply but:

Can you describe a reproduction recipe so that I can experience the same thing?

That is:

 1. steps to reproduce
 2. expected result
 3. actual result
 4. the difference and why it was unexpected

I suspect that this information is in your message, somewhere, but it is (understandably) unfocussed and I am having trouble pulling it out.

[...]

Ok, this part is something I might be able to help shed some light on.

Searching for 'unrelated' in "git help pull" finds:

       --allow-unrelated-histories
	   By default, git merge command refuses to merge histories that do not
	   share a common ancestor. This option can be used to override this
	   safety when merging histories of two projects that started their
	   lives independently. As that is a very rare occasion, no
	   configuration variable to enable this by default exists and will not
	   be added.

So that explains the "what" of that error message.

The "why" is a separate question.  Could you share output from

  git log --all --graph --decorate --oneline --simplify-by-decoration

and

  git status

to help us understand your current state?

Also, suggestions for improvements to the 'refusing to merge' message would be very welcome.

Thanks and hope that helps, Jonathan

1. steps to reproduce
=====================
```
Last login: Fri Feb 23 00:33:11 on ttys001
~ PATH variable not enhanced, no applications found in ~/Applications/*-latest

-bash:/Users/marcel:$ mkdir /tmp/$$ change to new directory '/tmp/2608'? [Y/n]

-bash:/tmp/2608:$ mkdir a.git change to new directory 'a.git'? [Y/n]

-bash:/tmp/2608/a.git:$ git init Initialized empty Git repository in /private/tmp/2608/a.git/.git/

-bash:/tmp/2608/a.git:$ touch foo

-bash:/tmp/2608/a.git:$ git add foo

-bash:/tmp/2608/a.git:$ git commit -m "foo" foo
[master (root-commit) ed191c4] foo
 1 file changed, 0 insertions(+), 0 deletions(-)
 create mode 100644 foo

-bash:/tmp/2608/a.git:$ cd - /tmp/2608

-bash:/tmp/2608:$ mkdir b.git change to new directory 'b.git'? [Y/n]

-bash:/tmp/2608/b.git:$ git init Initialized empty Git repository in /private/tmp/2608/b.git/.git/

-bash:/tmp/2608/b.git:$ touch bar

-bash:/tmp/2608/b.git:$ git add bar

-bash:/tmp/2608/b.git:$ git commit -m "bar" bar
[master (root-commit) 80b0355] bar
 1 file changed, 0 insertions(+), 0 deletions(-)
 create mode 100644 bar

-bash:/tmp/2608/b.git:$ cd - /tmp/2608

-bash:/tmp/2608:$ git clone a.git c Cloning into 'c'... done.

-bash:/tmp/2608:$ cd c

-bash:/tmp/2608/c:$ ll total 0 drwxr-xr-x 12 marcel wheel 384B 23 Feb 05:47 .git -rw-r--r-- 1 marcel wheel 0B 23 Feb 05:47 foo

-bash:/tmp/2608/c:$ git pull ../b.git/
warning: no common commits
remote: Counting objects: 3, done.
remote: Total 3 (delta 0), reused 0 (delta 0)
Unpacking objects: 100% (3/3), done.
From ../b
 * branch            HEAD       -> FETCH_HEAD
Successfully rebased and updated refs/heads/master.

-bash:/tmp/2608/c:$ ll total 0 drwxr-xr-x 14 marcel wheel 448B 23 Feb 05:47 .git -rw-r--r-- 1 marcel wheel 0B 23 Feb 05:47 bar

-bash:/tmp/2608/c:$ git reflog 80b0355 (HEAD -> master) HEAD@{0}: pull ../b.git/: checkout 80b03552466bc526b1130ce5ca4a991ba31a0546: returning to refs/heads/master 80b0355 (HEAD -> master) HEAD@{1}: pull ../b.git/: checkout 80b03552466bc526b1130ce5ca4a991ba31a0546 ed191c4 (origin/master, origin/HEAD) HEAD@{2}: clone: from /tmp/2608/a.git

-bash:/tmp/2608/c:$ git remote -v origin /tmp/2608/a.git (fetch) origin /tmp/2608/a.git (push)

-bash:/tmp/2608/c:$  git log --all --graph --decorate --oneline --simplify-by-decoration
* 80b0355 (HEAD -> master) bar
* ed191c4 (origin/master, origin/HEAD) foo
```

2. expected result
==================
just an error in case the too trees have no common ancestors

```
-bash:/tmp/2608/c:$ git pull ../b.git/
warning: no common commits
remote: Counting objects: 3, done.
remote: Total 3 (delta 0), reused 0 (delta 0)
Unpacking objects: 100% (3/3), done.
From ../b
 * branch            HEAD       -> FETCH_HEAD
fatal: refusing to merge unrelated histories
```

3. actual result
================
pulls out, removes all files from the first tree

4. the difference and why it was unexpected
===========================================
I can’t find words on it … it should not work but it did? somehow … with unexpected results to my local repository

it somehow seems to be an issue of my config, because resetting it, will not allow the pull as expected

``` -bash:/tmp/2608/c:$ GIT_CONFIG_NOSYSTEM=1 HOME=. git config -l core.repositoryformatversion=0 core.filemode=true core.bare=false core.logallrefupdates=true core.ignorecase=true core.precomposeunicode=true remote.origin.url=/tmp/2608/a.git remote.origin.fetch=+refs/heads/*:refs/remotes/origin/* branch.master.remote=origin branch.master.merge=refs/heads/master

-bash:/tmp/2608/c:$ GIT_CONFIG_NOSYSTEM=1 HOME=. git pull ../b.git/
warning: no common commits
remote: Counting objects: 3, done.
remote: Total 3 (delta 0), reused 0 (delta 0)
Unpacking objects: 100% (3/3), done.
From ../b
 * branch            HEAD       -> FETCH_HEAD
fatal: refusing to merge unrelated histories

-bash:/tmp/2608/c:$ git pull ../b.git/
From ../b
 * branch            HEAD       -> FETCH_HEAD
Successfully rebased and updated refs/heads/master.
```

the logs tells me he rebases ... ``` -bash:/tmp/2608/c:$ git config -l | grep merge diff.tool=p4merge merge.tool=p4merge merge.branchdesc=true merge.log=true branch.autosetupmerge=true branch.master.merge=refs/heads/master

-bash:/tmp/2608/c:$ git config -l | grep pull pull.rebase=preserve

-bash:/tmp/2608/c:$ git config -l | grep fetch fetch.recursesubmodules=true remote.origin.fetch=+refs/heads/*:refs/remotes/origin/* ```

I’m sorry, I just wanted to write down first any helpful information without - being late - having time to go into further investigations myself … hopefully to get some answers that „this is ok, you’re just stupid and didn’t read the spec/doc“ or „ok, this seems strange to me too, can we go forward in details and analysis“ ;)

```
-bash:/tmp/2608/c:$ git status
On branch master
Your branch and 'origin/master' have diverged,
and have 1 and 1 different commits each, respectively.
  (use "git pull" to merge the remote branch into yours)

nothing to commit, working tree clean

-bash:/tmp/2608/c:$ git branch -avv
* master                80b0355 [origin/master: ahead 1, behind 1] bar
  remotes/origin/HEAD   -> origin/master
  remotes/origin/master ed191c4 foo

```

just to say: the latter „refused pull“ is what I expect the first time too!

SUSPECTED / PROVEN to found a guilty setting
=============================================
```
-bash:/tmp/2608/c:$ GIT_CONFIG_NOSYSTEM=1 HOME=. git -c user.name="me" -c user.email=me@example.com pull -r ../b.git/
warning: no common commits
remote: Counting objects: 3, done.
remote: Total 3 (delta 0), reused 0 (delta 0)
Unpacking objects: 100% (3/3), done.
From ../b
 * branch            HEAD       -> FETCH_HEAD
First, rewinding head to replay your work on top of it...
Applying: foo

-bash:/tmp/2608/c:$ ll total 0 drwxr-xr-x 14 marcel wheel 448B 23 Feb 06:21 .git -rw-r--r-- 1 marcel wheel 0B 23 Feb 06:21 bar -rw-r--r-- 1 marcel wheel 0B 23 Feb 06:21 foo

``` shows me a quite different behavior, so solely rebase not seems the full problem BUT `--rebase=preserve` will .. o’man , really, is this intended?

Yes, I see, rebase is a harmful operation and yes, you might now tell me a fool to apply this per default

but why is this unrelated history check different from pull (merge) to rebase to rebase -p ? Can’t find a word on it https://git-scm.com/docs/git-rebase#git-rebase---preserve-merges

With regards, ~Marcel

On Fri, Feb 23, 2018 at 06:29:55AM +0100, "Marcel 'childNo͡.de' Trautwein" wrote:

Yeah, the bug seems to be in --preserve-merges. Here's an easier reproduction:

  git init
  >one && git add one && git commit -m one
  git checkout --orphan other
  git mv one two && git commit -m two

  git rebase --preserve-merges master

at which point we've dropped commit "two" and its files entirely.

I don't know much about how preserve-merges works. It looks like in the loop around git-rebase--interactive.sh:931 that we mark commit "two" with preserve=t, and so we _don't_ add it to the list of commits to pick.

I think that stems from the fact that it has no parent marked to be rewritten.

So something like this helps:

Hi Peff,

On Fri, 23 Feb 2018, Jeff King wrote:

Your analysis makes sense to me. Please note, though, that I would not consider myself a guru on preserve-merges. I think this mode is broken by design (you can blame me if you want).

Ciao, Dscho

On Tue, Feb 27, 2018 at 12:33:56AM +0100, Johannes Schindelin wrote:

I think that is doing the right thing for half of the problem. But there's something else funny where we do not include the "upstream" commits from the split history (i.e., we rebase onto nothing, whereas a normal "git rebase" with a split history will graft the two together).

I haven't figured that part out. But I do kind of wonder if we should simply punt early and say "what you're doing is crazy and unsupported". After all, the only known case of this was somebody doing it accidentally.

-Peff

Hi Peff,

On Wed, 28 Feb 2018, Jeff King wrote:

Let me ask to make sure I am understanding you correctly. Are you referring to "split history" as the case where the commit graph has *two* root commits?

If so: when you perform a merge-preserving rebase, then those two root commits will be recreated as new root commits, by design.

The non-merge-preserving mode cannot create two root commits, as it does not allow for introducing merge commits (and you'd need that to combine the two strands).

It is quite possible that I misunderstand completely, though. Care to enlighten me?

Ciao, Dscho

On Mon, Mar 05, 2018 at 03:49:13PM +0100, Johannes Schindelin wrote:

Yes, I mean two root commits. But especially when one is in the history to be rebased, and the other is in the "upstream" history.

So as a concrete example, if I have this repo:

  git init
  >one && git add one && git commit -m one
  git checkout --orphan other
  git mv one two && git commit -m two

and I do this:

  git rebase master

I end up with a two-commit history, with "two" on top of "one". That makes sense to me. Similarly if I instead do:

  git rebase -i master

the todo list has "pick two", and if I leave it as-is then I get the same history. But if I do:

  git rebase --preserve-merges master

I end up with a single-commit history, with only commit "one". That's wrong, because it threw away the history on the "other" branch.

If I apply the patch I showed earlier, then I get a single-branch history, but this time it contains only "two". That also seems wrong, because we didn't build on top of "master". I'd expect this command to give the same results as a non-merge-preserving rebase.

Does that make more sense?

-Peff