threads / discuss / 2862

git merge questions

Subject: git merge questions

## tl;dr

9 messages between Dec 16, 2005 and Dec 17, 2005.

replies: 8people: 3as markdown or json

Don Zickus· Dec 16, 2005, 20:05 UTC · lore

I now have a git merge question. I decided to try merging branches and was left with a situation in which the file was moved on the master branch and updated on my branch. Not sure how to properly integrate the changes from my branch.

For example: %git checkout -b test v2.6.14 %<manually apply all the 2.6.14.z stable patches> %<commit those patches> %git checkout -b test2 v2.6.15-rc4 %git pull . test

Now over the course of 2.6.15 the arch/ppc64 was renamed to arch/powerpc. Fine. The merge algorithms handled all the unchanged files properly. However arch/ppc64/Kconfig was modified and the merging was left unresolved. In fact there is no file no merge against (because it moved). So 'git-ls-files -u' only shows stage 1+3 (no stage 2, of course).

How do I merge those changes? I don't know enough about all the git commands to figure this out, especially how to take advantage of the stage 1, 2, and 3 files.

Thanks for any help, Don

Junio C Hamano· Dec 16, 2005, 20:41 UTC · re: Don Zickus · lore

Re: git merge questions

Don Zickus <dzickus@gmail.com> writes:
Show 10 quoted lines
> Now over the course of 2.6.15 the arch/ppc64 was renamed to
> arch/powerpc.  Fine. The merge algorithms handled all the unchanged
> files properly.  However arch/ppc64/Kconfig was modified and the
> merging was left unresolved.  In fact there is no file no merge
> against (because it moved).  So 'git-ls-files -u' only shows stage 1+3
> (no stage 2, of course).
>
> How do I merge those changes?  I don't know enough about all the git
> commands to figure this out, especially how to take advantage of the
> stage 1, 2, and 3 files.

I do not work on the kernel myself so I'll probably get the details wrong, but I am guessing here is what you have:

 - 2.6.14.z has arch/ppc64/Kconfig;
 - Your "test" branch has arch/ppc64/Kconfig.  You may or may
   not have changes since 2.6.14.z on that file;
 - "test2" branch has 2.6.15-rc4, which has the file at
   arch/powerpc/Kconfig;

Then, with the current branch being "test2", you pulled "test" into it.

 - Untouched [by whom???] files from arch/{ppc64,powerpc}/ were
   merged correctly and you find the result of merge under
   arch/powerpc --- after all you are applying changes on top of
   2.6.15-rc4 and you want them under arch/powerpc as the
   upstream has;
 - arch/ppc64/Kconfig was modified [by whom???] and merge was
   left unresolved.
I suspect these:
	ls -l arch/ppc64/Kconfig <1>
	git ls-files -s arch/ppc64/Kconfig <2>
        ls -l arch/powerpc/Kconfig arch/powerpc/Kconfig~* <3>
        git ls-files -s arch/powerpc/Kconfig <4>
<1> has the file from your "test" branch based on 2.6.14z;
<2> shows that same file;
<3> may show the three variants (I am not sure about this);
<4> has 2.6.15-rc4 version at stage 3 and merge base version at
    stage 1 (I am not absolutely certain what stage 1 has, and
    also am puzzled why stage 2 is not there --- if the
    recursive strategy figured out renames it should have the
    contents of arch/ppc64/Kconfig from test branch there).

The safest (however most primitive) way I can think of to go from here is:

1. figure out the merge-base Kconfig file's contents.
        $ mb=$(git merge-base refs/heads/test refs/heads/test2)
        $ git ls-tree $(mb) arch/powerpc/Kconfig arch/ppc/Kconfig
   I do not know which path the merge base has, but it should
   have either one of those.  Note the blob object name (call it
   $oSHA1).
2. extract three versions.
	$ git ls-tree refs/heads/test2 arch/powerpc/Kconfig
        $ git ls-tree refs/heads/test arch/powerpc/Kconfig
   Note the blob object name from these two as well (call them
   $aSHA1, and $bSHA1).
3. merge those three.
	$ git cat-file blob $oSHA1 >orig
        $ git cat-file blob $aSHA1 >from-test2
        $ git cat-file blob $bSHA1 >from-test
        $ merge from-test2 orig from-test

With luck from-test2 would contain a clean automerge result, or you will get <<< === >>> conflict markers. Resolve them in the file, and then:

4. resolve the index and remove cruft.
	$ cat from-test2 >arch/powerpc/Kconfig
        $ rm -f arch/ppc64/Kconfig orig from-test2 from-test
        $ git update-index --add --remove \
        	arch/ppc64/Kconfig arch/powerpc/Kconfig
5. if there is no other merge cruft, commit.
	$ git commit

What puzzles me is that I think it is supposed to have done all the above for you. Namely:

	$ ls -l arch/ppc64/Kconfig <1>
	$ git ls-files -s arch/ppc64/Kconfig <2>
        $ ls -l arch/powerpc/Kconfig arch/powerpc/Kconfig~* <3>
        $ git ls-files -s arch/powerpc/Kconfig <4>

<1> should not remain --- recursive merge could have notied that the file was moved. <2> ditto. <3> arch/powerpc/Kconfig should be there with possibly merge conflict markers. <4> stage1 with merge base version (possibly renamed), stage2 with test2 version, stage3 with your test version (definitely renamed).

and after editing arch/powerpc/Kconfig to resolve conflicts, you should be able to just say:

	$ git update-index arch/powerpc/Kconfig
	$ git commit
Don Zickus· Dec 16, 2005, 21:13 UTC · re: Junio C Hamano · lore

Re: git merge questions

On 12/16/05, Junio C Hamano <junkio@cox.net> wrote:
Show 16 quoted lines
> Don Zickus <dzickus@gmail.com> writes:
>
> > Now over the course of 2.6.15 the arch/ppc64 was renamed to
> > arch/powerpc.  Fine. The merge algorithms handled all the unchanged
> > files properly.  However arch/ppc64/Kconfig was modified and the
> > merging was left unresolved.  In fact there is no file no merge
> > against (because it moved).  So 'git-ls-files -u' only shows stage 1+3
> > (no stage 2, of course).
> >
> > How do I merge those changes?  I don't know enough about all the git
> > commands to figure this out, especially how to take advantage of the
> > stage 1, 2, and 3 files.
>
> I do not work on the kernel myself so I'll probably get the
> details wrong, but I am guessing here is what you have:
>
Sorry, I should have been more specific.  How about this:
1) 2.6.14-test branch modifies arch/ppc64/Kconfig
2) master branch renames arch/ppc64/Kconfig to arch/powerpc/Kconfig
3) 2.6.14-test and master branch share a common ancestor v2.6.14
4) Need to merge change arch/ppc64/Kconfig to arch/powerpc/Kconfig

$ git ls-files -u 100644 c658650af429672267409508b02b38754c11a40f 1 arch/ppc64/Kconfig 100644 8abf1118ebbd59954d098d87679114ffda0e75cb 3 arch/ppc64/Kconfig ***Notice no stage 2

$ ls -ld arch/p* drwxrwxr-x 9 dzickus dzickus 4096 Dec 16 14:12 arch/parisc drwxrwxr-x 11 dzickus dzickus 4096 Dec 16 14:26 arch/powerpc drwxrwxr-x 15 dzickus dzickus 4096 Dec 16 14:26 arch/ppc ***Notice no ppc64 (was removed in master branch)

$ ls -ld arch/powerpc/Kconfig* -rw-rw-r-- 1 dzickus dzickus 24279 Dec 16 14:26 arch/powerpc/Kconfig ***Notice no backup or side copy of Kconfig

There were other conflicts with this merge (ones that weren't automatically resolved) but those were easily merged using vi and searching for <<<. However this particular file is providing an unusual case.

Let me know if this helps or not.

Cheers, Don

Junio C Hamano· Dec 16, 2005, 21:16 UTC · re: Junio C Hamano · lore

Re: git merge questions

Junio C Hamano <junkio@cox.net> writes:
Show 16 quoted lines
> What puzzles me is that I think it is supposed to have done all
> the above for you.  Namely:
>
> 	$ ls -l arch/ppc64/Kconfig <1>
> 	$ git ls-files -s arch/ppc64/Kconfig <2>
>         $ ls -l arch/powerpc/Kconfig arch/powerpc/Kconfig~* <3>
>         $ git ls-files -s arch/powerpc/Kconfig <4>
>
> <1> should not remain --- recursive merge could have notied that
> the file was moved.
> <2> ditto.
> <3> arch/powerpc/Kconfig should be there with possibly merge
> conflict markers.
> <4> stage1 with merge base version (possibly renamed), stage2
> with test2 version, stage3 with your test version (definitely
> renamed).

I know what happened. That Kconfig file was modified between v2.6.14 and v2.6.15-rc4 beyond recognition, and the rename detection did not think it was renamed.

The merge has this message:
    CONFLICT (delete/modify): arch/ppc64/Kconfig deleted in HEAD and
    modified in e14ee1ed34e25df6ea93b0bfb1bc4138cd26bea2. Version
    e14ee1ed34e25df6ea93b0bfb1bc4138cd26bea2 of arch/ppc64/Kconfig
    left in tree.
and then, the digging I suggested yields these:
$ ls -l arch/{ppc64,powerpc}/Kconfig
-rw-rw-r--  1 junio src 24279 Dec 16 12:55 arch/powerpc/Kconfig
-rw-rw-r--  1 junio src 11027 Dec 16 12:58 arch/ppc64/Kconfig
$ git ls-files -s arch/ppc64/Kconfig arch/powerpc/Kconfig
100644 bb2efdd566a9d590d64184b10b097e4b7ed17e95 0       arch/powerpc/Kconfig
100644 c658650af429672267409508b02b38754c11a40f 1       arch/ppc64/Kconfig
100644 8abf1118ebbd59954d098d87679114ffda0e75cb 3       arch/ppc64/Kconfig
$ ls arch/{powerpc,ppc64}/Kconfig~*
ls: arch/powerpc/Kconfig~*: No such file or directory
ls: arch/ppc64/Kconfig~*: No such file or directory
$ git diff --theirs arch/ppc64/Kconfig
* Unmerged path arch/ppc64/Kconfig
diff --git a/arch/ppc64/Kconfig b/arch/ppc64/Kconfig

The merge algorithm thought your branch (that is, test2 which is
v2.6.15-rc4) removed the old ppc64/Kconfig path, but the other
branch (test1 which has diff between v2.6.14 and v2.6.14.4) made
updates to that file.  For the other path, it simply thought
your branch added a new file arch/powerpc/Kconfig while the
other branch did not do anything to that path between v2.6.14
and v2.6.14.4, so it merged it already.

The result you want in this case is to merge changes between
c65865 (stage1 of old path) and 8abf11 (stage3 of old path) into
bb2efd (the latest contents of the new path) and register it as
the result of merge for arch/powerpc/Kconfig, and remove
arch/ppc64/Kconfig.  So the sequence would be:

$ orig=$(git unpack-file c65865)
$ from_test=$(git unpack-file 8abf11)
$ merge $from_test $orig arch/powerpc/Kconfig
merge: warning: conflicts during merge

After resolving the conflict in $from_test file, and other conflicts:

$ ed $from_test
$ cat $from_tset >arch/ppc64/Kconfig
$ rm arch/powerpc/Kconfig
$ rm $orig $from_test
$ git update-index --add --remove arch/ppc64/Kconfig arch/powerpc/Kconfig
# also mark paths you hand-resolved such as Makefile, drivers/pcmcia/i82365.c
# etc. with "git update-index" here.
$ git commit

would commit the result of the merge.
Don Zickus· Dec 16, 2005, 21:35 UTC · re: Junio C Hamano · lore

Re: git merge questions

Show 48 quoted lines
> and then, the digging I suggested yields these:
>
> $ ls -l arch/{ppc64,powerpc}/Kconfig
> -rw-rw-r--  1 junio src 24279 Dec 16 12:55 arch/powerpc/Kconfig
> -rw-rw-r--  1 junio src 11027 Dec 16 12:58 arch/ppc64/Kconfig
> $ git ls-files -s arch/ppc64/Kconfig arch/powerpc/Kconfig
> 100644 bb2efdd566a9d590d64184b10b097e4b7ed17e95 0       arch/powerpc/Kconfig
> 100644 c658650af429672267409508b02b38754c11a40f 1       arch/ppc64/Kconfig
> 100644 8abf1118ebbd59954d098d87679114ffda0e75cb 3       arch/ppc64/Kconfig
> $ ls arch/{powerpc,ppc64}/Kconfig~*
> ls: arch/powerpc/Kconfig~*: No such file or directory
> ls: arch/ppc64/Kconfig~*: No such file or directory
> $ git diff --theirs arch/ppc64/Kconfig
> * Unmerged path arch/ppc64/Kconfig
> diff --git a/arch/ppc64/Kconfig b/arch/ppc64/Kconfig
>
> The merge algorithm thought your branch (that is, test2 which is
> v2.6.15-rc4) removed the old ppc64/Kconfig path, but the other
> branch (test1 which has diff between v2.6.14 and v2.6.14.4) made
> updates to that file.  For the other path, it simply thought
> your branch added a new file arch/powerpc/Kconfig while the
> other branch did not do anything to that path between v2.6.14
> and v2.6.14.4, so it merged it already.
>
> The result you want in this case is to merge changes between
> c65865 (stage1 of old path) and 8abf11 (stage3 of old path) into
> bb2efd (the latest contents of the new path) and register it as
> the result of merge for arch/powerpc/Kconfig, and remove
> arch/ppc64/Kconfig.  So the sequence would be:
>
> $ orig=$(git unpack-file c65865)
> $ from_test=$(git unpack-file 8abf11)
> $ merge $from_test $orig arch/powerpc/Kconfig
> merge: warning: conflicts during merge
>
> After resolving the conflict in $from_test file, and other conflicts:
>
> $ ed $from_test
> $ cat $from_tset >arch/ppc64/Kconfig
> $ rm arch/powerpc/Kconfig
> $ rm $orig $from_test
> $ git update-index --add --remove arch/ppc64/Kconfig arch/powerpc/Kconfig
> # also mark paths you hand-resolved such as Makefile, drivers/pcmcia/i82365.c
> # etc. with "git update-index" here.
> $ git commit
>
> would commit the result of the merge.
>

Wow. That makes sense then. All your digging techniques seem to be fairly straightforward. Now say I didn't know the name of the rename and I had to dig that up. Would the following be the right way or is there something easier?

%git log <renamed file> #grab the last commit id from here %git-diff-tree -p <last commit id> # search for the new file

Thanks again, Don

Junio C Hamano· Dec 16, 2005, 22:17 UTC · re: Don Zickus · lore

Re: git merge questions

Don Zickus <dzickus@gmail.com> writes:
Show 20 quoted lines
>> and then, the digging I suggested yields these:
>>
>> $ ls -l arch/{ppc64,powerpc}/Kconfig
>> -rw-rw-r--  1 junio src 24279 Dec 16 12:55 arch/powerpc/Kconfig
>> -rw-rw-r--  1 junio src 11027 Dec 16 12:58 arch/ppc64/Kconfig
>> $ git ls-files -s arch/ppc64/Kconfig arch/powerpc/Kconfig
>> 100644 bb2efdd566a9d590d64184b10b097e4b7ed17e95 0       arch/powerpc/Kconfig
>> 100644 c658650af429672267409508b02b38754c11a40f 1       arch/ppc64/Kconfig
>> 100644 8abf1118ebbd59954d098d87679114ffda0e75cb 3       arch/ppc64/Kconfig
>> ..
>> The result you want in this case is to merge changes between
>> c65865 (stage1 of old path) and 8abf11 (stage3 of old path) into
>> bb2efd (the latest contents of the new path) and register it as
>> the result of merge for arch/powerpc/Kconfig, and remove
>> arch/ppc64/Kconfig.  So the sequence would be:
>>
>> $ orig=$(git unpack-file c65865)
>> $ from_test=$(git unpack-file 8abf11)
>> $ merge $from_test $orig arch/powerpc/Kconfig
>> merge: warning: conflicts during merge

I suspect there might be a room for improvement to make this easier with a new command, if this becomes a common pattern.

Something like:
	$ git resolve-renamed-path arch/ppc64/Kconfig arch/powerpc/Kconfig

to mean "I want the result of this merge to rename ppc64/Kconfig to powerpc/Kconfig", perhaps? There are three cases: (1) we renamed they didn't --- this is the case we are looking at and there will be stage1 and stage3 but not stage2 for the old path, and stage0 for the new path; (2) they renamed we didn't --- this would happen if you pulled test2 into test, and there will be stage1 and stage2 but not stage3 for the old path, and stage0 for the new path; (3) both of us renamed.

The third case is handled by the merge command automatically. Old path will not remain to bother you even if the merge of the new path needs hand-resolving, so you do not need resolve-renamed-path command to deal with that case.

Show 6 quoted lines
> Now say I didn't know the name of the rename and I had to dig
> that up.  Would the following be the right way or is there
> something easier?
>
> %git log <renamed file>  #grab the last commit id from here
> %git-diff-tree -p <last commit id>  # search for the new file
That would work.

Or an explicit rename detection to see where many of the neighbouring paths moved (this is very expensive):

	mb=$(git merge-base test test2)
        git diff-tree -r --diff-filter=R -M -l0 --name-status $mb test2

Or the diff-tree between the merge base and your current tree and perhaps the other tree (this may give a lot of cruft):

	mb=$(git merge-base test test2)
        git diff-tree -r --diff-filter=A --name-status $mb test2
Johannes Schindelin· Dec 16, 2005, 23:58 UTC · re: Junio C Hamano · lore

Re: git merge questions

Hi,

just a thought: maybe in this case -- git fails to recognize a rename -- Pasky's idea would have some merit. You could then provide git with some extra information a la .git/info/grafts: "Even if you, git, do not believe it: this file *was* renamed from blah to blop".

Ciao, Dscho

Junio C Hamano· Dec 17, 2005, 01:32 UTC · re: Johannes Schindelin · lore

Re: git merge questions

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> just a thought: maybe in this case -- git fails to recognize a rename -- 
> Pasky's idea would have some merit. You could then provide git with some 
> extra information a la .git/info/grafts: "Even if you, git, do not believe 
> it: this file *was* renamed from blah to blop".

Yeah, except small details such as: where does it record it, and how does the information presented to the user, and how does the user tell git that information should be used?

What would be interesting would be to extend on this thing I just wrote:

    Something like:
        $ git resolve-renamed-path arch/ppc64/ arch/powerpc/
    to mean "I want the result of this merge to rename ppc64/Kconfig
    to powerpc/Kconfig", perhaps?

I think this would work very nicely even without rename detectino by the recursive strategy. What would happen with resolve strategy is that unchanged paths are removed from arch/ppc64 and added to arch/powerpc by the usual read-tree merge rules, and all paths (not just unrecognizable renames -- because resolve would not even try) that have been changed on the "test" branch would be in stage1+stage3 state in arch/ppc64, while the corresponding ones in arch/powerpc are collapsed ("only added in test2 branch") to stage0. The fictional resolve-renamed-path command (notice that I removed the explicit "Kconfig" from the sample command line) could go through the index file, looking for paths that arch/powerpc/ has stage0 and arch/ppc64 has stage1+3, and perform the renaming merge at that point.

Junio C Hamano· Dec 17, 2005, 03:48 UTC · re: Johannes Schindelin · lore

Re: git merge questions

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> just a thought: maybe in this case -- git fails to recognize a rename -- 
> Pasky's idea would have some merit.

It is not as simple as that. If you look at the output of the following command, you would understand why.

	$ git rev-list ^$(git merge-base test test2) test2 -- \
          arch/powerpc/Kconfig arch/ppc64/Kconfig |
          git diff-tree --pretty -C -r --stdin --abbrev --name-status \
          arch/powerpc/Kconfig arch/ppc64/Kconfig
The transition happened over time with multiple commits.

First ppc64/Kconfig was somewhat stripped of its contents, starting at this commit:

    diff-tree bcdd1ea... (from 5bfc826...)
    Author: Stephen Rothwell <sfr@canb.auug.org.au>
    Date:   Mon Sep 19 23:13:24 2005 +1000
        [PATCH] powerpc: Move arch/ppc*/oprofile/Kconfig to arch/powerpc
        These files are identical.
        Signed-off-by: Stephen Rothwell <sfr@canb.auug.org.au>
        Signed-off-by: Paul Mackerras <paulus@samba.org>
    M       arch/ppc64/Kconfig

and after a lot of hard work, powerpc/Kconfig gets created.

    diff-tree 14cf11a... (from e5baa39...)
    Author: Paul Mackerras <paulus@samba.org>
    Date:   Mon Sep 26 16:04:21 2005 +1000
        powerpc: Merge enough to start building in arch/powerpc.
        This creates the directory structure under arch/powerpc and a bunch
        of Kconfig files...
    A       arch/powerpc/Kconfig

After that both continues to exist for some time, and finally ppc64/Kconfig gets deleted with this one:

    diff-tree 7568cb4... (from c55377e...)
    Author: Paul Mackerras <paulus@samba.org>
    Date:   Mon Nov 14 17:30:17 2005 +1100
        powerpc: Move most remaining ppc64 files over to arch/powerpc
        Also deletes files in arch/ppc64 that are no longer used now that
        we don't compile with ARCH=ppc64 any more.
        Signed-off-by: Paul Mackerras <paulus@samba.org>
    M       arch/powerpc/Kconfig
    D       arch/ppc64/Kconfig

You cannot record "this is the rename" by attributing that information to one particular commit. Even looking at the commit history one by one you cannot really say powerpc/Kconfig is a rename of ppc64/Kconfig. I suspect that it is not really a rename --- from reading of the log messages, its original contents were moved around and scattered over to other Kconfig files in the tree, or along with other configuration pieces got consolidated into powerpc/Kconfig, or perhaps both.

← back to recent threads