git log --follow doesn't follow a rename over a merge

Hi All,

I'm trying, without success, to get git log to show the history of a file across a rename.  I'm using git version 1.7.5.4, and git blame has no such issues tracking the source of the lines.

This is part of a CVS migration.  In the repo a number of directories were committed atomically, but they originated from another CVS repository.  During the migration, I've grafted the history from the originating repository onto the destination repo's commit of the content.  Blame looks good, but log won't find the rename.

Git blame attributes 3 files:

dsf/org.eclipse.cdt.dsf/src/org/eclipse/cdt/dsf/debug/service/IBreakpoints.java plugins/org.eclipse.dd.dsf/src/org/eclipse/dd/dsf/debug/IBreakpoints.java plugins/org.eclipse.dd.dsf.debug/src/org/eclipse/dd/dsf/debug/service/IBreakpoints.java

Git log shows:

70237c3 Bug 315439 Fix up copyright 6a53ab4 [290074] - [services][breakpoints] IBreakpointDMData interface should extend IDMData 640102e partial fix for Bugzilla 265882 5b47187 Migrated DSF and DSF-GDB to the CDT project.

And it shows one fewer if I turn on --follow (with as many combinations of -M -C as I've tried):

70237c3 Bug 315439 Fix up copyright 6a53ab4 [290074] - [services][breakpoints] IBreakpointDMData interface should extend IDMData 640102e partial fix for Bugzilla 265882

My new merge commit is 5b47187 (when the content first arrived in the repo):

commit 5b471873662a87a77cfd854c98fca9c9948aa878
Merge: 4709306 c1e6da2
...

I can see the file in the parent of the merge:

... and the renamed file after the merge:

... (Neither file exists on the 4709 side)

Interestingly the tricks mentioned here (http://stackoverflow.com/questions/953481/restore-a-deleted-file-in-a-git-repo) for working out where a file was removed, produce no output:

But log does work for the file if follow is specified (output with --diff-filter is empty...):

10580b5 Updated copyright statements. 17585c3 [218577] Cleaned up IBreakpoints interface. ...

Have I done something wrong in grafting history like this, or is this just a problem with the way git log --follow tracks renames?

The repo's here if anyone's interested (it's 85M...): git://github.com/jamesblackburn/org.eclipse.cdt

Many thanks for any insight. Cheers, James

James Blackburn <jamesblackburn@gmail.com> writes:

[...]

The `--follow` option to git-log is more of a bolted-on hack rather than proper solution, see 750f7b6 (Finally implement "git log --follow", 2007-06-19), and it might not work in all cases.

On 16 June 2011 14:25, Jakub Narebski <jnareb@gmail.com> wrote:

Thanks for the reply Jakub.

I'm also interested in knowing what incantation I need to give to
actually get history of the file which was previously removed.  e.g. I
know the full path:
   plugins/org.eclipse.dd.dsf.debug/src/org/eclipse/dd/dsf/debug/service/IBreakpoints.java
used to exist, but without --follow, git log returns nothing.

Similarly I can't figure out how to find (without knowing in advance) which was the last commit which contained this version of IBreakpoints.java. i.e. which commit removed / renamed it?

Cheers, James

I see a lot of this has been discussed before... but just to add that Jeff's blame-log.sh: http://git.661346.n2.nabble.com/alternate-log-follow-idea-td1385917.html seems to work and show the full history of the file past the rename+merge!

I was also interested in whether there's a better way of re-connecting history than grafting one repository's history into the commit which made it visible in another? AFAICS the graft is no different to a merge commit which contains a rename in the merged node, so I'm guessing this is a reasonable thing to do?

Cheers, James

On Thu, Jun 16, 2011 at 06:01:25PM +0100, James Blackburn wrote:

You may want to also check out the "line-level history browser" work by Bo Yang and Thomas Rast, which is a similar idea, but done much better.

You can find it here:

  http://repo.or.cz/w/git/trast.git/shortlog/refs/heads/line-log-cleanup

or pull it with:

  git pull git://repo.or.cz/git/trast.git line-log-cleanup

Use on a whole file is something like:

  git log -L '1,$':yourfile

But note that it is a work in progress, and I seem to be able to get it to segfault on some simple tests.

Usually you would graft the tip of the old history to the root commit of the new history. So it's not a merge, but makes it look like a contiguous linear history near the graft. But that's only a per-repository thing. If you want to join two histories in the actual history graph, you would merge them; if one supersedes the other, then you would use the "ours" merge strategy to ignore the other side.

I'm not 100% sure I understand your question. Does that answer make sense?

-Peff

On Thu, Jun 16, 2011 at 02:31:02PM +0100, James Blackburn wrote:

Doesn't:

  git log -- plugins/org.eclipse.dd.dsf.debug/src/org/eclipse/dd/dsf/debug/service/IBreakpoints.java

do that?

-Peff

Jeff King <peff@peff.net> writes:

If the file emerged in one branch (either the primary or a side branch) in the past as a failed experiment and then got removed before merging back, i.e.

   past ---o----o-----o-----o-----o-----o-----o-----o--- now
            \                          /
             o----*----o----o----*----o
                  ^added         ^removed

then the merges are simplified away and you would not see it.

Perhaps simplify-merges option may help.

On Thu, Jun 16, 2011 at 10:59:23AM -0700, Junio C Hamano wrote:

Ah, right. The default simplification so often does what I want that I forget there are cases it can miss.

This test case shows full-history helping:

-- >8 --
commit() {
  echo $1 >$1 && git add $1 && git commit -m $1
}

git init repo && cd repo && commit one && commit two && commit three && git checkout -b side HEAD^ && commit four && commit five && git rm five && git commit -m "remove five" && commit six && git checkout master && git merge side && echo "==> default log (shows nothing)" && git --no-pager log -- five echo "==> full-history" && git --no-pager log --full-history -- five

On 16 June 2011 19:15, Jeff King <peff@peff.net> wrote:

This test shows what I've done:

commit() {
 echo $1 >>$1 && git add $1 && git commit -m $1
}

git init repo cd repo commit one commit two commit three

git symbolic-ref HEAD refs/heads/newroot rm * git rm --cached * commit four commit four commit four commit four commit four commit four commit five

git checkout master git merge --no-commit newroot

git mv four four2 commit six commit four2 commit five

git log --graph --oneline
* 7c4c441 five
* 02a7262 four2
*   6ac2fe0 six
|\
| * d4f5e35 five
| * 350d3e9 four
| * b975d48 four
| * 3bdbf38 four
| * 5b58da8 four
| * 9fb4f59 four
| * a6d1492 four
* b301c9c three
* 38865e2 two
* 9a9c689 one

At this point, only git-blame seems correct:

Correct

02a7262 four2 6ac2fe0 six

02a7262 four2

 <no output>

350d3e9 four b975d48 four 3bdbf38 four 5b58da8 four 9fb4f59 four a6d1492 four

is missing the deletion in the merge commit.

Log of 'five', which hasn't been renamed, looks ok.

James

On 16 June 2011 18:59, Junio C Hamano <gitster@pobox.com> wrote:

Magic, that works!

What I don't get is that I believe the file is removed in the 'merge' commit 'X', which is reachable from master, not in the feature branch:

 2002 ---o----o-----o-----o-----X-----o-----o-----o--- 2011         (CDT)
                                    /
 2006 ---o----o-----o-----o--   (2009)                                  (DSF)

2f1c904 [memory] DSDP-DD -> CDT initial commit 5b47187 Migrated DSF and DSF-GDB to the CDT project. 10580b5 Updated copyright statements. ...

So 5b48718 is X, and the file exists in the DSF parent of X: c1e6da2

I think 2f1c904 is entirely wrong too -- it's another graft I've done,
to a different repo, which isn't to do with DSF, and doesn't contain
IBreakpoints.java:
 > git show 2f1c904:plugins/org.eclipse.dd.dsf.debug/src/org/eclipse/dd/dsf/debug/service/IBreakpoints.java
fatal: Path 'plugins/org.eclipse.dd.dsf.debug/src/org/eclipse/dd/dsf/debug/service/IBreakpoints.java'
does not exist in '2f1c904'

Given the file was removed during my 'merge' graft in commit X, shouldn't it be visible in log without --simplify-merges?

Cheers, James

James Blackburn <jamesblackburn@gmail.com> writes:

The merge simplification logic does not treat earlier parent commits any differently than later parent commits of a merge (except when two parents are the same with respect to a given pathspec, and if the logic needs to pick only one, the earlier parent will be picked), and that is by design.

So the short answer is no.

On 16 June 2011 20:36, Junio C Hamano <gitster@pobox.com> wrote:

I think this is quite tough to grok. Git blame shows me a path, and git log -- <path> returns no output in the simple example I gave.

In this case --simplify-merges gives me a complete log for the path,
but man git-log tells me it should do the opposite:
       --simplify-merges
           Additional option to --full-history to remove some needless
merges from the resulting history, as there are
           no selected commits contributing to this merge.

I'm not sure how I would have figured out this switch might be just the trick to show the commits that touched this path, without your input.

As a naive user I expected:
  git log -- four
to tell we which commits changed the path 'four'. Especially those
commits which are reachable from my current HEAD.  I don't really
understand why one branch of history is worth following over another,
especially as the branch chosen, the older one, doesn't contain 'four'
at all.

Cheers, James

On 16 June 2011 19:41, James Blackburn <jamesblackburn@gmail.com> wrote:

This looks like a bug?   man git-log:
       [--] <path>...
           Show only commits that affect any of the specified paths.
To prevent confusion with options and branch names,
           paths may need to be prefixed with "-- " to separate them
from options or refnames.

git log --graph --oneline
* 6530823 six2
*   05512d0 nine
|\
| * 9917e1c seven
| * 76a230e six
| * a6d7961 six
* f7f43f9 four2
*   94392ac eight
|\
| * 5cf0b04 five
| * 5183657 four
* bd329af three
* 57aafee two
* 3d9ecdf one

git log --simplify-merges --oneline -- four 05512d0 nine 94392ac eight 5183657 four

but the merge commit at 'nine' has nothing to do with that path 'four'.

Reproduction steps:

commit() {
 echo $1 >>$1 && git add $1 && git commit -m $1
}

git init repo cd repo commit one commit two commit three

git symbolic-ref HEAD refs/heads/newroot rm * git rm --cached * commit four commit five

git symbolic-ref HEAD refs/heads/newnewroot rm * git rm --cached * commit six commit six commit seven

git checkout master git merge --no-commit newroot

git mv four four2 commit eight commit four2

git merge --no-commit newnewroot

git mv six six2 commit nine commit six2

Unfortunately that returuns no output.

If I add --follow, I get history: git log --follow --oneline -- plugins/org.eclipse.dd.dsf.debug/src/org/eclipse/dd/dsf/debug/service/IBreakpoints.java 10580b5 Updated copyright statements. ...

Where 10580 is a distant parent of c1e6da which was merged in to 5b47187.

On 16 June 2011 18:45, Jeff King <peff@peff.net> wrote:

In this case the commit is a 'merge' commit as history looks like:

  2002 ---o----o-----o-----o-----X-----o-----o-----o--- 2011         (CDT)
                                     /
  2006 ---o----o-----o-----o--   (2009)                                  (DSF)

CDT has existed since 2002. In 2006, at X, DSF was committed onto HEAD of CDT. Basically wholesale apart from the path (Java package) change (+some minor changes to 'import's).

So I created a graft from the commit which made it appear from CDT onto the HEAD of a DSF as it was when the import happened. Which makes X have two parents.

Cheers, James