Directory renames without breaking git log.

Hi folks,

in an effort to make Samba development easier, we're trying to merge the Samba3 and Samba4 branches into a single branch. In order to do so, we need to rename the "source" directories both Samba 3 and Samba 4 have (we're planning to use source3 and source4).

Unfortunately, the directories are big enough that git log stops to track the renamed files, so e.g. git log ./samba3 does not show the samba3 history. The history is not lost, of course, but it's way less intuitive to get it.

Here's how we merged the two branches:

$ mkdir samba-merged
$ cd samba-merged
$ git init
... (create COPYING, README and other top-level files, git add them)
$ git commit -m "Initial commit of merged samba"
$ git remote add git://git.samba.org/samba.git samba
$ git remote update
$ cp -a ~/samba3/source source3
$ cp -a ~/samba4/source source4
$ git add source3 source4
$ git write-tree
$ echo "merge branches" | git commit-tree <sha1 git write-tree retured> \
      -p <sha1 of the initial commit> \
      -p <sha1 of the current samba3 head> \
      -p <sha1 of the current samba4 head>
$ git reset --hard <sha1 returned by git commit-tree>
$ git log
... history is there as expected
$ git log samba3
... history is just the merge commit
$ git log samba4
... history is just the merge commit

Is there any way to fix this that doesn't involve changing the history with git-filter-branch?

Cheers, Kai

On Wed, Sep 3, 2008 at 2:38 PM, Kai Blin <kai@samba.org> wrote:

You can try setting diff.renamelimit to 0 in your ~/.gitconfig. See Linus's email here for a similar situation in the kernel: http://lwn.net/Articles/292948/

Thanks, Tarmigan

On Thursday 04 September 2008 02:16:24 Tarmigan wrote:

That doesn't seem to fix "git log path/to/file" cases. The really interesting part is that if I try git log --follow -M -C path/to/file, I don't get any history at all. (--follow is the culprit, if I remove that I at least get the merge commit)

git blame still works, and git log --sparse path/to/file works, of course. --sparse makes giving a path a bit pointless, of course, but we probably can live with that for time being. I'm still open for suggestions, of course. :)

Cheers, Kai

Kai Blin <kai@samba.org> writes:

Give both directories, like:

	"git log -- newdir olddir"

perhaps?

On Thursday 04 September 2008 21:49:27 Junio C Hamano wrote:

Better, but really ugly, as we'll have to keep doing this for the rest of the project's life to get the full history. And while it's all nice and fun for git log -- source3/configure.in source/configure.in, it's less fun for deeper paths.

We'll probably end up just doing a git-filter-branch renaming the samba3 source dir source3 from the beginning and the samba4 source dir source4 from the beginning, and then do the octopus merge. Without any paths changing, that should probably work. It's a bit annoying to break all external branches, but we only need to do this once, and people will only need to git-format-patch and git-am once, and we can provide a step-by-step guide for this as well.

Thanks for the feedback, though. :)

Cheers, Kai

Kai Blin <kai@samba.org> writes:

Yes, following across subtree merges _could_ be improved.

But another thing I should mention in this context is that you should not take --follow option (at least in the current form) too seriously.

I see it's been a while --- the last time I did this was October 2006 if I am not mistaken. It's time of the year I should point at one of the most important articles ever written on this mailing list:

    http://thread.gmane.org/gmane.comp.version-control.git/27/focus=217

After understanding what Linus envisioned back then, why what he said are important are important and why what he dismissed as uninteresting are indeed uninteresting, things to think about now are:

 - "blame" (especially with -C -C), as you found out already, does answer
   the more important question "where did this come from"; and

 - the question "log --follow $this_file" is asking is exactly "where did
   this file come from".  Remember what adjective was used for the
   question in the article?

I also should mention that --follow was done by Linus as a hack with known limitations.

Potential improvements to follow possible renames fully would involve:

 * allow not just a single path but a set of pathspecs to be recorded
   during --follow traversal;

 * allow the above information be associated with individual commits, not
   as a single global state in the traversal machinery;

 * enhance the logic to update the pathspecs information kept above when
   you hit renames while traversing the history.  An important part of
   this job involves inferring a wholesale rename of a directory by
   looking at many files moved from one place to another, which we
   currently do not do anywhere in git.

If you do all of the above, it will become a feature, not a hack with known limitation. But I do not think anybody so far thought it is worth the effort, only to answer the "where did this file come from" (ituot ue zaf m eqzeunxq cgqefuaz).

Personally I feel that this is slightly closer to "I might do this myself if I had infinite amount of time and I am really bored" than "I don't care; patches welcome" category.

Kai Blin <kai@samba.org> writes:

Unfortunately "git log --follow <filename>" works correctly only on relative simple histories. You are of course welcome to improve this part of git.

Simple workaround is to use "git log <file>" (optionally using --diff-filter) to get when file vanished, check using "git show" or "git whatchanged" on boundary commit, then use "git log -C -C -- <old name> <new name>"

(you're probably better off asking on the whole list, as I don't know much)

On Wed, Sep 3, 2008 at 11:53 PM, Kai Blin <kai@samba.org> wrote:

OK, I actually tried with your example, and see what you mean. It looks like follow does not work with this case. See http://permalink.gmane.org/gmane.comp.version-control.git/85766 for an illustration of --follow not working with the subtree merge strategy, which as far as I can tell is pretty much the same as this case. Also it sounds like follow only works with single files, and not directories: http://kerneltrap.org/mailarchive/git/2008/5/4/1714694

Why these do not work is way beyond my git knowledge. As far as I know (which is very little), follow could work for these cases, but it currently does not.

Glad that git blame works. If you haven't already, you should also try git gui blame too for better interactivity.

Thanks, Tarmigan