fast-import, merges, and file changes -- lack of clarity in docs, possible minor bug, or PEBKAC?

Hi,

It appears the documentation for fast-import does not specify the
basis on which a commit command's file changes (filemodify,
filedelete, filecopy, or filerename) are relative to.  For a non-merge
commit, it's obvious, and I thought that it would be obvious for a
merge commit too -- and then found out I had assumed the wrong answer.
 I assumed the file changes would be relative to the *combined* set of
files from the parents, whereas fast-import apparently treats them as
relative to the first (or 'from') parent.  (Since the combined set of
files may have conflicts, I'd expect it to be an error if the merge
commit didn't specify an appropriate filemodify command for at least
each of the filenames with conflicts.)

In case my description above isn't clear, I've got a simple example at the end of this email that has a file 'world' on one branch, a file 'bar' on another branch, and a merge of the two branches. In the merge commit, I do not need to mention the 'world' file; it will be included automatically. However, unless I mention the 'bar' file (by adding a 'M 100644 :1 bar' line to the end of the input), it will not be included in the merge commit.

Is this intentional? I believe we could change this behavior without breaking backward compatibility with older fast-export output streams, since such older streams would simply be providing redundant information. But is there a reason for this behavior that I'm missing?

Thanks, Elijah

Sample input file for git fast-import (which merges two commits but silently drops one of the files):

blob mark :1 data 4 foo

reset refs/heads/other commit refs/heads/other mark :2 author Elijah Newren <newren@gmail.com> 1234815015 -0700 committer Elijah Newren <newren@gmail.com> 1234815015 -0700 data 29 Initial commit, other branch M 100644 :1 bar

blob mark :3 data 3 hi

reset refs/heads/master commit refs/heads/master mark :4 author Elijah Newren <newren@gmail.com> 1234814943 -0700 committer Elijah Newren <newren@gmail.com> 1234814943 -0700 data 30 Initial commit, master branch M 100644 :3 world

commit refs/heads/master mark :5 author Elijah Newren <newren@gmail.com> 1234815029 -0700 committer Elijah Newren <newren@gmail.com> 1234815029 -0700 data 21 Merge branch 'other' from :4 merge :2

I think I've at least partially discovered the answer my own question(s), so I figured I'd document them in case anyone else was wondering...

On Mon, Feb 16, 2009 at 5:28 PM, Elijah Newren <newren@gmail.com> wrote:

<snip>

It is documented...though implicitly in discussion of other commands. In particular, "...a `merge` command may be used instead of `from` to start the commit with an empty tree" and later "If the `from` command is omitted when creating a new branch, the first `merge` commit will be the first ancestor of the current commit, and the branch will start out with no files." This implies that the files of a `merge` parent are ignored and any content must from that parent that is contained in the commit must be explicitly spelled out to be included.

Also, as far as backward compatibility, my suggestion would not be compatible; deletions involved in such merges would be handled by current fast-export streams by simply omitting the file in the list of file changes in the merge commit. My suggestion would require explicitly specified file deletion to ensure its removal.

So, it appears this is intentional, any behavior change such as I suggested would require different/new syntax, and would require support in fast-export too for my purposes. That certainly makes my git-fast-filter script (a script in a pipeline between fast-export and fast-import designed to act somewhat like filter-branch but with better performance on my large repositories) much harder. Oh, well.

Elijah