git-apply that handles rejects like merge conflicts

Hey,

I'm trying to get git-apply to apply patches, and let me handle the conflicts in a way I'm comfortable with -- by staging the "successful" hunks and leaving conflict markers in the working tree.

With the available flags, I seem to only be able to have successful hunks in the index, and rejected ones in patch-like .rej files.

Is there a way to accomplish this? If not, does anyone think it's a good idea?

-Ori

On Tue, Nov 08, 2011 at 12:10:48AM +0200, Ori Avtalion wrote:

No, I don't think there's a way to do it now. I do find the conflict markers the easiest way to mark something up. But most of my "git apply" use is through "git am", which knows the trick of falling back to a 3-way merge (see fall_back_3way in git-am.sh).

If it's an actual git diff, the same 3-way trick will yield good results, and it would be nice if it were easier to do that trick without calling "git am". But if it's not a git diff (i.e., missing the original blob information), then you won't be able to do that.

In the general case, you can't represent all failed hunks with conflict markers, can you? I'm thinking something where we couldn't find any relevant context. You know the lines from the original patch from the hunk header, so you can drop the failed content from the patch in the right spot. But how do you know how big a conflict marker to make for the "current" side? The same number of lines as were in the hunk? I think you'd end up with confusing conflict markers.

-Peff

On 11/08/2011 12:55 AM, Jeff King wrote:

I'm dealing with two codebases that have branched in the past, before any VCS was used, and now I'm tracking both separately with git. I'm trying to apply changes from one to the other with format-patch and git-am/apply. So yeah, no blob info.

Personally, I wouldn't object to having both "computable" conflicts, and the .rej files for hunks that lack context, but I see how that would be very confusing. :)

-Ori

Jeff King <peff@peff.net> writes:

Conflict markers come from the use of a 3-way merge, and if you were to do a 3-way merge, by definition, you would need some way to tell where the preimage of the patch and the target tree you are attempting to apply the patch forked from. That's done by fall-back-3way in "am -3".

You _could_ lift that logic out of "am -3", but I do not think it is worth the effort to do so (IOW, I do not see a reason to avoid "am -3").

If you do not want to create a commit for whatever reason, then you can "reset --soft" back.

On Mon, Nov 07, 2011 at 03:45:48PM -0800, Junio C Hamano wrote:

I think it would purely be "I have a patch produced by git diff, not by git format-patch". If you want to use "am -3", you would have to dress up your patch with mail headers.

In practice, this doesn't come up much for me. I think I was using "git diff >patch" as a poor-man's stash (and I did just stick some fake headers in, and "git reset HEAD^" afterwards). But maybe other workflows deal with this more.

But I think there are two questions:

  1. Should am's 3-way fallback be made more easily available to users
     of regular "apply"?

  2. Short of doing a 3-way merge, are there better ways to represent
     failed hunks in the patch target itself, rather than saving ".rej"
     files?

I'm actually not sure which one Ori was asking about.

-Peff

Jeff King <peff@peff.net> writes:

Me neither, but if I have to guess it would be the former. If there were a solution better than ".rej" in 2-way context, surely "patch" would have implemented it 10 years before we started ;-).

On 11/08/2011 07:46 AM, Jeff King wrote:

[ snip ]

I'm actually interested in both :)

Here's a copy of the description of my problem from another reply:

In answer to your first question

git-am is never part of this workflow as I'm trying to move patches between separate repositories with no shared root.

<rant> And, personally, I don't think git-am is named correctly as the only use-case I have for it is applying+committing single patches produced by format-patch and sent as individual files over some medium which isn't mboxes (I'm not that old-school). I never understood why git-apply can't do the commit and I have to instead use a tool with 'mail' in its name (Let's ignore the historical reasons) -- Shouldn't git-am be an mbox-reading wrapper around some more basic patch-applying tool? </rant>

I really want this as .rej files feel very un-git-like. However, after understanding the problems raised in this thread, I'm a bit more realistic :)

-Ori

On Tue, Nov 08, 2011 at 10:50:51PM +0200, Ori Avtalion wrote:

Isn't git-am the right tool for that? You format-patch out your commits in one repo, and then apply them in the other. No shared history is required; just the ability of the patches to actually be applied.

It _helps_ if you have the common base objects (the actual files, not the commits), since the git diffs carry the pre- and post-image file sha1s, which is what allows us to do a real 3-way merge. In that case, it is just a matter of making the objects from the first repo available to git during the moment you are applying in the second repo. You could do it by fetching the history of the first into a side-branch of the second, or even just by sharing object databases via the "alternates" mechanism.

git-am _is_ an mbox-reading wrapper around some more basic patch-applying tool. That tool is "git apply". I think what you are missing is that a single patch (or multiple patches) produced by format-patch _is_ an mbox. There is nothing wrong with:

  cd repo1 &&
  git format-patch -1 --stdout >../my.patch &&
  cd ../repo2 &&
  git am ../my.patch

There is no standard for representing commit metadata in the diff format. So git had to invent its own. It used rfc822 messages and mailboxes because it was simple and convenient, it mapped to what some people were already doing, and it means we don't need a separate tool for applying local commits versus ones that were emailed.

So the "m" is really for mbox, which happens to be git's format for storing one or more commits, including metadata. If you just forget that it's associated with mail, then I think you will be happy. :)

-Peff

On Mon, Nov 7, 2011 at 23:55, Jeff King <peff@peff.net> wrote:

GNU patch can produce conflict markers with the --merge option.

Bert

On Tue, Nov 08, 2011 at 09:52:26AM +0100, Bert Wesarg wrote:

Hmm. Yeah, it does work, but as I feared, it can produce pretty awful conflicts. Try this fairly straightforward setup (3 lines changed in the middle of a file):

  git init &&
  seq 1 10 >file && git add file && git commit -m base &&
  sed -i '3,5s/$/ master/' file && git commit -a -m master &&
  git checkout -b other HEAD^ &&
  sed -i '3,5s/$/ other/' file && git commit -a -m other

You can see what a real merge looks like:

  git merge master &&
  $EDITOR file

which is:

  1
  2
  <<<<<<< HEAD
  3 other
  4 other
  5 other
  =======
  3 master
  4 master
  5 master
  >>>>>>> master
  6
  7
  8
  9
  10

If you use "patch --merge", you get the same thing. Which is good. But now try it with 10 lines changed out of 100:

  rm -rf .git
  git init &&
  seq 1 100 >file && git add file && git commit -m base &&
  sed -i '50,60s/$/ master/' file && git commit -a -m master &&
  git checkout -b other HEAD^ &&
  sed -i '50,60s/$/ other/' file && git commit -a -m other

Doing a merge will get you the same sensible results. But "patch --merge" produces:

 ...
 45
 46
 <<<<<<<
 =======
 47
 48
 49
 50 master
 ...
 60 master
 61
 62
 63
 >>>>>>>
 47
 48
 49
 50 other
 51 other
 52 other
 53 other
 ...

which is not that helpful. Interestingly, I think it _should_ be able to do the same thing here as it did on the 3-line case. So I'm not sure why it doesn't.

But there are even more complex cases, like say "other" had added new lines of new content at the beginning of the file, and messed up the context lines that the patch was using. So I think in the general case, you will end up with patches like the latter one. Just shoving the patch hunk into the file with an empty preimage section. And that can even still be useful, but you are relying on line counts then. If they're off, it's going ot be quite confusing.

-Peff