{"thread":{"id":"26771","subject":"Merging split files","startedAt":"2011-03-18T13:22:36Z","lastAt":"2011-03-29T18:15:13Z","messageCount":4,"participants":["Stephen Bash","Jeff King"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"163633","messageId":"2495196.195017.1300454556155.JavaMail.root@mail.hq.genarts.com","threadId":"26771","inReplyTo":"31155742.183989.1300374518689.JavaMail.root@mail.hq.genarts.com","subject":"Merging split files","fromName":"Stephen Bash","fromEmail":"bash@genarts.com","sentAt":"2011-03-18T13:22:36Z","receivedAt":"2011-03-18T13:22:36Z","isPatch":false,"sender":{"key":"bash@genarts.com","avatar":null},"body":"Hi all-\n\nIn our previous release foo.cxx contained both the base class and a few subclasses.  Since then the number of subclasses has grown, and we've split foo.cxx (base and sub-classes) into foo-base.cxx (base class) and foo-defs.cxx (sub-classes).  Since the release, we've had a few bug fixes in foo.cxx on the maintenance branch, and need to merge those back to development.  When I did the merge Git identified foo.cxx as moved to foo-defs.cxx, which worked for most changes, but a few needed to be in foo-base.cxx.  In this case it was a pretty trivial manual resolution, but is there a method for handling merges of split files?\n\nThanks,\nStephen\n"},{"id":"164579","messageId":"20110329151623.GD10771@sigill.intra.peff.net","threadId":"26771","inReplyTo":"2495196.195017.1300454556155.JavaMail.root@mail.hq.genarts.com","subject":"Re: Merging split files","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2011-03-29T15:16:23Z","receivedAt":"2011-03-29T15:16:23Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Fri, Mar 18, 2011 at 09:22:36AM -0400, Stephen Bash wrote:\n\n> In our previous release foo.cxx contained both the base class and a\n> few subclasses.  Since then the number of subclasses has grown, and\n> we've split foo.cxx (base and sub-classes) into foo-base.cxx (base\n> class) and foo-defs.cxx (sub-classes).  Since the release, we've had a\n> few bug fixes in foo.cxx on the maintenance branch, and need to merge\n> those back to development.  When I did the merge Git identified\n> foo.cxx as moved to foo-defs.cxx, which worked for most changes, but a\n> few needed to be in foo-base.cxx.  In this case it was a pretty\n> trivial manual resolution, but is there a method for handling merges\n> of split files?\n\nI don't think there is currently a good way to do this automatically.\n\nThe problem is that the closest merge-recursive gets to understanding\ncontent movement is that it considers whole file renames. So it sees\n\"foo.cxx became foo-defs.cxx\", and applies changes to foo.cxx to\nfoo-defs.cxx, but it has no clue that foo-base.cxx. So at the very\nleast, it would need to represent \"foo.cxx has split into foo-base.cxx\nand foo-defs.cxx\", which is not something it can currently handle. But\nmore than that, you want to know _which_ parts moved to each file.\n\nSo I think the most flexible thing is to forget file renames at all.\nThey are just a rough version of the general idea of content movement.\nIn theory, we should be able to see that the content we changed in\nfoo.cxx no longer exists, and then start looking for similar content\nelsewhere. Not similar _files_, but for the chunk of content that is\nchanged between the merge base and the maintenance (and some surrounding\ncontext), find where that bit of content went. And then try to merge our\nchanges into that new bit of content.\n\nOne problem is that when it fails, it fails pretty hard. With file\nrenames, your changes at least usually ends up in the right file (your\npresent problem excluded), and you get some textual mess to clean up.\nBut with content-level renaming, I suspect in conflict cases we would\nend up with no clue where the result goes (because the conflict means we\ncan't easily match up the content for similarity), and have to stick it\nin the deleted file. On the other hand, it might simply work to keep\nexpanding the amount of context we consider for content similarity until\nwe find a match, which eventually would end up considering the whole\nfile, and generalize to a file rename.\n\nImplementing that inside of merge-recursive is likely to be pretty nasty\n(even the current file-rename code is already pretty nasty). But it may\nbe possible to prototype something that runs after we hit the conflicted\nstate, like mergetool.\n\nI definitely think it's an interesting area to work in, but I would have\nto give it a lot of thought.\n\n-Peff\n"},{"id":"164581","messageId":"3752347.282743.1301416397164.JavaMail.root@mail.hq.genarts.com","threadId":"26771","inReplyTo":"20110329151623.GD10771@sigill.intra.peff.net","subject":"Re: Merging split files","fromName":"Stephen Bash","fromEmail":"bash@genarts.com","sentAt":"2011-03-29T16:33:17Z","receivedAt":"2011-03-29T16:33:17Z","isPatch":false,"sender":{"key":"bash@genarts.com","avatar":null},"body":"Jeff-\n\nThanks for taking the time to think about this.  More inline...\n\n----- Original Message -----\n> From: \"Jeff King\" <peff@peff.net>\n> To: \"Stephen Bash\" <bash@genarts.com>\n> Cc: git@vger.kernel.org\n> Sent: Tuesday, March 29, 2011 11:16:23 AM\n> Subject: Re: Merging split files\n>\n> On Fri, Mar 18, 2011 at 09:22:36AM -0400, Stephen Bash wrote:\n> \n> > In our previous release foo.cxx contained both the base class and a\n> > few subclasses. Since then the number of subclasses has grown, and\n> > we've split foo.cxx (base and sub-classes) into foo-base.cxx (base\n> > class) and foo-defs.cxx (sub-classes). Since the release, we've had\n> > a\n> > few bug fixes in foo.cxx on the maintenance branch, and need to\n> > merge\n> > those back to development. When I did the merge Git identified\n> > foo.cxx as moved to foo-defs.cxx, which worked for most changes, but\n> > a\n> > few needed to be in foo-base.cxx. In this case it was a pretty\n> > trivial manual resolution, but is there a method for handling merges\n> > of split files?\n> \n> I don't think there is currently a good way to do this automatically.\n> \n> The problem is that the closest merge-recursive gets to understanding\n> content movement is that it considers whole file renames. ...\n> \n> So I think the most flexible thing is to forget file renames at all.\n\nI agree that would be the best solution long term. (\"Git doesn't track files, Git tracks content\".  Think I heard that somewhere before...)\n\nThat being said, the back seat drivers in the office here (i.e. me and everyone else that knows almost nothing about the internals of merge recursive!) thought maybe a middle ground is teach merge recursive to do copy detection along with rename detection.  Then the algorithm would have a (relatively small?) list of candidate files to check for hunks.  You still have to deal with the similarity score in some corner cases, but hopefully since all we want is candidate files the process is relatively insensitive to the similarity threshold.\n\nAm I way off the deep end now?  I'm not lying when I say I know *nothing* about the merge implementations.\n\n> I definitely think it's an interesting area to work in, but I would\n> have to give it a lot of thought.\n\nIt's a \"corner case\" that I seem to have run into a lot in my work experience, so if the Git community can actually make a good solution work it will be a major win in my book.\n\nThanks again!\n\nStephen\n"},{"id":"164600","messageId":"20110329181513.GA20784@sigill.intra.peff.net","threadId":"26771","inReplyTo":"3752347.282743.1301416397164.JavaMail.root@mail.hq.genarts.com","subject":"Re: Merging split files","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2011-03-29T18:15:13Z","receivedAt":"2011-03-29T18:15:13Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Tue, Mar 29, 2011 at 12:33:17PM -0400, Stephen Bash wrote:\n\n> > The problem is that the closest merge-recursive gets to understanding\n> > content movement is that it considers whole file renames. ...\n> > \n> > So I think the most flexible thing is to forget file renames at all.\n> \n> I agree that would be the best solution long term. (\"Git doesn't track\n> files, Git tracks content\".  Think I heard that somewhere before...)\n\nExactly. :) I think that is a tricky project, though, and in the\nmeantime, I wouldn't be opposed to a more file-based solution if it\ngenerates good results.\n\n> That being said, the back seat drivers in the office here (i.e. me and\n> everyone else that knows almost nothing about the internals of merge\n> recursive!) thought maybe a middle ground is teach merge recursive to\n> do copy detection along with rename detection.  Then the algorithm\n> would have a (relatively small?) list of candidate files to check for\n> hunks.  You still have to deal with the similarity score in some\n> corner cases, but hopefully since all we want is candidate files the\n> process is relatively insensitive to the similarity threshold.\n\nThis was something I gave some thought to recently in this other thread:\n\n  http://thread.gmane.org/gmane.comp.version-control.git/169944\n\nthough I came to the conclusion in that case that break-rewriting was a\nmuch better match for that particular case. Namely, we see that content\nhas been renamed, so we make sure to merge changes to the \"original\"\ncontent with each other, no matter whether the changes happened in the\nrenamed path or the original. And similarly, we merge changes left over\nfrom any \"new\" content that has replaced the original (which, in the\npure rename case, is just empty, but with break-rewriting we might have\nsome dissimilar content at the old path). We know that the \"new\" content\ncan't be related to the \"old\" content, because to find a rename, we\nwould have to have triggered the \"break\" by finding that the content is\ndissimilar.\n\nCopy detection has to deal with that, but harder. :) I see two major\nchallenges:\n\n  1. One source file may go to multiple destinations. So instead of\n     saying \"oops, I should be doing the merge with this other, renamed\n     content\", you have to pick a best one (either through heuristic, or\n     even per-hunk by trying each hunk in turn). And this means you're\n     interacting deeply with the content-level 3-way merger. I haven't\n     looked at that code at all, so I don't know how feasible that is.\n\n     And you have to accept that you may pick wrong, or even that there\n     may be no right answer. If I do \"cp foo bar; cp foo baz; rm foo\",\n     and then modify \"foo\" on another branch, the choice of merging\n     changes to \"bar\" versus \"baz\" is going to be arbitrary.\n\n  2. Because it's a copy and not a rename, your source file may still\n     exist and be a candidate for applying content to. And that violates\n     the break-rewrite rename logic I mentioned above, which is that old\n     content goes with old content and new content goes with new\n     content. We're not sure which the source file is for a given hunk.\n\n     I think that may not be a big deal, though. We already have to deal\n     with the hard part in (1), which is finding _which_ copy is the\n     right place for a given bit of content. So this may just simplify\n     to adding the source file (if it still exists) as another possible\n     place to merge changes to, and it is another case of (1) (though\n     obviously we should prefer merging to the original pathname if it\n     is still there, rather than a copy).\n\n> Am I way off the deep end now?  I'm not lying when I say I know\n> *nothing* about the merge implementations.\n\nNo, I don't think you're off the deep end. But then, I don't know that\nmuch about the merge code, either. :)\n\n-Peff\n"}]}