Re: Upstream merging and conflicts (was Re: Using cvs2git to track an external CVS project)

On Fri, 3 Jun 2005, Martin Langhoff wrote:

Yes, and right now that "manual" part is actualyl fairly high. I'll fix that thing up (today, I hope), making nontrivial merges much easier.

Well, there's asome support for keeping a peripheral branch in sync, but part of it ends up having to try to just merge into it from the main branch every once in a while.

It all really boils down to "merge often, merge small". Git encourages that behaviour. Git concentrates on trivial merges, because the git philosophy is that if you let two trees diverge too much, your problems are not with the merge, but with other things.

NOTE! This does _not_ mean that you can't do big changes. You can very much do a branch that does _huge_ changes, and as long as that branch keeps merging with the original, it should work out reasonably well. Every merge "re-bases" the work, so you'll never have to merge old changes. You may end up with a fair number of manual things to fix up each time (and see my comment on the manual fixup problems right now), but at least the thing should hopefully be incremental.

But yeah, if you actually do major re-organization, then the system absolutely needs tons more smarts in the automated parts, since right now it has no automated merging for renames etc things (things it _could_ detect, but doesn't do). There's really several layers of merging you can automate, and git right now only automates the very very lowest stuff.

In other words, it should be possible to make git do pretty well even for complex branches, but no, right now the automated parts don't even try. And it's still _all_ geared towards development that ends up being merged reasonably often - you can merge in one direction for a while to keep the pain incremental, but yes, you do need to merge the other way _eventually_.

It's true that you can't skip patches, since a merge is always all-or-nothing, and the global history really requires that (in a very fundamental way). However, you _can_ merge from the HEAD, and then say "that patch is no longer relevant in this tree", and remove it. Then you'll never need to worry about that patch any more, because you've merged it and in your merged result, it no longer exists).

That said, I don't claim that this is what you always want to do. In fact, we have this exact issue in the kernel, and I think it's a great thing to use a combination of a patch-based system _and_ git. You use git for the "core", you use the patch-based system for the more fluid stuff. In the case of the kernel, the patch-based system that seems to be used the most is "quilt", but that's a detail. The big issue is that git is simply part of a bigger process:

Yes. I think this is fundamental. "git" needs to converge. It's how git works. git also "crystallizes" the history and makes it unbreakable. Both of these things are _wonderful_ for what they mean, but both of these fundamental issues are _horrible_ for other reasons.

So it's not a one-size-fits-all. git is designed to work around the kind of stuff I do - my job in many ways is exactly to "crystallize" the work that happens around me. I'm the ugly speck of dirt around which a beautiful snowflake forms.

But to get the pieces that crystallize to form, you need all those unruly patches floating around freely as a gas, and feeding the crystallization process. And git is not suitable _at_all_ for that phase.

So I don't think that you should necessarily think of git as "the" souce control management in the big picture. It's how you handle _one_ phase of development, and the constraints it puts on that phase are both wonderful and horrible, and thus you really should see the git phase as being the solid core, but no more.

So git (apart from the use of "ephemeral branches for testing") is _not_ very conductive to wild development. I think git is actually a wonderful way of doing the wild development too, but only in a microscopic sense: you can use git either for the "big picture" crystal, or you can use git for the "I want to keep track of what I did" on a small scale, but then in between those things you'd need to have a patch-manager or something.

In other words, I'd expect that a _lot_ of git usage is of the type:
 - clone a throw-away tree from the standard repositories
 - do random things in it, merge with the standard repo every once in a 
   while. This is the "little picture".
 - export the result as a patch when you're happy with it, and use 
   somethign else to keep track of the patch until it can be merged into 
   the "big picture".

So I don't think any development effort that is big enough necessarily wants to use git as the _only_ way of doing development and merging stuff. The kernel certainly does not. Not now, and not in the long run.

(We know about the long run, because these issues were largely trye of BK too, although BK handled especially metadata merges much much better. But even with BK, we always ended up having 50% or more of the actual development end results going around as patches, and BK was really the way to crystallize the end result. It shouldn't come as a surprise that git does the same - git was designed not so much as a technical BK replacement, but as a replacement for that _process_ we had for BK).

Final note: I do believe that the kernel is kind of strange. I doubt anybody else ended up using BK the way we did, and I suspect _most_ BK users used BK as a better CVS, where BK was the primary - and only - thing that kept track of patches.

Whether the kernel model is applicable to anything else, I dunno. 

		Linus