How to pull only a few files from one branch to another?

I have a long-running topic branch. I have fixed a few nits on the master branch that I would like on the topic branch. How do I pull in only a few files from the head of the master branch?

I tried all sorts of incantations (I am running 1.5.0-rc2), including:

git pull . origin foo.cc git pull origin foo.cc git pull foo.cc master git pull . master foo.cc git pull master foo.cc

Bill

On Tue, 23 Jan 2007, Bill Lear wrote:

Git *very* fundamentally tracks project state, not file state. Which means that you very much can NOT try to "merge a file". It is a senseless operation in git, and in fact, any SCM that allows it pretty much is doomed to be a total piece of sh*t (*).

So if you want to get just one file, you cannot see it as a "project-level merge", and you literally have to handle that file as an individual file and merge it as such, and then you can commit the result as a regular commit (but it will _not_ show up as real "git merge").

So in git, the rule is always that you track the whole state of the project. It's a project tracker, not a "file tracker".

That said, if you just want to get the fixes from one branch, and merge it, git does allow you to do it, although there isn't any "porcelain". You'd basically have to do it by hand, a few different ways

 (1) If you just want to commit the state of a file AS IT IS in another 
     branch, you can just do

	git checkout other-branch filename-goes-here
	git commit

     which will just do it for you.

 (2) However, more commonly, what you actually *want* to do is to merge 
     the changes in the other branch into the current branch (which has 
     its own set of changes), and then things get more interesting and 
     more complex.

     If you have just one simple merge-base, you'd do something like

	git merge-one-file \
		$(git merge-base HEAD other):filename \
		HEAD:filename \
		other:filename \
		filename

     (where those last three things are just the file mode). And then just 
     commit the end result.

     Which is just total git magic. If you understand the above command
     line, I have nothing more to teach you.

Maybe somebody could actually test the above one-liner magic thing, and explain how it works. And maybe we could write porcelain to do this ;)

NOTE NOTE NOTE! I don't actually think "git merge-one-file" really works correctly as-is for your usage case. It _kind_of_ gets the result you want, but it's not really written to be used like that, and you'll get an error from the

	git-checkout-index -f --stage=2 -- "$4" &&
		cat "$src1" >"$4"

stage because stage 2 didn't exist ("git-merge-one-file" is really only supposed to be called by a real merge).

But it's _almost_ usable that way. Maybe Dscho or Junio wants to make it work the extra final mile.

Anyway, the above (2) kind of works, but if you want to avoid the warning and want to avoid the magic, you would really have to do something like

	git cat-file blob $(git-merge-base HEAD other):filename > .base
	git cat-file blob other:filename > .other
	git merge-file filename .base .other

which will mostly do what you wanted (without complaints), and is what the git-merge-one-file magic actually boils down to.

Not exactly pretty. If this is something people want to do often, we'd need to generate some porcelain for it.

But if you understand the above three lines, you're actually doing really really well already. And as a "yes, git can do it" kind of demonstration, it may even be useful.

FINAL NOTE! After you do file-level merging like this, don't expect future merges to that file to be easy if you continue to make changes! This is _why_ git doesn't do file-level merges natively - namely it really doesn't work well together with future merges.

To keep merges working after more than just one merge, you really need to either do _all_ merges on a file-per-file level (which is insane and doesn't scale and has serious problems) or you need to do merges at a "project level" (which is how git fundamentally works).

		Linus

(*) And I'm not saying that just because git doesn't do it. It's much more fundamental than that. Once you start doing per-file branching and merging, you've basically screwed yourself, and you'll never be able to work on the project as a "whole project" any more - you no longer have a well-defined history that actually is the history of the whole project.

On Tue, 23 Jan 2007, Linus Torvalds wrote:

That comment made no sense. I removed the file modes entirely, and forgot to remove the comment. It turns out that you can just drop the file modes entirely from the invocation of least the current "git-merge-one-file", so I did that, to keep it cleaner and simpler.

Anyway, the way "git merge-one-file" works, you pass it:

 - the SHA1 of the "base object"

	$(git merge-base HEAD other):filename

 - the SHA1 of the "current branch version of the file"

	HEAD:filename

 - the SHA1 of the "other branch version of the file"

	other:filename

 - the filename as you want the end result to be in the index

	filename

 - normally, you'd pass the file modes for the three cases too

	0644 0644 0644

   but those are the ones I skipped because the shell script actually only 
   uses them to see if they are different (so dropping them means that it 
   all works fine - the empty arguments are all the same).

but additionally, since "git merge-one-file" expects to be in the middle of a real merge (and not just some faked-out single-file merging thing) it also expects the index to be populated in stage2 with the current branch version of the file - which it won't be outside of a merge. Which is why it ends up complaining.

It would be kind of cool to have the whole "merge-recursive" logic (which can handle multiple ancestors etc - unlike the above) able to do this, but that's a whole 'nother kettle of fish.

			Linus

Hi,

On Tue, 23 Jan 2007, Linus Torvalds wrote:

You'd lose all the renaming logic when doing it strictly on a file level. So I suggest opening a sidebranch, doing the merge, and just pick the new file as you proposed with "git checkout -f sidebranch file".

Ciao, Dscho

Hi,

On Tue, 23 Jan 2007, Linus Torvalds wrote:

I don't really see the point, but yes, it would be easy enough (at least in the builtin version I sent out last night).

Note that the checkout-index is used _purely_ to set the correct mode (at least as far as I can tell). *Clickety-click* well, seems you had something different in mind on June 8th, 2005. "make sure that the full pathname is created". This, of course, is missing from the builtin merge-one-file (yet).

I fully expect people to agree that this is not the way to go. I think cherry-picking is what you were after, Bill, correct?

Having said that, I find myself picking changes from a side-branch (with too many one-liner commits) with this command:

	git diff -R sidebranch file1 file2.. | git apply --index

Ciao, Dscho

On Wednesday, January 24, 2007 at 11:16:35 (+0100) Johannes Schindelin writes:

I can't answer confidently, but I think that this is correct.

Bill

On Tue, Jan 23, 2007 at 08:11:38PM -0800, Linus Torvalds wrote:

In fact, I came some time ago on a workflow which could be seen as quite similar to this issue.

The problem was about merging a new upstream release in a local branch, when both branches had heavy changes. Indeed this tree was not using GIT but CVS, with upstream tarballs imported on a branch - I just asked myself how it could be best handled with GIT, and could only come to the conclusion that something in GIT was still missing.

What was particular about this tree, is that we are several people working on it, namely developpers taking care of the app, and me as build manager taking care of the build mechanics as well as kernel+OS. So the task was logically divided: one dev would merge the app, and I would merge everything else. As you see, it's far from one-file merges, but the problem is quite similar.

The idea which I came up with was inspired by the "partial merge" feature in PRCS, which I had never used or even understood the purpose before that date.

Basically, it was that if some set of files could be merged somewhat independently from the rest, and we don't want to get GIT attempt to merge them again when finally merging the whole, a commit of a partial merge would somehow record that some files had already been merged. And I happenned to think that it could be sufficient to create a commit that would not be a merge commit itself (since it is not a full-project merge), but which would instead reference "subcommits" for the relevant parts of the tree that were merged during that iteration.

I realize that solution would not really be perfect, as there is always some coupling between the code and build stuff in a project. And sometimes we could want to get finer granularity than the file level. But that's so far my best guess at finding a solution.

How would you handle such a situation ?

Best regards,

Yann Dirson wrote:

This I think would be best handled by (future/prototype) submodules support.

I wonder if fake-recording resolutions in git-rerere would help in that case...

On Wed, Jan 24, 2007 at 09:46:49PM +0100, Jakub Narebski wrote:

That could be used for some cases, but it's harder to use to avoid getting a Makefile merged with the rest of the subtree.

Hm. Looks like that would require being able to communicate rerere results from one workspace to another.

Yann Dirson wrote:

Another solution would be to record partial merge (using "ours" strategy for files we don't want to merge), then --amend it.

On Wed, 24 Jan 2007, Yann Dirson wrote:

Yes, it's basically the same thing.

And the sad part is that you can easily come up with tons of examples of where this makes sense, and indeed there are no fundamental problems at all as long as the "per-file" (or "file group" - the thing doesn't really change) history is strictly a perfect subset of the "whole project" history topology.

The reason it fundamentally doesn't work in the long run is that people invariably will then do things that do _not_ have the same "history topology". You'll have one file that has some of its history mixed in two different branches that _globally_ share none of that per-file history, so then the global history no longer matches the history of that file, and the "topology" of the history cannot be mapped from one to the other.

So what do I mean by that?

What git does is to track the history as it is shared by ALL files in the whole project (or, the way I prefer to think about it: it's not about independent files at all, it's _always_ about the whole collection).

This makes sense, because what does "history" actually mean? Forget about what it means as a word in English, and concentrate on what its meaning is strictly from a technical standpoint. Why does history matter? Why don't we just have a set of commits in date order?

"set of commits in date order" is useless) is because it gives us a COMMON ANCESTOR! And *that* is the only thing that matters.

Now, what is fundamentally wrong with doing per-file history? 

Now, if you've followed this argument, you should go "Aaahh! Obvious!".

The thing that is _fundamentally_ wrong with per-file history is that it breaks the one AND ONLY point of having history in the first place. There is no well-defined "common ancestor" notion on a repository level any more. You've made the history meaningless on a repo level, and it's now only meaningful on a file level.

So whats' wrong with that? You could still do merges, file by file, and just do the common ancestry that way, couldn't you?

Yes you can. It's just that if you do, you're a total moron, and you're back to the dark ages and playing with CVS (and possibly SVN - it's too early to say yet, since SVN can't do merges and history AT ALL right now as far as I know - it has the same merge logic as CVS has, I think, which is to say that it doesn't really understand the point of having history in the first place).

Which gets me back to where I started: you could (at least in in theory) allow a situation where you allowed file-level merges AS LONG AS THEY NEVER CLASH WITH THE GLOBAL HISTORY.

In other words, I could imagine starting a branch for a particular file (and when I say "file", it can be any arbitrary subset of the full state), and having file-level history at that level, but you must NEVER then merge that file across into _another_ global branch, because if you do, then the "global history" has lost all value, since it's no longer actually global history any more.

So you have to choose: do you want to track things file by file, or do you want to track the whole project. You literally can't say "I want to do both!". Because tracking history file by file breaks the whole concept of tracking global history, the moment you start merging individual files across branch boundaries.

I really don't think that people understand how fundamentally broken the whole concept is.

I think git should help you combine partial state between different branches, but you should always realize that IT IS NOT A MERGE! It's more like a "cherry-pick" - except you cherry-pick multiple commits "in space" instead of cherry-picking one commit "in time".

(The way I personally view git, "space" is the workspace - aka contents - while "time" is the history, aka the relationship of commits")

So it's perfectly ok to take data from other branches and include them in your current one. Nobody doubts that, and indeed, some forms of it we already have nice tools to help you do that ("cherry-pick" in particular).

And yes, doing a simple

	git diff commit..othercommit filename | 
		git-apply --index && git commit

is really just a way to "cherry-pick" the data when it's located in "space" instead of in "time" (ie we restrict it to a particular region of the workspace, and cherry-pick the work we did over a long time: this is 100% equivalent to "git cherry-pick", which does it the other way around: it cherry-picks the work restricted "in time", but unstrstricted "in space").

But when you do this YOU MUST NOT CALL IT A MERGE!

Because you _by_definition_ don't actually do the ONE thing that is the whole point of a merge: the result does not become a common ancestor of the result.

So when you cherry-pick, you don't merge: you just create a bug-standard commit. It may contain data from another branch, but it is NOT a merge of the other branch.

See above. Just create them as individual commits, and perhaps point to where the data came from in the commit comment, but don't try to think they are merges.

Btw, to explain my point perhaps even more fully, let me give an example of the "reverse" situation:

	git merge -s ours

This actually IS a merge, even though we don't actually make ANY CHANGES AT ALL to the tree, and we don't take any actual data at all from the other branch, and just state: "the result of the merge is always the current branch contents".

So why is it a merge? Exactly for the same reason that a partial-file "merge" is NOT a merge. It's a merge because the whole point of the "ours" strategy is to say "This is now going to be the common ancestor for these two states going forward". And that is the _definition_ of a merge, since that's the whole (and ONLY) point of having history in the first place.

So it all really boils down to a simple question: "What is the meaning of 'history'?"

Once you understand what history is all about, you understand why a "merge" of a few files isn't actually a merge at all, but just a cherry-pick.

(And btw, that doesn't mean that we wouldn't use the "merge" program to do it. The way we actually implement "cherry-pick" is to physically do a "merge" of the contents, because passing patches forwards and backwards is just stupid. So the issue is somewhat confused by the different "levels" of meaning of the word "merge". There's the pure "content merge", which is a purely technical method of combining data, and which is usually just a three-way RCS-merge. And then there is the "merge commit", which tells you something about HISTORY, which is a lot more fundamental and a lot more important).

You can often do the "data merge" by hand. It's the least of your problems, and it's not even very interesting. The _interesting_ part of a git merge is what it means for history!

Doing cherry-picking (whichever kind you want to) is fine, but you have to realize that it also means that you will have a potentially harder time to do a "true merge" of the two branches later. Once your per-file history doesn't match the whole-project history, you end up losing a lot of the automatic goodness of simple merges, and you get back to the old CVS merge hell.

But the good news is that at least it won't be any _worse_ than what a lot of people have long since learnt about CVS merging. It will be "hell" only when compared to what you can do when you don't play games, and when you merge nicely.

			Linus

On Tue, 23 Jan 2007, Bill Lear wrote:

You don't pull in a few files, you apply the changes made in a few commits:

  git diff HEX_OF_NIT_FIX^ HEX_OF_NIT_FIX | git apply

If there's other stuff in the nit-fixing commit, shame on you, but you can edit the patch before applying it to remove everything that's not what you want.

(Incidentally, I think "git diff ^ {commit}" should be made to do "git diff {commit}^ {commit}"; i.e., if there is a single other revision provided, interpret a modifier not applied to anything as applying to that revision, in the "what else could that possibly mean?" department.)

	-Daniel
*This .sig left intentionally blank*

On Wednesday, January 24, 2007 at 00:29:00 (-0500) Daniel Barkalow writes:

Good technique.  Thank you for sharing.

I had been thinking about trying something along the lines of Junio's "Separating topic branches" posted on the "GIT Howto Index" page. I may have tried that had I been braver.

I probably should have thought ahead and made this fix on a branch, merged it into my master branch and then into my topic branch ... I think.

Bill

Bill Lear <rael@zopyra.com> wrote:

Yes. That is standard "good practice" with Git. Unfortunately it does require a little planning ahead if the base to start on isn't immediately obvious (you need to think "I want this here, and here, and here.. and compute the merge base").

Bill Lear wrote:

Perhaps you can just rebase that one fix 'back' onto the base. Then merge it into both.

-apw

Daniel Barkalow <barkalow@iabervon.org> writes:

Or just:

	git cherry-pick HEX_OF_NIT_FIX

possibly followed by tweaking the result and running:

	git commit --amend

Daniel Barkalow wrote:

"git diff <commit>^!" doesn't work?

On Wed, 24 Jan 2007, Jakub Narebski wrote:

It generates the right result, but it hasn't become something I automatically think of. :) For that matter, "git diff <commit>{^,}" also works, but, again, I don't think of it instantly.

	-Daniel
*This .sig left intentionally blank*