Dangers of working on a tracking branch

We are about to switch to git 1.5 from git 1.4.4.1. I cannot remember if someone posted about this, but what is the danger of working on a tracking branch --- there are abundant cautions about doing this, but I can't recall and can't find the reason this is bad.

Bill

On Thu, 15 Feb 2007, Bill Lear wrote:

A tracking branch is supposed to be a local mirror of what is available remotely. If you commit local changes to it then you break that model.

This is why with Git 1.5.0 tracking branches are in a different namespace by default to which you cannot commit.

Nicolas

On Thursday, February 15, 2007 at 16:00:23 (-0500) Nicolas Pitre writes:

Ok, so I break the model, what is the harm in that? Can I no longer pull from or push to the remote branch? Do I corrupt something locally? Does something else break? I'm trying to formulate an explanation to our users why the 1.5 way is superior and I can't just say "if you do that you break the model".

Bill

On Thursday, February 15, 2007 at 15:21:38 (-0600) Bill Lear writes:

BTW, my ignorant assumption is that for merging to take place, git requires two branches, so 'git pull' will fetch into the tracking branch and then merge onto your ... uh ... whatever the name of the working version of your tracking branch is.

BTW, again, why does git clone not have an option to just create all of the "working versions" (better name needed) of tracking branches?

Bill

On Thu, Feb 15, 2007 at 03:21:38PM -0600, Bill Lear wrote:

The commits you make will not actually go onto that tracking branch; they will be part of a "detached HEAD" (that is, your HEAD doesn't point to _any_ branch). Once you check out a different branch, you will potentially lose those commits (actually, they will still be available through your reflog, but you will have to know to look for them there).

That's more or less correct. You don't really have to have two branches; doing a 'git pull /path/to/repo branch' will fetch the branch into the temporary FETCH_HEAD name, and merge from that. But yes, the way a raw 'git pull' will work is to first fetch all tracking branches into refs/remotes/origin/*, and then merge from whichever is defined by your config.

And they're usually called "local branches" if you differentiating them from remote tracking branches, or just "branches" otherwise.

I don't recall anybody asking for it, yet. I think the idea is that those branches would just be clutter. If you want to work on something, it's easy enough to just start a local version of the branch:

  git checkout -b topic origin/topic

-Peff

On Thursday, February 15, 2007 at 16:43:52 (-0500) Jeff King writes:

Is this really the way 1.4.4.1 works? I have (mistakenly) been working on my tracking branch, committing to it, pushing it, pulling in from elsewhere, shifting to new branches, etc., and I haven't lost anything, and can't see what harm I've done...

Sure, it is easy, but it's surprising to (our) users when they do a clone and can't "jump right in", and have to spend 3 seconds doing the above...

Bill

On Thu, Feb 15, 2007 at 03:53:01PM -0600, Bill Lear wrote:

Sorry, I thought you were talking about v1.5. v1.4.* does not have the detached HEAD feature, and the default layout is to keep tracking and local branches in the same namespace. This means that you can checkout and commit to a tracking branch just as a local branch, and all changes will be saved. It will only bite you later, when you try to fetch into that tracking branch and realize that the fetch is not a fast-forward (remember the troubles you were having with fetches and pushes last week? Those were caused by working on the tracking branches).

Those branches were moved to a separate namespace in v1.5 so that it's harder to accidentally commit to them.

Fair enough. It should be pretty simple to implement; why don't you try working up a patch? :)

-Peff

On Thursday, February 15, 2007 at 16:58:16 (-0500) Jeff King writes:

Ok, just wanted confirmation of my (in)sanity.

Well, I seriously thought about it, but the git-clone code is pretty dense. It might be educational though, and since I'm the only one who wants it ... might be worth a shot.

Thank you for your help.

Bill

Bill Lear <rael@zopyra.com> wrote:

Careful.

Lets say you patch git-clone to create these branches under refs/heads, and also under refs/remotes/origin. Now someone else modifies one of those branches, and you do a git-fetch to get the latest. The tracking branch under refs/remotes/origin would update, but not the one in refs/heads. Which means "jumping right in" may actually cost you a lot more time, because you are now starting on something that is several days old.

On Thursday, February 15, 2007 at 17:09:18 (-0500) Shawn O. Pearce writes:

Granted, but this would be the same as creating the branch by hand, then going rock-climbing over the weekend, while my colleagues toiled, and coming back on Monday to find 15 checkins on that branch, right?

At which point I could .... rebase?  Do a pull?

Bill

Bill Lear <rael@zopyra.com> wrote:

Yes.  Although if you pull something like that, your colleagues
may also be muttering things about you near the water cooler...
So Git may be the least of your problems.  ;-)
 

Rebase or pull, either would work. Rebase would clutter the history less (no merge) but also tosses some history (no merge). Its a tossup.

All I was trying to say was this may be more likely to happen, as you will clone, work in that repository for a couple of weeks, then get told to look at the `frotz` branch, as it has all the cool stuff you were looking for. You peek at refs/heads/frotz, and it has some cool stuff, but its 2 weeks out of date. Unless you remember to pull `origin/frotz` to `frotz` first, you might spend a while looking at stale code.

On Thu, Feb 15, 2007 at 04:58:16PM -0500, Jeff King wrote:

Re-reading this, it sounds a little harsh. I meant less "screw you, if you want it, make a patch" and more "I don't have time to work on this now, but I would be happy to help guide you in making a patch."

-Peff

On Thursday, February 15, 2007 at 17:06:34 (-0500) Jeff King writes:

Understood perfectly the first time, even with the smiley. :)

Bill

Bill Lear wrote:

You cannot fetch or push to the remote branch if you made some commits on the tracking branch ("broke the model") and remote branch has other commits. You can force the issue, but then either remote (for push) or local (for fetch) commits would be lost.

Sidenote: for some branches you have to skip fast-forward check, because
they have rewritten history (e.g. pu branch in git.git).
 

But one of those branches can be temporary "branch" FETCH_HEAD (if you use "git pull <URL>" syntax, and do not use tracking (remote) branches).

Because usually you work with one branch with tracking (remote) + local branch workflow; I think with multiple branches you usually use temporary feature branches...

On Friday, February 16, 2007 at 03:00:04 (+0100) Jakub Narebski writes:

Ok, I'm trying to come up with an experiment that verifies this, so I can give a concrete example to our developers.

I don't seem to be able to get it to fail, but I sure remember having severe problems with this in practice. Here is my attempt, again with git 1.4.4.1:

# Create my repo and add a file A % mkdir my_repo % cd my_repo % git init-db defaulting to local storage area % echo A > A % git add A && git commit -a -m "Add A" Committing initial tree f53c91092dbda83f3565e78c285f3e2ab0cfd968

# Create a peer repo and add a file B % cd .. % mkdir peer_repo % cd peer_repo/ % git init-db defaulting to local storage area % echo B > B % git add B && git commit -a -m "Add B" Committing initial tree bfcfa4ca04d80d4b092e022ad163e82ca0f4a34f

# Create a topic branch in peer repo and add file C % git checkout -b topic % echo C > C % git add C && git commit -a -m "Add C"

# Go back to my repo and fetch peer's topic branch
% cd ../my_repo/
% git fetch ../peer_repo topic:topic
warning: no common commits
remote: Generating pack...
remote: Done counting 6 objects.
remote: Deltifying 6 objects.
remote: /6) done/6) done
Unpacking 6 objects
remote: Total 6, written 6 (delta 0), reused 0 (delta 0)
 100% (6/6) done
* refs/heads/topic: storing branch 'topic' of ../peer_repo
  commit: af3ab53

# In my repo, checkout topic and change B % git checkout topic % echo "Change B" >> B % git commit -a -m "Change B"

# Go back to peer repo and change C % cd ../peer_repo/ % echo "Change C" >> C % git commit -a -m "Change C"

# Go back to my repo and pull peer's topic branch
% cd ../my_repo
% git pull ../peer_repo topic:topic
remote: Generating pack...
remote: Done counting 5 objects.
remote: Result has 3 objects.
remote: Deltifying 3 objects.
remote:  100% (3/3) done
remote: Total 3, written 3 (delta 0), reused 0 (delta 0)
Unpacking 3 objects
 100% (3/3) done
* refs/heads/topic: not updating to non-fast forward branch 'topic' of ../peer_repo
  old...new: d3cf979...9e36a9a
Trying really trivial in-index merge...
Wonderful.
In-index merge
 C |    1 +
 1 files changed, 1 insertions(+), 0 deletions(-)

So, why does it say "not updating to non-fast forward branch", yet it does the merge and gets the changes anyway?

# Seem to have gotten the changes into my_repo anyway:
% git branch
  master
* topic
% cat B C
B
Change B
C
Change C

I'm trying to come up with a white-board presentation I can give to the group this afternoon and they are a contumacious bunch.

Bill

On Fri, Feb 16, 2007 at 09:13:22AM -0600, Bill Lear wrote:

Because your pull command really means "merge in the topic branch from peer_repo, and while you're at it, store it in my local tracking branch topic". Remember that pull is really a fetch+merge. But the fetch is actually doing _two_ things: putting the fetched branch into FETCH_HEAD, and putting it in into refs/heads/topic. The latter fails (because of a non-fastforward), but pull actually uses the FETCH_HEAD results to do the merge.

Yes, this seems overly complex for what you're doing, but the reason for FETCH_HEAD is to support pulls when you _don't_ have a tracking branch at all (i.e., 'git pull ../peer_repo topic').

-Peff

On Friday, February 16, 2007 at 10:21:30 (-0500) Jeff King writes:

Ok, fair enough, but then I guess I'm back to my original question: how can I give a concrete demonstration to our developers that this is a bad thing?

This is not 100% required, so if you are tired of answering my incessant questions, feel free to decline. I will be able to get our group to move forward, simply because we need to try to stay current, and there are lots of improvements in git besides this issue.

Bill

On Fri, Feb 16, 2007 at 09:27:45AM -0600, Bill Lear wrote:

I think it will always work with the example you gave, because you are simultaneously fetching into the tracking branch (which fails) and merging from FETCH_HEAD (which succeeds) into that same tracking branch.

At best, though, the tracking branch you have is pointless (since you immediately overwrite it anyway). The point of a tracking branch generally is to allow you to do operations against your peer's idea of the branch (e.g., diffing against upstream's version of "topic"). But you can't do that, because "topic" always contains your topic, not upstream's. In effect, your pull becomes one without a tracking branch at all.

This will also get you on a push, where there is no merging at all, just a fast-forward (or failure). IIRC, you ran into problems before because you were trying to push into your public repo from your private, but the two had divergent branches. So I think to illustrate the problem you had before, you actually need an intermediate repo which has you fetch from and push to.

-Peff

On 2/16/07, Bill Lear <rael@zopyra.com> wrote:

It no longer works with recent git, as of v1.4.4.1-37-gd25c26e. Now git-fetch exit with a non-zero status when fast-forward check fails, so the merge does not happen.

Santi

On Fri, 16 Feb 2007, Bill Lear wrote:

Try this:

	# create first repository
	mkdir foo
	cd foo
	git init-db
	echo FOO > biz
	git add biz
	git commit -a -m "commit 1 in foo"

	# clone this repository elsewhere
	cd ..
	git clone foo bar
	cd bar

	# modify the tracking branch (origin in this case)
	git checkout origin
	echo BAR > biz
	git commit -a -m "modify tracking branch in bar"

	# go back to original repo and modify it
	cd ../foo
	echo BAZ > biz
	git commit -a -m "modify master branch in foo"

	# now back to the cloned repo
	cd ../bar
	git fetch	# it fails
	git push	# itfails
	git pull	# it almost performs the merge but...

Because you now have a conflict to resolve, you might wish to inspect what the remote state actually is. But because you modified your origin branch you don't have any pristine version of what the remote has, except maybe the MERGE_HEAD. But if you work in a complex project, MERGE_HEAD will not stay there for long if you move around.

Imagine that you want to move to another branch, say master, because right now you don't feel like resolving this ultra complex merge:

	git checkout master	# it fails (unresolved merge)
	git checkout -f		# succeeds, discarding the merge

But at this point you still don't have any way to look at the remote's content. If instead the origin branch was untouched and the work had occurred in the master branch, then you could git-log the origin branch, you could git-diff your master branch with the origin branch, you could even checkout the origin content to test it, etc. But since you added commits on top of the origin branch then you cannot do any of that because it is impossible to update the origin branch with the remote stuff. Well you can force it of course:

	git fetch -f

But by doing so you lose all the work you committed on top of origin.

BTW I checked out v1.4.4.1 to verify the above operations..... and this GIT version really feels odd compared to v1.5.0

Nicolas

On Friday, February 16, 2007 at 11:34:01 (-0500) Nicolas Pitre writes:

Ok, that's basically what I was looking for --- incontrovertible evidence from "them" (a git expert --- not me). Thanks very much Nicolas.

Bill

On Thu, 15 Feb 2007, Bill Lear wrote:

If you commit on top of a tracking branch, then you won't be ablt to update that branch with remote changes because the tracking branch will contain local changes that the remote doesn't have. In other words, the remote end won't be able to determine the set of changes you are missing to send you only those missing changes.

At that point it is still possible to do the reverse, i.e. push your local changes to the remote then both local and remote branches will be in sync. But that works only if the remote was not updated by someone else. If the remote was updated by someone else, then you fall into the same situation as if you committed on top of a tracking branch and you try to fetch, because it is then impossible for you to determine what the remote is lacking since you won't find the remote changes in your local repository. You are then stuck in a dead lock.

Now there is of course a way to force a push or a fetch even if the above scenarios occur. But if you force the push then the remote changes that you don't have will be lost. If you force a fetch then your local changes that the remote doesn't have will be lost.

This is why it is better to have a tracking branch that is modified only by fetching remote changes only. When you make a local copy of that branch and commit changes to that work branch, then you can push those changes to update the remote branch. And if the remote branch has changed then you have the possibility of updating the tracking branch with the remote changes you don't have yet, merge them into your local copy with your work, and reattempt the push. But that works fine only if the tracking branch is always a subset of what the remote has.

Nicolas