Re: git merging

[ Git mailing list cc'd, since this is a general question that maybe 
  others have wondered about ]

On Fri, 17 Jun 2005, Jens Axboe wrote:

You can just simply remove the .git/refs/heads/branch-name file (obviously after having made sure that ".git/HEAD" points to something else), and then you can run "git prune", which will remove all objects that aren't referenced (directly or indirectly) by anything in ".git/refs/**".

Be very very careful, though. "git prune" is a dangerous script, since it will remove objects from the object store. If you had something that you don't have a proper reference for, or if you give "git prune" a list of references manually and it's not complete, "git prune" will happily remove objects that you didn't mean for it to remove.

The "git prune" script is really trivial, you can see for yourself what "git-prune-script" does.

NOTE! It's entirely possible to not actually delete the old branch, but just rename it, and use it as a reference while you build up the new branch with the same contents (but maybe different commit log messages, or maybe just re-ordering the commits in the new version). To rename a branch, just rename the ".git/refs/heads/<branch-name>" file to something else.

So you may decide to remove (and prune) the old branch only after you've re-created a new version of it, for example. That way you can use git itself to extract the contents of the branch you want to re-do, which is often easier.

Btw, if you have a recent version of "gitk" instealled, you can do

	gitk $(ls .git/revs/heads)

and you'll get a full graphical view of _all_ your branches, which can be very instructive.

		Linus

On Fri, Jun 17 2005, Linus Torvalds wrote:

That worked! Thanks.

Yeah that's very handy. I love the way the branches work, with just the sha in the file name. So easy to manipulate.

Linus Torvalds wrote:

I've been too lazy to report this, but...

last time I tried it, git-prune-script ate valid objects, for my multi-branch trees. This was 100% reproducible. Luckily, the objects it ate were all in the main branch, so an rsync-without-delete from linux-2.6.git/objects would restore the missing data.

This was verified by repeated runs of
	git-fsck-cache		# OK
	git-prune-script
	git-fsck-cache		# spits out tons of errors
	rsync ...		# restore deleted objects
	git-fsck-cache		# OK

I use hardlinks on the local disk, between several multi-branch repos, if that matters.

	Jeff

On Fri, 17 Jun 2005, Jeff Garzik wrote:

Ok. The most likely reason is that your main branch is not a symlink to ".git/refs/***" at all, but just a regular ".git/HEAD" file.

"git prune" will by default _only_ look for references in .git/refs/, and if you have any references anywhere else, you need to do all your refs by hand.

There might be a bug in the ref-lookup code, of course, but it's more likely that your refs just aren't where fsck expects them.

		Linus

Linus Torvalds wrote:

This is definitely not the case; my .git/HEAD is _always_ a symlink.

My git-switch-tree script, attached, demonstrates how .git/HEAD symlink is retargetted to the specified branch. My workflow depends on .git/HEAD being a symlink.

I'll see if I can reproduce with the latest git.

	Jeff

#!/bin/sh

if [ "x$1" != "x" ]
then
	if [ ! -f .git/refs/heads/$1 ]
	then
		echo Branch $1 not found.
		exit 1
	fi

	( cd .git && rm -f HEAD && ln -s refs/heads/$1 HEAD )
fi

git-read-tree -m HEAD && git-checkout-cache -q -f -u -a

On Fri, 17 Jun 2005, Jeff Garzik wrote:

Ok. Are you sure that you gave the same arguments (or rather, lack of arguments) to both fsck and "git prune"? The thing is, they are both really the same thing, so I'm pretty surprised. If git prune says something is unreachable, then git-fsck-cache shouldn't complain about it being gone, because one just depends on the other..

Btw, you can now do the same thing more safely and guarantee that it doesn't overwrite any old information by using

	git-read-tree -m -u <old-head> <new-head>

which basically switches from "old" to "new", and verifies that all the old index contents were valid in "old-head", and that any file that was dirty is not different in "new-head".

Your old script would silently overwrite any dirty state in your working directory, and drop anything that you had done a git-update-cache on but not committed.

Now, you may have _depended_ on that behaviour as a way to just reset the tree to a known state, but if so, I'd suggest using

	git-read-tree --reset HEAD && git-checkout-cache -q -f -u -a

for that instead (which will also throw away any partial merges).

So for the "switch" case, you might make it be something like

	if [ ! -f .git/refs/heads/$1 ]
	then
		echo "Branch '$1' not found"
		exit 1;
	fi
	git-read-tree -m -u HEAD "heads/$1" && ln -sf refs/heads/$1 .git/HEAD

which should do the right thing.

Totally untested, of course ;)

[ But the two-tree read-tree is definitely not untested: this is how we 
  do a safe "fast forward" in the git-resolve-script, which really ends up 
  being the exact same thing: it "switches" from one head to another ].

		Linus

On Fri, Jun 17 2005, Linus Torvalds wrote:

Seeing something weird here as well, my setup is similar to Jeffs in that I have branches in refs/heads/* and HEAD a symlink to the active one.

I committed a bad patch, so wanting to reverse that change I did an

# echo sha_of_previous_commit > .git/HEAD # git-read-tree -m HEAD && git-checkout-cache -q -f -u -a

which, if I understand correctly, should put me back where I was before. So continuing commits, checking the tree now gives me:

axboe@nelson:[.]l/git/linux-2.6-block.git $ git prune error: cannot map sha1 file c39ae07f393806ccf406ef966e9a15afc43cc36a bad sha1 entry '5dc01c595e6c6ec9ccda4f6f69c131c0dd945f8c' axboe@nelson:[.]l/git/linux-2.6-block.git $ git-fsck-cache error: cannot map sha1 file c39ae07f393806ccf406ef966e9a15afc43cc36a bad object in tag 5dc01c595e6c6ec9ccda4f6f69c131c0dd945f8c bad sha1 entry '5dc01c595e6c6ec9ccda4f6f69c131c0dd945f8c'

Running git prune again gives me the same output. What is wrong?

Hi, Jens Axboe wrote:

What does "git-cat-file -t" say on these two -- are they botth vald commit objects?

On Mon, Jun 20 2005, Matthias Urlichs wrote:

axboe@nelson:[.]l/git/linux-2.6-block.git $ git-cat-file -t c39ae07f393806ccf406ef966e9a15afc43cc36a error: cannot map sha1 file c39ae07f393806ccf406ef966e9a15afc43cc36a fatal: git-cat-file c39ae07f393806ccf406ef966e9a15afc43cc36a: bad file axboe@nelson:[.]l/git/linux-2.6-block.git $ git-cat-file -t 5dc01c595e6c6ec9ccda4f6f69c131c0dd945f8c tag

[ Daniel put on the To: list to see if he can confirm or deny my theory ]

On Mon, 20 Jun 2005, Jens Axboe wrote:

That's the 2.6.11 "tree" object.

Very interesting. You have my "v2.6.11-tree" tag file pointing the 2.6.11 tree object, but you seem to not have that 2.6.11 tree itself.

Actually, judging from the fact that you got this error _during_ the prune, maybe you never had it in that repository in the first place?

In particular, if you don't use "rsync", but instead use one of the "optimized pull" things to create a repository (ie git-http-pull or git-ssh-pull), I think your newly pulled tree will always miss anything that isn't a head. And the 2.6.11 tree is a special case: it's a pure "tree" object without any commit at all pointing to it, just a single tag that points directly to the tree.

Anyway, the fact that you're missing the original 2.6.11 tree doesn't really matter, so it's not a huge deal per se. You can re-populate it with an "rsync -avz --ignore-existing", but clearly something seems to be wrong.

Daniel: I think git-ssh-pull has problems with any refs that aren't
commits. I also noticed earlier that I can't use git-ssh-push -w to write
a tag - it is apparently unhappy because it would write a "tag" object,
and it only wants to write references to "commit" objects. The same thing
might be true about the special "tag to a tree" object?

Jeff, do you remember (did you check?) if your prune problems might also have been about the tag to a tree?

		Linus

On Mon, 20 Jun 2005, Linus Torvalds wrote:

You're correct; it's actually that pull.c has never gotten extended to look at the object you specify, determine what it is, and then behave appropriately. Basically, it needs a "process_unknown()" which does the appropriate thing, and pull() needs to call that instead of process_commit(). It should also get a "process_tag()" which handles tags (calling process_unknown() on their objects). I should be able to do it Tuesday evening or Wednesday night, if nobody else gets it before then.

	-Daniel
*This .sig left intentionally blank*

Hi, Daniel Barkalow wrote:

It also needs a request queue.

The beauty of rsync() is that it doesn't depend on a low-latency wire to work efficiently. Right now, pull() needs one round trip per object, and that's way too much.

I hope to have some free time for that this week, but no promises. :-/

On Mon, Jun 20 2005, Linus Torvalds wrote:

I can't tell exactly, but I'm fairly sure this is a new error. I have two git trees locally - one which is just an rsync of your kernel.org tree, the other which is based off that (with a clone of the fs dir) and has multiple branches for developments. The main tree did not have this problem, while the development tree does/did.

I pulled with rsync manually from kernel.org, and that did fix things up for me. The main tree is rsync'ed, but the development tree gets the changes with /opt/kernel/git/linux-2.6/.git/ as the url given to git-pull-script.

On Mon, 20 Jun 2005, Jens Axboe wrote:

Ok, that explains it. Since you're using a regular local filename, the pull will be using "git-local-pull", which will only fetch the objects directly needed. And doesn't understand the tag-to-tree thing, so doesn't fetch the tree (or possibly you just copied the tags by hand totally outside of the regular pull?)

		Linus

On Mon, Jun 20 2005, Linus Torvalds wrote:

Isn't that a little 'end user' confusing from a usability point of view, that it behaves differently depending on which pull script it ends up using in the end?

I guess I can just always use rsync even for local trees. And use it directly, so I always have everything :)

On Tue, 21 Jun 2005, Jens Axboe wrote:

Yes, I'm not claiming it's good, I'm just claiming it explains what you see ;)

So it's a bug, but I'm hoping somebody else will fix it. I've not actually worked on that "intelligent pull" functionality at all (aside from making the ssh-push thing work for my limited use).

			Linus