Re: Please undo "Use git-merge instead of git-resolve in

Linus schreib:

... "origin".  OK.

Und, Petr schreib auch:

Wait.

For me, this paragraph suddenly turned on one missing light: The default construction of repository branches/heads _mismatches_ names on local and remote ends: "origin" local came from "master" remote. Did I miss reading that somewhere else? (Likely.)

And I sat through the Great Remote Name Discussion of '05 ("How is working on arbitrary remote heads supposed to work in Cogito") but I just didn't get it back then.

(This is an intentional asymmetry, right?  Distributed systems, right?)

In any case, I just tried this:

    git fetch rsync://www.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.gi\t master:origin

And got this mess:

    sent 18136 bytes  received 2731519 bytes  60431.98 bytes/sec
    total size is 97584183  speedup is 35.49
    rsync: link_stat "/scm/linux/kernel/git/torvalds/linux-2.6.git/objects/info/alt\ernates" (in pub) failed: No such file or directory (2)
    rsync error: some files could not be transferred (code 23) at main.c(1173)
    * non-commit: 3fd07d3bf0077dcc0f5a33d2eb1938ea050da8da
      branch 'master' of rsync://www.kernel.org/pub/scm/linux/kernel/git/torvalds/l\inux-2.6
    * refs/heads/origin: does not fast forward to branch 'master' of rsync://www.ke\rnel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6;
      leaving it in 'refs/heads/origin.remote'

And this is due to kernel.org being Not Quite Right, right?

Which points out one of the other points of frustration that I feel should be addressed eventually: A whole section about "What To Do When It Goes Wonky" needs to be written.

OK, so it didn't merge? Now what? What got left where? How do I recover? What bits are in my tree, and what bits are in the Index, and what bits are in the Object store now?

OK, so it didn't download it left you "refs/heads/rigin.remote". What should I do with it now? And later, should I re-execute the same "git fetch" command and hope it recovers and patches the pieces together? Should I do a round of house cleaning before attempting to re-run some (the same?) command?

Things of that nature.

And more as I get further too. :-)

Thanks! jdl

Dear diary, on Thu, Sep 22, 2005 at 08:32:45PM CEST, I got a letter where Jon Loeliger <jdl@freescale.com> told me that...

FWIW, with Cogito, interrupted or failed fetch can be safely rerun, no extra recovery procedure is required. It *seems* that this holds for git-fetch as well.

On Thu, September 22, 2005 3:10 pm, Petr Baudis said:

Petr,

Why doesn't cogito just use the git fetch/pull commands? Why does it need anything special? It seems like cogito is doing more than just being an ease-of-use layer above git.

Sean

"Sean" <seanlkml@sympatico.ca> writes:

The way this question is posed is quite unfair to Pasky -- it makes him look needlessly bad.

The simple reason is because Cogito had its own richer fetch/pull first. The development of git aware pack transfer protocols by Linus and the list discussion for multi-head pushes and pulls came much later, which resulted in the current 'git fetch/pull' interface.

On Thu, September 22, 2005 6:22 pm, Junio C Hamano said:

Will try to pose it differently then, because it was not meant to make him "look bad". In a different email Pasky seemed to be musing that cogito might do away with fast forward merges. This too seemed like a decision best left to the plumbing, so i have been wondering how Pasky views cogito's relationship to git.

But the immediate question really was, wouldn't it be better if cogito used the same code paths that git uses for push/pull/fetch? Is there a reason that this isn't possible?

Yes, cogito had it first but once this functionality gets pushed down into git (where it's been for a while now) it makes a lot of sense for the procelain layers to use it. That way the functionality only has to be maintained in one place and nobody has to guess what transports work with cogito or git etc. But perhaps there are reasons that this just isn't possible with the cogito code, i dunno.

Sean

On Thu, 22 Sep 2005, Jon Loeliger wrote:

Right. It's intentional. The "master" branch tends to be the main one for everybody, but _my_ "master" branch is clearly _your_ "linus" branch, and _your_ "master" branch would be the "jon" branch for me.

So it's not "asymmetric". It _is_ symmetric, but it's symmetric the same way "left" and "right" are symmetric when facing each other - my left is your right, your left is my right. It's a symmetry, but it's not an _identity_.

And obviously, nobody is going to have every possible branch anyway. You just name the ones you care about (and that, in turn, _will_ break the symmetry ;)

Ok, ignore that error.

Ok, 3fd07d3bf0077dcc0f5a33d2eb1938ea050da8da is my current head, so that part looks good. However, the "non-commit" part doesn't look good. It _should_ say "committish".

What does

	git-cat-file -t 3fd07d3bf0077dcc0f5a33d2eb1938ea050da8da

say in your tree? I suspect you don't have that object at all, and that the mirrors are still not in sync (ie I suspect you get a "error: unable to find 3fd..")

This, btw, is why I hate the rsync protocol. Quite frankly, it sucks. It was wonderful for bootstrapping git when we didn't have anything else, but dammit, it's horrid. It has no git knowledge at all, does _zero_ sanity checking, and leaves the user going "huh" when it fails.

Yes. It tries to verify that the new commit it stuffs into "origin" is a proper superset of the old one (so that you're not losing any data), and it fails, since the new commit doesn't even exist in your object list yet.

Hey, you seem to have a good example of Wonky, maybe you want to write it?

The good news is, I don't think there's anything to recover. It refused to write the new head, because it wasn't valid yet.

Actually, the fact that it even left it as a head might cause problems (because now you have a reference to something that doesn't exist), so you are probably best off just removing the .git/refs/heads/origin.remote file entirely.

You just need to try again when the mirroring is doing better.

Oh. And I think you might want to change that "rsync:" to "http:" instead. It's going to be slower, but at least git-http-fetch should be more careful, since it's fetching one object at a time, and validating what it fetches that way. The rsync: protocol just does everything totally blidly (which makes it fairly efficient, but also means that when things go wrong, they are _really_ confusing).

		Linus

Dear diary, on Thu, Sep 22, 2005 at 09:25:34PM CEST, I got a letter where Sean <seanlkml@sympatico.ca> told me that...

As Junio already explained, Cogito had those commands earlier - so the main reason was simply that I didn't manage to do it yet. It is not so easy to keep up with the latest GIT stuff _and_ enhance Cogito at the same time, so I simply didn't get to really consider that yet. :-)

That said, this is a non-comprehensive list of the factors in my consideration:

Pros (not so much of them, but they are big):
	* Already supports alternates and remotes
	* Do it once, stop caring :-) - only single instance of the code
	  does the stuff (this is obviously the motivation)

Cons:
	* cg-fetch can do with symlinked object database
	* cg-fetch will hardlink if possible when fetching locally
	* cg-fetch will fetch the tags if possible
		(this is actually a bit controversial - we should have
		the private/public tags distinction, that's another
		thing I simply didn't get to yet)
	* cg-fetch has cute progress info
		(perhaps git-fetch output could be just fed to that)
	* cg-fetch won't do useless fetches when we are already
	  up-to-date
	* I'm not sure if I could call git-fetch to do the initial
	  fetch during clone
	* The amount of work to initially convert it might be comparable
	  with implementing the current new stuff
	* I really dislike the git-fetch coding style
		This is not that I'd like to force my coding style
		customs on anyone and Junio might well feel the same
		about Cogito's code, but I would have to support and
		fix bugs in git-fetch if I used it.
	* I'm obviously not too happy to throw away the rather big
	  amount of code I spent a lot of time on and which is already
	  pretty well debugged and tuned, I think (that's not to say
	  that git-fetch is buggy... ;-)

The cons may seem mostly minor stuff which is quite fixable, and that's right, it's just show that there is plenty of stuff to do before git-fetch will be superset of cg-fetch. Now because of the coding style, I might as well just implement the alternates and remotes stuff to cg-fetch, which would be less work and less painful for me in the short term.

I'll convert cg-object-id to use git-rev-parse after the release so that you can't say I'm an old-fashioned freak refusing to use any of the new GIT stuff. :^)

Petr Baudis <pasky@suse.cz> writes:

Do you mean ".git/object" in the repository you are fetching into is a symlink to somewhere, or something else?

True, git-fetch and git-clone try to use pack protocols by default, which is debatable. Adding -l (similar to what git-clone does) would not be too hard, if enough people want it.

Care to explain? Perhaps you are talking about rsync transport (in which case I would not be surprised)?

    $ git-init-db && git fetch http://kernel.org/pub/scm/git/git.git

should work.

There is another minor "con", if you include git-clone in the discussion. Over http transport, it refuses to run against a remote repository that does not have info/refs and info/packs, even if it is not packed. I think cg-clone uses recursive wget for directory traversal and does not have to use them.

Dear diary, on Fri, Sep 23, 2005 at 11:34:16AM CEST, I got a letter where Junio C Hamano <junkio@cox.net> told me that...

Yes, exactly that. You get it by doing cg-clone -l /local/path. O(1) cloning, but it has its downsides and dangers which I tried to describe in the cg-clone documentation.

Just that if the reference pointer did not change, you don't try to pull any objects. Speeds things up somewhat, and now it should always work (well, cg-fetch has -f which makes it always try objects as well, in case something went wrong anyway).

Good.

cg-clone just basically does cg-init && cg-fetch, and I'd really want to keep it on that. So effectively, cg-clone over http just runs git-http-fetch and trusts it to do the right thing (reasonably fast).

Recursive wget is used only for fetching tags (and it actually seems to work nicely, when directory listing is enabled, obviously). Yes, I should teach cg-fetch about info/refs too.

On Fri, 23 Sep 2005, Petr Baudis wrote:

That's actually how I use git, and it works for me. (To be precise, I have:

 ~/git/project.git/{objects,refs,info}

and I have

 ~/working/git/current-work/, with .git/{objects,refs,info} symlinks to 
                              the above.

git fetch works fine, with the exception that remotes aren't shared between working trees.

I assume you mean with rsync? The current git-http-fetch won't try any objects if the ref didn't change (or, for that matter, if we have a different ref that's the same), and the old one would make a local check for the objects before trying to downlaod anything.

	-Daniel
*This .sig left intentionally blank*

Petr Baudis <pasky@suse.cz> writes:

git-clone -l -s does optimization similar to that in spirit, but it does not create a repository with symlinked .git/object pointing at /some/where/original/repo/.git/object. Instead it uses alternates to borrow from the other; newly created objects are always stored locally.

But I thought you were talking about fetch. git-fetch should be able to pull into a repository whose .git/object happens to be a symlink pointing at somewhere else just fine.

Both git-aware protocol and commit walker peek at the remote ref and stop without downloading objects if it is known to us, so I think we are OK.

Of course, the above 'git fetch' only fetches remote HEAD and stores it in .git/FETCH_HEAD; if you truly want to 'clone', you could first run git-ls-remote to peek the remote refs and then give them as storing refspecs to git fetch, doing something like this (modulo renaming 'master' over there to 'origin' here):

	mkdir foo && cd foo && git-init-db
	refspec=$(git-ls-remote $repo | \
	        sed -n -e 's/^[0-9a-f]*	//' -e 's|refs/.*|&:&|p')
	git fetch $repo $refspec