# Re: Please undo "Use git-merge instead of git-resolve in 11 messages from 2005-09-22 to 2005-09-24. Participants: Jon Loeliger, Petr Baudis, Sean, Linus Torvalds, Junio C Hamano, Daniel Barkalow. Thread: https://gitlist.dev/t/1916 ## Jon Loeliger, 2005-09-22 18:32 Subject: Re: Please undo "Use git-merge instead of git-resolve in Message-ID: URL: https://gitlist.dev/e/E1EIVsD-0001Hu-9m%40jdl.com ``` Linus schreib: > > git fetch rsync:..../linus-2.6-git : > > will fetch the branch (ie you'd usually use "master") from my tree > and write it to the branch on your tree. ... "origin". OK. Und, Petr schreib auch: > Then it creates an 'origin' head, and will copy all the history from > the remote repository's 'master' head there. So this head exists to > reflect the state of the remote repository. The important point is > that it is called 'origin' in our new repository, even through it > corresponds to a 'master' head in the old repository. This is normal - > you can name your local heads whatever you want. Wait. For me, this paragraph suddenly turned on one missing light: The default construction of repository branches/heads _mismatches_ names on local and remote ends: "origin" local came from "master" remote. Did I miss reading that somewhere else? (Likely.) And I sat through the Great Remote Name Discussion of '05 ("How is working on arbitrary remote heads supposed to work in Cogito") but I just didn't get it back then. (This is an intentional asymmetry, right? Distributed systems, right?) In any case, I just tried this: git fetch rsync://www.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.gi\t master:origin And got this mess: sent 18136 bytes received 2731519 bytes 60431.98 bytes/sec total size is 97584183 speedup is 35.49 rsync: link_stat "/scm/linux/kernel/git/torvalds/linux-2.6.git/objects/info/alt\ernates" (in pub) failed: No such file or directory (2) rsync error: some files could not be transferred (code 23) at main.c(1173) * non-commit: 3fd07d3bf0077dcc0f5a33d2eb1938ea050da8da branch 'master' of rsync://www.kernel.org/pub/scm/linux/kernel/git/torvalds/l\inux-2.6 * refs/heads/origin: does not fast forward to branch 'master' of rsync://www.ke\rnel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6; leaving it in 'refs/heads/origin.remote' And this is due to kernel.org being Not Quite Right, right? Which points out one of the other points of frustration that I feel should be addressed eventually: A whole section about "What To Do When It Goes Wonky" needs to be written. OK, so it didn't merge? Now what? What got left where? How do I recover? What bits are in my tree, and what bits are in the Index, and what bits are in the Object store now? OK, so it didn't download it left you "refs/heads/rigin.remote". What should I do with it now? And later, should I re-execute the same "git fetch" command and hope it recovers and patches the pieces together? Should I do a round of house cleaning before attempting to re-run some (the same?) command? Things of that nature. And more as I get further too. :-) Thanks! jdl ``` ## Petr Baudis, 2005-09-22 19:10 Subject: Re: Please undo "Use git-merge instead of git-resolve in Message-ID: <20050922191058.GM21019@pasky.or.cz> URL: https://gitlist.dev/e/20050922191058.GM21019%40pasky.or.cz In-Reply-To: ``` Dear diary, on Thu, Sep 22, 2005 at 08:32:45PM CEST, I got a letter where Jon Loeliger told me that... > Which points out one of the other points of frustration > that I feel should be addressed eventually: A whole section > about "What To Do When It Goes Wonky" needs to be written. > > OK, so it didn't merge? Now what? What got left where? > How do I recover? What bits are in my tree, and what bits > are in the Index, and what bits are in the Object store now? > > OK, so it didn't download it left you "refs/heads/rigin.remote". > What should I do with it now? And later, should I re-execute > the same "git fetch" command and hope it recovers and patches > the pieces together? Should I do a round of house cleaning > before attempting to re-run some (the same?) command? FWIW, with Cogito, interrupted or failed fetch can be safely rerun, no extra recovery procedure is required. It *seems* that this holds for git-fetch as well. -- Petr "Pasky" Baudis Stuff: http://pasky.or.cz/ VI has two modes: the one in which it beeps and the one in which it doesn't. ``` ## Sean, 2005-09-22 19:25 Subject: Re: Please undo "Use git-merge instead of git-resolve in Message-ID: URL: https://gitlist.dev/e/BAYC1-PASMTP0510EEC44C7F787F27215AAE970%40CEZ.ICE In-Reply-To: <20050922191058.GM21019@pasky.or.cz> ``` On Thu, September 22, 2005 3:10 pm, Petr Baudis said: > FWIW, with Cogito, interrupted or failed fetch can be safely rerun, no > extra recovery procedure is required. It *seems* that this holds for > git-fetch as well. Petr, Why doesn't cogito just use the git fetch/pull commands? Why does it need anything special? It seems like cogito is doing more than just being an ease-of-use layer above git. Sean ``` ## Linus Torvalds, 2005-09-22 21:12 Subject: Re: Please undo "Use git-merge instead of git-resolve in Message-ID: URL: https://gitlist.dev/e/Pine.LNX.4.58.0509221355330.2553%40g5.osdl.org In-Reply-To: ``` On Thu, 22 Sep 2005, Jon Loeliger wrote: > > For me, this paragraph suddenly turned on one missing light: > The default construction of repository branches/heads _mismatches_ > names on local and remote ends: "origin" local came from "master" > remote. Did I miss reading that somewhere else? (Likely.) > > And I sat through the Great Remote Name Discussion of '05 > ("How is working on arbitrary remote heads supposed to work in Cogito") > but I just didn't get it back then. > > (This is an intentional asymmetry, right? Distributed systems, right?) Right. It's intentional. The "master" branch tends to be the main one for everybody, but _my_ "master" branch is clearly _your_ "linus" branch, and _your_ "master" branch would be the "jon" branch for me. So it's not "asymmetric". It _is_ symmetric, but it's symmetric the same way "left" and "right" are symmetric when facing each other - my left is your right, your left is my right. It's a symmetry, but it's not an _identity_. And obviously, nobody is going to have every possible branch anyway. You just name the ones you care about (and that, in turn, _will_ break the symmetry ;) > In any case, I just tried this: > > git fetch rsync://www.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git master:origin > > And got this mess: > > sent 18136 bytes received 2731519 bytes 60431.98 bytes/sec > total size is 97584183 speedup is 35.49 > rsync: link_stat "/scm/linux/kernel/git/torvalds/linux-2.6.git/objects/info/alternates" (in pub) failed: No such file or directory (2) > rsync error: some files could not be transferred (code 23) at main.c(1173) Ok, ignore that error. > * non-commit: 3fd07d3bf0077dcc0f5a33d2eb1938ea050da8da > branch 'master' of rsync://www.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6 Ok, 3fd07d3bf0077dcc0f5a33d2eb1938ea050da8da is my current head, so that part looks good. However, the "non-commit" part doesn't look good. It _should_ say "committish". What does git-cat-file -t 3fd07d3bf0077dcc0f5a33d2eb1938ea050da8da say in your tree? I suspect you don't have that object at all, and that the mirrors are still not in sync (ie I suspect you get a "error: unable to find 3fd..") This, btw, is why I hate the rsync protocol. Quite frankly, it sucks. It was wonderful for bootstrapping git when we didn't have anything else, but dammit, it's horrid. It has no git knowledge at all, does _zero_ sanity checking, and leaves the user going "huh" when it fails. > * refs/heads/origin: does not fast forward to branch 'master' of rsync://www.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6; > leaving it in 'refs/heads/origin.remote' > > And this is due to kernel.org being Not Quite Right, right? Yes. It tries to verify that the new commit it stuffs into "origin" is a proper superset of the old one (so that you're not losing any data), and it fails, since the new commit doesn't even exist in your object list yet. > Which points out one of the other points of frustration > that I feel should be addressed eventually: A whole section > about "What To Do When It Goes Wonky" needs to be written. Hey, you seem to have a good example of Wonky, maybe you want to write it? > OK, so it didn't merge? Now what? What got left where? > How do I recover? What bits are in my tree, and what bits > are in the Index, and what bits are in the Object store now? The good news is, I don't think there's anything to recover. It refused to write the new head, because it wasn't valid yet. Actually, the fact that it even left it as a head might cause problems (because now you have a reference to something that doesn't exist), so you are probably best off just removing the .git/refs/heads/origin.remote file entirely. You just need to try again when the mirroring is doing better. Oh. And I think you might want to change that "rsync:" to "http:" instead. It's going to be slower, but at least git-http-fetch should be more careful, since it's fetching one object at a time, and validating what it fetches that way. The rsync: protocol just does everything totally blidly (which makes it fairly efficient, but also means that when things go wrong, they are _really_ confusing). Linus ``` ## Junio C Hamano, 2005-09-22 22:22 Subject: Re: Please undo "Use git-merge instead of git-resolve in Message-ID: <7vk6h8lp3k.fsf@assigned-by-dhcp.cox.net> URL: https://gitlist.dev/e/7vk6h8lp3k.fsf%40assigned-by-dhcp.cox.net In-Reply-To: ``` "Sean" writes: > Why doesn't cogito just use the git fetch/pull commands? Why does it > need anything special? It seems like cogito is doing more than just > being an ease-of-use layer above git. The way this question is posed is quite unfair to Pasky -- it makes him look needlessly bad. The simple reason is because Cogito had its own richer fetch/pull first. The development of git aware pack transfer protocols by Linus and the list discussion for multi-head pushes and pulls came much later, which resulted in the current 'git fetch/pull' interface. ``` ## Sean, 2005-09-22 22:54 Subject: Re: Please undo "Use git-merge instead of git-resolve in Message-ID: URL: https://gitlist.dev/e/BAYC1-PASMTP0529AB50E9E3F8BACFE1EFAE970%40CEZ.ICE In-Reply-To: <7vk6h8lp3k.fsf@assigned-by-dhcp.cox.net> ``` On Thu, September 22, 2005 6:22 pm, Junio C Hamano said: > "Sean" writes: > >> Why doesn't cogito just use the git fetch/pull commands? Why does it >> need anything special? It seems like cogito is doing more than just >> being an ease-of-use layer above git. > > The way this question is posed is quite unfair to Pasky -- it > makes him look needlessly bad. Will try to pose it differently then, because it was not meant to make him "look bad". In a different email Pasky seemed to be musing that cogito might do away with fast forward merges. This too seemed like a decision best left to the plumbing, so i have been wondering how Pasky views cogito's relationship to git. But the immediate question really was, wouldn't it be better if cogito used the same code paths that git uses for push/pull/fetch? Is there a reason that this isn't possible? > The simple reason is because Cogito had its own richer > fetch/pull first. The development of git aware pack transfer > protocols by Linus and the list discussion for multi-head pushes > and pulls came much later, which resulted in the current 'git > fetch/pull' interface. Yes, cogito had it first but once this functionality gets pushed down into git (where it's been for a while now) it makes a lot of sense for the procelain layers to use it. That way the functionality only has to be maintained in one place and nobody has to guess what transports work with cogito or git etc. But perhaps there are reasons that this just isn't possible with the cogito code, i dunno. Sean ``` ## Petr Baudis, 2005-09-23 09:10 Subject: Re: Please undo "Use git-merge instead of git-resolve in Message-ID: <20050923091012.GA10255@pasky.or.cz> URL: https://gitlist.dev/e/20050923091012.GA10255%40pasky.or.cz In-Reply-To: <34462.10.10.10.28.1127417134.squirrel@linux1> ``` Dear diary, on Thu, Sep 22, 2005 at 09:25:34PM CEST, I got a letter where Sean told me that... > On Thu, September 22, 2005 3:10 pm, Petr Baudis said: > > > FWIW, with Cogito, interrupted or failed fetch can be safely rerun, no > > extra recovery procedure is required. It *seems* that this holds for > > git-fetch as well. > > Petr, > > Why doesn't cogito just use the git fetch/pull commands? Why does it > need anything special? It seems like cogito is doing more than just > being an ease-of-use layer above git. As Junio already explained, Cogito had those commands earlier - so the main reason was simply that I didn't manage to do it yet. It is not so easy to keep up with the latest GIT stuff _and_ enhance Cogito at the same time, so I simply didn't get to really consider that yet. :-) That said, this is a non-comprehensive list of the factors in my consideration: Pros (not so much of them, but they are big): * Already supports alternates and remotes * Do it once, stop caring :-) - only single instance of the code does the stuff (this is obviously the motivation) Cons: * cg-fetch can do with symlinked object database * cg-fetch will hardlink if possible when fetching locally * cg-fetch will fetch the tags if possible (this is actually a bit controversial - we should have the private/public tags distinction, that's another thing I simply didn't get to yet) * cg-fetch has cute progress info (perhaps git-fetch output could be just fed to that) * cg-fetch won't do useless fetches when we are already up-to-date * I'm not sure if I could call git-fetch to do the initial fetch during clone * The amount of work to initially convert it might be comparable with implementing the current new stuff * I really dislike the git-fetch coding style This is not that I'd like to force my coding style customs on anyone and Junio might well feel the same about Cogito's code, but I would have to support and fix bugs in git-fetch if I used it. * I'm obviously not too happy to throw away the rather big amount of code I spent a lot of time on and which is already pretty well debugged and tuned, I think (that's not to say that git-fetch is buggy... ;-) The cons may seem mostly minor stuff which is quite fixable, and that's right, it's just show that there is plenty of stuff to do before git-fetch will be superset of cg-fetch. Now because of the coding style, I might as well just implement the alternates and remotes stuff to cg-fetch, which would be less work and less painful for me in the short term. I'll convert cg-object-id to use git-rev-parse after the release so that you can't say I'm an old-fashioned freak refusing to use any of the new GIT stuff. :^) -- Petr "Pasky" Baudis Stuff: http://pasky.or.cz/ VI has two modes: the one in which it beeps and the one in which it doesn't. ``` ## Junio C Hamano, 2005-09-23 09:34 Subject: Re: Please undo "Use git-merge instead of git-resolve in Message-ID: <7vslvwcelj.fsf@assigned-by-dhcp.cox.net> URL: https://gitlist.dev/e/7vslvwcelj.fsf%40assigned-by-dhcp.cox.net In-Reply-To: <20050923091012.GA10255@pasky.or.cz> ``` Petr Baudis writes: > * cg-fetch can do with symlinked object database Do you mean ".git/object" in the repository you are fetching into is a symlink to somewhere, or something else? > * cg-fetch will hardlink if possible when fetching locally True, git-fetch and git-clone try to use pack protocols by default, which is debatable. Adding -l (similar to what git-clone does) would not be too hard, if enough people want it. > * cg-fetch won't do useless fetches when we are already > up-to-date Care to explain? Perhaps you are talking about rsync transport (in which case I would not be surprised)? > * I'm not sure if I could call git-fetch to do the initial > fetch during clone $ git-init-db && git fetch http://kernel.org/pub/scm/git/git.git should work. There is another minor "con", if you include git-clone in the discussion. Over http transport, it refuses to run against a remote repository that does not have info/refs and info/packs, even if it is not packed. I think cg-clone uses recursive wget for directory traversal and does not have to use them. ``` ## Petr Baudis, 2005-09-23 09:57 Subject: Re: Please undo "Use git-merge instead of git-resolve in Message-ID: <20050923095742.GC10255@pasky.or.cz> URL: https://gitlist.dev/e/20050923095742.GC10255%40pasky.or.cz In-Reply-To: <7vslvwcelj.fsf@assigned-by-dhcp.cox.net> ``` Dear diary, on Fri, Sep 23, 2005 at 11:34:16AM CEST, I got a letter where Junio C Hamano told me that... > Petr Baudis writes: > > > * cg-fetch can do with symlinked object database > > Do you mean ".git/object" in the repository you are fetching > into is a symlink to somewhere, or something else? Yes, exactly that. You get it by doing cg-clone -l /local/path. O(1) cloning, but it has its downsides and dangers which I tried to describe in the cg-clone documentation. > > * cg-fetch won't do useless fetches when we are already > > up-to-date > > Care to explain? Perhaps you are talking about rsync transport > (in which case I would not be surprised)? Just that if the reference pointer did not change, you don't try to pull any objects. Speeds things up somewhat, and now it should always work (well, cg-fetch has -f which makes it always try objects as well, in case something went wrong anyway). > > * I'm not sure if I could call git-fetch to do the initial > > fetch during clone > > $ git-init-db && git fetch http://kernel.org/pub/scm/git/git.git > > should work. Good. > There is another minor "con", if you include git-clone in the > discussion. Over http transport, it refuses to run against a > remote repository that does not have info/refs and info/packs, > even if it is not packed. I think cg-clone uses recursive wget > for directory traversal and does not have to use them. cg-clone just basically does cg-init && cg-fetch, and I'd really want to keep it on that. So effectively, cg-clone over http just runs git-http-fetch and trusts it to do the right thing (reasonably fast). Recursive wget is used only for fetching tags (and it actually seems to work nicely, when directory listing is enabled, obviously). Yes, I should teach cg-fetch about info/refs too. -- Petr "Pasky" Baudis Stuff: http://pasky.or.cz/ VI has two modes: the one in which it beeps and the one in which it doesn't. ``` ## Daniel Barkalow, 2005-09-23 21:07 Subject: Re: Please undo "Use git-merge instead of git-resolve in Message-ID: URL: https://gitlist.dev/e/Pine.LNX.4.63.0509231648310.23242%40iabervon.org In-Reply-To: <20050923095742.GC10255@pasky.or.cz> ``` On Fri, 23 Sep 2005, Petr Baudis wrote: > Dear diary, on Fri, Sep 23, 2005 at 11:34:16AM CEST, I got a letter > where Junio C Hamano told me that... > > Petr Baudis writes: > > > > > * cg-fetch can do with symlinked object database > > > > Do you mean ".git/object" in the repository you are fetching > > into is a symlink to somewhere, or something else? > > Yes, exactly that. You get it by doing cg-clone -l /local/path. O(1) > cloning, but it has its downsides and dangers which I tried to describe > in the cg-clone documentation. That's actually how I use git, and it works for me. (To be precise, I have: ~/git/project.git/{objects,refs,info} and I have ~/working/git/current-work/, with .git/{objects,refs,info} symlinks to the above. git fetch works fine, with the exception that remotes aren't shared between working trees. > > > * cg-fetch won't do useless fetches when we are already > > > up-to-date > > > > Care to explain? Perhaps you are talking about rsync transport > > (in which case I would not be surprised)? > > Just that if the reference pointer did not change, you don't try to pull > any objects. Speeds things up somewhat, and now it should always work > (well, cg-fetch has -f which makes it always try objects as well, in > case something went wrong anyway). I assume you mean with rsync? The current git-http-fetch won't try any objects if the ref didn't change (or, for that matter, if we have a different ref that's the same), and the old one would make a local check for the objects before trying to downlaod anything. -Daniel *This .sig left intentionally blank* ``` ## Junio C Hamano, 2005-09-24 06:19 Subject: Re: Please undo "Use git-merge instead of git-resolve in Message-ID: <7virwr2dka.fsf@assigned-by-dhcp.cox.net> URL: https://gitlist.dev/e/7virwr2dka.fsf%40assigned-by-dhcp.cox.net In-Reply-To: <20050923095742.GC10255@pasky.or.cz> ``` Petr Baudis writes: > Dear diary, on Fri, Sep 23, 2005 at 11:34:16AM CEST, I got a letter > where Junio C Hamano told me that... >> Petr Baudis writes: >> >> > * cg-fetch can do with symlinked object database >> >> Do you mean ".git/object" in the repository you are fetching >> into is a symlink to somewhere, or something else? > > Yes, exactly that. You get it by doing cg-clone -l > /local/path. git-clone -l -s does optimization similar to that in spirit, but it does not create a repository with symlinked .git/object pointing at /some/where/original/repo/.git/object. Instead it uses alternates to borrow from the other; newly created objects are always stored locally. But I thought you were talking about fetch. git-fetch should be able to pull into a repository whose .git/object happens to be a symlink pointing at somewhere else just fine. > Just that if the reference pointer did not change, you don't try to pull > any objects. Both git-aware protocol and commit walker peek at the remote ref and stop without downloading objects if it is known to us, so I think we are OK. >> $ git-init-db && git fetch http://kernel.org/pub/scm/git/git.git >> >> should work. > > Good. Of course, the above 'git fetch' only fetches remote HEAD and stores it in .git/FETCH_HEAD; if you truly want to 'clone', you could first run git-ls-remote to peek the remote refs and then give them as storing refspecs to git fetch, doing something like this (modulo renaming 'master' over there to 'origin' here): mkdir foo && cd foo && git-init-db refspec=$(git-ls-remote $repo | \ sed -n -e 's/^[0-9a-f]* //' -e 's|refs/.*|&:&|p') git fetch $repo $refspec ```