{"thread":{"id":"4528","subject":"Re: 2.6.17-rc6-mm2","startedAt":"2006-06-16T01:14:09Z","lastAt":"2006-06-20T03:22:39Z","messageCount":11,"participants":["Goo GGooo","Linus Torvalds","Uwe Zeisberger","H. Peter Anvin","bert hubert","Junio C Hamano","Michal Ludvig"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"21863","messageId":"ef5305790606151814i252c37c4mdd005f11f06ceac@mail.gmail.com","threadId":"4528","inReplyTo":"ef5305790606142040r5912ce58kf9f889c3d61b2cc0@mail.gmail.com","subject":"Re: 2.6.17-rc6-mm2","fromName":"Goo GGooo","fromEmail":"googgooo@gmail.com","sentAt":"2006-06-16T01:14:09Z","receivedAt":"2006-06-16T01:14:09Z","isPatch":false,"sender":{"key":"googgooo@gmail.com","avatar":null},"body":"On 6/15/06, Goo GGooo wrote:\n> Andrew Morton wrote:\n>\n> > - To fetch an -mm tree using git, use (for example)\n> >\n> > git fetch git://git.kernel.org/pub/scm/linux/kernel/git/smurf/linux-trees.git\n> > v2.6.16-rc2-mm1\n>\n> I'm not able to get -mm tree from GIT. In\n> http://git.kernel.org/.../smurf/linux-trees.git/refs/tags/ I can see\n> the most recent tags like v2.6.17-rc6-mm2 but cg-clone\n> http://git.kernel.org/.../smurf/linux-trees.git gives me only\n> 2.6.16-rc3 :(\n>\n> I tried \"cg-fetch v2.6.17-rc6-mm2\" which seemed to fetch some more\n> tags, then played with git-checkout & friends but still can't get the\n> most recent source tree.\n\nAll right, finally this worked out:\ngit pull rsync://git.kernel.org/pub/scm/linux/kernel/git/smurf/linux-trees.git \\\n tag v2.6.17-rc6-mm2\n\nStrange enough with http:// instead of rsync:// I got some message\nabout nonexistent tag.\n\nNow when I try git pull with http:// again it says the tree is up to\ndate. However with git:// it started downloading more things and tags.\n\nThat's confusing - I believed all protocols should behave the same way...?\n\nGoo\n"},{"id":"21865","messageId":"Pine.LNX.4.64.0606151937360.5498@g5.osdl.org","threadId":"4528","inReplyTo":"ef5305790606151814i252c37c4mdd005f11f06ceac@mail.gmail.com","subject":"Re: 2.6.17-rc6-mm2","fromName":"Linus Torvalds","fromEmail":"torvalds@osdl.org","sentAt":"2006-06-16T02:46:18Z","receivedAt":"2006-06-16T02:46:18Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Fri, 16 Jun 2006, Goo GGooo wrote:\n> \n> That's confusing - I believed all protocols should behave the same way...?\n\nNot really. The primary protocol is the native git one, and the others try \nto do a best effort, but the http protocol really can't do a very good \njob unless the server side has run \"git update-server-info\" to help the \nhttp client along.\n\nI suspect that the -mm git tree simply doesn't do that. In fact, even the \nmain tree didn't use to do it, but I finally just broke down and added the \nproper hook to make it always do it automatically when I push.\n\n(In case Andrew wants to do that, the way to do it is:\n\n\techo -e \"#!/bin/sh\\nexec git-update-server-info\" > hooks/post-update\n\tchmod +x hooks/post-update\n\ninside the git repository - all it will do is always execute that script, \nand this \"git-update-server-info\", after you've updated the repo).\n\nFinally, the rsync protocol just copies all objects over, and since it \ndoesn't even know _which_ objects it is getting, it doesn't do the normal \ntag following that the native git protocol does.\n\nSo to recap:\n - http is fundamentally weaker, and needs some server-side help to work\n - rsync is fine for the initial clone, but doesn't actually know what \n it's doing, so the end result can actually even be a corrupted \n repository, because you happened to rsync just as it was updating.\n - the native git protocol generally should be considered the golden \n standard, where the other ones are just fallbacks in case of problems \n (like firewalls that don't let git:// through, or more commonly hosted \n servers that don't do the git protocol at all).\n\nWhich hopefully clarifies the issue a bit.\n\n\t\tLinus\n"},{"id":"21870","messageId":"ef5305790606152249n2702873fy7b708d9c47c78470@mail.gmail.com","threadId":"4528","inReplyTo":"Pine.LNX.4.64.0606151937360.5498@g5.osdl.org","subject":"Re: 2.6.17-rc6-mm2","fromName":"Goo GGooo","fromEmail":"googgooo@gmail.com","sentAt":"2006-06-16T05:49:00Z","receivedAt":"2006-06-16T05:49:00Z","isPatch":false,"sender":{"key":"googgooo@gmail.com","avatar":null},"body":"On 6/16/06, Linus Torvalds wrote:\n\n> So to recap:\n> - http is fundamentally weaker, and needs some server-side help to work\n> - rsync is fine for the initial clone, but doesn't actually know what\n> it's doing, so the end result can actually even be a corrupted\n> repository, because you happened to rsync just as it was updating.\n> - the native git protocol generally should be considered the golden\n> standard, where the other ones are just fallbacks in case of problems\n> (like firewalls that don't let git:// through, or more commonly hosted\n> servers that don't do the git protocol at all).\n>\n> Which hopefully clarifies the issue a bit.\n\nThanks for explanation. Unfortunately I can't use git:// with \"git\npull\" (at least in git-1.3.2). First it does some traffic, that\nsuddenly stops - I guess the server starts doing *something*, perhaps\npreparing the update for me or whatnot. After a pretty long while it\nsends some more data but in the meanwhile my ADSL router dropped the\nNAT entry and git sits on my side waiting for data forever. Recently I\ntried the same on a system with direct Inet connection and that worked\njust fine.\n\nI suggest adding SO_KEEPALIVE option on the git socket.\n\nGoo\n"},{"id":"21872","messageId":"Pine.LNX.4.64.0606152335130.5498@g5.osdl.org","threadId":"4528","inReplyTo":"ef5305790606152249n2702873fy7b708d9c47c78470@mail.gmail.com","subject":"Re: 2.6.17-rc6-mm2","fromName":"Linus Torvalds","fromEmail":"torvalds@osdl.org","sentAt":"2006-06-16T06:39:35Z","receivedAt":"2006-06-16T06:39:35Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Fri, 16 Jun 2006, Goo GGooo wrote:\n> \n> Thanks for explanation. Unfortunately I can't use git:// with \"git\n> pull\" (at least in git-1.3.2). First it does some traffic, that\n> suddenly stops - I guess the server starts doing *something*, perhaps\n> preparing the update for me or whatnot.\n\nYeah, for a big pull, the server will have to think about the objects it \nis going to send you.\n\n> I suggest adding SO_KEEPALIVE option on the git socket.\n\nActually, the really irritating thing is that we actually generate all \nthese nice status updates, which just makes pulling and cloning a lot more \ncomfortable, because you actually see what is going on, and what to \nexpect. \n\nExcept they only work over ssh, where we have a separate channel (for \nstderr), and with the native git protocol all that nice status work just \ngets flushed to /dev/null :(\n\nDang. It's literally the most irritating part of the thing: the protocol \nitself is exactly the same whether you go over ssh:// or over git://, but \nthat visual information about what is going on is missing, and it's \nsurprisingly important from a usability standpoint.\n\nAnd in your case, the usability downside actually turned into a real \naccessibility bug.\n\nOh, well.\n\n\t\tLinus\n"},{"id":"21879","messageId":"20060616124010.GB13884@informatik.uni-freiburg.de","threadId":"4528","inReplyTo":"ef5305790606152249n2702873fy7b708d9c47c78470@mail.gmail.com","subject":"Re: 2.6.17-rc6-mm2","fromName":"Uwe Zeisberger","fromEmail":"zeisberg@informatik.uni-freiburg.de","sentAt":"2006-06-16T12:40:10Z","receivedAt":"2006-06-16T12:40:10Z","isPatch":false,"sender":{"key":"u.kleine-koenig@pengutronix.de","avatar":"https://gravatar.com/avatar/354b5e3ceb2806a2f1e1e382ac29ddbdad18288654da62b61eb13583a857eee7?d=mp&s=160"},"body":"Hello,\n\n> I suggest adding SO_KEEPALIVE option on the git socket.\nI suggest to do this \"manually\", that is send an dummy (or status)\npackage every x seconds. Then the server could detect if a cloning\nclient disconnected and stop generating the pack file.\n\n(Currently I see from time to time a git server process (IIRC\ngit-pack-objects) that creates a packfile and only when it's done fails\nto send it.)\n\nBest regards\nUwe\n\n-- \nUwe Zeisberger\n\nhttp://www.google.com/search?q=30+hours+and+4+days+in+seconds\n"},{"id":"21909","messageId":"44931AFD.4070809@zytor.com","threadId":"4528","inReplyTo":"Pine.LNX.4.64.0606152335130.5498@g5.osdl.org","subject":"Re: 2.6.17-rc6-mm2","fromName":"H. Peter Anvin","fromEmail":"hpa@zytor.com","sentAt":"2006-06-16T20:56:29Z","receivedAt":"2006-06-16T20:56:29Z","isPatch":false,"sender":{"key":"hpa@zytor.com","avatar":null},"body":"Linus Torvalds wrote:\n> \n> Actually, the really irritating thing is that we actually generate all \n> these nice status updates, which just makes pulling and cloning a lot more \n> comfortable, because you actually see what is going on, and what to \n> expect. \n> \n> Except they only work over ssh, where we have a separate channel (for \n> stderr), and with the native git protocol all that nice status work just \n> gets flushed to /dev/null :(\n> \n> Dang. It's literally the most irritating part of the thing: the protocol \n> itself is exactly the same whether you go over ssh:// or over git://, but \n> that visual information about what is going on is missing, and it's \n> surprisingly important from a usability standpoint.\n> \n\nPerhaps we shouldn't rely on stderr, and instead have a backchannel as part of the \nprotocol itself. After all, the protocol already does packetization, so all it needs is a \nreliable way to pick out the error/status packets; we could even combine that with a \nmachine-readable code (like SMTP et al) that could get interpreted by the other side as \nneeded.\n\n\t-hpa\n"},{"id":"21922","messageId":"20060616224406.GA10451@outpost.ds9a.nl","threadId":"4528","inReplyTo":"Pine.LNX.4.64.0606152335130.5498@g5.osdl.org","subject":"Re: 2.6.17-rc6-mm2","fromName":"bert hubert","fromEmail":"bert.hubert@netherlabs.nl","sentAt":"2006-06-16T22:44:06Z","receivedAt":"2006-06-16T22:44:06Z","isPatch":false,"sender":{"key":"bert.hubert@netherlabs.nl","avatar":null},"body":"On Thu, Jun 15, 2006 at 11:39:35PM -0700, Linus Torvalds wrote:\n\n> Except they only work over ssh, where we have a separate channel (for \n> stderr), and with the native git protocol all that nice status work just \n> gets flushed to /dev/null :(\n\nIt won't help passing firewalls one bit, but you might consider using SCTP\nwith multiple datastreams for this - theoretically :-)\n\n\tBert\n\n-- \nhttp://www.PowerDNS.com Open source, database driven DNS Software \nhttp://netherlabs.nl Open and Closed source services\n"},{"id":"21924","messageId":"7v3be4d87y.fsf@assigned-by-dhcp.cox.net","threadId":"4528","inReplyTo":"44931AFD.4070809@zytor.com","subject":"Re: 2.6.17-rc6-mm2","fromName":"Junio C Hamano","fromEmail":"junkio@cox.net","sentAt":"2006-06-16T22:52:17Z","receivedAt":"2006-06-16T22:52:17Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"H. Peter Anvin\" writes:\n\n> Perhaps we shouldn't rely on stderr, and instead have a backchannel as\n> part of the protocol itself.\n\nConcurred. This was one of the thing I was planning to do\nanyway.\n"},{"id":"21928","messageId":"Pine.LNX.4.64.0606161720590.5498@g5.osdl.org","threadId":"4528","inReplyTo":"44931AFD.4070809@zytor.com","subject":"Re: 2.6.17-rc6-mm2","fromName":"Linus Torvalds","fromEmail":"torvalds@osdl.org","sentAt":"2006-06-17T00:22:01Z","receivedAt":"2006-06-17T00:22:01Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Fri, 16 Jun 2006, H. Peter Anvin wrote:\n> \n> Perhaps we shouldn't rely on stderr, and instead have a backchannel as part of\n> the protocol itself.\n\nAbsolutely. I'm just irritated at myself for not going that way in the \nfirst place, but when I originally wrote it, I had my eyes on other \nissues, and the nice status updates got added later..\n\n\t\tLinus\n"},{"id":"22096","messageId":"44976506.8040205@logix.cz","threadId":"4528","inReplyTo":"Pine.LNX.4.64.0606152335130.5498@g5.osdl.org","subject":"Re: 2.6.17-rc6-mm2","fromName":"Michal Ludvig","fromEmail":"michal@logix.cz","sentAt":"2006-06-20T03:01:26Z","receivedAt":"2006-06-20T03:01:26Z","isPatch":false,"sender":{"key":"michal@logix.cz","avatar":null},"body":"Linus Torvalds wrote:\n> \n> On Fri, 16 Jun 2006, Goo GGooo wrote:\n> \n>> I suggest adding SO_KEEPALIVE option on the git socket.\n> \n> Actually, the really irritating thing is that we actually generate all \n> these nice status updates, which just makes pulling and cloning a lot more \n> comfortable, because you actually see what is going on, and what to \n> expect. \n> \n> Except they only work over ssh, where we have a separate channel (for \n> stderr), and with the native git protocol all that nice status work just \n> gets flushed to /dev/null :(\n\nOpenBSD has CVS access to their repos over SSH even for anonymous users.\nCould something similar be set up on git.kernel.org as well?\n\n> And in your case, the usability downside actually turned into a real \n> accessibility bug.\n\nSame issue here. Thanks for the hint. Attached is a patch against git\n1.4.0 that solves it perfectly in my case.\n\nSysctl settings (for keepalive every 10 sec):\nnet.ipv4.tcp_keepalive_intvl=10\nnet.ipv4.tcp_keepalive_time=10\n\nMichal\n\n\nSet SO_KEEPALIVE option on native git:// sockets.\n\nSigned-off-by: Michal Ludvig \n\nIndex: git-1.4.0/connect.c\n===================================================================\n--- git-1.4.0.orig/connect.c\n+++ git-1.4.0/connect.c\n@@ -331,7 +331,7 @@ static int git_tcp_connect_sock(char *ho\n \tchar *colon, *end;\n \tchar *port = STR(DEFAULT_GIT_PORT);\n \tstruct addrinfo hints, *ai0, *ai;\n-\tint gai;\n+\tint gai, option;\n \n \tif (host[0] == '[') {\n \t\tend = strchr(host + 1, ']');\n@@ -363,6 +363,10 @@ static int git_tcp_connect_sock(char *ho\n \t\t\t\tai->ai_socktype, ai->ai_protocol);\n \t\tif (sockfd < 0)\n \t\t\tcontinue;\n+\n+\t\toption = 1;\n+\t\tsetsockopt(sockfd, SOL_SOCKET, SO_KEEPALIVE, &option, sizeof(option));\n+\n \t\tif (connect(sockfd, ai->ai_addr, ai->ai_addrlen) < 0) {\n \t\t\tclose(sockfd);\n \t\t\tsockfd = -1;\n@@ -392,7 +396,7 @@ static int git_tcp_connect_sock(char *ho\n \tstruct hostent *he;\n \tstruct sockaddr_in sa;\n \tchar **ap;\n-\tunsigned int nport;\n+\tunsigned int nport, option;\n \n \tif (host[0] == '[') {\n \t\tend = strchr(host + 1, ']');\n@@ -433,6 +437,9 @@ static int git_tcp_connect_sock(char *ho\n \t\tsa.sin_port = htons(nport);\n \t\tmemcpy(&sa.sin_addr, *ap, he->h_length);\n \n+\t\toption = 1;\n+\t\tsetsockopt(sockfd, SOL_SOCKET, SO_KEEPALIVE, &option, sizeof(option));\n+\n \t\tif (connect(sockfd, (struct sockaddr *)&sa, sizeof sa) < 0) {\n \t\t\tclose(sockfd);\n \t\t\tsockfd = -1;\n"},{"id":"22098","messageId":"Pine.LNX.4.64.0606192016210.5498@g5.osdl.org","threadId":"4528","inReplyTo":"44976506.8040205@logix.cz","subject":"Re: 2.6.17-rc6-mm2","fromName":"Linus Torvalds","fromEmail":"torvalds@osdl.org","sentAt":"2006-06-20T03:22:39Z","receivedAt":"2006-06-20T03:22:39Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Tue, 20 Jun 2006, Michal Ludvig wrote:\n> \n> OpenBSD has CVS access to their repos over SSH even for anonymous users.\n> Could something similar be set up on git.kernel.org as well?\n\nI suspect the kernel.org people would prefer not to. And I'm almost \ncertain that others don't want to. It would really be much better if the \ngit protocol itself just had a sideband channel. Oh, well.\n\n\t\t\tLinus\n"}]}