Hi all,
I faced a problem when trying to clone git://gitorious.org/qt/qt.git
this works: git clone -n git://gitorious.org/qt/qt.git repo
this doesn't: git clone -n git://gitorious.org/qt/qt.git repo >log fatal: The remote end hung up unexpectedly fatal: early EOF fatal: index-pack failed
I reproduced the error in the following versions: git version 1.6.0.4 git version 1.6.4.2 git version 1.5.4.7
Any clues? Thanks
Aloisio Almeida
Stefan NaeweRe: Cannot clone redirecting stdout
On 9/11/2009 12:33 AM, Aloisio wrote:
> Hi all, > > I faced a problem when trying to clone git://gitorious.org/qt/qt.git > > this works: > git clone -n git://gitorious.org/qt/qt.git repo > > this doesn't: > git clone -n git://gitorious.org/qt/qt.git repo >log > fatal: The remote end hung up unexpectedly > fatal: early EOF > fatal: index-pack failed > > I reproduced the error in the following versions: > git version 1.6.0.4 > git version 1.6.4.2 > git version 1.5.4.7 > > Any clues? > Thanks
I'd say:
Take a look at /proc/<pid-of-git>/fd
fd 0 (stdout) is linked through a pipe to 'git index-pack'. Redirecting this link breaks everything.
Regards, Stefan
Stefan NaeweRe: Cannot clone redirecting stdout
On 9/11/2009 8:39 AM, Stefan Naewe wrote:
> On 9/11/2009 12:33 AM, Aloisio wrote: >> Hi all, >> >> I faced a problem when trying to clone git://gitorious.org/qt/qt.git >> >> this works: >> git clone -n git://gitorious.org/qt/qt.git repo >> >> this doesn't: >> git clone -n git://gitorious.org/qt/qt.git repo >log >> fatal: The remote end hung up unexpectedly >> fatal: early EOF >> fatal: index-pack failed >> >> I reproduced the error in the following versions: >> git version 1.6.0.4 >> git version 1.6.4.2 >> git version 1.5.4.7 >> >> Any clues? >> Thanks > > I'd say: > > Take a look at /proc/<pid-of-git>/fd > > fd 0 (stdout) is linked through a pipe to 'git index-pack'. > Redirecting this link breaks everything.
stdout is of course fd 1 ;-)
Regards, Stefan
-- /dev/random says: Never trust a computer you can't lift. - Stan Masor
Re: Cannot clone redirecting stdout
Hi list!
Aloisio wrote:
> this doesn't: > git clone -n git://gitorious.org/qt/qt.git repo >log > fatal: The remote end hung up unexpectedly > fatal: early EOF > fatal: index-pack failed
What are you trying to store in that file? The only line I observe being printed to stdout while cloning is this one:
Initialized empty Git repository in [...]/repo/.git/
Everything else (progress status) is printed to stderr and I presume that any error messages would also go to stderr too, if any. So possibly you would be happy with this:
git clone -n git://gitorious.org/qt/qt.git repo 2> log
which works fine here with git-1.6.4.2. Though I agree that I would expect redirecting stdout to work.
Cheers, jlh
Jeff KingRe: Cannot clone redirecting stdout
On Thu, Sep 10, 2009 at 07:33:21PM -0300, Aloisio wrote:
> I faced a problem when trying to clone git://gitorious.org/qt/qt.git > > this works: > git clone -n git://gitorious.org/qt/qt.git repo > > this doesn't: > git clone -n git://gitorious.org/qt/qt.git repo >log > fatal: The remote end hung up unexpectedly > fatal: early EOF > fatal: index-pack failed
I can reproduce the problem here. But after staring at the strace for a long time, I don't think the problem is on the client side. The remote end _does_ hang up unexpectedly.
Looking at what we send, the only difference between the redirected and unredirected case I could find is that we send the "no-progress" flag to the server, which then hangs up on us instead of sending us the pack. Which makes no sense.
Confused, -Peff
Jeff KingRe: Cannot clone redirecting stdout
On Fri, Sep 11, 2009 at 06:23:29AM -0400, Jeff King wrote:
> > I faced a problem when trying to clone git://gitorious.org/qt/qt.git > > > > this works: > > git clone -n git://gitorious.org/qt/qt.git repo > > > > this doesn't: > > git clone -n git://gitorious.org/qt/qt.git repo >log > > fatal: The remote end hung up unexpectedly > > fatal: early EOF > > fatal: index-pack failed > > I can reproduce the problem here. But after staring at the strace for a > long time, I don't think the problem is on the client side. The remote > end _does_ hang up unexpectedly. > > Looking at what we send, the only difference between the redirected and > unredirected case I could find is that we send the "no-progress" flag to > the server, which then hangs up on us instead of sending us the pack. > Which makes no sense.
I did a little more testing, and I can't reproduce the problem against a local git-daemon. I tried using several versions for the server, going all the way back to v1.5.0, which pre-dates no-progress, and all of them worked just fine.
So I am inclined to think there is something non-standard or broken at gitorious.org. I'm cc'ing support@gitorious to see if they have any comment.
-Peff
Johan SørensenRe: Cannot clone redirecting stdout
On Fri, Sep 11, 2009 at 3:51 PM, Jeff King <peff@peff.net> wrote:
> On Fri, Sep 11, 2009 at 06:23:29AM -0400, Jeff King wrote: > >> > I faced a problem when trying to clone git://gitorious.org/qt/qt.git >> > >> > this works: >> > git clone -n git://gitorious.org/qt/qt.git repo >> > >> > this doesn't: >> > git clone -n git://gitorious.org/qt/qt.git repo >log >> > fatal: The remote end hung up unexpectedly >> > fatal: early EOF >> > fatal: index-pack failed >> >> I can reproduce the problem here. But after staring at the strace for a >> long time, I don't think the problem is on the client side. The remote >> end _does_ hang up unexpectedly. >> >> Looking at what we send, the only difference between the redirected and >> unredirected case I could find is that we send the "no-progress" flag to >> the server, which then hangs up on us instead of sending us the pack. >> Which makes no sense. > > I did a little more testing, and I can't reproduce the problem against a > local git-daemon. I tried using several versions for the server, going > all the way back to v1.5.0, which pre-dates no-progress, and all of them > worked just fine. > > So I am inclined to think there is something non-standard or broken at > gitorious.org. I'm cc'ing support@gitorious to see if they have any > comment.
Some quick tests seem to indicate it's related to the fact that our wonderful little fork+exec git-daemon[1] (which is different from the one distributed with git) exec's to "git-upload-pack --strict --timeout=30 /path/to/repo". Now, why exactly that'll trigger when the no-progress flag is given I'm not sure of. The daemon itself execs as soon as it figures out what repo the client requested, so apart from the timeout the only thing it reacts to is the header (the "${headersize}git-upload-pack project/repo.git\0host=gitorious.org\0" part).
We also do redirect stderr to /dev/null for reasons I cannot remember (so probably not good ones), that may be related as well. Well run some more tests...
[1]: http://gitorious.org/gitorious/mainline/blobs/master/script/git-daemon
> > -Peff > -- > To unsubscribe from this list: send the line "unsubscribe git" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html >
Jeff KingRe: Cannot clone redirecting stdout
On Fri, Sep 11, 2009 at 05:46:23PM +0200, Johan Sørensen wrote:
> Some quick tests seem to indicate it's related to the fact that our
> wonderful little fork+exec git-daemon[1] (which is different from the
> one distributed with git) exec's to "git-upload-pack --strict
> --timeout=30 /path/to/repo". Now, why exactly that'll trigger when the
> no-progress flag is given I'm not sure of. The daemon itself execs as
> soon as it figures out what repo the client requested, so apart from
> the timeout the only thing it reacts to is the header (the
> "${headersize}git-upload-pack project/repo.git\0host=gitorious.org\0"
> part).
>
> We also do redirect stderr to /dev/null for reasons I cannot remember
> (so probably not good ones), that may be related as well. Well run
> some more tests...Ah. I have a theory. If I do a clone of git://gitorious.org/qt/qt.git, the counting/compressing stages take a long time (I timed it at 1m40 before it actually sends any data). And looking at upload-pack.c, we leave the 30-second alarm set while creating the pack. Meaning we die 30 seconds into creating the pack.
When progress is being displayed, however, the progress timer actually uses SIGALRM, as well. So we are constantly resetting the timer and it never goes off.
And we should be able to test this theory. How long does it take for the failure case to fail:
$ time git clone -n git://gitorious.org/qt/qt.git repo >log fatal: The remote end hung up unexpectedly fatal: early EOF fatal: index-pack failed
real 0m31.106s user 0m0.000s sys 0m0.012s
Hmm. Suspicious. :)
So that implies to me a few things:
1. You guys should really pack your repos, as you are wasting over a
minute of CPU time every time somebody clones this repo. 2. Upload-pack has what I consider a bug. The --timeout should be
suspended while we are actually crunching numbers to create the
pack. We probably don't want it when sending the pack, either, as
people with slow connections (or big repos) will get timed out
during the send. Probably we just want to apply it to times when we
are waiting to get the list of refs from the client. 3. Upload-pack and the progress code are both using the global alarm
timer and signal, and that is papering over the bug in (2) when
progress is enabled. I'm not sure of the simplest way of having
those interact. But maybe we can just ignore it, because we
probably don't want to using the --timeout alarm during the packing
phase, anyway.-Peff
Jeff KingRe: Cannot clone redirecting stdout
On Fri, Sep 11, 2009 at 12:05:10PM -0400, Jeff King wrote:
> Ah. I have a theory. If I do a clone of git://gitorious.org/qt/qt.git, > the counting/compressing stages take a long time (I timed it at 1m40 > before it actually sends any data). And looking at upload-pack.c, we > leave the 30-second alarm set while creating the pack. Meaning we die 30 > seconds into creating the pack. > > When progress is being displayed, however, the progress timer actually > uses SIGALRM, as well. So we are constantly resetting the timer and it > never goes off.
Hmm. Actually, this is not quite right. It looks like we call out to pack-objects as an external program, so there is no conflict with the signal. And we do proxy the output of pack-objects, which will keep our timer resetting every time we see a chunk of output. But pack-objects produces no output during the deltification phase, unless progress is turned on. So we still hit our timeout in upload-pack during that phase.
So our options are:
1. Turn off the timer during deltification, which could mean that it
would potentially go forever. But it's not controlled by the user.
I think the 'timeout' feature is really about the client just
opening the connection and sitting. 2. Keep progress on during deltification, but just throw it away
instead of relaying it if no-progress is in effect. 3. Accept that hitting the timeout during deltification _should_ cause
it to die. In that case, then the case with progress is wrong, and
we should stop resetting the timer just because we got some
progress output from pack-objects. But this may be redefining the
intent of --timeout. I don't know what the original intent was, or
what users of the feature are expecting.-Peff
Daniel BarkalowRe: Cannot clone redirecting stdout
On Fri, 11 Sep 2009, Jeff King wrote:
> On Fri, Sep 11, 2009 at 12:05:10PM -0400, Jeff King wrote: > > > Ah. I have a theory. If I do a clone of git://gitorious.org/qt/qt.git, > > the counting/compressing stages take a long time (I timed it at 1m40 > > before it actually sends any data). And looking at upload-pack.c, we > > leave the 30-second alarm set while creating the pack. Meaning we die 30 > > seconds into creating the pack. > > > > When progress is being displayed, however, the progress timer actually > > uses SIGALRM, as well. So we are constantly resetting the timer and it > > never goes off. > > Hmm. Actually, this is not quite right. It looks like we call out to > pack-objects as an external program, so there is no conflict with the > signal. And we do proxy the output of pack-objects, which will keep our > timer resetting every time we see a chunk of output. But pack-objects > produces no output during the deltification phase, unless progress is > turned on. So we still hit our timeout in upload-pack during that > phase. > > So our options are: > > 1. Turn off the timer during deltification, which could mean that it > would potentially go forever. But it's not controlled by the user. > I think the 'timeout' feature is really about the client just > opening the connection and sitting. > > 2. Keep progress on during deltification, but just throw it away > instead of relaying it if no-progress is in effect. > > 3. Accept that hitting the timeout during deltification _should_ cause > it to die. In that case, then the case with progress is wrong, and > we should stop resetting the timer just because we got some > progress output from pack-objects. But this may be redefining the > intent of --timeout. I don't know what the original intent was, or > what users of the feature are expecting.
You don't remember October 2005? HPA introduced it in 960decc, which has a pretty good explanation: we doesn't want to get DoS'd if clients just send SYNs. So it's supposed to time out only if we spend that long waiting for a protocol item from the client.
-Daniel *This .sig left intentionally blank*