# Re: [PATCH] git-svn: error out when the SVN connection fails during a fetch 9 messages from 2006-11-28 to 2006-12-06. Participants: Florian Weimer, Eric Wong, Junio C Hamano. Thread: https://gitlist.dev/t/43365 ## Eric Wong, 2006-11-28 22:06 Subject: [PATCH] git-svn: error out when the SVN connection fails during a fetch Message-ID: <20061128220605.GA1253@localdomain> URL: https://gitlist.dev/e/20061128220605.GA1253%40localdomain ``` finish_report does seem to return a useful value indicating success or failure, so we'll just set a flag when close_edit is called (it is not called on failures, nor is abort_edit) and check the flag before proceeding. Thanks to Pazu for pointing this out. Signed-off-by: Eric Wong --- git-svn.perl | 9 ++++++++- 1 files changed, 8 insertions(+), 1 deletions(-) diff --git a/git-svn.perl b/git-svn.perl index 7942bba..c3ad5ec 100755 --- a/git-svn.perl +++ b/git-svn.perl @@ -2941,6 +2941,9 @@ sub libsvn_fetch_delta { $reporter->set_path('', $last_rev, 0, @lock, $pool); $reporter->finish_report($pool); $pool->clear; + unless ($ed->{git_commit_ok}) { + die "SVN connection failed somewhere...\n"; + } libsvn_log_entry($rev, $author, $date, $msg, [$last_commit]); } @@ -3195,6 +3198,9 @@ sub libsvn_new_tree { $reporter->set_path('', $rev, 1, @lock, $pool); $reporter->finish_report($pool); $pool->clear; + unless ($ed->{git_commit_ok}) { + die "SVN connection failed somewhere...\n"; + } } else { open my $gui, '| git-update-index -z --index-info' or croak $!; libsvn_traverse($gui, '', $SVN->{svn_path}, $rev); @@ -3501,7 +3507,8 @@ sub abort_edit { sub close_edit { my $self = shift; - close $self->{gui} or croak; + close $self->{gui} or croak $!; + $self->{git_commit_ok} = 1; $self->SUPER::close_edit(@_); } -- 1.4.4.1.ge151 ``` ## Eric Wong, 2006-11-28 22:24 Subject: Re: [PATCH] git-svn: error out when the SVN connection fails during a fetch Message-ID: <20061128222442.GC1369@localdomain> URL: https://gitlist.dev/e/20061128222442.GC1369%40localdomain In-Reply-To: <20061128220605.GA1253@localdomain> ``` Eric Wong wrote: > finish_report does seem to return a useful value indicating success > or failure, so we'll just set a flag when close_edit is called > (it is not called on failures, nor is abort_edit) and check > the flag before proceeding. Also, this only affects the delta fetcher. When using the non-delta fetch: the SVN libraries will just segfault :) -- ``` ## Florian Weimer, 2006-12-03 18:23 Subject: Re: [PATCH] git-svn: error out when the SVN connection fails during a fetch Message-ID: <871wnget3b.fsf@mid.deneb.enyo.de> URL: https://gitlist.dev/e/871wnget3b.fsf%40mid.deneb.enyo.de In-Reply-To: <20061128220605.GA1253@localdomain> ``` * Eric Wong: > finish_report does seem to return a useful value indicating success > or failure, so we'll just set a flag when close_edit is called > (it is not called on failures, nor is abort_edit) and check > the flag before proceeding. It seems that this needs some kind of fine-tuning. Now that git-svn uses HTTP keepalive connections, you get a HTTP request error once you run into the server-side request limit. It seems a bit excessive to ``` ## Eric Wong, 2006-12-04 08:52 Subject: Re: [PATCH] git-svn: error out when the SVN connection fails during a fetch Message-ID: <20061204085253.GA31047@soma> URL: https://gitlist.dev/e/20061204085253.GA31047%40soma In-Reply-To: <871wnget3b.fsf@mid.deneb.enyo.de> ``` Florian Weimer wrote: > * Eric Wong: > > > finish_report does seem to return a useful value indicating success > > or failure, so we'll just set a flag when close_edit is called > > (it is not called on failures, nor is abort_edit) and check > > the flag before proceeding. > > It seems that this needs some kind of fine-tuning. Now that git-svn > uses HTTP keepalive connections, you get a HTTP request error once you > run into the server-side request limit. It seems a bit excessive to > stop completely in this case. Does the following patch help? From: Eric Wong Date: Mon, 4 Dec 2006 00:51:16 -0800 Subject: [PATCH] git-svn: avoid network timeouts for long-running fetches Long-running fetches run inside children to avoid memory leaks. When we refork, the connection in the parent can be idle for a long time; attempting to reuse it in the next child can result in timeouts. Signed-off-by: Eric Wong --- git-svn.perl | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/git-svn.perl b/git-svn.perl index d0bd0bd..747daf0 100755 --- a/git-svn.perl +++ b/git-svn.perl @@ -459,6 +459,7 @@ sub fetch_lib { $min = $max + 1; $max += $inc; $max = $head if ($max > $head); + $SVN = libsvn_connect($SVN_URL); } restore_index($index); return { revision => $last_rev, commit => $last_commit }; -- ``` ## Florian Weimer, 2006-12-04 08:56 Subject: Re: [PATCH] git-svn: error out when the SVN connection fails during a fetch Message-ID: <87zma4yr71.fsf@mid.deneb.enyo.de> URL: https://gitlist.dev/e/87zma4yr71.fsf%40mid.deneb.enyo.de In-Reply-To: <20061204085253.GA31047@soma> ``` * Eric Wong: > Does the following patch help? Don't think so. The issue is not timing-related. I've seen a failure every 1000 requests, which suggests to me that it's hitting the ``` ## Eric Wong, 2006-12-04 09:05 Subject: Re: [PATCH] git-svn: error out when the SVN connection fails during a fetch Message-ID: <20061204090549.GB30316@hand.yhbt.net> URL: https://gitlist.dev/e/20061204090549.GB30316%40hand.yhbt.net In-Reply-To: <87zma4yr71.fsf@mid.deneb.enyo.de> ``` Florian Weimer wrote: > * Eric Wong: > > > Does the following patch help? > > Don't think so. The issue is not timing-related. I've seen a failure > every 1000 requests, which suggests to me that it's hitting the > MaxKeepAliveRequests limit configured on the server. Actually, that's exactly what this patch should fix. git-svn restarts a child every 1000 revisions to avoid memory usage from going through the roof. -- ``` ## Florian Weimer, 2006-12-06 17:01 Subject: Re: [PATCH] git-svn: error out when the SVN connection fails during a fetch Message-ID: <87psax3qmb.fsf@mid.deneb.enyo.de> URL: https://gitlist.dev/e/87psax3qmb.fsf%40mid.deneb.enyo.de In-Reply-To: <20061204090549.GB30316@hand.yhbt.net> ``` * Eric Wong: > Florian Weimer wrote: >> * Eric Wong: >> >> > Does the following patch help? >> >> Don't think so. The issue is not timing-related. I've seen a failure >> every 1000 requests, which suggests to me that it's hitting the >> MaxKeepAliveRequests limit configured on the server. > > Actually, that's exactly what this patch should fix. git-svn restarts a > child every 1000 revisions to avoid memory usage from going through the > roof. ``` ## Junio C Hamano, 2006-12-06 19:01 Subject: Re: [PATCH] git-svn: error out when the SVN connection fails during a fetch Message-ID: <7v64co4zo0.fsf@assigned-by-dhcp.cox.net> URL: https://gitlist.dev/e/7v64co4zo0.fsf%40assigned-by-dhcp.cox.net In-Reply-To: <87psax3qmb.fsf@mid.deneb.enyo.de> ``` Florian Weimer writes: > * Eric Wong: > >> Florian Weimer wrote: >>> * Eric Wong: >>> >>> > Does the following patch help? >>> >>> Don't think so. The issue is not timing-related. I've seen a failure >>> every 1000 requests, which suggests to me that it's hitting the >>> MaxKeepAliveRequests limit configured on the server. >> >> Actually, that's exactly what this patch should fix. git-svn restarts a >> child every 1000 revisions to avoid memory usage from going through the >> roof. > > Oh. My (limited) testing of the patch confirms that. Ok, so this means I should apply this one, right? -- >8 -- From: Eric Wong Date: Mon, 4 Dec 2006 00:51:16 -0800 Subject: [PATCH] git-svn: avoid network timeouts for long-running fetches Long-running fetches run inside children to avoid memory leaks. When we refork, the connection in the parent can be idle for a long time; attempting to reuse it in the next child can result in timeouts. Signed-off-by: Eric Wong --- git-svn.perl | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/git-svn.perl b/git-svn.perl index d0bd0bd..747daf0 100755 --- a/git-svn.perl +++ b/git-svn.perl @@ -459,6 +459,7 @@ sub fetch_lib { $min = $max + 1; $max += $inc; $max = $head if ($max > $head); + $SVN = libsvn_connect($SVN_URL); } restore_index($index); return { revision => $last_rev, commit => $last_commit }; -- 1.4.4.1.g6129 ``` ## Eric Wong, 2006-12-06 19:45 Subject: Re: [PATCH] git-svn: error out when the SVN connection fails during a fetch Message-ID: <20061206194530.GA24986@localdomain> URL: https://gitlist.dev/e/20061206194530.GA24986%40localdomain In-Reply-To: <7v64co4zo0.fsf@assigned-by-dhcp.cox.net> ``` Junio C Hamano wrote: > Florian Weimer writes: > > > * Eric Wong: > > > >> Florian Weimer wrote: > >>> * Eric Wong: > >>> > >>> > Does the following patch help? > >>> > >>> Don't think so. The issue is not timing-related. I've seen a failure > >>> every 1000 requests, which suggests to me that it's hitting the > >>> MaxKeepAliveRequests limit configured on the server. > >> > >> Actually, that's exactly what this patch should fix. git-svn restarts a > >> child every 1000 revisions to avoid memory usage from going through the > >> roof. > > > > Oh. My (limited) testing of the patch confirms that. > > Ok, so this means I should apply this one, right? Yes, please do. Thanks. -- ```