Re: [PATCH 0/2] allow git-svn fetching to work using serf

From: David Rothenberger <daveroth@acm.org>
Date: Jul 7, 2013, 03:44 UTC
Message-ID: <51D8E40F.2020008@acm.org>
In-Reply-To: <20130707002804.GF30132@google.com>

On 7/6/2013 5:28 PM, Jonathan Nieder wrote:

Show 11 quoted lines

> David Rothenberger wrote:
>> On 7/5/2013 8:41 PM, Kyle McKay wrote:
> 
>>> Daniel Shahaf has suggested also setting
>>> "servers:global:http-bulk-updates=on".
>>
>> I have a patch that does this, but since turning on bulk updates has
>> a possible performance penalty, I prefer your approach. 
> 
> I assume that's because http-bulk-updates defeats caching.  If so,
> makes sense.

I believe that "bulk updates" means that serf makes one request for a lot of information and receives it all in one big response. In "skelta" mode, serf makes a single request for a single piece of information. The serf authors feel this can lead to improved overall throughput because they can pipeline these requests and have multiple connections open at the same time.

The downside, though, is that serf will do multiple open_file calls in parallel as it descends down sibling directories.

> It's still not clear to me how we know that ra_serf driving the editor
> in a non depth-first manner is the problem here.  Has that explanation
> been confirmed somehow?

I did do a trace of "git svn fetch" and observed this non-depth-first traversal. It certainly causes the failure we've observed.

> Is there a simple explanation of why violating the depth-first
> constraint would lead to multiple blob (i.e., file, not directory)
> deltas being opened in a row without an intervening close?

I believe serf is doing the following for a number of files in parallel:
 1. open_file
 2. apply_textdelta
 3. change_file_prop, change_file_prop, ...
 4. close_file

-- 
David Rothenberger  ----  daveroth@acm.org

Nusbaum's Rule:
        The more pretentious the corporate name, the smaller the
        organization.  (For instance, the Murphy Center for the
        Codification of Human and Organizational Law, contrasted
        to IBM, GM, and AT&T.)