From: Junio C Hamano <gitster@pobox.com>
Date: Wed, 12 Feb 2014 20:06:50 GMT
Subject: Re: Make the git codebase thread-safe
Message-ID: <xmqqr478m6xx.fsf@gitster.dls.corp.google.com>
In-Reply-To: <CAHOQ7J8gvwpwJV2mBPDaARu3cQ54-ZDQ6iGOwKuJRr9Z+XBL7g@mail.gmail.com>

Stefan Zager <szager@chromium.org> writes:

> ...  I used the Very Sleepy profiler
> to see where all the time was spent on Windows: 55% of the time was
> spent in OpenFile, and 25% in CloseFile (both in win32).

This is somewhat interesting.

When we check things out, checkout_paths() has a list of paths to be
checked out, and iterates over them and call checkout_entry().

I wonder if you can:

 - introduce a version of checkout_entry() that takes file
   descriptors to write to;

 - have an asynchronous helper threads that pre-open the paths to be
   written out and feed <ce, file descriptor to be written> to a
   queue;

 - restructure that loop so that it reads the <ce, file descriptor
   to be written> from the queue, performs the actual writing out,
   and then feeds <file descriptor to be closed> to another queue; and

 - have another asynchronous helper threads that reads <file
   descriptor to be closed> from the queue and close them.

Calls to write (and preparation of data to be written) will then
remain single-threaded, but it sounds like that codepath is not the
bottleneck in your measurement, so....