Re: [PATCH] Enable index-pack threading in msysgit.

From: Duy Nguyen <pclouds@gmail.com>
Date: Mar 20, 2014, 01:25 UTC
Message-ID: <CACsJy8D01gxeSoxgD9QkLgS85B=Af8aXytuqn6CBkF+_7WjZtQ@mail.gmail.com>
In-Reply-To: <20140319213556.2FC3D4062B@wince.sfo.corp.google.com>

On Thu, Mar 20, 2014 at 4:35 AM, Stefan Zager <szager@chromium.org> wrote:

Show 11 quoted lines

> This adds a Windows implementation of pread.  Note that it is NOT
> safe to intersperse calls to read() and pread() on a file
> descriptor.  According to the ReadFile spec, using the 'overlapped'
> argument should not affect the implicit position pointer of the
> descriptor.  Experiments have shown that this is, in fact, a lie.
>
> To accomodate that fact, this change also incorporates:
>
> http://article.gmane.org/gmane.comp.version-control.git/196042
>
> ... which gives each index-pack thread its own file descriptor.

If the problem is mixing read() and pread() then perhaps it's enough to do

output_fd = dup(output_fd);

after pack_fd is set in open_pack_file(), to make sure that fixup_pack_header_footer() has its own file handle. If that works, we don't need one pack_fd per thread.

compat/mmap.c uses pread() and its bad interaction with read() could turn it into a nightmare. Fortunately Windows (except Cygwin) does not use this implementation. Not sure if we should make a note about this.

It makes me wonder if sliding mmap window (like we do for pack access in sha1_file.c) would be better than pread(). index-pack used to do mmap() [1] in the past with poor performance but I don't think sliding window was mentioned.

[1] http://thread.gmane.org/gmane.comp.version-control.git/34741/focus=34832

Show 10 quoted lines

> --- a/config.mak.uname
> +++ b/config.mak.uname
> @@ -474,7 +474,6 @@ ifeq ($(uname_S),NONSTOP_KERNEL)
>  endif
>  ifneq (,$(findstring MINGW,$(uname_S)))
>         pathsep = ;
> -       NO_PREAD = YesPlease
>         NEEDS_CRYPTO_WITH_SSL = YesPlease
>         NO_LIBGEN_H = YesPlease
>         NO_POLL = YesPlease

What about the "ifeq ($(uname_S),Windows)" section? I think MSVC and MinGW builds share a lot of code.

-- 
Duy