Re: [PATCH] Limit file descriptors used by packs

From: Shawn Pearce <spearce@spearce.org>
Date: Mar 1, 2011, 14:58 UTC
Message-ID: <AANLkTik=FiwsQUg89MRXZX1-jR-fkF7uyJAimSXVSLvR@mail.gmail.com>
In-Reply-To: <7vwrkiex62.fsf@alter.siamese.dyndns.org>

On Tue, Mar 1, 2011 at 06:24, Junio C Hamano <gitster@pobox.com> wrote:

Show 11 quoted lines

> "Shawn O. Pearce" <spearce@spearce.org> writes:
>
>> In this particular part of C Git, if we are bumping up against the
>> hard pack_max_fds limit we're already into some pretty difficult
>> computation. Trying to push the rlimit higher in order to avoid
>> close/open calls as we cycle through fds isn't really going to make
>> a huge difference on end-user latency for the command to finish
>> its task. So maybe we are better off honoring the rlim_cur that we
>> inherited from the user/environment.
>
> Let's step back a bit.

...

> For a .pack that fits inside a single pack window, however, can't we close
> the file descriptor immediately after mmap() it to obtain a sole window
> into it?

Yes. And its unrelated to this patch. You can still run out of file descriptors because you have a lot of large packs. :-)

I've considered this in the past, but avoided it because I thought the unuse_one_window() function might become more complex. But its not, we can just keep popping windows until the condition is met, which for a file descriptor is that we are below the limit.

I'll send a follow-up patch that builds on top of this one to close the pack fd if the entire thing fits into one window.

-- 
Shawn.