Re: [PATCH] compat: Add another rudimentary poll() emulation
- From
Erik Faye-Lund <kusmabite@googlemail.com>
- Date
- May 27, 2010, 12:57 UTC
- Message-ID
- <AANLkTiliJFXWKXnksQryvAivadrkTUeZ1Wu7FkUGm2YZ@mail.gmail.com>
- In-Reply-To
- <AANLkTinX8nK68rZtN5dwJ-fGQm4gR2G84xo9raxb4vLY@mail.gmail.com>
On Thu, May 27, 2010 at 2:36 PM, Marko Kreen <markokr@gmail.com> wrote:
Show 50 quoted lines
> On 5/27/10, Erik Faye-Lund <kusmabite@googlemail.com> wrote:
>> On Thu, May 27, 2010 at 1:39 PM, Marko Kreen <markokr@gmail.com> wrote:
>> > On 5/27/10, Erik Faye-Lund <kusmabite@googlemail.com> wrote:
>> >> On Thu, May 27, 2010 at 12:10 PM, Jonathan Nieder <jrnieder@gmail.com> wrote:
>> >> > Implement the subset of poll() semantics needed by git in terms of
>> >> > select(), for use by the Interix port. Inspired by commit 6ed807f
>> >> > (Windows: A rudimentary poll() emulation, 2007-12-01).
>> >> >
>> >>
>> >>
>> >> A possible problem with this approach is that the maximum number of
>> >> file descriptors poll can handle limited by RLIMIT_NOFILE, whereas the
>> >> maximum number of file descriptors select can handle is limited by
>> >> FD_SETSIZE.
>> >>
>> >> I don't think this is a big problem in reality, though - both values
>> >> seem to be pretty high in most implementations. And IIRC git-daemon is
>> >> the only one who needs more than 2, and it doesn't even check
>> >> RLIMIT_NOFILE.
>> >>
>> >> If we decide to go this route, perhaps it'd make sense to change to
>> >> this code for Windows also? Our Windows-implementation of poll() has
>> >> some annoying limitations...
>> >
>> > Example of poll() compat without FD_SETSIZE limit:
>> >
>> > http://github.com/markokr/plproxy-dev/blob/master/src/poll_compat.c
>> >
>>
>>
>> How does this code convince FD_SET() that the buffer has increased? It
>> looks to me like it depends on a specific FD_SET() implementation...
>> For instance, Windows' FD_SET() implementation is like this:
>>
>> #define FD_SET(fd, set) do { \
>> if (((fd_set FAR *)(set))->fd_count < FD_SETSIZE) \
>> ((fd_set FAR *)(set))->fd_array[((fd_set FAR
>> *)(set))->fd_count++]=(fd);\
>> } while(0)
>>
>> ...so unless another set is passed in, it won't add any more fds once
>> fd_count reaches FD_SETSIZE.
>>
>> Also, FD_SETSIZE is 64 on Windows. IIRC it's 1024 on Linux, so it is
>> much more likely that we encounter this issue on Windows than on
>> Linux, at least ;)
>
> Hm, good catch. Seems such compat poll() cannot be done without
> OS-specific hacks.
>Perhaps getrlimit() could overridden to return FD_SETSIZE for both the soft and hard limit when asking about RLIMIT_NOFILE? In such cases, anyone who passes nfds above FD_SETSIZE hasn't consulted RLIMIT_NOFILE, and should be outside the standard. But your point might have been about a limitless poll()-implementation, like the code you linked tried to achieve. In that context, no. I doubt it's possible to do in a robust fashion.
For git, I don't think this is necessary, though. As said, I think git-daemon is the only call-site for poll where nfds can be above 2. And git-daemon's default max-connection is 32, which shouldn't cause much problems. There's the theoretical problem of someone setting --max-connections above their platform's limit, but I don't think we need to touch that ;)
> Do you know perhaps what other OS-es have non-bitmap fd_set?
No, I don't have much low-level file-descriptor knowledge about other OS'es than Windows, really.
-- Erik "kusma" Faye-Lund