git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCHv2 2/7] xread: poll on non blocking fds

From
Stefan Beller <sbeller@google.com>
Date
Dec 17, 2015, 20:51 UTC
Message-ID
<CAGZ79kYHjH7QgChn6_GU9iLrtyouFQFuMAZKADZofgHjBS73-A@mail.gmail.com>
In-Reply-To
<56731E19.7050504@web.de>
On Thu, Dec 17, 2015 at 12:42 PM, Torsten Bögershausen <tboegi@web.de> wrote:
Show 72 quoted lines
> On 17.12.15 21:22, Stefan Beller wrote:
>> On Thu, Dec 17, 2015 at 12:12 PM, Torsten Bögershausen <tboegi@web.de> wrote:
>>> On 16.12.15 01:04, Stefan Beller wrote:
>>>> The man page of read(2) says:
>>>>
>>>>   EAGAIN The file descriptor fd refers to a file other than a socket
>>>>        and has been marked nonblocking (O_NONBLOCK), and the read
>>>>        would block.
>>>>
>>>>   EAGAIN or EWOULDBLOCK
>>>>        The file descriptor fd refers to a socket and has been marked
>>>>        nonblocking (O_NONBLOCK), and the read would block.  POSIX.1-2001
>>>>        allows either error to be returned for this case, and does not
>>>>        require these constants to have the same value, so a portable
>>>>        application should check for both possibilities.
>>>>
>>>> If we get an EAGAIN or EWOULDBLOCK the fd must have set O_NONBLOCK.
>>>> As the intent of xread is to read as much as possible either until the
>>>> fd is EOF or an actual error occurs, we can ease the feeder of the fd
>>>> by not spinning the whole time, but rather wait for it politely by not
>>>> busy waiting.
>>>>
>>>> We should not care if the call to poll failed, as we're in an infinite
>>>> loop and can only get out with the correct read().
>>> I'm not sure if this is valid under all circumstances:
>>> This is what "man poll" says under Linux:
>>> []
>>>  ENOMEM There was no space to allocate file descriptor tables.
>>> []
>>> And this is from Mac OS, ("BSD System Calls Manual")
>>> ERRORS
>>>      Poll() will fail if:
>>>
>>>      [EAGAIN]           Allocation of internal data structures fails.  A sub-
>>>                         sequent request may succeed.
>>> And this is opengroup:
>>> http://pubs.opengroup.org/onlinepubs/9699919799//functions/poll.html:
>>> [EAGAIN]
>>>     The allocation of internal data structures failed but a subsequent request may succeed.
>>>
>>> read() may return EAGAIN, but poll() may fail to allocate memory, and fail.
>>> Is it always guaranteed that the loop is terminated?
>>
>> In case poll fails (assume a no op for it), the logic should not have
>> changed by this patch?
>>
>> Looking closely:
>>
>>>>       while (1) {
>>>>               nr = read(fd, buf, len);
>>>> -             if ((nr < 0) && (errno == EAGAIN || errno == EINTR))
>>>> -                     continue;
>>>> +             if (nr < 0) {
>>>> +                     if (errno == EINTR)
>>>> +                             continue;
>>>> +                     if (errno == EAGAIN || errno == EWOULDBLOCK) {
>>>> +                             struct pollfd pfd;
>>>> +                             pfd.events = POLLIN;
>>>> +                             pfd.fd = fd;
>>>> +                             /*
>>>> +                              * it is OK if this poll() failed; we
>>>> +                              * want to leave this infinite loop
>>>> +                              * only when read() returns with
>>>> +                              * success, or an expected failure,
>>>> +                              * which would be checked by the next
>>>> +                              * call to read(2).
>>>> +                              */
>>>> +                             poll(&pfd, 1, -1);
>>
>> Or do you mean to insert another continue in here?
> I was thinking that we run into similar loop as before:
> read() returns -1; errno = EAGAIN  /* No data to read */
which is expected for non blocking fds,
> poll() returns -1; errno = EAGAIN /* poll failed. If the fd was OK, the failure may be temporaly,
>                                     as much as poll() can see.
>                                     But most probably we run out ouf memory */

Before this patch we would not have asked poll, but had just a continue here, so I think we need to have it here again no matter of the return code of the poll.

If poll determines it is low on memory, this should not make this function fail, we can still do as good as we did before by just asking read repeatedly again, though?

So I'd be convinced now we'd want to have:
    poll(&pfd, 1, -1); /* this is only buying time
                        for the fd to deliver data, in case it fails
                        we don't care but just fall back to old
                        behavior before this patch with busy spinning*/
    continue;
Show 22 quoted lines
>
> So the code would look like this:
>
>    if (!poll(&pfd, 1, -1))
>       return -1;
>
>
>>
>
>>>> +                     }
>>>> +             }
>>>>               return nr;
>>>>       }
>>>>  }
>>>>
>>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe git" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>
>
Previous: Torsten BögershausenNext: Jeff King
Message 7 of 19 in “[PATCHv2 0/7] Rerolling sb/submodule-parallel-fetch for the time after 2.7”
  1. Stefan BellerDec 16, 2015
  2. 1/7 submodule.c: write "Fetching submodule <foo>" to stderrStefan Beller, Dec 16, 2015
  3. 2/7 xread: poll on non blocking fdsStefan Beller, Dec 16, 2015
  4. Torsten BögershausenDec 17, 2015
  5. Stefan BellerDec 17, 2015
  6. Torsten BögershausenDec 17, 2015
  7. Stefan BellerDec 17, 2015
  8. Jeff KingDec 18, 2015
  9. Jeff KingDec 18, 2015
  10. Torsten BögershausenDec 18, 2015
  11. Jeff KingDec 18, 2015
  12. Jeff KingDec 18, 2015
  13. 3/7 strbuf: add strbuf_read_once to read without blockingStefan Beller, Dec 16, 2015
  14. 4/7 sigchain: add command to pop all common signalsStefan Beller, Dec 16, 2015
  15. 5/7 run-command: add an asynchronous parallel child processorStefan Beller, Dec 16, 2015
  16. 6/7 fetch_populated_submodules: use new parallel job processingStefan Beller, Dec 16, 2015
  17. 7/7 submodules: allow parallel fetching, add tests and documentationStefan Beller, Dec 16, 2015
  18. Stefan BellerDec 16, 2015
  19. Junio C HamanoDec 16, 2015

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.