git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 01/10] strbuf: Add strbuf_read_noblock

From
Jeff King <peff@peff.net>
Date
Sep 17, 2015, 18:02 UTC
Message-ID
<20150917180205.GA29985@sigill.intra.peff.net>
In-Reply-To
<xmqqoah1hqxs.fsf@gitster.mtv.corp.google.com>
On Thu, Sep 17, 2015 at 10:54:39AM -0700, Junio C Hamano wrote:
Show 15 quoted lines
> OK.  Trying to repurpose strbuf_read() for non-blocking FD was a
> silly idea to begin with, as it wants to read thru to the EOF, and
> setting FD explicitly to O_NONBLOCK is a sign that the caller wants
> to grab as much as possible and does not want to wait for the EOF.
> 
> So assuming we want strbuf_read_nonblock(), what interface do we
> want from it?  We could:
> 
>  * Have it grab as much as possible into sb as long as it does not
>    block?
> 
>  * Have it grab reasonably large amount into sb, and not blocking is
>    an absolute requirement, but the function is not required to read
>    everything that is available on the FD (i.e. the caller is
>    expected to loop)?

I think we are crossing emails, but I would definitely argue for the latter.

Show 10 quoted lines
> If we choose the latter, then your "EAGAIN? EOF?" becomes easier,
> no?  We only have to do a single call to xread(), and then we:
> 
>  - get EAGAIN or EWOULDBLOCK; leave sb as-is, set errno==EAGAIN and
>    return -1.
> 
>  - get something (in which case that is not an EOF yet); append to
>    sb, return the number of bytes.
> 
>  - get EOF; leave sb as-is, return 0.
Yes, exactly.
Show 10 quoted lines
> ssize_t strbuf_read_nonblock(struct strbuf *sb, int fd, size_t hint)
> {
> 	strbuf_grow(sb, hint ? hint : 8192);
> 	ssize_t want = sb->alloc - sb->len - 1;
> 	ssize_t got = xread_nonblock(fd, sb->buf + sb->len, want);
> 	if (got < 0) {
> 		if (errno == EWOULDBLOCK)
> 			errno = EAGAIN; /* make life easier for the caller */
> 		return got;
> 	}

I like the normalizing of errno, but that should probably be part of xread_nonblock(), I would think.

> 	sb->len += got;
> 	sb->buf[sb->len] = '\0';
Use strbuf_setlen() here?
> 	return got;

If "got == 0", we naturally do not change sb->len at all, and the strbuf is left unchanged. But do we want to de-allocate what we allocated in strbuf_grow() above? That is what strbuf_read() does, but I think it is even less likely to help anybody here. With the original strbuf_read(), you might do:

  if (strbuf_read(&buf, fd, hint) <= 0)
	return; /* got nothing */

but because the nature of strbuf_read_nonblock() is to call it from a loop, you'd want to strbuf_release() when you leave the loop anyway.

-Peff
Previous: Junio C HamanoNext: Stefan Beller
Message 17 of 37 in “fetch submodules in parallel and a preview on parallel "submodule update"”
  1. 00/10 fetch submodules in parallel and a preview on parallel "submodule update"Stefan Beller, Sep 17, 2015
  2. 01/10 strbuf: Add strbuf_read_noblockStefan Beller, Sep 17, 2015
  3. Junio C HamanoSep 17, 2015
  4. Jeff KingSep 17, 2015
  5. Junio C HamanoSep 17, 2015
  6. Stefan BellerSep 17, 2015
  7. Jeff KingSep 17, 2015
  8. Junio C HamanoSep 17, 2015
  9. Jeff KingSep 17, 2015
  10. Stefan BellerSep 17, 2015
  11. Jeff KingSep 17, 2015
  12. Stefan BellerSep 17, 2015
  13. Jeff KingSep 17, 2015
  14. Stefan BellerSep 17, 2015
  15. Junio C HamanoSep 17, 2015
  16. Junio C HamanoSep 17, 2015
  17. Jeff KingSep 17, 2015
  18. Stefan BellerSep 17, 2015
  19. 02/10 run-command: factor out return value computationStefan Beller, Sep 17, 2015
  20. Jeff KingSep 17, 2015
  21. 03/10 run-command: add an asynchronous parallel child processorStefan Beller, Sep 17, 2015
  22. Junio C HamanoSep 17, 2015
  23. Stefan BellerSep 17, 2015
  24. Junio C HamanoSep 18, 2015
  25. Stefan BellerSep 18, 2015
  26. 04/10 fetch_populated_submodules: use new parallel job processingStefan Beller, Sep 17, 2015
  27. 05/10 submodules: Allow parallel fetching, add tests and documentationStefan Beller, Sep 17, 2015
  28. 06/10 git submodule update: Redirect any output to stderrStefan Beller, Sep 17, 2015
  29. Eric SunshineSep 17, 2015
  30. Stefan BellerSep 17, 2015
  31. 07/10 git submodule update: pass --prefix only with a non empty prefixStefan Beller, Sep 17, 2015
  32. Eric SunshineSep 17, 2015
  33. 08/10 git submodule update: cmd_update_recursiveStefan Beller, Sep 17, 2015
  34. 09/10 git submodule update: cmd_update_recursiveStefan Beller, Sep 17, 2015
  35. Eric SunshineSep 17, 2015
  36. 10/10 git submodule update: cmd_update_fetchStefan Beller, Sep 17, 2015
  37. Jacob KellerSep 17, 2015

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.