git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: fatal: Out of memory, getdelim failed under NFS mounts

From
Yaroslav Halchenko <yoh@onerussian.com>
Date
Aug 10, 2017, 20:29 UTC
Message-ID
<20170810202942.s232yr5stjxmmz4t@hopa.kiewit.dartmouth.edu>
In-Reply-To
<20170810200502.rutab4z3ft7gcpjz@sigill.intra.peff.net>
On Thu, 10 Aug 2017, Jeff King wrote:
> On Thu, Aug 10, 2017 at 09:58:37PM +0200, René Scharfe wrote:
> > > So the function is returning -1 and leaving ENOMEM in errno on
> > > Yaroslav's system.
> > > I wonder if we are truly hitting out of memory, though.  The same
> > > symptom could bee seen if getdelim() does not touch errno when it
> > > returns -1, but some other system call earlier set it to ENOMEM,
> > > for example.
> > That can happen when the end of a file is reached; getdelim() returns
> > -1 and leaves errno unchanged.  So we need to set errno to 0 just
> > before that call.
> Good catch. That's a bug in my original conversoin of
> strbuf_getwholeline().

I think this did it! at least on this simple test... yet to test a bit more in those scenarios we ran into it before while testing git-annex.

commit 36ef5e3ad2c187d3be664c33dbc8c06e59bceaf4 (HEAD -> bf-seterrno0)
Author: Yaroslav O. Halchenko <yhalchen@smaug.cns.dartmouth.edu>
Date:   Thu Aug 10 20:26:47 2017 +0000
    BF: set errno to 0 before getdelim call to get unbiased assesment later
diff --git a/strbuf.c b/strbuf.c
index 89d22e3b0..323c49ceb 100644
--- a/strbuf.c
+++ b/strbuf.c
@@ -476,6 +476,7 @@ int strbuf_getwholeline(struct strbuf *sb, FILE *fp, int term)
        /* Translate slopbuf to NULL, as we cannot call realloc on it */
        if (!sb->alloc)
                sb->buf = NULL;
+       errno = 0;
        r = getdelim(&sb->buf, &sb->alloc, term, fp);
 
        if (r > 0) {
-- 
Yaroslav O. Halchenko
Center for Open Neuroscience     http://centerforopenneuroscience.org
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834                       Fax: +1 (603) 646-1419
WWW:   http://www.linkedin.com/in/yarik        
Previous: Jeff KingNext: René Scharfe
Message 9 of 21 in “fatal: Out of memory, getdelim failed under NFS mounts”
  1. Yaroslav HalchenkoAug 9, 2017
  2. René ScharfeAug 10, 2017
  3. Yaroslav HalchenkoAug 10, 2017
  4. René ScharfeAug 10, 2017
  5. Junio C HamanoAug 10, 2017
  6. Yaroslav HalchenkoAug 10, 2017
  7. René ScharfeAug 10, 2017
  8. Jeff KingAug 10, 2017
  9. Yaroslav HalchenkoAug 10, 2017
  10. strbuf: clear errno before calling getdelim(3)René Scharfe, Aug 10, 2017
  11. Jeff KingAug 10, 2017
  12. Yaroslav HalchenkoAug 10, 2017
  13. Junio C HamanoAug 10, 2017
  14. Simon RuderichAug 11, 2017
  15. René ScharfeAug 11, 2017
  16. Simon RuderichAug 12, 2017
  17. René ScharfeAug 12, 2017
  18. René ScharfeAug 12, 2017
  19. Jeff KingAug 13, 2017
  20. René ScharfeAug 13, 2017
  21. Jeff KingAug 13, 2017

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.