git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: git status OOM on mmap of large file

From
Joey Hess <id@joeyh.name>
Date
Jan 24, 2019, 17:05 UTC
Message-ID
<20190124170544.GB29200@kitenet.net>
In-Reply-To
<20190124121427.GB4949@sigill.intra.peff.net>
Jeff King wrote:
Show 6 quoted lines
> I didn't experiment with the smudge side, but I think it uses the same
> apply_filter() code. Which means that yes, it would try to store the
> 11GB in memory before writing it out. And I agree writing it out to a
> file and moving it directly into place is the sanest option there. If
> that doesn't work, spooling to a tempfile and then streaming it into
> place would also work.

I have seen that buffering on the smudge side, yes. In git-annex I happen to use the smudge filter in an unusual way that avoids that being a problem but I think it would affect git-lfs.

-- 
see shy jo
Previous: Jeff KingNext: Jeff King
Message 4 of 12 in “git status OOM on mmap of large file”
  1. Joey HessJan 22, 2019
  2. brian m. carlsonJan 24, 2019
  3. Jeff KingJan 24, 2019
  4. Joey HessJan 24, 2019
  5. Jeff KingJan 24, 2019
  6. Duy NguyenJan 24, 2019
  7. Joey HessJan 24, 2019
  8. Jeff KingJan 24, 2019
  9. Jeff KingJan 24, 2019
  10. avoid unncessary malloc of whole file sizeJoey Hess, Jan 24, 2019
  11. Junio C HamanoJan 24, 2019
  12. Jeff KingJan 24, 2019

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.