git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v4] compat: Fix read() of 2GB and more on Mac OS X

From
Linus Torvalds <torvalds@linux-foundation.org>
Date
Aug 19, 2013, 22:51 UTC
Message-ID
<CA+55aFzAQjxB7HkDqR6_3wdex1t1Tbrf5CeUVyiVm=DRyDVhhQ@mail.gmail.com>
In-Reply-To
<7E527329-230E-4954-9942-8BB0935ACE4D@gmail.com>
On Mon, Aug 19, 2013 at 2:56 PM, Kyle J. McKay <mackyle@gmail.com> wrote:
>
> The fact that the entire file is read into memory when applying the filter
> does not seem like a good thing (see #7-#10 above).

Yeah, that's horrible. Its likely bad for performance too, because even if you have enough memory, it blows everything out of the L2/L3 caches, and if you don't have enough memory it obviously causes other problems.

So it would probably be a great idea to make the filtering code able to do things in smaller chunks, but I suspect that the patch to chunk up xread/xwrite is the right thing to do anyway.

              Linus
Previous: Kyle J. McKayNext: Junio C Hamano
Message 25 of 37 in “xread(): Fix read error when filtering >= 2GB on Mac OS X”
  1. xread(): Fix read error when filtering >= 2GB on Mac OS XSteffen Prohaska, Aug 17, 2013
  2. John KeepingAug 17, 2013
  3. Torsten BögershausenAug 17, 2013
  4. Johannes SixtAug 17, 2013
  5. Jonathan NiederAug 17, 2013
  6. Kyle J. McKayAug 17, 2013
  7. Jonathan NiederAug 17, 2013
  8. compat: Fix read() of 2GB and more on Mac OS XSteffen Prohaska, Aug 19, 2013
  9. John KeepingAug 19, 2013
  10. Steffen ProhaskaAug 19, 2013
  11. Johannes SixtAug 19, 2013
  12. Stefan BellerAug 19, 2013
  13. Johannes SixtAug 19, 2013
  14. Steffen ProhaskaAug 19, 2013
  15. compat: Fix read() of 2GB and more on Mac OS XSteffen Prohaska, Aug 19, 2013
  16. Eric SunshineAug 19, 2013
  17. Junio C HamanoAug 19, 2013
  18. compat: Fix read() of 2GB and more on Mac OS XSteffen Prohaska, Aug 19, 2013
  19. Linus TorvaldsAug 19, 2013
  20. Steffen ProhaskaAug 19, 2013
  21. Junio C HamanoAug 19, 2013
  22. Junio C HamanoAug 19, 2013
  23. Linus TorvaldsAug 19, 2013
  24. Kyle J. McKayAug 19, 2013
  25. Linus TorvaldsAug 19, 2013
  26. Junio C HamanoAug 27, 2013
  27. 0/2 Fix IO of >=2GB on Mac OS X by limiting IO chunksSteffen Prohaska, Aug 20, 2013
  28. 1/2 xread, xwrite: Limit size of IO, fixing IO of 2GB and more on Mac OS XSteffen Prohaska, Aug 20, 2013
  29. Junio C HamanoAug 20, 2013
  30. Torsten BögershausenAug 21, 2013
  31. 2/2 Revert "compate/clipped-write.c: large write(2) fails on Mac OS X/XNU"Steffen Prohaska, Aug 20, 2013
  32. 0/2 Fix IO >= 2GB on Mac, fixed typoSteffen Prohaska, Aug 21, 2013
  33. 1/2 xread, xwrite: Limit size of IO, fixing IO of 2GB and more on Mac OS XSteffen Prohaska, Aug 21, 2013
  34. 2/2 Revert "compate/clipped-write.c: large write(2) fails on Mac OS X/XNU"Steffen Prohaska, Aug 21, 2013
  35. Junio C HamanoAug 21, 2013
  36. Johannes SixtAug 19, 2013
  37. Torsten BögershausenAug 19, 2013

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.