git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH] Fix random fast-import errors when compiled with NO_MMAP

From
Charles Bailey <charles@hashpling.org>
Date
Jan 18, 2008, 08:42 UTC
Message-ID
<20080118084201.GA14763@hashpling.org>
In-Reply-To
<alpine.LFD.1.00.0801172013270.2957@woody.linux-foundation.org>
On Thu, Jan 17, 2008 at 08:27:08PM -0800, Linus Torvalds wrote:
Show 32 quoted lines
> 
> 
> On Thu, 17 Jan 2008, Shawn O. Pearce wrote:
> >
> > fast-import was relying on the fact that on most systems mmap() and
> > write() are synchronized by the filesystem's buffer cache.  We were
> > relying on the ability to mmap() 20 bytes beyond the current end
> > of the file, then later fill in those bytes with a future write()
> > call, then read them through the previously obtained mmap() address.
> > 
> > This isn't always true with some implementations of NFS, but it is
> > especially not true with our NO_MMAP=YesPlease build time option used
> > on some platforms.
> 
> In fact, even with mmap(), it's not guaranteed. There are really crappy 
> mmap implementations out there, partly due to bad CPU design (virtual CPU 
> caches without coherency), but more often due to total crap OS.
> 
> (Yeah, Linux did count in that area at some point. Long ago. Early 90's. 
> Maybe)
> 
> I think HP-UX used to have non-coherent mmap for the longest time, due to 
> carrying around some totally crap memory management based on some ancient 
> BSD version that everybody else (including the BSD's) had long since 
> jettisoned.
> 
> That said, I suspect any unix you can run today (without calling it a 
> retro setup) probably has coherent-enough mmap. The possible virtual cache 
> coherency issue is unlikely to be able to trigger this (and not relevant 
> on any sane hardware anyway).
> 
> 				Linus

I've just checked the Mac OS X build and it looks like there is a mmap and git is indeed using it, so this is obviously an example of a "really crappy" mmap implementation.

This adds more ammunition to the fight against the whole Mac OS X is powered/built/based on UNIX myth.

Charles.
Previous: Linus TorvaldsNext: Linus Torvalds
Message 2 of 13 in “Re: [PATCH] Fix random fast-import errors when compiled with NO_MMAP”
  1. Linus TorvaldsJan 18, 2008
  2. Charles BaileyJan 18, 2008
  3. Linus TorvaldsJan 18, 2008
  4. Junio C HamanoJan 19, 2008
  5. Linus TorvaldsJan 19, 2008
  6. Shawn O. PearceJan 21, 2008
  7. Junio C HamanoJan 18, 2008
  8. Shawn O. PearceJan 21, 2008
  9. Johannes SixtJan 18, 2008
  10. Charles BaileyJan 18, 2008
  11. Junio C HamanoJan 18, 2008
  12. Charles BaileyJan 18, 2008
  13. Junio C HamanoJan 18, 2008

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.