git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH] Fix random fast-import errors when compiled with NO_MMAP

From
Linus Torvalds <torvalds@linux-foundation.org>
Date
Jan 18, 2008, 17:08 UTC
Message-ID
<alpine.LFD.1.00.0801180842350.2957@woody.linux-foundation.org>
In-Reply-To
<20080118084201.GA14763@hashpling.org>
On Fri, 18 Jan 2008, Charles Bailey wrote:
> 
> I've just checked the Mac OS X build and it looks like there is a mmap
> and git is indeed using it, so this is obviously an example of a
> "really crappy" mmap implementation.
Looking closer, this is not necessarily the case here.

Git uses MAP_PRIVATE, because that whole pack-file mapping was really *meant* to map an existing read-only pack-file, and fast-import seems to really be screwing with it.

It so happens that Linux has a particularly clean and streamlined VM, and if you do only reads to a MAP_PRIVATE mapping on a normal filesystem, you'll always be as coherent as with MAP_SHARED because Linux will simply map in the page cache pages directly.

But this is definitely not portable, and the git fast-import mmap window usage before Shawn's patch it was simply wrong.

So in this case, it really was git that was crap.

It just happened to work because the Linux mmap handling is just generally pretty sane. It probably also worked fine on pretty much any other modern UNIX (ie Solaris).

I'm not quite sure what OS X does to MAP_PRIVATE mappings, but if OS X is still based on Mach (with FreeBSD just as a single-server on top), I suspect that may be why it broke on OS X. The Mach VM is insanely complex and does really odd things.

But the fact is, without MAP_SHARED, you shouldn't expect things to be coherent, even if they often will be (especially for PROT_READ).

Btw, even with Shawn's patch, I wonder if the index_data usage is correct.
			Linus
Previous: Charles BaileyNext: Junio C Hamano
Message 3 of 13 in “Re: [PATCH] Fix random fast-import errors when compiled with NO_MMAP”
  1. Linus TorvaldsJan 18, 2008
  2. Charles BaileyJan 18, 2008
  3. Linus TorvaldsJan 18, 2008
  4. Junio C HamanoJan 19, 2008
  5. Linus TorvaldsJan 19, 2008
  6. Shawn O. PearceJan 21, 2008
  7. Junio C HamanoJan 18, 2008
  8. Shawn O. PearceJan 21, 2008
  9. Johannes SixtJan 18, 2008
  10. Charles BaileyJan 18, 2008
  11. Junio C HamanoJan 18, 2008
  12. Charles BaileyJan 18, 2008
  13. Junio C HamanoJan 18, 2008

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.