git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 11/17] Fully activate the sliding window pack access.

From
Johannes Schindelin <johannes.schindelin@gmx.de>
Date
Dec 24, 2006, 01:23 UTC
Message-ID
<Pine.LNX.4.63.0612240159270.19693@wbgn013.biozentrum.uni-wuerzburg.de>
In-Reply-To
<Pine.LNX.4.64.0612231207400.3671@woody.osdl.org>
Hi,
On Sat, 23 Dec 2006, Linus Torvalds wrote:
Show 8 quoted lines
> On Sat, 23 Dec 2006, Junio C Hamano wrote:
> > 
> > I have been thinking that we should perhaps change Cygwin
> > default to NO_MMAP.  Safer but slower would be better than not
> > working correctly for people on FAT32.
> 
> The thing is, with a smaller pack access window, it might not even be 
> slower. I don't know just _how_ many hoops cygwin jumps through for mmap,

AFAIK Windows API _does_ have an equivalent to mmap(), which just is not as easy to use.

[/me asks my friend Google]

Yes. The function is called CreateFile(), and it takes a _filename_, not a file descriptor. Which does not matter much, since Windows is braindead enough not to be able to have files deleted which are still opened. Mind you, renaming is no problem...

So, Cygwin has to keep record of the file name (and AFAICT it necessarily fails if mmap() is called with a file descriptor of a file which has been renamed _externally_ after opening) of the file descriptor.

Cygwin also has to reopen after a fork(), since Windows does not have fork(), and it has to be simulated by a CreateProcess() and reestablishing existing resources. That is the reason a fork() is really, really expensive on Windows.

> and maybe mmap under cygwin is actually perfectly fine, but at the same 
> time I do suspect that UNIX mmap semantics are a lot harder to emulate 
> than just a regular "pread()", so it's quite possible that by avoiding 
> mmap you could avoid a lot of complex cygwin code.
But should we really cater for a braindead API?

At the time, I _hated_ the fact that we had to change HEAD into a regular file, just because people feared that Windows could not handle symbolic links (and mind you, Cygwin handles them quite gracefully, using .lnk files).

Besides, IIRC you said yourself that mmap() is better than pread() when seeking a lot. And pack file accesses _are_ the perfect example for such a use case, no?

> And yes, making it all work on FAT32 would obviously be a good thing.

If really needed, we could make a hack for Windows, which just adds a new config option automagically set by init-db (on Windows only), and which is checked in gitfakemmap() and gitfakeunmmap(). (BTW I don't like the naming anymore; a good cleanup would be to name them git_mmap() and git_unmmap()).

Having said that, if somebody is really dedicated to working on Windows, she could overcome the mentioned problems. There's got to be a way to avoid the error on FAT32 without using NO_MMAP. Don't look at me, I hate Windows, I will not fix it.

Ciao, Dscho

Previous: Linus TorvaldsNext: Shawn Pearce
Message 17 of 25 in “Replace unpack_entry_gently with unpack_entry.”
  1. 1/17 Replace unpack_entry_gently with unpack_entry.Shawn O. Pearce, Dec 23, 2006
  2. 2/17 Introduce new config option for mmap limit.Shawn O. Pearce, Dec 23, 2006
  3. 3/17 Refactor packed_git to prepare for sliding mmap windows.Shawn O. Pearce, Dec 23, 2006
  4. 4/17 Use off_t for index and pack file lengths.Shawn O. Pearce, Dec 23, 2006
  5. 5/17 Create read_or_die utility routine.Shawn O. Pearce, Dec 23, 2006
  6. 6/17 Refactor how we open pack files to prepare for multiple windows.Shawn O. Pearce, Dec 23, 2006
  7. 7/17 Replace use_packed_git with window cursors.Shawn O. Pearce, Dec 23, 2006
  8. 8/17 Loop over pack_windows when inflating/accessing data.Shawn O. Pearce, Dec 23, 2006
  9. 9/17 Document why header parsing won't exceed a window.Shawn O. Pearce, Dec 23, 2006
  10. 10/17 Unmap individual windows rather than entire files.Shawn O. Pearce, Dec 23, 2006
  11. 11/17 Fully activate the sliding window pack access.Shawn O. Pearce, Dec 23, 2006
  12. Linus TorvaldsDec 23, 2006
  13. Eric BlakeDec 23, 2006
  14. Johannes SchindelinDec 24, 2006
  15. Junio C HamanoDec 23, 2006
  16. Linus TorvaldsDec 23, 2006
  17. Johannes SchindelinDec 24, 2006
  18. Shawn PearceDec 24, 2006
  19. Shawn PearceDec 24, 2006
  20. 12/17 Load core configuration in git-verify-pack.Shawn O. Pearce, Dec 23, 2006
  21. 13/17 Ensure core.packedGitWindowSize cannot be less than 2 pages.Shawn O. Pearce, Dec 23, 2006
  22. 14/17 Improve error message when packfile mmap fails.Shawn O. Pearce, Dec 23, 2006
  23. 15/17 Support unmapping windows on 'temporary' packfiles.Shawn O. Pearce, Dec 23, 2006
  24. 16/17 Create pack_report() as a debugging aid.Shawn O. Pearce, Dec 23, 2006
  25. 17/17 Test suite for sliding window mmap implementation.Shawn O. Pearce, Dec 23, 2006

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.