git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 0/17] Sliding window mmap for packfiles.

From
Andy Whitcroft <apw@shadowen.org>
Date
Jan 2, 2007, 15:28 UTC
Message-ID
<459A7A21.4060901@shadowen.org>
In-Reply-To
<20061224094959.GA7814@spearce.org>
Shawn Pearce wrote:
Show 36 quoted lines
> Francis Moreau <francis.moro@gmail.com> wrote:
>> On 12/24/06, Shawn Pearce <spearce@spearce.org> wrote:
>>> However with this series even a 32 bit OS which only permits
>>> processes to have at most 2 GiB of address space (2 GiB split
>>> between kernel space and userspace) can access packfiles up
>>> to 4 GiB in size.  That seems to be the split most OSes wind
>>> up using, if they didn't push it out to 3.2 GiB like Linux
>>> and Solaris have done.
>>>
>> Does it still needed for 64 bit OS ?
> 
> Not really.  Almost any reasonable 64 bit OS which is also running
> a Git compiled for 64 bit userspace would be able to mmap multiple
> 4 GiB packfiles without this series.
>  
>> if not, can the overhead (if there is a significant one) implied by
>> your rework be avoid for such cases ?
> 
> The overhead is rather low.  I did try hard to make it only a handful
> of machine instructions worth of additional work, and even then I
> tried to ammortize those over relatively large blocks of data to
> reduce the impact.  But yes, there is an overhead over the current
> shipping version of Git.
> 
> However at least some of the overhead can be avoided by setting
> core.packedGitWindowSize and core.packedGitLimit to higher values.
> This will allow the implementation to mmap() larger windows of the
> packfiles and retain a greater number of windows in memory at once.
> 
> If core.packedGitWindowSize is larger than your largest packfile
> then most of the code will just "shutoff" and won't get in the way.
> Its default is 32 MiB (see Documentation/config.txt).
> 
> I think the additional overhead added by this series is neglible
> and worth the more graceful degredation it allows when virtual
> address space becomes limited.

You now change the default size based on NO_MMAP, could you not just bump the window size to 4GiB on 64 bit?

-apw
Previous: Shawn PearceNext: Linus Torvalds
Message 8 of 9 in “Sliding window mmap for packfiles.”
  1. 0/17 Sliding window mmap for packfiles.Shawn O. Pearce, Dec 23, 2006
  2. Junio C HamanoDec 23, 2006
  3. Shawn PearceDec 23, 2006
  4. Francis MoreauDec 24, 2006
  5. Shawn PearceDec 24, 2006
  6. Francis MoreauDec 24, 2006
  7. Shawn PearceDec 24, 2006
  8. Andy WhitcroftJan 2, 2007
  9. Linus TorvaldsDec 24, 2006

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.