[PATCH] Increase packedGit{Limit,WindowSize} on 64 bit systems.

Shawn O. Pearce· Jan 5, 2007, 03:28 UTC · lore

If we have a 64 bit address space we can easily afford to commit a larger amount of virtual address space to pack file access. So on these platforms we should increase the default settings of core.packedGit{Limit,WindowSize} to something that will better handle very large projects.

Thanks to Andy Whitcroft for pointing out that we can safely increase these defaults on such systems.

Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
---
 Documentation/config.txt |   16 ++++++++++------
 git-compat-util.h        |   10 ++++++++--
 2 files changed, 18 insertions(+), 8 deletions(-)

Show changes to 2 files +18 −8

Documentation/config.txt, git-compat-util.h

diff --git a/Documentation/config.txt b/Documentation/config.txt
index e4ee52f..b4aae0d 100644
--- a/Documentation/config.txt
+++ b/Documentation/config.txt
@@ -125,9 +125,12 @@ core.packedGitWindowSize::
 	more quickly.  Smaller window sizes will negatively affect
 	performance due to increased calls to the operating system's
 	memory manager, but may improve performance when accessing
-	a large number of large pack files.  Default is 32 MiB,
-	which should be reasonable for all users/operating systems.
-	You probably do not need to adjust this value.
+	a large number of large pack files.
++
+Default is 1 MiB if NO_MMAP was set at compile time, otherwise 32
+MiB on 32 bit platforms and 1 GiB on 64 bit platforms.  This should
+be reasonable for all users/operating systems.  You probably do
+not need to adjust this value.
 +
 Common unit suffixes of 'k', 'm', or 'g' are supported.
 
@@ -136,9 +139,10 @@ core.packedGitLimit::
 	from pack files.  If Git needs to access more than this many
 	bytes at once to complete an operation it will unmap existing
 	regions to reclaim virtual address space within the process.
-	Default is 256 MiB, which should be reasonable for all
-	users/operating systems, except on the largest projects.
-	You probably do not need to adjust this value.
++
+Default is 256 MiB on 32 bit platforms and 8 GiB on 64 bit platforms.
+This should be reasonable for all users/operating systems, except on
+the largest projects.  You probably do not need to adjust this value.
 +
 Common unit suffixes of 'k', 'm', or 'g' are supported.
 
diff --git a/git-compat-util.h b/git-compat-util.h
index f243b86..55456da 100644
--- a/git-compat-util.h
+++ b/git-compat-util.h
@@ -97,11 +97,17 @@ extern int git_munmap(void *start, size_t length);
 #else /* NO_MMAP */
 
 #include <sys/mman.h>
-#define DEFAULT_PACKED_GIT_WINDOW_SIZE (32 * 1024 * 1024)
+#define DEFAULT_PACKED_GIT_WINDOW_SIZE \
+	(sizeof(void*) >= 8 \
+		?  1 * 1024 * 1024 * 1024 \
+		: 32 * 1024 * 1024)
 
 #endif /* NO_MMAP */
 
-#define DEFAULT_PACKED_GIT_LIMIT (256 * 1024 * 1024)
+#define DEFAULT_PACKED_GIT_LIMIT \
+	(sizeof(void*) >= 8 \
+		?   8 * 1024 * 1024 * 1024 \
+		: 256 * 1024 * 1024)
 
 #ifdef NO_SETENV
 #define setenv gitsetenv

-- 
1.5.0.rc0.gce9e

Junio C Hamano· Jan 5, 2007, 06:01 UTC · re: Shawn O. Pearce · lore

Re: [PATCH] Increase packedGit{Limit,WindowSize} on 64 bit systems.

"Shawn O. Pearce" <spearce@spearce.org> writes:

Show 5 quoted lines

> If we have a 64 bit address space we can easily afford to commit
> a larger amount of virtual address space to pack file access.
> So on these platforms we should increase the default settings of
> core.packedGit{Limit,WindowSize} to something that will better
> handle very large projects.

Hmmmm.  What's the reasoning behind this?

We have more than enough virtual memory anyway, we do not bother with our own mmap limit -- we will let the operating system worry about it.

If that is the reasoning, I have a feeling that we might want to be even more agressive. If you have a 1.5GB pack, wouldn't you rather map the whole thing in a single window, instead of splitting that into two?

Currently we are limited (by pack offset) to 4GB per pack, so raising the window max to 4GB might make sense.

On the total size of vm space, I am wondering what would happen if we make this unbounded. You certainly notice mmap() failure and fall back to recycle other windows even when your total usage is under the limit, don't you? Your later patches in the series even unmap the mapped but unused window lazily to make room for xmalloc() and friends, so I suspect it might even make it simpler not to have this limit especially on larger systems.

patchIncrease packedGit{Limit,WindowSize} on 64 bit systems.