threads / discuss / 10777

Reducing the memory footprint

Subject: Reducing the memory footprint

## tl;dr

4 messages between Nov 9, 2007 and Nov 10, 2007.

replies: 3people: 2as markdown or json

Jon Smirl· Nov 9, 2007, 23:38 UTC · lore

I'm using this config file:

[core]
        repositoryformatversion = 0
        filemode = true
        bare = false
        logallrefupdates = true
        packedGitWindowSize = 1M
        packedGitLimit = 1M
        deltaBaseCacheLimit = 1M
[pack]
        windowMemory = 1M
        deltaCacheSize = 1M

And I have NO_MMAP compiled in.

git is still using over 200MB of memory or address space, my process gets killed either way.

Ideas on how to reduce the foot print more?

-- 
Jon Smirl
jonsmirl@gmail.com

Brian Downing· Nov 10, 2007, 00:53 UTC · re: Jon Smirl · lore

Re: Reducing the memory footprint

On Fri, Nov 09, 2007 at 06:38:00PM -0500, Jon Smirl wrote:

Show 10 quoted lines

> I'm using this config file:
> 
> [pack]
>         windowMemory = 1M
>         deltaCacheSize = 1M
> 
> And I have NO_MMAP compiled in.
> 
> git is still using over 200MB of memory or address space, my process
> gets killed either way.

I'm assuming it's dying on repacking since you included the pack parameters.

How big is your biggest object? Even with pack.windowMemory, it still keeps the last object around to try and delta against (in other words, the window only shrinks to size 1), which means you have to have room for it and its delta index.

-bcd

Jon Smirl· Nov 10, 2007, 01:05 UTC · re: Brian Downing · lore

Re: Reducing the memory footprint

On 11/9/07, Brian Downing <bdowning@lavos.net> wrote:

Show 19 quoted lines

> On Fri, Nov 09, 2007 at 06:38:00PM -0500, Jon Smirl wrote:
> > I'm using this config file:
> >
> > [pack]
> >         windowMemory = 1M
> >         deltaCacheSize = 1M
> >
> > And I have NO_MMAP compiled in.
> >
> > git is still using over 200MB of memory or address space, my process
> > gets killed either way.
>
> I'm assuming it's dying on repacking since you included the pack
> parameters.
>
> How big is your biggest object?  Even with pack.windowMemory, it still
> keeps the last object around to try and delta against (in other words,
> the window only shrinks to size 1), which means you have to have room
> for it and its delta index.

It's a Linux kernel repository. Git receive-pack is going over 200MB and getting zapped. I don't understand why the process is so large. I am compiled with -DNO_MMAP.

I think I have a achieved a work around. I rsync'd in my last several weeks of changes. Now I can 'git push' small amounts of changes without getting killed.

I'm begging dreamhost to simply install git. Installed commands don't get zapped.

-- 
Jon Smirl
jonsmirl@gmail.com

Jon Smirl· Nov 10, 2007, 06:07 UTC · re: Jon Smirl · lore

Re: Reducing the memory footprint

On 11/9/07, Jon Smirl <jonsmirl@gmail.com> wrote:

Show 24 quoted lines

> On 11/9/07, Brian Downing <bdowning@lavos.net> wrote:
> > On Fri, Nov 09, 2007 at 06:38:00PM -0500, Jon Smirl wrote:
> > > I'm using this config file:
> > >
> > > [pack]
> > >         windowMemory = 1M
> > >         deltaCacheSize = 1M
> > >
> > > And I have NO_MMAP compiled in.
> > >
> > > git is still using over 200MB of memory or address space, my process
> > > gets killed either way.
> >
> > I'm assuming it's dying on repacking since you included the pack
> > parameters.
> >
> > How big is your biggest object?  Even with pack.windowMemory, it still
> > keeps the last object around to try and delta against (in other words,
> > the window only shrinks to size 1), which means you have to have room
> > for it and its delta index.
>
> It's a Linux kernel repository. Git receive-pack is going over 200MB
> and getting zapped.  I don't understand why the process is so large. I
> am compiled with -DNO_MMAP.

I believe I must not have installed everything correctly with my NO_MMAP build. After debugging for a while and fixing things I'm able to do a push now in about 80MB of memory.

Show 11 quoted lines

> I think I have a achieved a work around. I rsync'd in my last several
> weeks of changes. Now I can 'git push' small amounts of changes
> without getting killed.
>
> I'm begging dreamhost to simply install git. Installed commands don't
> get zapped.
>
> --
> Jon Smirl
> jonsmirl@gmail.com
>

-- 
Jon Smirl
jonsmirl@gmail.com