From: Eric Wong <normalperson@yhbt.net>
Date: Wed, 23 Sep 2015 00:35:16 GMT
Subject: Re: [PATCH] git-svn: make batch mode optional for git-cat-file
Message-ID: <20150923003516.GA6086@dcvr.yhbt.net>
In-Reply-To: <20150923001350.GA22266@dcvr.yhbt.net>

Eric Wong <normalperson@yhbt.net> wrote:
> Victor Leschuk <vleschuk@accesssoftek.com> wrote:
> > The thing is that git-cat-file keeps growing during work when running
> > in "batch" mode. See the figure attached: it is for cloning a rather
> > small repo (1 hour to clone about ~14000 revisions). However the clone
> > of a large repo (~280000 revisions) took about 2 weeks and
> > git-cat-file has outgrown the parent perl process several times
> > (git-cat-file - ~3-4Gb, perl - 400Mb).

How much of that is anonymous memory, though?
(pmap $PID_OF_GIT_CAT_FILE)

Running the following on the Linux kernel tree I had lying around:

(for i in $(seq 100 200); do git ls-files | sed -e "s/^/HEAD~$i:/"; done)|\
  git cat-file --batch >/dev/null

Reveals about 510M RSS in top, but pmap says less than 20M of that
is anonymous.  So the rest are mmap-ed packfiles; that RSS gets
transparently released back to the kernel under memory pressure.