threads / discuss / 27456

gitk normal use on linux repo and severe memory pressure

Subject: gitk normal use on linux repo and severe memory pressure

## tl;dr

4 messages between May 25, 2011 and May 25, 2011.

replies: 3people: 3as markdown or json

Martin Langhoff· May 25, 2011, 15:20 UTC · lore

An anecdotical report: Right today, I fired up gitk on a linux-2.6.39 checkout to review some new patches. While discussing the patches with other developers on irc, my machine (arguably, RAM-constrained at 1GB) started to hit swap heavily.

Took a couple of minutes to kill it and get the machine back to a usable state. This is on F-14, git v1.7.4.4 .

Linus' early instinct that people care about the recent patches (so gitk and git log better show them immediately) was spot on. Now the problem is that gitk continues to walk history regardless of user interest. It gobbled up a ton of memory while I never went deeper than 20 commits.

I am tempted to set a global config option limiting gitk to a couple thousand commits, but it's not really the right solution. If I am trawling older history, I'll hit the limit and have to close gitk and reopen overriding the parameter.

Are there efforts afoot to teach gitk to read in _some_ history and pause, continuing if the user scrolls down? Not sure if searches are done in-memory in gitk -- which probably complicates things.

cheers,

-- 
 martin.langhoff@gmail.com
 martin@laptop.org -- Software Architect - OLPC
 - ask interesting questions
 - don't get distracted with shiny stuff  - working code first
 - http://wiki.laptop.org/go/User:Martinlanghoff

Andreas Schwab· May 25, 2011, 17:01 UTC · re: Martin Langhoff · lore

Re: gitk normal use on linux repo and severe memory pressure

Martin Langhoff <martin.langhoff@gmail.com> writes:

> I am tempted to set a global config option limiting gitk to a couple
> thousand commits, but it's not really the right solution. If I am
> trawling older history, I'll hit the limit and have to close gitk and
> reopen overriding the parameter.

You can start gitk with --max-count=N and if you need more commits you can edit the current view to increase that number. No need to restart gitk.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Martin Langhoff· May 25, 2011, 17:17 UTC · re: Andreas Schwab · lore

Re: gitk normal use on linux repo and severe memory pressure

On Wed, May 25, 2011 at 1:01 PM, Andreas Schwab <schwab@linux-m68k.org> wrote:

> You can start gitk with --max-count=N

Or --since=2.weeks.ago. Of course I can -- and sometimes do. But gitk defaults to OOMing my box if I forget to do so on a large repo.

> and if you need more commits you
> can edit the current view to increase that number.  No need to restart
> gitk.

Ah, that's news to me! I just tried it and it's... awkward. In fact, the dialog box seems to be missing an 'apply' button.

-- 
 martin.langhoff@gmail.com
 martin@laptop.org -- Software Architect - OLPC
 - ask interesting questions
 - don't get distracted with shiny stuff  - working code first
 - http://wiki.laptop.org/go/User:Martinlanghoff

Jeff King· May 25, 2011, 18:27 UTC · re: Martin Langhoff · lore

Re: gitk normal use on linux repo and severe memory pressure

On Wed, May 25, 2011 at 11:20:24AM -0400, Martin Langhoff wrote:

> An anecdotical report: Right today, I fired up gitk on a linux-2.6.39
> checkout to review some new patches. While discussing the patches with
> other developers on irc, my machine (arguably, RAM-constrained at 1GB)
> started to hit swap heavily.

Yeah, gitk on linux-2.6 tops out at around 1.6GB on my machine. Ouch. That's 6-7K per commit. In git.git, it's similar (7-8K per commit). For comparison, all of "git log --pretty=raw" for linux-2.6 is only about 180M, or ~0.7K per commit.

Obviously there is going to be some overhead with organizing the data, but that is really quite a lot.

> Are there efforts afoot to teach gitk to read in _some_ history and
> pause, continuing if the user scrolls down? Not sure if searches are
> done in-memory in gitk -- which probably complicates things.

Not that I know of. But it seems like there might be some room to optimize gitk's storage. I know absolutely nothing about how commits are stored now, though, so it's possible I'm completely wrong.

-Peff