gitk performance questions/issues

Since the gitk is being discussed, I'd like to see if I'm understanding correctly and hitting problems or just not doing things correctly. (This is using a self-compiled "git version 1.5.2.2".)

[Caveat: this is on a repository that is evolves using my own scripts which mimic the git shell scripts in calling the low-level programs. I'm 99.999% sure it's generating a fully valid repo, but I might be missing something. Actual generating script is at http://www.personal.rdg.ac.uk/~sis05dst/chronoversion.htm ]

1.  With gitk --all -n 256 on my repo it consistently
takes 12s for the window to appear and 21s for the
"I'm working" cursor to change to a normal cursor.
This is on a dual Xeon machine, /proc/cpuinfo
excerpt

model name : Intel(R) Xeon(TM) CPU 3.00GHz stepping : 3 cpu MHz : 2992.496 cache size : 2048 KB

running x86-64 Fedora 7, packed repo has a 12M .git dir. Is this the expected start-up time for such a configuration? The output of time git-rev-list --parents --topo-order --all>/dev/null is

real 0m0.048s user 0m0.044s sys 0m0.005s

2. Startup time isn't an excessive problem. However,
I often don't seem to be getting the "diff with parent"
being displayed once I click on the another commit. If
I click on several in sequence I get the error
message

"error getting diffs: couldn't fork child process: not enough memory"

The diffs are relatively small (consecutive commits are generated hourly automatically) and a command line git-diff generates them instantly.

3. Is "gitk --all -n 256" the best way to say "show me
relatively recent stuff" or should I be using different
options for limiting things?

Many thanks for any insight,

On Mon, 13 Aug 2007, David Tweed wrote:

There is something seriously wrong there.

For me, on the kernel archive (which is much bigger than yours), the window appears immediately, and the "working cursor" takes about two seconds to disappear.

Your CPU is fast enough..

.. and you're apparently fully packed, and do not even have all that big a repository.

So:

Absolutely not.

Ok, so it's not the revision generation that takes time. 

You have some *serious* problem somewhere. I think your tcl/tk is screwed up somehow.

Alternatively, maybe your repo has some odd structure (do you have tons and tons of branches or tags?) and "gitk" ends up having some problem with that. What does

	git rev-parse --all | wc -l

say? Various git tools have had performance problems with thousands of branches or tags in the past.

One thing to try: I think that "git gc" back in the 1.5.2.2 days didn't pack branches by default - and if you have lots and lots of branches, you should try to make sure that they are packed, using

	git pack-refs --all --prune

(newer git versions will do this automatically in "git gc" - while v1.5.2.2 will only pack the tags by default, iirc).

Yeah, it sounds like git itself has no problems, and it's literally gitk that shows some strange behaviour.

That's a perfectly good thing to do. You don't need the "-n", you can just do "-256". That said, if you actually want commits to be in date order, use

	gitk -d --all -256

instead, which tells gitk to use "--date-order" instead, which is often nicer (the graph ends up being a bit messier, but it gives you a more correct view of relative dates when you have multiple branches!

		Linus

On 8/13/07, Linus Torvalds <torvalds@linux-foundation.org> wrote:

Ah, that's it: I've got
$ wc .git/packed-refs
  1915   3832 130178 .git/packed-refs
and 35 currently unpacked tags and temporarily moving that
file away, gitk now starts up virtually instantly. My usage generates
a tag per commit which is probably excessive. For completeness,
$ git rev-parse --all | wc -l
1957

Thinking about it, I suppose even if you're only looking at the last 256 commits you've got to look through all the tags to see whether or not they refer to something within that window, so it's not unreasonable for it to affect startup time.

The bit that is a quite surprising to me is that once the window has finished initialising, clicking on one of the blue dots now (ie without .git/packed-refs file) now generates the diffs instantly as well. It's not obvious to me why the existence/non-existence of tags should affect displaying-diff performance.

Anyway, now I know I can rearrange things not to need anything like as many tags.

Many thanks for the help,

David Tweed writes:

Gitk uses a Tcl associative array for that, which should be very fast.

Could you try this: go to the Edit->Preferences window and turn off "Display nearby tags". Then with your ~2000 refs in place, see how long gitk takes to start up and to display a diff.

If doing that makes it fast, then I suspect that my topology caching patch will also help. That's waiting on an answer from Junio or Linus to my question about making git rev-list ignore negated non-existent refs.

Also, if it's possible to give me a copy of your repo, that would help.

Paul.

On 8/14/07, Paul Mackerras <paulus@samba.org> wrote:

Right, I've restored the file of 1915 packed tags (but not the 35 loose tags). Using DNT for "display nearby tags"

DNT on: consistently 11s to window appear, 20s to normal cursor DNT off: consistently 11s to window appear but normal cursor immediately

Regarding getting the diffs, I think that was "user misunderstanding" and I'm doubting if I definitely did click on the blue circles yesterday. Clicking on the any part yellow tag marker displays the SHA of the tag object (which I'd confused with the start of an unfinished commit diff); for some reason I'd thought it would give the diff-with-parent. Clicking on either the blue circle or "commit headline text" does bring up the diff instantly even with the packed refs file. Sorry for the confusion.

I'll privately mail Paul about getting it; if anyone else wants a copy of the repository in order look at startup performance issues mail me. (The most embarassing thing in there is just some astonishingly crap code, but I'd prefer not to blanket publish it.)