Re: how to speed up "git log"?

From: Linus Torvalds <torvalds@linux-foundation.org>
Date: Feb 12, 2007, 04:20 UTC
Message-ID: <Pine.LNX.4.64.0702112009440.8424@woody.linux-foundation.org>
In-Reply-To: <200702120041.27419.bruno@clisp.org>

On Mon, 12 Feb 2007, Bruno Haible wrote:

Show 8 quoted lines

> Hello Johannes,
> 
> Thanks for the helpful answer.
> 
> > Yes, because there were only 147 commits which changed the file. But git 
> > looked at all commits to find that.
> 
> Ouch.

This should become a FAQ.

Git simply DOES NOT HAVE per-file history. And having it is actually a BUG in other systems.

Not having per-file history is what allows git to do

	git log directory-or-file-set

ratehr than being able to track just one file. You can't do it sanely with per-file history (because to tie the per-file histories back together in a logical sequence, you need the global history to sort it again!)

So:

 - git is "slow" on single-file things, because such things DON'T EVEN 
   EXIST in git!

   When you do "git log <path-limiter>", itreally always ends up being a 
   full git log.

 - but this is fundamentally what allows you to track multiple directories 
   well. It's what makes things like "gitk drivers/scsi/" actually work, 
   where you really can see the history for a random *collection* of 
   files. Nobody else can do it, afaik, and git just considers a single 
   filename to be a case of the "random collection of files".

The example I gave to corecode was to do

	gitk builtin-rev-list.c
	gitk builtin-rev-parse.c
	gitk builtin-rev-parse.c builtin-rev-list.c

adn realize that doing the history for two files together is NOT AT ALL EQUIVALENT to doing the history for those files individually and stitching it together.

(The reason the above is a great example is that both of the files alone have a very simple linear history, but when you look at the *combined* history you actually see concurrent development, and merges: you see merge commits that simply don't "exist" when only looking at the history of one of them separately).

Show 7 quoted lines

> Is there some other concept or command that git offers? I'm in the situation
> where I know that 'tr' in coreutils version 5.2.1 had a certain bug and
> version 6.4 does not have the bug, and I want to review all commits that
> are relevant to this. I know that the only changes in tr.c are relevant
> for this, and I'm interested in a display of the minimum amount of relevant
> commit messages. If "git log" is not the right command for this question,
> which command is it?

Do

	git log v5.2.1..v6.4 -- tr.c

(or whatever your tag-names for releases are) where you can limit the log generation cost by giving the beginning commit. But yeah, it *will* look at the whole history in between, so if there is a long long history between v5.2.1 and v6.4, you'll still end up using reasonable amounts of CPU.

Show 6 quoted lines

> > Probably the mmap() problem. Does it go away when you use git 1.5.0-rc4?
> 
> No, it became even worse: git-1.5.0-rc4 is twice as slow as git-1.4.4 for
> this command:
>   git-1.4.4: 25 seconds real time, 24 seconds of CPU time (12 user, 12 system)
>   git-1.5.0: 50 seconds real time, 39 seconds of CPU time (20 user, 19 system)

That's an interesting fact in itself. Do you have the repo available somewhere?

Yes, some of the operations can be improved upon by not wasting quite so much time uncompressing stuff, so we could at least help this a bit. But that's a long-term thing. The slowdown is bad, and that probably has some simple explanation.

		Linus