Re: how to speed up "git log"?

From: Bruno Haible <bruno@clisp.org>
Date: Feb 12, 2007, 02:02 UTC
Message-ID: <200702120302.00576.bruno@clisp.org>
In-Reply-To: <200702120059.17676.robin.rosenberg.lists@dewire.com>

Thanks for the responses.

Robin Rosenberg wrote:

> Since you know that you are not interested in the whole history, you can limit your scan.
> 
> git log COREUTILS-5_2_1..COREUTILS-6_4 src/tr.c

Thanks, that indeed does the trick: it reduces the time from 33 sec to 11 sec.

To reduce the time even more, and to allow more flexibility among the search criteria (e.g. "I need the commits from date X to date Y, on this file set, from anyone except me"), I would need to connect git to a database. git cannot store all kinds of indices and reverse mappings to allow all kinds of queries; that's really a classical database application area.

Show 6 quoted lines

> > No, it became even worse: git-1.5.0-rc4 is twice as slow as git-1.4.4 for
> > this command:
> >   git-1.4.4: 25 seconds real time, 24 seconds of CPU time (12 user, 12 system)
> >   git-1.5.0: 50 seconds real time, 39 seconds of CPU time (20 user, 19 system)
> 
> Could the UTF-8 stuff have anything to do with this?

Actually, no. Brown paper bag on me for doing benches in different conditions. The timing difference is an effect of the buffer cache / page cache:

  - After the second repetition of the command (i.e. when all files are cached
    in RAM), the timings are
        25 seconds real time, 24 seconds of CPU time (13 user, 11 system)
    both in git-1.4.4 and -1.5.0-rc4.

  - After unmounting and remounting the disk containing the repository (i.e.
    when none of the files are cached in RAM), the timings are
        49 seconds real time, 38 seconds of CPU time (20 user, 18 system)

Sorry for the false alarm.

Bruno