Re: Git blame performance on files with a lot of history

From: Clement Moyroud <clement.moyroud@gmail.com>
Date: Dec 17, 2018, 20:43 UTC
Message-ID: <CABXAcUyiJOcZwjSyvCKJC0vKDRbSP21sY_Fa7-WfmQHsczYbzQ@mail.gmail.com>
In-Reply-To: <CAGyf7-EZaiVBnoBfRJXAK7KdaXzPaDFVocUrB8LQNU_Hi16u7g@mail.gmail.com>

did hOn Fri, Dec 14, 2018 at 11:10 AM Bryan Turner <bturner@atlassian.com> wrote:

Show 7 quoted lines

>
> After you converted the repository from CVS to Git, did you run a manual repack?
>
> The process of converting a repository from another SCM often results
> in poor delta chain selections which result in a repository that's
> unnecessarily large on disk, and/or performs quite slowly.
>

Yep I did a repack, using 'git repack -A -d --pack-kept-objects'. On NFS it'd be even worse, because of all the small objects Git would have to go through.

Show 7 quoted lines

> Something like `git repack -Adf --depth=50 --window=200` discards the
> existing delta chains and chooses new ones, and may result in
> significantly improved performance. A smaller depth, like --depth=20,
> might result in even more performance improvement, but may also make
> the repository larger on disk; you'll need to find the balance that
> works for you.
>

I re-ran with 'git repack -Adf --depth=20 --window=200' and that did
help quite a bit:
  > time git blame master --  important/file.C > /tmp/foo
  Blaming lines: 100% (33179/33179), done.
  git blame master -- important/file.C > /tmp/foo 50.70s user 0.55s
system 99% cpu 51.298 total

That's roughly a 3x improvement, great. I just need another 10x one and we'll be in business :) Based on experiments with the Bloom filter, maybe that'll help enough to get our users on board.

Cheers,

Clément