Re: Poor performance of git describe in big repos

From: Alex Bennée <kernel-hacker@bennee.com>
Date: Jun 3, 2013, 08:39 UTC
Message-ID: <CAJ-05NO2reGkboet1c2kYy0Y7xzkb9K45mTdCLq_AU7dp1OTNw@mail.gmail.com>
In-Reply-To: <20130531161710.GB1365@sigill.intra.peff.net>

On 31 May 2013 17:17, Jeff King <peff@peff.net> wrote:

Show 26 quoted lines

> On Fri, May 31, 2013 at 12:27:11PM +0200, Thomas Rast wrote:
>
>> Thomas Rast <trast@inf.ethz.ch> writes:
>>
>> > However, if that turns out to be the culprit, it's not fixable
>> > currently[1].  Having commits with insanely long messages is just, well,
>> > insane.
>> >
>> > [1]  unless we do a major rework of the loading infrastructure, so that
>> > we can teach it to load only the beginning of a commit as long as we are
>> > only interested in parents and such
>>
>> Actually, Peff, doesn't your commit parent/tree pointer caching give us
>> this for free?
>
> It does. You can test it from the "jk/metapacks" branch at
> git://github.com/peff/git. After building, you'd need to do:
>
>   $ git gc
>   $ git metapack --all --commits
>
> in the target repository. You can check that it's working because "git
> rev-list --all --count" should be an order of magnitude faster. You may
> need to add "save_commit_buffer = 0" in any commands you are checking,
> though, as the optimization can only kick in if parse_commit does not
> want to save the buffer as a side effect.

Is this a command line argument? The tools don't seem to think so.

Anyway it seems to make a marginal difference to my case:

09:08 ajb@sloy/x86_64 [work.git] >time git --no-pager describe --long --tags ajb-build-test-5225-2-gdc0b771

real 0m14.105s user 0m12.409s sys 0m1.660s 09:11 ajb@sloy/x86_64 [work.git] >git gc Counting objects: 399436, done. Delta compression using up to 4 threads. Compressing objects: 100% (110874/110874), done. Writing objects: 100% (399436/399436), done. Total 399436 (delta 281538), reused 398357 (delta 280493) Checking connectivity: 399436, done. 09:12 ajb@sloy/x86_64 [work.git] >git metapack --all --commits 09:13 ajb@sloy/x86_64 [work.git] >time git --no-pager describe --long --tags ajb-build-test-5225-2-gdc0b771

real 0m12.781s user 0m11.669s sys 0m1.080s 09:32 ajb@sloy/x86_64 [work.git] >time git --no-pager describe --long --tags ajb-build-test-5225-2-gdc0b771

real 0m12.768s user 0m11.817s sys 0m0.908s 09:33 ajb@sloy/x86_64 [work.git] >time git --no-pager describe --long --tags ajb-build-test-5225-2-gdc0b771

real 0m12.642s user 0m11.705s sys 0m0.904s

Show 7 quoted lines

>
> I also looked into trying to just read the beginning part of a commit[1],
> but it turned out not to be all that much of an improvement.
>
> -Peff
>
> [1] http://article.gmane.org/gmane.comp.version-control.git/212301

-- 
Alex, homepage: http://www.bennee.com/~alex/