git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: Poor performance of git describe in big repos

From
ABAlex Bennée <kernel-hacker@bennee.com>
Date
May 30, 2013, 12:29 UTC
Message-ID
<CAJ-05NM9EhikDBP0izqWrnLbZW6RcHq_cH-20YTE08SZw5fjqA@mail.gmail.com>
In-Reply-To
<20130530114808.GD17475@serenity.lan>

The repo is a fairly hairy one as it includes two historically un-related but content related repos which I'm the process of cherry-picking stuff across.

11:58 ajb@sloy/x86_64 [work.git] >git count-objects -v count: 493 size: 4572 in-pack: 399307 packs: 1 size-pack: 1930755 prune-packable: 0 garbage: 0 size-garbage: 0

This was after a repack which did have slight negative effect on performance. The pack file is:

13:27 ajb@sloy/x86_64 [work.git] >ls -lh ./.git/objects/pack/* -r--r--r-- 1 ajb cvs 11M May 30 11:56 ./.git/objects/pack/pack-a9ba133a6f25ffa74c3c407e09ab030f8745b201.idx -r--r--r-- 1 ajb cvs 1.9G May 30 11:56 ./.git/objects/pack/pack-a9ba133a6f25ffa74c3c407e09ab030f8745b201.pack

I ran perf on it and the top items in the report where:
 41.58%   git  libcrypto.so.1.0.0  [.] 0x6ae73
 33.96%   git  libz.so.1.2.3.4     [.] 0xe0ec
 10.39%   git  libz.so.1.2.3.4     [.] adler32
  2.03%   git  [kernel.kallsyms]   [k] clear_page_c

So I'm guessing it's spending a lot of non-cache efficient time un-packing and processing the deltas?

-- Alex.

On 30 May 2013 12:48, John Keeping <john@keeping.me.uk> wrote:
Show 10 quoted lines
> On Thu, May 30, 2013 at 11:38:32AM +0100, Alex Bennée wrote:
>> One factor might be the size of my repo (.git is around 2.4G). Could
>> this just be due to computational cost of searching through large
>> packs to walk the commit chain? Is there any way to make this easier
>> for git to do?
>
> What does "git count-objects -v" say for your repository?
>
> You may find that performance improves if you repack with "git gc
> --aggressive".
-- 
Alex, homepage: http://www.bennee.com/~alex/
Previous: John KeepingNext: Duy Nguyen
Message 29 of 33 in “Poor performance of git describe in big repos”
  1. Alex BennéeMay 30, 2013
  2. Ramkumar RamachandraMay 30, 2013
  3. Alex BennéeMay 30, 2013
  4. Ramkumar RamachandraMay 30, 2013
  5. Alex BennéeMay 30, 2013
  6. Ramkumar RamachandraMay 30, 2013
  7. Thomas RastMay 30, 2013
  8. Alex BennéeMay 30, 2013
  9. Thomas RastMay 30, 2013
  10. Thomas RastMay 30, 2013
  11. Antoine PelisseMay 30, 2013
  12. John KeepingMay 30, 2013
  13. Alex BennéeMay 31, 2013
  14. Thomas RastMay 31, 2013
  15. Alex BennéeMay 31, 2013
  16. Thomas RastMay 31, 2013
  17. Alex BennéeMay 31, 2013
  18. Alex BennéeJun 3, 2013
  19. Junio C HamanoJun 3, 2013
  20. Junio C HamanoJun 3, 2013
  21. Thomas RastMay 31, 2013
  22. Jeff KingMay 31, 2013
  23. Alex BennéeJun 3, 2013
  24. Jeff KingJun 3, 2013
  25. John KeepingMay 31, 2013
  26. Alex BennéeMay 31, 2013
  27. John KeepingMay 31, 2013
  28. John KeepingMay 30, 2013
  29. Alex BennéeMay 30, 2013
  30. Duy NguyenMay 30, 2013
  31. Alex BennéeMay 30, 2013
  32. Duy NguyenMay 30, 2013
  33. Alex BennéeMay 30, 2013

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.