git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: Poor performance of git describe in big repos

From
ABAlex Bennée <kernel-hacker@bennee.com>
Date
May 31, 2013, 08:49 UTC
Message-ID
<CAJ-05NNgVPukJchskVv9oL7-9p+txC0g_SXfHne-mwF327Q_3Q@mail.gmail.com>
In-Reply-To
<20130531083252.GA1072@serenity.lan>
On 31 May 2013 09:32, John Keeping <john@keeping.me.uk> wrote:
Show 24 quoted lines
> On Fri, May 31, 2013 at 09:14:49AM +0100, Alex Bennée wrote:
>> On 30 May 2013 20:30, John Keeping <john@keeping.me.uk> wrote:
>> > On Thu, May 30, 2013 at 06:21:55PM +0200, Thomas Rast wrote:
>> >> Alex Bennée <kernel-hacker@bennee.com> writes:
>> >>
>> >> > On 30 May 2013 16:33, Thomas Rast <trast@inf.ethz.ch> wrote:
>> >> >> Alex Bennée <kernel-hacker@bennee.com> writes:
>> > <snip>
>> >> > Will it be loading the blob for every commit it traverses or just ones that hit
>> >> > a tag? Why does it need to load the blob at all? Surely the commit
>> >> > tree state doesn't
>> >> > need to be walked down?
>> >>
>> >> No, my theory is that you tagged *the blobs*.  Git supports this.
>>
>> Wait is this the difference between annotated and non-annotated tags?
>> I thought a non-annotated just acted like references to a particular
>> tree state?
>
> No, this is something slightly different.  In Git there are four types
> of object: tag, commit, tree and blob.  When you have a heavyweight tag,
> the tag reference points at a tag object (which in turn points at
> another object).  With a lightweight tag, the tag reference typically
> points at a commit object.
I think this is the case in my repo.
Show 7 quoted lines
> However, there is no restriction that says that a tag object must point
> to a commit or that a lightweight tag must point at a commit - it is
> equally possible to point directly at a tree or a blob (although a lot
> less common).
>
> Thomas is suggesting that you might have a tag that does not point at a
> commit but instead points to a blob object.

It's looking like I just have some very heavy commits. One data point I probably should have mentioned at the beginning is this was a converted CVS repo and I'm wondering if some of the artifacts that introduced has contributed to this.

Show 13 quoted lines
>> > You can see if that is the case by doing something like this:
>> >
>> >     eval $(git for-each-ref --shell --format '
>> >         test $(git cat-file -t %(objectname)^{}) = commit ||
>> >         echo %(refname);')
>> >
>> > That will print out the name of any ref that doesn't point at a
>> > commit.
>>
>> Hmm that didn't seem to work.
>
> You mean there was no output?  In that case it's likely that all your
> references do indeed point at commits.
Correct.
Show 14 quoted lines
>
>>                               But looking at the output by hand I
>> certainly have a mix of tags that are commits vs tags:
>>
>>
>> 09:08 ajb@sloy/x86_64 [work.git] >git for-each-ref | grep "refs/tags"
>> | grep "commit" | wc -l
>> 1345
>> 09:12 ajb@sloy/x86_64 [work.git] >git for-each-ref | grep "refs/tags"
>> | grep -v "commit" | wc -l
>> 66
>
> This means that you have 1345 lightweight tags and 66 heavyweight tags,
> assuming that all of the lines that don't say "commit" do say "tag".
Yep all commits and tags, nothing else
> By the way, I don't remember if you said which version of Git you're
> using.  If it's an older version then it's possible that something has
> changed.
I'm running the GIT stable PPA:

09:38 ajb@sloy/x86_64 [work.git] >git --version git version 1.8.3

Although I have also tested with the latest git.git maint. I'm happy to try master if it's likely to have changed.

-- 
Alex, homepage: http://www.bennee.com/~alex/
Previous: John KeepingNext: John Keeping
Message 26 of 33 in “Poor performance of git describe in big repos”
  1. Alex BennéeMay 30, 2013
  2. Ramkumar RamachandraMay 30, 2013
  3. Alex BennéeMay 30, 2013
  4. Ramkumar RamachandraMay 30, 2013
  5. Alex BennéeMay 30, 2013
  6. Ramkumar RamachandraMay 30, 2013
  7. Thomas RastMay 30, 2013
  8. Alex BennéeMay 30, 2013
  9. Thomas RastMay 30, 2013
  10. Thomas RastMay 30, 2013
  11. Antoine PelisseMay 30, 2013
  12. John KeepingMay 30, 2013
  13. Alex BennéeMay 31, 2013
  14. Thomas RastMay 31, 2013
  15. Alex BennéeMay 31, 2013
  16. Thomas RastMay 31, 2013
  17. Alex BennéeMay 31, 2013
  18. Alex BennéeJun 3, 2013
  19. Junio C HamanoJun 3, 2013
  20. Junio C HamanoJun 3, 2013
  21. Thomas RastMay 31, 2013
  22. Jeff KingMay 31, 2013
  23. Alex BennéeJun 3, 2013
  24. Jeff KingJun 3, 2013
  25. John KeepingMay 31, 2013
  26. Alex BennéeMay 31, 2013
  27. John KeepingMay 31, 2013
  28. John KeepingMay 30, 2013
  29. Alex BennéeMay 30, 2013
  30. Duy NguyenMay 30, 2013
  31. Alex BennéeMay 30, 2013
  32. Duy NguyenMay 30, 2013
  33. Alex BennéeMay 30, 2013

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.