Re: git annotate runs out of memory

From: Pierre Habouzit <madcoder@debian.org>
Date: Dec 11, 2007, 19:34 UTC
Message-ID: <20071211193407.GC20644@artemis.madism.org>
In-Reply-To: <4aca3dc20712111109y5d74a292rf29be6308932393c@mail.gmail.com>

On Tue, Dec 11, 2007 at 07:09:03PM +0000, Daniel Berlin wrote:

Show 22 quoted lines

> On 12/11/07, Linus Torvalds <torvalds@linux-foundation.org> wrote:
> >
> >
> > On Tue, 11 Dec 2007, Daniel Berlin wrote:
> > >
> > > This seems to be a common problem with git. It seems to use a lot of
> > > memory to perform common operations on the gcc repository (even though
> > > it is faster in some cases than hg).
> >
> > The thing is, git has a very different notion of "common operations" than
> > you do.
> >
> > To git, "git annotate" is just about the *last* thing you ever want to do.
> > It's not a common operation, it's a "last resort" operation. In git, the
> > whole workflow is designed for "git log -p <pathnamepattern>" rather than
> > annotate/blame.
> >
> I understand this, and completely agree with you.
> However, I cannot force GCC people to adopt completely new workflow in
> this regard.
> The changelog's are not useful enough (and we've had huge fights over
> this) to do git log -p and figure out the info we want.

> Looking through thousands of diffs to find the one that happened to
> your line is also pretty annoying.

  If the question you want to answer is "what happened to that line"
then using git annotate is using a big hammer for no good reason.

git log -S'<put the content of the line here>' -- path/to/file.c

will give you the very same answer, pointing you to the changes that added or removed that line directly. It's not a fast command either, but it should be less resource hungry than annotate that has to do roughly the same for all lines whereas you're interested in one only.

The direct plus here, is that git log output is incremental, so you have answers about the first diffs quite quick, which let you examine the first answers while the rest is still being computed.

Unlike git annotate, this also allow you to restrict the revisions where it searches to a range where you know this happened, which makes it almost instantaneous in most cases.

Of course, if the line is ' free(p);\n' then you will probably have quite a few false positives, but with the path restriction, I assume this will still be quite accurate.

What is important here is to know what is the real question the GCC programmers want to answer to. It seems to me that `blame` is an overkill for the underlying issue.

Note that it does not justifies the current memory consumption that just looks bad and wrong to me, but this aims at finding a way to answer your question doing just what you need to answer it and not gazillions of other things :)

-- 
·O·  Pierre Habouzit
··O                                                madcoder@debian.org
OOO                                                http://www.madism.org