Re: [RFC PATCH 0/5] line-log: towards a more responsive, incremental 'git log -L'

From: SZEDER Gábor <szeder.dev@gmail.com>
Date: Apr 24, 2020, 09:02 UTC
Message-ID: <20200424090241.GA5925@szeder.dev>
In-Reply-To: <459f2978-03f1-a58f-0dec-d0376fec5201@gmail.com>

On Thu, Apr 09, 2020 at 08:17:41AM -0400, Derrick Stolee wrote:

Show 30 quoted lines

> On 8/18/2019 2:27 PM, SZEDER Gábor wrote:
> > Line-level log performs a preprocessing step in
> > prepare_revision_walk(), during which it filters and rewrites history
> > to keep only commits modifying the given line range.  This
> > preprocessing causes significant delay before the first commit is
> > shown, wastes CPU time when the user asks only for a few commits, and
> > does parent rewriting with no way to turn it off.
> > 
> > This patch series addresses these issues by integrating line-level log
> > filtering into the revision walking machinery and making it work
> > together with generation number-based topo-ordering (though for now
> > only in the case when the user doesn't explicitly asks for parent
> > rewriting, which is probably the common case).
> > 
> > The first two patches are quite straightforward (and arguably somewhat
> > unrelated), but the rest deals with history traversal and parent
> > rewriting, which I don't usually do, hence the RFC.
> > 
> > 
> > SZEDER Gábor (5):
> >   completion: offer '--(no-)patch' among 'git log' options
> >   line-log: remove unused fields from 'struct line_log_data'
> >   t4211-line-log: add tests for parent oids
> >   line-log: more responsive, incremental 'git log -L'
> >   line-log: try to use generation number-based topo-ordering
> 
> Hi Szeder,
> 
> I was taking inventory of our issues especially around history now
> that the changed-path Bloom filters are close to wrapping up.

Well, I'm about to stir it up over the weekend...

> What's
> the status on this RFC? Looking at it now, I understand the situation
> better and could help review a bit more than before. Do you have more
> context as to the situation on this series?

Sadly, I haven't touched this patch series since then, other than rebasing it on top of new releases once or twice, but since v2.23 not even that. I think I run into some conflicts and was not in the mood to resolve them, because with a2bb801f6a (line-log: avoid unnecessary full tree diffs, 2019-08-21) the performance benefits are much lower, so it was not that pressing...

I think patch 4 in itself is not really the right way to integrate line-log into the revision walking machinery:

  - Line-log follows full-file renames, but it doesn't actually use
    '--follow', but rather implements its own logic to detect them.
    This logic is in some ways better, than '--follow', notably it can
    follow multiple paths at once, while '--follow' only allows a
    single path.
    I think the rename following logic should be extracted from
    line-log, and it should be used to implement '--follow', removing
    some of its restrictions.

  - Line-log should then be ported to use the revamped '--follow'.

  - And then it's finally time for something like that patch 4, and to
    have some "fun" with making explicitly requested parent rewriting
    work (I can only remember that whenever I tried to make that work
    my brain started to hurt :)
 
Anyway, I think the first three patches are worth having.