Re: Calculating tree nodes

From: Junio C Hamano <gitster@pobox.com>
Date: Sep 4, 2007, 17:39 UTC
Message-ID: <7vbqcinxdb.fsf@gitster.siamese.dyndns.org>
In-Reply-To: <20070904062629.GZ18160@spearce.org>

"Shawn O. Pearce" <spearce@spearce.org> writes:

Show 8 quoted lines

> There's nothing stopping us from creating additional indexes.
> ...
> But we can also store the notes alongside the commits in the
> packfile, so that if the data for the commit has been paged in
> by the kernel then the note data is also most likely in memory,
> and if not, is in the read-ahead queue.  Clustering the notes
> alongside the commits makes access to them even faster, as we
> don't need to consult an external hash to locate the position.

I would agree with your main thrust "nobody prevents you from building additional index", but on a tangent, I am skeptical about adding too much to pack v4. Especially "clustering the notes" part.

Many operations (like "git log" that is not path limited) do not even need trees. The current packfile format has commits at the beginning without any associated trees, and they are stored in traversal order (modulo delta-base requirements can move base object earlier), which is geared toward optimizing such a common case.

Now, hopefully many operations do not need notes either, although notes themselves can store _anything_ so each of them could be large and/or each commit could have large number of them. I suspect clustering notes along with the commit they annotate would break the locality of access for common case.