Re: Git's database structure

From: Julian Phillips <julian@quantumfyre.co.uk>
Date: Sep 5, 2007, 15:54 UTC
Message-ID: <Pine.LNX.4.64.0709051648400.3189@reaper.quantumfyre.co.uk>
In-Reply-To: <9e4733910709050837o61a2dedfpc5f72a239b1cb8e3@mail.gmail.com>

On Wed, 5 Sep 2007, Jon Smirl wrote:

Show 12 quoted lines

> On 9/5/07, Andreas Ericsson <ae@op5.se> wrote:
>> Jon Smirl wrote:
>>>
>>> The path name field needs to be moved back into the blobs to support
>>> alternative indexes. For example I want an index on the Signed-off-by
>>> field. I use this index to give me the SHAs for the blobs
>>> Signed-off-by a particular person. In the current design I have no way
>>> of recovering the path name for these blobs other than a brute force
>>> search following every path looking for the right SHA.
>>>
>>
>> Ah, there we go. A use-case at last :)

But not a brilliant one. You sign off on commits not blobs. So you go from the sign-off to paths, then to blobs. There is no need to go from blob to path unless you deliberately introduce such a need.

Show 13 quoted lines

>>
>> So now we have a concrete problem that we can formulate thus:
>> "How can one create a database listing the relationship between 'signers'
>> and blobs?"
>>
>> So the second question: Do you seriously argue that git should take a
>> huge performance loss on its common operations to accommodate a need that
>> I suspect very few people have?
>
> Why do you keep jumping to a performance loss? Both schemes will have
> an index based on paths. The problem is how those indexes are
> constructed, not the existence of the index. Moving the paths into the
> blobs in no way prevents you from creating an index on that field.

But moving the path into the blob _IS_ the perfomance hit. You lose the ability to tell the two files have the same content _without even looking at the blob_. This is one of the core parts of making git operations blindingly fast. You can't throw that out, and then say that there is no performance hit.

You keep talking about abstract database performance - but git is not an abstract database. It has very specific common usage patterns, and is optomisied to handle them.

Show 8 quoted lines

>
> The problem is that the SHAs have been intertwined with the tree
> nodes. This blending has made it impossible to create other indexes on
> the blobs.
>
> The path index in the flat scheme will probably look just like tree
> nodes do today but these new tree nodes won't be intertwined with the
> SHAs.

And you will have to prove that diff/merge etc. don't become very much slower before you get buy in.

-- 
Julian

  ---
Many receive advice, few profit by it.
 		-- Publilius Syrus