Re: RFC: Flat directory for notes, or fan-out? Both!

From: Sam Vilain <sam@vilain.net>
Date: Feb 11, 2009, 05:05 UTC
Message-ID: <49925CAB.505@vilain.net>
In-Reply-To: <alpine.LFD.2.00.0902101953170.3590@localhost.localdomain>

Linus Torvalds wrote:

Show 17 quoted lines

>> The only case where it hurts is when you want to merge. Nothing else
>> should care. So, if a merge of these note trees sees two different trie
>> sizes then it can convert the shorter one to the longer length first,
>> and then try the merge again. So you get the pain, but only once. And
>> when a project decides that its split is too small, it can split then
>> and it should "silently" spread out to others.
>>     
>
> But what's the advantage of the added complexity?
>
> The non-fixed trie only helps for the case that doesn't matter - just a 
> few annotations. If you have a thousand annotations or less, you _really_ 
> don't care. Whatever you do will be fine.
>
> So the whole thing only matters once you have tens of thousands of 
> entries, and then you do want to have fan-out. No?
>

Yeah. I see your point and you may be right, that a 12/28 split hurts no-one, if we take this to the benchmarks. There's certainly savings in terms of total object count for the small users by using a smaller split.

I just already wrote the code to handle an arbitrary split for the features written so far[1]. If *I* can write it, in C, it means it must not be that complicated ;-)

So it comes down to how complicated things are when merging happens. If 12 is fixed in stone this is simple, because there are no chances for discrepancies. But refs/notes/commits still needs special treatment to be fetched, because it is not under refs/heads/* and you wouldn't normally have a working tree to resolve conflicts.

So I think probably the most productive thing to do is for me to write the code to handle the merge as I described above, once the code to handle pulling in - and merging - notes at 'git fetch' time is written. Then we can see whether it's that much of a complication.

To bench this we need the current builtin-log implementation to be re-written to be lazy. Which means we can't put it in the next release unless someone writes that. However my proposal means that we can release as we are and not care, and let some code - which I hope I have shown isn't *that* complicated, really - deal with it in a later release, and not break backwards compatibility.

Sam.

1. see message <1233455960.17688.122.camel@maia.lan>