Re: Show of hands, how many set USE_NSEC

From: Shawn O. Pearce <spearce@spearce.org>
Date: Aug 8, 2008, 17:52 UTC
Message-ID: <20080808175247.GH9152@spearce.org>
In-Reply-To: <alpine.LFD.1.10.0808081027590.3462@nehalem.linux-foundation.org>

Linus Torvalds <torvalds@linux-foundation.org> wrote:

Show 6 quoted lines

> On Fri, 8 Aug 2008, Shawn O. Pearce wrote:
> > 
> > Really I'd just like to scrap the entire DIRC file format and do
> > it over again.  Having the flat namespace is nuts.
> 
> I really disagree, because you have no clue about performance.

Actually, I do have half a clue.  I just haven't convinced you of
that yet.  Something to aspire to.

> The flat file format is absolutely _critical_ for performance.

 
I know, I agree.

> The point is, the index file absolutely *HAS* to be a single file in order 
> to perform well in a big project. Otherwise there's no point, and you 
> might as well just use a git "tree" object for everything.

Yes, it _has_ to be a single file, at the root of the directory tree. I agree with that design decision entirely.

That single file however does not need to be structured internallyy
the way it is.

> Now, if you talk about the _sorting_ order (as opposed to the flatness of 
> the file), I could probably agree. The sort order was probably a mistake. 
> That said, we're stuck with it.  You can't change it without changing the 
> tree object format, so it's not just an "local index file" format issue.

I was talking about something like the 'TREE' extension. If we used a format such as:

	index_file:: tree
	tree:: entry_count sha1_id record*
	record:: mode pathlen path (tree | file)
	file:: sha1_id ctime mtime ....

And sorted the entries within each tree record by their path (or path+mode in "Git style") then we wind up with all records in the index file in the same order they are today, and we don't have the big redundant path strings like "src/a.c", "src/b.c", "src/c.c".

This may actually create a _smaller_ index file, resulting in a few less minor page faults as we read it in.

-- 
Shawn.