Re: [PATCH] multi item packed files

From: Linus Torvalds <torvalds@osdl.org>
Date: Apr 22, 2005, 16:22 UTC
Message-ID: <Pine.LNX.4.58.0504220916060.2344@ppc970.osdl.org>
In-Reply-To: <200504212016.16729.mason@suse.com>

On Thu, 21 Apr 2005, Chris Mason wrote:

> 
> We can sort by the files before reading them in, but even if we order things 
> perfectly, we're spreading the io out too much across the drive.

No we don't.

It's easy to just copy the repository in a way where this just isn't true: you sort the objects by how far they are from the current HEAD, and you just copy the repository in that order ("furthest" objects first - commits last).

That's what I meant by defragmentation - you can actually do this on your own, even if your filesystem doesn't support it.

Do it twice a year, and I pretty much guarantee that your performance will stay pretty constant over time. The one exception is fsck, which doesn't seek in "history order".

And this works exactly because: 
 - we don't do no steenking delta's, and don't have deep "chains" of data 
   to follow. The longest chain we ever have is just a few deep, and it's 
   trivial to just encourage the filesystem to have recent things together.
 - we have an append-only mentality.

In fact, it works for exactly the same reason that makes us able to drop old history if we want to. We essentially "drop" the history to another part of the disk.

		Linus