Re: Creating objects manually and repack

From: Rogan Dawes <discard@dawes.za.net>
Date: Aug 4, 2006, 16:39 UTC
Message-ID: <44D37845.5010009@dawes.za.net>
In-Reply-To: <9e4733910608040841v7f4f27efra63e5ead2656e07@mail.gmail.com>

Jon Smirl wrote:

Show 14 quoted lines

> On 8/4/06, Linus Torvalds <torvalds@osdl.org> wrote:
>> I'd suggest against it, but you can (and should) just repack often enough
>> that you shouldn't ever have gigabytes of objects "in flight". I'd have
>> expected that with a repack every few ten thousand files, and most files
>> being on the order of a few kB, you'd have been more than ok, but
>> especially if you have large files, you may want to make things "every 
>> <n>
>> bytes" rather than "every <n> files".
> 
> How about forking off a pack-objects and handing it one file name at a
> time over a pipe. When I hand it the next file name I delete the first
> file. Does pack-objects make multiple passes over the files? This
> model would let me hand it all 1M files.
>

I'd imagine that this would not necessarily save you a lot, if you have to write it to disk, and then read it back again. Your only chance here is if you stay in the buffer, and avoid actually writing to disk at all.

Of course, using a ramdisk/tmpfs for your object directories might be enough to save you. Just use a symlink to tmpfs for the objects directory, and leave the pack files on persistent storage.

That doesn't answer your question about how many passes pack-objects does. Nicholas Pitre should be able to answer that.

Rogan