Re: Efficiency of initial clone from server
- From
Jon Smirl <jonsmirl@gmail.com>
- Date
- Feb 12, 2007, 05:17 UTC
- Message-ID
- <9e4733910702112117o53630946ja50572c7c7f2b2c1@mail.gmail.com>
- In-Reply-To
- <20070212051108.GB699@spearce.org>
On 2/12/07, Shawn O. Pearce <spearce@spearce.org> wrote:
Show 31 quoted lines
> Jon Smirl <jonsmirl@gmail.com> wrote: > > But pack to the original point, can't the server check and see if it > > has write access so that it can keep the fully packed tree? I've just > > caused kernel.org to needlessly repack the wireless-dev tree a dozen > > times playing with this clone command. If it didn't have to keep > > repacking for the clone, clone would be a lot faster. > > We probably could. > > I have actually been thinking about another problem that is > somewhat related. We cannot put more than 4 GiB of data into a > single packfile, due to the current index size limitation, or more > than 2^32-1 objects into one packfile, due to the header nr_objects > field size. > > Right now we are sending a single packfile down to the client, > even if the remote server end has the repository broken down into > a couple of packfiles (such as "really old historical stuff" and > "active stuff from this year"). If we could send more than one > packfile to the client in a single stream, we could still keep the > file size limitations. > > We can also avoid this huge repack case on the server. Because it > could just send all of the packfiles that it already has, followed > by whatever is loose which wasn't in a prior packfile. And no > write access required. > > Of course, we still could do the optimization of caching the > packfile, but I'm not sure how well that would work on kernel.org, > as I understand the trees are owned by the devs which created them > while the git daemon is probably not running as their UNIX user.
I didn't want to cache the packfile, instead I wanted to repack the repository and then copy the resulting pack file down the wire. A clone would just be a trigger to make sure everything in the repo was packed (maybe into multiple packs) before starting to send anything. Doing it this way means that everyone benefits from the packing.
> > -- > Shawn. >
-- Jon Smirl jonsmirl@gmail.com