From: Jon Smirl Date: Mon, 12 Feb 2007 05:17:19 GMT Subject: Re: Efficiency of initial clone from server Message-ID: <9e4733910702112117o53630946ja50572c7c7f2b2c1@mail.gmail.com> In-Reply-To: <20070212051108.GB699@spearce.org> On 2/12/07, Shawn O. Pearce wrote: > Jon Smirl wrote: > > But pack to the original point, can't the server check and see if it > > has write access so that it can keep the fully packed tree? I've just > > caused kernel.org to needlessly repack the wireless-dev tree a dozen > > times playing with this clone command. If it didn't have to keep > > repacking for the clone, clone would be a lot faster. > > We probably could. > > I have actually been thinking about another problem that is > somewhat related. We cannot put more than 4 GiB of data into a > single packfile, due to the current index size limitation, or more > than 2^32-1 objects into one packfile, due to the header nr_objects > field size. > > Right now we are sending a single packfile down to the client, > even if the remote server end has the repository broken down into > a couple of packfiles (such as "really old historical stuff" and > "active stuff from this year"). If we could send more than one > packfile to the client in a single stream, we could still keep the > file size limitations. > > We can also avoid this huge repack case on the server. Because it > could just send all of the packfiles that it already has, followed > by whatever is loose which wasn't in a prior packfile. And no > write access required. > > Of course, we still could do the optimization of caching the > packfile, but I'm not sure how well that would work on kernel.org, > as I understand the trees are owned by the devs which created them > while the git daemon is probably not running as their UNIX user. I didn't want to cache the packfile, instead I wanted to repack the repository and then copy the resulting pack file down the wire. A clone would just be a trigger to make sure everything in the repo was packed (maybe into multiple packs) before starting to send anything. Doing it this way means that everyone benefits from the packing. > > -- > Shawn. > -- Jon Smirl jonsmirl@gmail.com