Re: [kernel.org users] Re: auto-packing on kernel.org? please?

From: Daniel Barkalow <barkalow@iabervon.org>
Date: Oct 17, 2005, 19:13 UTC
Message-ID: <Pine.LNX.4.63.0510171348370.23242@iabervon.org>
In-Reply-To: <20051016213341.GF5509@reactrix.com>

On Sun, 16 Oct 2005, Nick Hengeveld wrote:

Show 10 quoted lines

> On Sun, Oct 16, 2005 at 09:56:49AM -0700, Junio C Hamano wrote:
> 
> > That's what the .idx file is for, except that after you fetch
> > the range, you may find you would need something else that the
> > object is delta against.
> 
> Would it make sense to load the pack indexes for each base up front,
> and then fetch individual objects from a pack if they exist in one of
> a base's pack indexes?  In such a case, it may not even make sense to
> try fetching the object directly first.

At the start, you have the option of either fetching the list of packs or the object. There are three cases:

 1) the object isn't available separately; we need to fetch the list of 
    packs to find it in a pack.
 2) there aren't any new packs; we need to fetch the object individually.
 3) the object is present both individually and in a pack.

(2) is more common than (1), because we don't repack every update. (3) doesn't happen at all, currently, because we prune after packing. So it makes most sense to try the object at once.

On the other hand, the parallel code should probably do both at the same time, since it can, and it only causes notable latency, not bandwidth. We probably also ought to speculatively get any new index files in parallel with whatever else we're doing, since it is likely that we'll need some pack at some point, and then we'll need all the index files to decide what pack to get.

> What are the circumstances under which it makes more sense to fetch the
> whole pack rather than fetching individual objects from it?

I'm not sure there's a good way of deciding without a plan for what conditions cause there to be a choice.

	-Daniel
*This .sig left intentionally blank*