Re: Resumable clone/Gittorrent (again) - stable packs?

From: Jeff King <peff@peff.net>
Date: Jan 7, 2011, 21:56 UTC
Message-ID: <20110107215631.GA10343@sigill.intra.peff.net>
In-Reply-To: <20110107214501.GA29959@LK-Perkele-VI.localdomain>

On Fri, Jan 07, 2011 at 11:45:01PM +0200, Ilari Liusvaara wrote:

Show 8 quoted lines

> > I'm not clear in your example what ~/repositories/linux-2.6 is. Is it a
> > repo? In that case, isn't that basically the same as --reference? Or is
> > it a local mirror list?
> 
> Yes, it is a repo. No, it isn't the same as --reference. It is list
> of mirrors to try first before connecting to final repository and can
> be any type of repository URL (local, true smart transport, smart HTTP,
> dumb HTTP, etc...)

OK, I understand what you mean. I was thrown off by your example using a local repository (in which case you probably would want --reference to save disk space, unless the burden of alternates management is too much).

So yeah, I think we are on the same page, except that you were proposing to pass the mirror directly, and I was proposing passing a mirror file which would contain a list of mirrors. Yours is much simpler and would probably be what people want most of the time.

Show 8 quoted lines

> > If the latter, then yeah, I think it is a good idea. Clients should
> > definitely be able to ignore, override, or add to mirror lists provided
> > by servers. The server can provide hints about useful mirrors, but it is
> > up to the client to decide which methods are useful to it and which
> > mirrors are closest.
> 
> This is essentially adding mirrors to mirror list (modulo that mirrors
> are not assumed to be complete).

I think there should always be an assumption that mirrors are not necessarily complete. That is necessary for bundle-like mirrors to be feasible, since updating the bundle for every commit defeats the purpose.

It would be nice for there to be a way for some mirrors to be marked as "should be considered complete and authoritative", since we can optimize out the final check of the master in that case (as well as for future fetches). But that's a future feature. My plan was to leave space in the mirror list for arbitrary metadata of that sort.

Show 14 quoted lines

> > Of course there are some servers who will want to do more than hint
> > (e.g., the gentoo case where they really don't want people cloning from
> > the main machine). For those cases, though, I think it is best to
> > provide the hint and to reject clients who don't follow it (e.g., by
> > barfing on somebody who tries to do a full clone). You have to implement
> > that rejection layer anyway for older clients.
> 
> With option like this, a client could do:
> 
> git clone --use-mirror=http://git.example.org/base/foo git://git.example.org/foo
> 
> To first grab stuff via HTTP (well-packed dumb HTTP is very light on the
> server) and then continue via git:// (now much cheaper because client is
> relatively up to date).

Yes, exactly.

-Peff