Re: Bringing a bit more sanity to $GIT_DIR/objects/info/alternates?

From: Michael Haggerty <mhagger@alum.mit.edu>
Date: Aug 5, 2012, 09:38 UTC
Message-ID: <501E3F04.4050902@alum.mit.edu>
In-Reply-To: <7vmx2a3pif.fsf@alter.siamese.dyndns.org>

On 08/05/2012 06:56 AM, Junio C Hamano wrote:

Show 5 quoted lines

> The "alternates" mechanism [...]
> The UI for this mechanism however has some room for improvement, and
> we may want to start improving it for the next release after the
> upcoming Git 1.7.12 (or even Git 2.0 if the change is a large one
> that may be backward incompatible but gives us a vast improvement).

> Here are some random thoughts as a discussion starter. [...]

[...]

Show 8 quoted lines

>     - Make the distinction between a regular repository and an object
>       store that is meant to be used for object sharing stronger.
>
>       Perhaps a configuration item "core.objectstore = readonly" can
>       be introduced, and we forbid "clone -s" from pointing at a
>       repository without such a configuration.  We also forbid object
>       pruning operations such as "gc" and "repack" from being run in
>       a repository marked as such.

Must the repository necessarily be "readonly"? It seems that it would be permissible to push new objects to such a repository; just not to delete existing objects. Thus maybe another term would be better to describe such a repository, like "appendonly" or "noprune" or even something more abstract like "donor".

I have some other crazy ideas for making the concept even more powerful:

* Support remote alternate repositories.  Local repository obtains 
missing objects from the remote as needed.  This would probably be 
insanely inefficient without also supporting...

* Lazy copying of "borrowed" objects to the local repository.  Any 
object fetched from the alternate object store is copied to the local 
object store.

Together, I think that these two features would give fully-functional shallow clones.

Such alternates could even be chained together: for example, keep a single local lazy clone of the upstream repository somewhere on your site or on your computer, and use that as read-through cache for other clones.

* To help manage local disk space, allow intelligent curation of the 
objects kept in the local store when they are also available in the 
alternate.  The criteria for what to keep could be things like 
"revisions with depth <= 20 on branches X, Y/*, and Z"; "objects that 
have been accessed within the last 3 months", "all tag objects 
refs/tags/release-*".  It should be possible to cull objects not meeting 
the criteria with or without actively fetching all objects meeting the 
criteria.  Probably the criteria would be stored in the configuration to 
be reused (and perhaps run as part of "git gc").

This would cure a lot of "storing big, non-deltaable files" pain because big blobs could be stored on a central server without multiplying the size of every clone.

Michael

-- 
Michael Haggerty
mhagger@alum.mit.edu