Re: Avery Pennarun's git-subtree?

From: Marc Branchaud <marcnarc@xiplink.com>
Date: Jul 23, 2010, 15:19 UTC
Message-ID: <4C49B31F.8000102@xiplink.com>
In-Reply-To: <AANLkTimOb2VjYI21wQsC64lm4HsVPwpRWd1twIUBnbJ3@mail.gmail.com>

On 10-07-22 03:41 PM, Avery Pennarun wrote:

> 
> 1) Sometimes I want to clone only some subdirs of a project
> 2) Sometimes I don't want the entire history because it's too big.
> 3) Super huge git repositories start to degrade in performance.

The reason we turned to submodules is precisely to deal with repository size.
 Our code base encompasses the entire FreeBSD tree plus different versions of
the Linux kernel, along with various third-party libraries & apps.  You don't
need everything to build a given product (a FreeBSD product doesn't use any
Linux kernels, for example) but because all the products share common code we
need to be able to branch and tag the common code along with the uncommon code.

So a straight "git clone" that would need to fetch all of FreeBSD plus 4 different Linux kernels and check all that out is a major problem, especially for our automated build system (which could definitely be implemented better, but still). In truth it's the checkout that takes the most time by far, though commands like git-status also take inconveniently long.

We chose git-submodule over git-subtree mainly because git-submodule lets us selectively checkout different parts of our code. (AFAIK sparse checkouts aren't yet an option.) We didn't really consider git-subtree because it's not an official part of git, and we didn't want to have to teach (and nag) all our developers to install and maintain it in addition to keeping up with git itself. Besides, git-submodule's collection-of-independent-repos model works fairly well in our situation, though the implementation could definitely be improved (and Jens's list is a really good start).

Neither submodule nor subtree really solves our situation, but right now git-submodule is the only thing "official" git offers to manage loosely-coupled code. It would be nice to see git-submodule added to the toolkit, but it would be even nicer if git had better ways to deal with "vast" repositories.

Another tool folks should keep in mind in this discussion is 'repo' which Google built for the Android project. Android's code base is also too vast to work well in a single git repository, and I don't think subtrees or submodules would be a good match for them either.

M.