Re: Composing git repositories

From: Ramkumar Ramachandra <artagnon@gmail.com>
Date: Mar 27, 2013, 11:49 UTC
Message-ID: <CALkWK0kNH2A4eLML22RTofarR3MB++OECiNXMi-bWLLMWK1GAg@mail.gmail.com>
In-Reply-To: <7vmwtqt8rs.fsf@alter.siamese.dyndns.org>

Junio C Hamano wrote:

Show 7 quoted lines

> So you have to stash it somewhere.  We could have made it to move
> them to $HOME/.safeplace or somewhere totally unrelated to the
> superproject.  So in that sense, the repositories are *not* owned by
> the superproject in any way.  However, you are working within the
> context of the superproject on the submodule after all, and
> somewhere under $GIT_DIR/ of the superorject is not too wrong a
> place to use as such a safe place.

Thanks for the explanation. The paths in .git/modules are unnecessary ugly and unwieldy, especially in the case of multiple levels of nesting: I'll look into converting it to a flat structure using <repository name>.git, while handling name conflicts. I'll also look into adding a feature to relocate this/ using an object store from an existing clone.

> Look for floating submodules in the list archive.

The most relevant message thread I could find was [1], back from 2011.
 You argue that floating submodules are Wrong, and that there is no
real usecase for it.

Like I explained earlier, I'm looking at one tool that solves a superset of the problems mr, repo, submodules, subtrees, and other tools solve. I really like the way repo allows me to work on a meta project like Android or ChromiumOS, but hate that it allows for zero composition.

To move forward, I have the following design thoughts (elaborating on my previous email):

1. If .gitmodules is tracked like a normal file, it is absolutely
impossible to tell the possible dependencies of the superproject
without cloning it entirely, and looking at the .gitmodules file in
each of the branches.  Can't we have it as a special ref instead, so I
can `git fetch` just that ref to figure out the dependencies?

2. True composition requires that I be able to specify the entire
manifest (for nested submodules) in the toplevel .gitmodules, or break
it up as I see fit.  This is currently impossible, and brings us back
to #1: the manifest for b/, b/c are in the toplevel repo's special
ref, and I need to fetch c's special ref to figure out what d is (or
error out if no such thing exists).

3. True floating submodules are impossible, because a change in the
submodule means a change in the commit object referenced by the
superproject's tree object; diff-tree will see that some content has
changed in the repository.  We can represent that diff however we want
(using diff.submodule), but we can't change the fact that the change
has to be committed.  Fixing this will require nothing short of
introducing a new kind of object (say "submodule" object which can be
a concrete SHA-1 or point to a ref).

Do you think thinking about these things is worthwhile? I see more complaints like [2] as git adoption in the industry increases, but we have no solution: we can't make git scale to super-large repositories, and we have no real way to compose smaller repositories. Hacks like repo sadden me.

[1]: http://thread.gmane.org/gmane.comp.version-control.git/185164 [2]: http://thread.gmane.org/gmane.comp.version-control.git/189776