[RFC] Referring to a submodule state recorded in a supermodule from within the submodule

Hi,

I have a stand-alone project, "foo", that I work on myself. The "foo" project is included as a submodule in two other projects, "bar" and "baz", that I don't have any direct affiliation with.

Semi-regularly, I like to keep tabs on bar and baz, to see what versions of foo they are using, what changes they have made to foo, and if there are things I could pick up from them, or maybe even things they could learn from eachother.

Doing this currently is quite tedious:
1. Clone/Fetch bar and initialize/update its foo submodule
2. Clone/Fetch baz and initialize/update its foo submodule
3. Set up remotes bar_foo and baz_foo in my main foo repo,
   pointing to bar/foo and baz/foo, respectively. Fetch.
4. Create tags bar_foo_current and baz_foo_current pointing
   to the foo SHA1 sum recorded in baz and baz, respectively.
5. Start comparing bar_foo_current and baz_foo_current to
   eachother, and to my own master branch.

Now, bring this into a larger company setting. There are many (~100) different foo-type projects owned by (~50) different developers. These projects are included as submodules by several (~20) different bar/baz-type projects. In addition, each bar/baz-type project has multiple (~3) branches using various versions of the foo-type submodules.

Finally, throw in questions like "What are the differences in submodule FOO between branch X of BAR, and branch Y of BAZ?", for random values of FOO, X, BAR, Y, and BAZ. It is apparent that the above approach just doesn't cut it.

Ironically, these questions are fairly easily answered by our current 
CVS setup (which applies the supermodule tags/branches to the entire 
source tree (including submodules)):
1. cd FOO
2. cvs diff -r BAR_X -r BAZ_Y

What I'd like, is some way to refer to the state of a repo as specified by the appropriate submodule blob in a superproject commit. Ideally this should be done remotely as well, so that I don't have to clone the entire superproject just to get at the appropriate submodule blob.

Crude Proposal:

Define a new "git submodule" subcommand that takes three arguments:
   <superURL> <tree-ish> <submoduleName>
The command does the following steps:
1. Locate git repo at <superURL>
2. Resolve <tree-ish> to the tree object within the git repo in #1
3. Locate .gitmodules within the tree object in #2
4. Lookup <submoduleName> in .gitmodules to find its path and URL
5. Locate the submodule blob from the path in #4 within the tree from #2
6. Record the object name (SHA1) stored in the submodule blob from #4
7. Fetch the object name (#7) from the submodule repo (its URL was
   found in #4) into the local git repo
8. Store a reference to the fetched object

Given the above command, the tedious steps described at the top can be 
reduced to the following (which is reasonably close to the equivalent 
CVS commands):
1. git submodule magic-command url/to/bar master foo
2. git submodule magic-command url/to/baz master foo
3. Start comparing the fetched refs to eachother, and
   to my own master branch.

If this sounds reasonable, I'd be happy to start coding the above proposal.

Have fun! :)

...Johan

Johan Herland <johan@herland.net> writes:

If I am reading you right and you are only interested in the part "foo" in these projects, there is something wrong with the setup of "bar" and "baz".

The submodule mechanism is designed to bind an independent project on its own as a subdirectory of another project. It seems to me that the problem is that "bar" and "baz" projects do not give direct access to clone "foo" part of them for you or other people.

On Wednesday 05 November 2008, Junio C Hamano wrote:

No, they do. I can clone foo directly from bar/baz's server. The problem is that I cannot quickly get at which revision of foo is used by bar/baz without tedious cloning and interacting with the superrepos.

I basically want to quickly answer questions like "What are the differences between bar's foo and baz's foo (and my own foo)?"

...Johan

On Wed, 5 Nov 2008, Johan Herland wrote:

Set up remotes in your foo:

[remote "bar"]
	url = ...
	fetch = refs/heads/*:refs/remotes/bar/*
[remote "bar-foo"]
	url = ...
	fetch = refs/heads/*:refs/remotes/in-bar/*

You should be able to do "git fetch bar" (you really don't want to merge this, but git is fine with unrelated histories in the same repo...); "git fetch bar-foo".

Then I'd expect "git log master..bar/master:foo" to work [*1*].

bar/master is the commit in the superproject, "foo" is the path in its tree that reaches the submodule, so "bar/master:foo" is whatever's there, which turns out to be a commit, which is the commit you're interested in.

I think the only annoying things are that you need to find out from bar/master:.gitmodules what to fetch as the branch that's related to your local project, and you need to do two fetches each time, and you have to download the whole superproject state in order to find out one reference from it. (But this also lets you look at the history of which versions they incorporated over time, so you can see "they adopted something at one point, rolled back for a while, cherry-picked a change, and then updated to a version with the feature fixed.")

[*1*] This may not actually work; I don't know if we give up on interpreting a object name when we're looking for a commit and start to see a path. I'm pretty sure that git will not understand "refs/remotes/bar/master:path/to/foo:Makefile" (which could name the Makefile in the root directory of the commit in the path/to/foo location in the current value of bar/master, but will actually look for a "foo:Makefile" filename; it probably ought to work with a '/' before the second ':', though, since that is unambiguous)

	-Daniel
*This .sig left intentionally blank*