[RFC] What to you think about a loose status for submodules?

Hi,

for some time now I have been thinking about submodules which are not checked out by default. So for example if you have a project which consist of the submodules:

project/core
       /help
       /app
...

Consider the situation that 'help' is really big because it is a user friendly application ;)

So you usually do not need the help folder to develop the application but it still is tied to a certain revision.

For such a workflow I would like to implement what I call 'loose' submodules. Where a

  git clone project.git
  cd project
  git submodule init && git submodule update

would omit the 'help' folder. But in case I specify it directly like

  git submodule init help

it would update to the recorded revision.

Of course the relation would be configurable. E.g.:

  git config submodule."name".relation loose

and the opposite as

  git config submodule."name".relation tight

Initially the implementation would only deal with the initialization case. As a second step I would like to deal with the situation whether a submodule should be shown as changed or not. What do you think about such an extension?

cheers Heiko

Heiko Voigt <hvoigt@hvoigt.net> writes:

I thought a blanket "submodule init/update" wasn't even a recommended practice for this exact reason. We tried to keep the default not to gratuitously populate and checkout all submodule repositories, but probably what you are trying to do was made more difficult by mistake because people who wanted the other behaviour pushed too hard?

Defaulting to "do not populate and checkout unless explicitly asked" sounds like the right thing to do, and if we broke it, it should be corrected, I think. Shouldn't it be a simple matter of teaching "--all" option to "submodule init" (and "update") to let them blindly affect all submodules, while making the default not to do anything?

I do not think this should be a project-wide configuration that is recorded in .gitmodules; if you are "help documentation" participant to the project you would want "help" submodule, and other people will want different submodules.

It would probably make more sense to introduce the notion of "module groups", similar to the way "remote update <group>" can name a group of remotes to affect. Then documentation people can say

    submodule init doc && submodule update

if .gitmodules file records the mapping from "doc" to one or more submodules (e.g. "help" and "doc"). If we are going to take this route, it would still make sense to teach "--all" to "submodule init" and perhaps default to init the "default" group if one exists, instead of making the parameterless "init" a no-op as I suggested earlier.

But it is quite a long time since I looked at git-submodule.sh so please take the above with a healthy dose of salt.

On Wed, Oct 21, 2009 at 01:23:19PM -0700, Junio C Hamano wrote:

Currently if a project has submodule its either use all submodules or none. There is nothing in between.

Not completely. As said above I want something 'in between' because some small submodule might be necessary to compile the project. And they are submodules because they are shared code, so more than one project is using them. And then there are optional submodules like pugins or documentation which are not needed for compilation.

That makes sense. If I understand it correctly the group definitions would then be stored in .gitmodules ?

Thinking further we could also name the group definition 'view'. In the notion of different kind of views for a project. That way a doc developer would get a set of options which make more sense for him (thinking about extending this to sparse checkout). But first lets consider submodules.

I like this idea.

I am not sure what is the best way to create such a group mapping using our config files though. I would like to allow multiple groups/views to have the submodules like:

[submodule "doc"]
	path = Documentation
	views = doc

[submodule "help"]
	path = help
	views = doc

[submodule "core"]
	path = core
	views = doc default

[submodule "app"]
	path = app
	views = default

What do you think?

cheers Heiko

Heiko Voigt <hvoigt@hvoigt.net> writes:

That, or alternatively:

    [submodule "doc"]
        path = Documentation

    [submodule "help"]
        path = help

    [submodule "core"]
        path = core

    [submodule "app"]
        path = app

    [view "default"]
        modules = core app

    [view "doc"]
        modules = core doc help

Or even (mimicking the way how "git remote" defines remote groups):

    [submodule "doc"]
        path = Documentation

    [submodule "help"]
        path = help

    [submodule "core"]
        path = core

    [submodule "app"]
        path = app

    [submodule]
        default = core app
        doc = core doc help

On Thu, Oct 22, 2009 at 12:58:04PM -0700, Junio C Hamano wrote:

Even though there has not been much more work on this item the idea has become much clearer for me since the last discussion. Now that sparse checkout is in master I would like to generalize this idea a little further.

How about adding this to .gitattributes so it can be used for any file/directory? E.g.:

.gitattributes: /core view=default, doc /app view=default /Documentation view=doc *.loc view=doc, default

which could then  be utilized by

  git checkout --view=doc

for example. If no view is given we default to everything.

Does the current sparse implementation work with submodule entries as well? Then it could be even more straightforward to implement than the previous idea.

cheers Heiko

Heiko Voigt <hvoigt@hvoigt.net> writes:

Selective init/update of submodules is a fundamental design element and the presense of "sparse checkout" does not affect it.

In the remainder of your message

you are talking about a change that would also apply to a project that does not have submodules at all, and you would want to involve Nguyễn in the discussion.

I have no strong opinion if attributes is the right vehicle to carry this information, except for this part:

What the project ships as grouping suggestions to cloners is merely a small part of the information on submodules (others such as URL and name are already found in .gitmodules, and .gitmodules are designed to be extensible by allowing other "config"-ish items in it). Separating only the grouping info into a different file (I don't particularly care if you use .gitattributes or something else; as long as it is not .gitmodules, the argument is the same) is not such a good idea.