tracking submodules out of main directory.

> We can obviously solve this by doing trees of submodules just reflecting
> the trees of dependency but it create somme problems.
>
>   1. In project 4 I have 2 times project 1 and 3 times libraries 1 and 2
>      And 2 times library 4.
>   2. It is a wast of space.
>   3. Different version of the same libraries or projects could be used.
>   4. when linking object, multiples objects will export the same symbols

> One possible working tree organization may look like this:
> 
> 	-+- lib1
>          +- project1/Makefile -- refers to ../lib1
>          +- project2/Makefile -- refers to ../lib1

> An interesting point your situation raises is that there is no direct way
> to express module dependencies in .gitmodules file right now, I think.
> Ideally you would want "submodule init project1" to infer automatically
> that project1 needs lib1 and run "submodule init lib1" for you. My gut
> feeling is that it belongs to .gitmodules of the superproject

> Am 27.06.2011 18:51, schrieb Junio C Hamano:
> > One possible working tree organization may look like this:
> > 
> > 	-+- lib1
> >          +- project1/Makefile -- refers to ../lib1
> >          +- project2/Makefile -- refers to ../lib1
> 
> This is what we do at work and it works really well for us. The
> possible downside (that you can't tie project1 and project2 to a
> specific version of lib1 in their own repo) is not a real problem
> in our experience, as the superproject ties the correct combination
> together.
> 

> Le lundi 27 juin 2011 à 20:14 +0200, Jens Lehmann a écrit :
>> Am 27.06.2011 18:51, schrieb Junio C Hamano:
>>> One possible working tree organization may look like this:
>>>
>>> 	-+- lib1
>>>          +- project1/Makefile -- refers to ../lib1
>>>          +- project2/Makefile -- refers to ../lib1
>>
>> This is what we do at work and it works really well for us. The
>> possible downside (that you can't tie project1 and project2 to a
>> specific version of lib1 in their own repo) is not a real problem
>> in our experience, as the superproject ties the correct combination
>> together.
>>
> 
> That is a good starting point.
> I have used this method for some little projects and it work great.
> 
> But I had never find a mean to handle the case of project1 depending on
> both lib1 and lib2.

> Am 27.06.2011 20:52, schrieb henri GEIST:
> > Le lundi 27 juin 2011 à 20:14 +0200, Jens Lehmann a écrit :
> >> Am 27.06.2011 18:51, schrieb Junio C Hamano:
> >>> One possible working tree organization may look like this:
> >>>
> >>> 	-+- lib1
> >>>          +- project1/Makefile -- refers to ../lib1
> >>>          +- project2/Makefile -- refers to ../lib1
> >>
> >> This is what we do at work and it works really well for us. The
> >> possible downside (that you can't tie project1 and project2 to a
> >> specific version of lib1 in their own repo) is not a real problem
> >> in our experience, as the superproject ties the correct combination
> >> together.
> >>
> > 
> > That is a good starting point.
> > I have used this method for some little projects and it work great.
> > 
> > But I had never find a mean to handle the case of project1 depending on
> > both lib1 and lib2.
> 
> Hmm, but isn't that covered by having lib1, lib2 & project1 right next
> to each other in your superproject?
> 

> Am 27.06.2011 18:51, schrieb Junio C Hamano:
>> One possible working tree organization may look like this:
>> 
>> 	-+- lib1
>>          +- project1/Makefile -- refers to ../lib1
>>          +- project2/Makefile -- refers to ../lib1
> ...
>> An interesting point your situation raises is that there is no direct way
>> to express module dependencies in .gitmodules file right now, I think.
>> Ideally you would want "submodule init project1" to infer automatically
>> that project1 needs lib1 and run "submodule init lib1" for you. My gut
>> feeling is that it belongs to .gitmodules of the superproject
>
> That is where this is handled now, but having a submodule refer to a
> submodule outside of it as a dependency is an interesting thought. But
> as that only matters at the moment you add project1 (and it won't compile
> because ../lib1 is missing, which can easily handled by: "oh, then I have
> to add lib1 as a submodule to the superproject too"), ...

> Jens Lehmann <Jens.Lehmann@web.de> writes:
> 
>> Am 27.06.2011 18:51, schrieb Junio C Hamano:
>>> One possible working tree organization may look like this:
>>>
>>> 	-+- lib1
>>>          +- project1/Makefile -- refers to ../lib1
>>>          +- project2/Makefile -- refers to ../lib1
>> ...
>>> An interesting point your situation raises is that there is no direct way
>>> to express module dependencies in .gitmodules file right now, I think.
>>> Ideally you would want "submodule init project1" to infer automatically
>>> that project1 needs lib1 and run "submodule init lib1" for you. My gut
>>> feeling is that it belongs to .gitmodules of the superproject
>>
>> That is where this is handled now, but having a submodule refer to a
>> submodule outside of it as a dependency is an interesting thought. But
>> as that only matters at the moment you add project1 (and it won't compile
>> because ../lib1 is missing, which can easily handled by: "oh, then I have
>> to add lib1 as a submodule to the superproject too"), ...
> 
> That is what I called "there is no direct way". Wouldn't it be nicer if
> the .gitmodules file in the superproject said something like
> 
> 	[module "project one"]
> 		path = project1
>         	url = ...
>                 depends = lib1
> 	[module "lib1"]
>         	path = lib1
>                 url = ...
> 
> and then "git submodule init project1" run by the end user implied running
> also "git submodule init lib1"?

> Am 27.06.2011 21:05, schrieb Junio C Hamano:
> > Jens Lehmann <Jens.Lehmann@web.de> writes:
> > 
> >> Am 27.06.2011 18:51, schrieb Junio C Hamano:
> >>> One possible working tree organization may look like this:
> >>>
> >>> 	-+- lib1
> >>>          +- project1/Makefile -- refers to ../lib1
> >>>          +- project2/Makefile -- refers to ../lib1
> >> ...
> >>> An interesting point your situation raises is that there is no direct way
> >>> to express module dependencies in .gitmodules file right now, I think.
> >>> Ideally you would want "submodule init project1" to infer automatically
> >>> that project1 needs lib1 and run "submodule init lib1" for you. My gut
> >>> feeling is that it belongs to .gitmodules of the superproject
> >>
> >> That is where this is handled now, but having a submodule refer to a
> >> submodule outside of it as a dependency is an interesting thought. But
> >> as that only matters at the moment you add project1 (and it won't compile
> >> because ../lib1 is missing, which can easily handled by: "oh, then I have
> >> to add lib1 as a submodule to the superproject too"), ...
> > 
> > That is what I called "there is no direct way". Wouldn't it be nicer if
> > the .gitmodules file in the superproject said something like
> > 
> > 	[module "project one"]
> > 		path = project1
> >         	url = ...
> >                 depends = lib1
> > 	[module "lib1"]
> >         	path = lib1
> >                 url = ...
> > 
> > and then "git submodule init project1" run by the end user implied running
> > also "git submodule init lib1"?
> 
> And if lib1 happens to have another dependency, that will be initialized
> too? That would make life much easier for users who only want certain
> submodules populated to work on, as they won't have to chase compile
> errors anymore until they found all necessary submodules ... very nice.
> 

> Le lundi 27 juin 2011 à 21:40 +0200, Jens Lehmann a écrit :
>> Am 27.06.2011 21:05, schrieb Junio C Hamano:
>>> Jens Lehmann <Jens.Lehmann@web.de> writes:
>>>
>>>> Am 27.06.2011 18:51, schrieb Junio C Hamano:
>>>>> One possible working tree organization may look like this:
>>>>>
>>>>> 	-+- lib1
>>>>>          +- project1/Makefile -- refers to ../lib1
>>>>>          +- project2/Makefile -- refers to ../lib1
>>>> ...
>>>>> An interesting point your situation raises is that there is no direct way
>>>>> to express module dependencies in .gitmodules file right now, I think.
>>>>> Ideally you would want "submodule init project1" to infer automatically
>>>>> that project1 needs lib1 and run "submodule init lib1" for you. My gut
>>>>> feeling is that it belongs to .gitmodules of the superproject
>>>>
>>>> That is where this is handled now, but having a submodule refer to a
>>>> submodule outside of it as a dependency is an interesting thought. But
>>>> as that only matters at the moment you add project1 (and it won't compile
>>>> because ../lib1 is missing, which can easily handled by: "oh, then I have
>>>> to add lib1 as a submodule to the superproject too"), ...
>>>
>>> That is what I called "there is no direct way". Wouldn't it be nicer if
>>> the .gitmodules file in the superproject said something like
>>>
>>> 	[module "project one"]
>>> 		path = project1
>>>         	url = ...
>>>                 depends = lib1
>>> 	[module "lib1"]
>>>         	path = lib1
>>>                 url = ...
>>>
>>> and then "git submodule init project1" run by the end user implied running
>>> also "git submodule init lib1"?
>>
>> And if lib1 happens to have another dependency, that will be initialized
>> too? That would make life much easier for users who only want certain
>> submodules populated to work on, as they won't have to chase compile
>> errors anymore until they found all necessary submodules ... very nice.
>>
> 
> That is right.
> But could also be done with the .gitmodules in project1 containing
> 
> 	[module "lib1"]
>         	path = ../lib1
>                 url = ...
> 
> and making implicite if the module is describe in .gitmodules that
> mean ./ depend on it.

> >>>
> >>> That is what I called "there is no direct way". Wouldn't it be nicer if
> >>> the .gitmodules file in the superproject said something like
> >>>
> >>> 	[module "project one"]
> >>> 		path = project1
> >>>         	url = ...
> >>>                 depends = lib1
> >>> 	[module "lib1"]
> >>>         	path = lib1
> >>>                 url = ...
> >>>
> >>> and then "git submodule init project1" run by the end user implied running
> >>> also "git submodule init lib1"?
> >>
> >> And if lib1 happens to have another dependency, that will be initialized
> >> too? That would make life much easier for users who only want certain
> >> submodules populated to work on, as they won't have to chase compile
> >> errors anymore until they found all necessary submodules ... very nice.
> >>
> > 
> > That is right.
> > But could also be done with the .gitmodules in project1 containing
> > 
> > 	[module "lib1"]
> >         	path = ../lib1
> >                 url = ...
> > 
> > and making implicite if the module is describe in .gitmodules that
> > mean ./ depend on it.
> 
> But that would be much more complicated than just recording that in the
> superproject's .gitmodules. In a Git repo everything lives in a directory
> tree, if you want to reference something outside of that tree things will
> get very complicated ... and I'm not sure the benefit is worth the hassle.
> 

> Jens Lehmann <Jens.Lehmann@web.de> writes:
> 
> > Am 27.06.2011 18:51, schrieb Junio C Hamano:
> >> One possible working tree organization may look like this:
> >> 
> >> 	-+- lib1
> >>          +- project1/Makefile -- refers to ../lib1
> >>          +- project2/Makefile -- refers to ../lib1
> > ...
> >> An interesting point your situation raises is that there is no direct way
> >> to express module dependencies in .gitmodules file right now, I think.
> >> Ideally you would want "submodule init project1" to infer automatically
> >> that project1 needs lib1 and run "submodule init lib1" for you. My gut
> >> feeling is that it belongs to .gitmodules of the superproject
> >
> > That is where this is handled now, but having a submodule refer to a
> > submodule outside of it as a dependency is an interesting thought. But
> > as that only matters at the moment you add project1 (and it won't compile
> > because ../lib1 is missing, which can easily handled by: "oh, then I have
> > to add lib1 as a submodule to the superproject too"), ...
> 
> That is what I called "there is no direct way". Wouldn't it be nicer if
> the .gitmodules file in the superproject said something like
> 
> 	[module "project one"]
> 		path = project1
>         	url = ...
>                 depends = lib1
> 	[module "lib1"]
>         	path = lib1
>                 url = ...
> 
> and then "git submodule init project1" run by the end user implied running
> also "git submodule init lib1"?
> 
> 

> Le lundi 27 juin 2011 à 12:05 -0700, Junio C Hamano a écrit :
>> Jens Lehmann <Jens.Lehmann@web.de> writes:
>>
>>> Am 27.06.2011 18:51, schrieb Junio C Hamano:
>>>> One possible working tree organization may look like this:
>>>>
>>>> 	-+- lib1
>>>>          +- project1/Makefile -- refers to ../lib1
>>>>          +- project2/Makefile -- refers to ../lib1
>>> ...
>>>> An interesting point your situation raises is that there is no direct way
>>>> to express module dependencies in .gitmodules file right now, I think.
>>>> Ideally you would want "submodule init project1" to infer automatically
>>>> that project1 needs lib1 and run "submodule init lib1" for you. My gut
>>>> feeling is that it belongs to .gitmodules of the superproject
>>>
>>> That is where this is handled now, but having a submodule refer to a
>>> submodule outside of it as a dependency is an interesting thought. But
>>> as that only matters at the moment you add project1 (and it won't compile
>>> because ../lib1 is missing, which can easily handled by: "oh, then I have
>>> to add lib1 as a submodule to the superproject too"), ...
>>
>> That is what I called "there is no direct way". Wouldn't it be nicer if
>> the .gitmodules file in the superproject said something like
>>
>> 	[module "project one"]
>> 		path = project1
>>         	url = ...
>>                 depends = lib1
>> 	[module "lib1"]
>>         	path = lib1
>>                 url = ...
>>
>> and then "git submodule init project1" run by the end user implied running
>> also "git submodule init lib1"?
>>
>>
> 
> This could be a way if .gitmodules can contain something like
> 
> 	[module "project one"]
> 		path = project1
>         	url = ...
>                 depends = lib1	"123456"
> 	[module "project two"]
> 		path = project2
>         	url = ...
>                 depends = lib1	"abcdef"
> 	[module "lib1"]
>         	path = lib1
>                 url = ...

>> This could be a way if .gitmodules can contain something like
>>
>> 	[module "project one"]
>> 		path = project1
>>         	url = ...
>>                 depends = lib1	"123456"
>> 	[module "project two"]
>> 		path = project2
>>         	url = ...
>>                 depends = lib1	"abcdef"
>> 	[module "lib1"]
>>         	path = lib1
>>                 url = ...
> 
> But there is no need for that as "123456" and "abcdef" are already present
> as the sha1 values for the gitlinks "lib1" and "lib2".

> Am 27.06.2011 23:51, schrieb henri GEIST:
> > Le lundi 27 juin 2011 à 12:05 -0700, Junio C Hamano a écrit :
> >> Jens Lehmann <Jens.Lehmann@web.de> writes:
> >>
> >>> Am 27.06.2011 18:51, schrieb Junio C Hamano:
> >>>> One possible working tree organization may look like this:
> >>>>
> >>>> 	-+- lib1
> >>>>          +- project1/Makefile -- refers to ../lib1
> >>>>          +- project2/Makefile -- refers to ../lib1
> >>> ...
> >>>> An interesting point your situation raises is that there is no direct way
> >>>> to express module dependencies in .gitmodules file right now, I think.
> >>>> Ideally you would want "submodule init project1" to infer automatically
> >>>> that project1 needs lib1 and run "submodule init lib1" for you. My gut
> >>>> feeling is that it belongs to .gitmodules of the superproject
> >>>
> >>> That is where this is handled now, but having a submodule refer to a
> >>> submodule outside of it as a dependency is an interesting thought. But
> >>> as that only matters at the moment you add project1 (and it won't compile
> >>> because ../lib1 is missing, which can easily handled by: "oh, then I have
> >>> to add lib1 as a submodule to the superproject too"), ...
> >>
> >> That is what I called "there is no direct way". Wouldn't it be nicer if
> >> the .gitmodules file in the superproject said something like
> >>
> >> 	[module "project one"]
> >> 		path = project1
> >>         	url = ...
> >>                 depends = lib1
> >> 	[module "lib1"]
> >>         	path = lib1
> >>                 url = ...
> >>
> >> and then "git submodule init project1" run by the end user implied running
> >> also "git submodule init lib1"?
> >>
> >>
> > 
> > This could be a way if .gitmodules can contain something like
> > 
> > 	[module "project one"]
> > 		path = project1
> >         	url = ...
> >                 depends = lib1	"123456"
> > 	[module "project two"]
> > 		path = project2
> >         	url = ...
> >                 depends = lib1	"abcdef"
> > 	[module "lib1"]
> >         	path = lib1
> >                 url = ...
> 
> But there is no need for that as "123456" and "abcdef" are already present
> as the sha1 values for the gitlinks "lib1" and "lib2".
> 

> That is what I called "there is no direct way". Wouldn't it be nicer if
> the .gitmodules file in the superproject said something like
>
>        [module "project one"]
>                path = project1
>                url = ...
>                depends = lib1
>        [module "lib1"]
>                path = lib1
>                url = ...
>
> and then "git submodule init project1" run by the end user implied running
> also "git submodule init lib1"?

> On 27 June 2011 22:05, Junio C Hamano <gitster@pobox.com> wrote:
>> That is what I called "there is no direct way". Wouldn't it be nicer if
>> the .gitmodules file in the superproject said something like
>>
>>        [module "project one"]
>>                path = project1
>>                url = ...
>>                depends = lib1
>>        [module "lib1"]
>>                path = lib1
>>                url = ...
>>
>> and then "git submodule init project1" run by the end user implied running
>> also "git submodule init lib1"?
> 
> This is a very nice idea. In my workflow, I find it that I more often
> need to clone a repo _including_ its submodules, because the top-level
> project won't compile without them. If we had a way to specify
> dependencies on submodules, `git pull` could automatically init and
> update them.

> If a user really wants to clone only the top-level repo without
> submodules, git could provide him with an option for `git pull` (like
> `git pull --shallow`) to do just that. I think this second scenario is
> less common, so it is more reasonable to have a '--shallow' option for
> it, instead of '--recursive' counterpart.

> > This pach :
> <snip>
> >   - Still forbids to add anything else.
> 
> Why? If you let submodules live outside the tree I don't see any reason why
> regular files shouldn't live there too (Disclaimer: I d not think that would
> be a good idea either ;-).
> 

> What you want looks like this:
> 
> -+- lib1    #registered as submodule of project1 *and* project2 but not here
>  +- project1            # submodule of the superproject
>  |  +- ../lib1
>  +- project2            # submodule of the superproject
>     +- ../lib1
> 

> You are opening a can of worms by having two different repos point to the same
> submodule living in a third repo (which also happens to be their superproject
> and must somehow ignore it). You'll have two SHA1s for a single submodule;
> "git submodule foreach --recursive" will have interesting results too; and so
> on. Not good.
> 

> Le jeudi 28 juillet 2011 à 18:48 +0200, Jens Lehmann a écrit :
>> Am 28.07.2011 10:57, schrieb henri GEIST:
>>> In the current code it was not possible to add a gitlink to a repository
>>> outside of the main repository.
> 
> Of course that is why I have done the patch.
> 
>> Me thinks this is a *feature* this patch removes (as I understand it it was
>> a major design decision that everything /inside/ a directory is controlled
>> by git).
> 
> That is still the case.

>>> This pach :
>> <snip>
>>>   - Still forbids to add anything else.
>>
>> Why? If you let submodules live outside the tree I don't see any reason why
>> regular files shouldn't live there too (Disclaimer: I d not think that would
>> be a good idea either ;-).
> 
> there is regular file in the submodule repository but they are controled
> by the submodule itself. Then the main repository do not need to handle
> them.
> 
> Just like you I think that regular file should not be touche outside of
> the repository by git.

>> What you want looks like this:
>>
>> -+- lib1    #registered as submodule of project1 *and* project2 but not here
>>  +- project1            # submodule of the superproject
>>  |  +- ../lib1
>>  +- project2            # submodule of the superproject
>>     +- ../lib1
> 
> In fact no.
> What I want is to have this:
> 
> -+- lib1
>  +- project1
>  +- project2
> 
> With :
>  - lib1 not knowing anything about projects 1 & 2
>  - project1 not knowing anything about project2 and vice et versa
>  - project1 knowing that it need lib1 in version N.
>  - project2 knowing that it need lib1 in version M.

> I know that means a conflict in the required lib version but it is
> required by the independence of projects 1 & 2 which do not have to know
> what happen in the other one.
> 
> And in fact it is just what I want, it enable me if I decide to work on
> an optional "BigProject" depending on both project 1 & 2.
> 
> Then If lib1 is in version M:
>  - a git status in project2 will say nothing
>  - a git status in project1 will say
>    "modified:   ../lib1 (modified content)
>  - a git status in BigProject will say
>    "modified:   ../project1 (modified content)
> 
> Then I know that I need to update project1 to work with the last version
> M of lib1.

>> You are opening a can of worms by having two different repos point to the same
>> submodule living in a third repo (which also happens to be their superproject
>> and must somehow ignore it). You'll have two SHA1s for a single submodule;
>> "git submodule foreach --recursive" will have interesting results too; and so
>> on. Not good.
> 
> As I just said before it is my purpose to do it like that.

> Let say a concret exemple
> 
> 3 different teams work on libtiff, libpng, and libjpeg they are totally
> unrelated.
> 
> One more team is working on the "gimp". And they need those 3 libs in
> specific versions not necessarily there heads.
> 
> One other unrelated team is working on "gqview" and need the same libs
> in other specifics versions (Why should they know what te gimp team
> does)
> 
> Neither "gimp" and "gqview" project will contain directory with those
> libs inside. They just depend on them.
> 
> And the last team work on the gnome project which need the "gimp" and
> "gqview". It will be this team witch have to care about having both
> "gimp" and "gqview" sharing the same libs version>
> And has well the gnome project will not contain "gqview" and "gimp" in
> its own tree.
> It will also depend on them.

>> What about solving that with a "ln -s ../lib1" in "project1" and "project2"
>> (you seem to need that for your build environment) and adding the submodule
>> "lib1" to the superproject just like "project1" and "project2"?
>>
> 
> For my build environment I do not use simlinks.
> I use tu put :
> 
> #include "../lib1/lib1.h"
> in project1/project1.c
> 
> And even if I do not face this problem myself, simlinks do not work so
> well on Windows.

> Am 29.07.2011 11:39, schrieb henri GEIST:
> > Le jeudi 28 juillet 2011 à 18:48 +0200, Jens Lehmann a écrit :
> >> Am 28.07.2011 10:57, schrieb henri GEIST:
> > 
> > It is not a matter of disabling any control of git in its own
> > repository.
> > It is just a matter of adding inside the git repository a reference
> > (dependency) to an other git repository.
> 
> ... which you want to have *outside* of the containing repository!

> >>> This pach :
> >> <snip>
> >>>   - Still forbids to add anything else.
> >>
> >> Why? If you let submodules live outside the tree I don't see any reason why
> >> regular files shouldn't live there too (Disclaimer: I d not think that would
> >> be a good idea either ;-).
> > 
> > there is regular file in the submodule repository but they are controled
> > by the submodule itself. Then the main repository do not need to handle
> > them.
> > 
> > Just like you I think that regular file should not be touche outside of
> > the repository by git.
> 
> No, I think we should /either/ allow gitlinks *and* files to live outside of
> the repository /or/ *none* of them, as they have a lot in common (and I'm
> working on making them to behave as similar as possible). And in the use case
> you describe here it is totally irrelevant if you dependency consists of a
> directory tree in a submodule (= a bunch of files) or just a single file (say
> a header containing project wide definitions).
> 

> > Because in this case it is not just a reference that is managed but the
> > file itself. And this way there is a risk to overwrite some data not
> > under revision control outside of the repository.
> 
> You have the same risk when a gitlink points outside, as a submodule is a
> way of controlling a bunch of files through that reference. And the file
> would be under version control in the repository where it is registered, no?
> 

> > And in fact it is just what I want, it enable me if I decide to work on
> > an optional "BigProject" depending on both project 1 & 2.
> > 
> > Then If lib1 is in version M:
> >  - a git status in project2 will say nothing
> >  - a git status in project1 will say
> >    "modified:   ../lib1 (modified content)
> >  - a git status in BigProject will say
> >    "modified:   ../project1 (modified content)
> > 
> > Then I know that I need to update project1 to work with the last version
> > M of lib1.
> 
> Maybe no update for project1 is needed, because M only contains a bugfix
> which doesn't even need a recompilation of project1. But now you need to
> add a commit to project1 nonetheless with a message like "Updated lib1
> with a bugfix which is needed by project2" which makes your idea of
> independent submodules break down.
> 

> >> You are opening a can of worms by having two different repos point to the same
> >> submodule living in a third repo (which also happens to be their superproject
> >> and must somehow ignore it). You'll have two SHA1s for a single submodule;
> >> "git submodule foreach --recursive" will have interesting results too; and so
> >> on. Not good.
> > 
> > As I just said before it is my purpose to do it like that.
> 
> I understood that, but what are you proposing to do to solve all the
> problems your approach introduces? You can't just hand wave them away.
> 

> > Let say a concret exemple
> > 
> > 3 different teams work on libtiff, libpng, and libjpeg they are totally
> > unrelated.
> > 
> > One more team is working on the "gimp". And they need those 3 libs in
> > specific versions not necessarily there heads.
> > 
> > One other unrelated team is working on "gqview" and need the same libs
> > in other specifics versions (Why should they know what te gimp team
> > does)
> > 
> > Neither "gimp" and "gqview" project will contain directory with those
> > libs inside. They just depend on them.
> > 
> > And the last team work on the gnome project which need the "gimp" and
> > "gqview". It will be this team witch have to care about having both
> > "gimp" and "gqview" sharing the same libs version>
> > And has well the gnome project will not contain "gqview" and "gimp" in
> > its own tree.
> > It will also depend on them.
> 
> Cool, that is a real life example resembling what we have a my dayjob. But
> a "gimp" and "gqview" project will only have dependencies like "use libpng
> of version 1.2.3 or newer (because we need a feature/bugfix introduced
> there)" and won't be tied to a special version of that library. This means
> they need a dependency like "SHA1 or newer" instead of "exactly this SHA1".
> 

> > It is just the same with aptitude on debian.
> > Each package know there dependency by themselves, does not contain there
> > dependencies, and do not need a bigger superpackage to tell them what
> > are there own dependencies.
> 
> And this is a very good point for the "version x.yy-z *or newer*" argument,
> they are /never/ tied to the /exact/ x.yy-z version, as that would make the
> dependencies pretty much unusable. They use a "newer than x.yy-z" scheme.
> 

> 
> > And Still I realy want to have every project knowing there own
> > dependency by themselves and not needing an external superproject to
> > tell them what they need.
> 
> I want to have that too! I'm just convinced using a gitlink to achieve that
> is wrong in so many ways. I'd rather prefer to express such dependencies in
> something like a config file, and I believe they should not be as strict as
> "I need exactly that version" but rather like "this version or newer (and by
> the way: we of course only tested that specific version ;-)". These
> dependencies could then be checked and displayed by git status.
> 

> Le samedi 30 juillet 2011 à 16:16 +0200, Jens Lehmann a écrit :
>> Am 29.07.2011 11:39, schrieb henri GEIST:
>>> Le jeudi 28 juillet 2011 à 18:48 +0200, Jens Lehmann a écrit :
>>>> Am 28.07.2011 10:57, schrieb henri GEIST:
>>>
>>> It is not a matter of disabling any control of git in its own
>>> repository.
>>> It is just a matter of adding inside the git repository a reference
>>> (dependency) to an other git repository.
>>
>> ... which you want to have *outside* of the containing repository!
> 
> yes
> 
>> That will then be registered in other git repositories too in your model,
>> which gets rid of the "one file/submodule, one repo" assumption we now have
>> and will introduce ambiguities which are *really* hard to handle.
> 
> I am sorry, I am not a native English speaker. This sentence is to
> complex for me. And google translator is of no help in this case.

> But I agree the step is really weak before enabling to put any regular
> file outside of the directory.
> I do not see any reasonable workflow (to my eyes) for it but' maybe some
> day someone will came with a justifiable workflow which need it. we will
> never know.
> 
> But in this case we need solve some questions :
>   - Will we extend git status signaling untracked files out of the
>     repository ?

>>> Because in this case it is not just a reference that is managed but the
>>> file itself. And this way there is a risk to overwrite some data not
>>> under revision control outside of the repository.
>>
>> You have the same risk when a gitlink points outside, as a submodule is a
>> way of controlling a bunch of files through that reference. And the file
>> would be under version control in the repository where it is registered, no?
> 
> I agree on this point.
> 
> But they are still confined in an another git repository not
> disseminated all over the file system.
> And it never corrupt this pointed repository. just ask it to do by it's
> own regular git commands.

> I can do a second patch to prevent git submodule command to make clones
> outside of the repository.
> It will requires the user to do those clones manually.
> In fact this is already what I do.
> My only use of this is to track dependencies.

>>> And in fact it is just what I want, it enable me if I decide to work on
>>> an optional "BigProject" depending on both project 1 & 2.
>>>
>>> Then If lib1 is in version M:
>>>  - a git status in project2 will say nothing
>>>  - a git status in project1 will say
>>>    "modified:   ../lib1 (modified content)
>>>  - a git status in BigProject will say
>>>    "modified:   ../project1 (modified content)
>>>
>>> Then I know that I need to update project1 to work with the last version
>>> M of lib1.
>>
>> Maybe no update for project1 is needed, because M only contains a bugfix
>> which doesn't even need a recompilation of project1. But now you need to
>> add a commit to project1 nonetheless with a message like "Updated lib1
>> with a bugfix which is needed by project2" which makes your idea of
>> independent submodules break down.
> 
> In fact I work ni the world of "high integrity programming" then It is
> just what I need.
> If there is a bugfix in any library, used by the program it is no more
> the same program.
> I need the "SHA1" to correspond to the exact and complete source code
> involved in my executable.
> 
> And this way the "SHA1" of the project sign the "SHA1" of the
> libraries. 

>>>> You are opening a can of worms by having two different repos point to the same
>>>> submodule living in a third repo (which also happens to be their superproject
>>>> and must somehow ignore it). You'll have two SHA1s for a single submodule;
>>>> "git submodule foreach --recursive" will have interesting results too; and so
>>>> on. Not good.
>>>
>>> As I just said before it is my purpose to do it like that.
>>
>> I understood that, but what are you proposing to do to solve all the
>> problems your approach introduces? You can't just hand wave them away.
> 
> There is some solutions :
> 
>   - First it is one more **feature** if it does not correspond to your
>     work flow it does not prevent you to work exactly the way you did
>     until now.
> 
>   - Second if you want to use the feature but not want to have the
>     conflict **feature** (for me it is one), just put the independent
>     project with there libs in different directory
> 
>       -+- foo -+- lib1     (in version N)
>        |       +- project1
>        |
>        +- bar -+- lib1     (in version M)
>                +- project2
> 
>   - Third if you really need to have project 1 & 2 in the same
>     directory foo, that means they are needed by a third BigProject in
>     the same directory foo depending on project 1 & 2.
>     And then you really need git to declare a conflict.

>>> Let say a concret exemple
>>>
>>> 3 different teams work on libtiff, libpng, and libjpeg they are totally
>>> unrelated.
>>>
>>> One more team is working on the "gimp". And they need those 3 libs in
>>> specific versions not necessarily there heads.
>>>
>>> One other unrelated team is working on "gqview" and need the same libs
>>> in other specifics versions (Why should they know what te gimp team
>>> does)
>>>
>>> Neither "gimp" and "gqview" project will contain directory with those
>>> libs inside. They just depend on them.
>>>
>>> And the last team work on the gnome project which need the "gimp" and
>>> "gqview". It will be this team witch have to care about having both
>>> "gimp" and "gqview" sharing the same libs version>
>>> And has well the gnome project will not contain "gqview" and "gimp" in
>>> its own tree.
>>> It will also depend on them.
>>
>> Cool, that is a real life example resembling what we have a my dayjob. But
>> a "gimp" and "gqview" project will only have dependencies like "use libpng
>> of version 1.2.3 or newer (because we need a feature/bugfix introduced
>> there)" and won't be tied to a special version of that library. This means
>> they need a dependency like "SHA1 or newer" instead of "exactly this SHA1".
> 
> It is useful and simpler to work like this but could introduce some
> bugs.

> The "gimp" team has tested it with libpng 1.2.3 and maybe know that it
> did not work with previous versions but if they do not have any crystal
> ball they never know if newer versions will not break something.
> In fact I doubt that the first version of gimp will work with the last
> version of libpng.

>>> It is just the same with aptitude on debian.
>>> Each package know there dependency by themselves, does not contain there
>>> dependencies, and do not need a bigger superpackage to tell them what
>>> are there own dependencies.
>>
>> And this is a very good point for the "version x.yy-z *or newer*" argument,
>> they are /never/ tied to the /exact/ x.yy-z version, as that would make the
>> dependencies pretty much unusable. They use a "newer than x.yy-z" scheme.
> 
> It is an other feature that the one I need.
> But it is a good idea.
> 
> Nothing prevent us to make a patch to add a new test in git status to
> see if the current SHA1 in the libpng repository has the SHA1 of the
> gitlink in the gimp in its ancestor.

>>> And Still I realy want to have every project knowing there own
>>> dependency by themselves and not needing an external superproject to
>>> tell them what they need.
>>
>> I want to have that too! I'm just convinced using a gitlink to achieve that
>> is wrong in so many ways. I'd rather prefer to express such dependencies in
>> something like a config file, and I believe they should not be as strict as
>> "I need exactly that version" but rather like "this version or newer (and by
>> the way: we of course only tested that specific version ;-)". These
>> dependencies could then be checked and displayed by git status.
> 
> It effectively could be in a config file it seem good to me as well.

> Am 30.07.2011 23:55, schrieb henri GEIST:
> > Le samedi 30 juillet 2011 à 16:16 +0200, Jens Lehmann a écrit :
> >> Am 29.07.2011 11:39, schrieb henri GEIST:
> >>> Le jeudi 28 juillet 2011 à 18:48 +0200, Jens Lehmann a écrit :
> >>>> Am 28.07.2011 10:57, schrieb henri GEIST:
> > 
> >> That will then be registered in other git repositories too in your model,
> >> which gets rid of the "one file/submodule, one repo" assumption we now have
> >> and will introduce ambiguities which are *really* hard to handle.
> > 
> > I am sorry, I am not a native English speaker. This sentence is to
> > complex for me. And google translator is of no help in this case.
> 
> Your proposal of letting multiple gitlinks in different repos point to the
> same submodule will break the assumption that each file is only handled by
> a single git repo.

> > But I agree the step is really weak before enabling to put any regular
> > file outside of the directory.
> > I do not see any reasonable workflow (to my eyes) for it but' maybe some
> > day someone will came with a justifiable workflow which need it. we will
> > never know.
> > 
> > But in this case we need solve some questions :
> >   - Will we extend git status signaling untracked files out of the
> >     repository ?
> 
> I don't think that would work well.

> 
> >   - What will do git-clean ? it is already dangerous inside the
> >     repository. and it will be worst if it can access outside of it.
> 
> Hopefully git clean will learn the --recurse-submodules option in the not
> too distant future, then you will have just the same danger for the files
> inside a submodule.

> > But they are still confined in an another git repository not
> > disseminated all over the file system.
> > And it never corrupt this pointed repository. just ask it to do by it's
> > own regular git commands.
> 
> The only difference here is that a submodule can contain more than one file,
> but you can corrupt those files just as easily as a single file using git
> commands.

> > I can do a second patch to prevent git submodule command to make clones
> > outside of the repository.
> > It will requires the user to do those clones manually.
> > In fact this is already what I do.
> > My only use of this is to track dependencies.
> 
> But gitlinks are more than simple dependencies, they are followed! "git
> submodule", status, diff and fetch already follow them. push is learning
> that right now. checkout, reset, merge and friends are being taught that
> too (see the enhance_submodule branch in my github repo for the current
> state). So a gitlink is more than just a simple reference, it is followed
> by a lot of commands and the submodule it points to is manipulated by
> those commands. We had a patch for "git archive --recurse-submodules" on
> the list, what will that do when used in "project1"?

> > In fact I work in the world of "high integrity programming" then It is
> > just what I need.
> > If there is a bugfix in any library, used by the program it is no more
> > the same program.
> > I need the "SHA1" to correspond to the exact and complete source code
> > involved in my executable.
> > 
> > And this way the "SHA1" of the project sign the "SHA1" of the
> > libraries. 
> 
> I cannot believe you want single commits in your "Gimp" repo for every
> combination of distributions and library versions where someone said
> "this works". This is insane and won't scale at all.
> 

> What you do is that each distribution tests their combination of programs
> and libraries and says "that works". And that is why the only sane way to
> record this "high integrity programming" test result is in the superproject
> (= distribution) and not in each of the program repositories.
> 
> I also see that it would be cool when a program could record "I do work with
> that library version, if you use another you are on your own". But it will
> never say "I only work with *this* specific library version", which is what
> your proposal is trying to do.

> >> I understood that, but what are you proposing to do to solve all the
> >> problems your approach introduces? You can't just hand wave them away.
> > 
> > There is some solutions :
> > 
> >   - First it is one more **feature** if it does not correspond to your
> >     work flow it does not prevent you to work exactly the way you did
> >     until now.
> > 
> >   - Second if you want to use the feature but not want to have the
> >     conflict **feature** (for me it is one), just put the independent
> >     project with there libs in different directory
> > 
> >       -+- foo -+- lib1     (in version N)
> >        |       +- project1
> >        |
> >        +- bar -+- lib1     (in version M)
> >                +- project2
> > 
> >   - Third if you really need to have project 1 & 2 in the same
> >     directory foo, that means they are needed by a third BigProject in
> >     the same directory foo depending on project 1 & 2.
> >     And then you really need git to declare a conflict.
> 
> No you don't. You just need to git to tell you: this is not the version I
> was tested against, repeat the tests to be sure.
> 

> >> Cool, that is a real life example resembling what we have a my dayjob. But
> >> a "gimp" and "gqview" project will only have dependencies like "use libpng
> >> of version 1.2.3 or newer (because we need a feature/bugfix introduced
> >> there)" and won't be tied to a special version of that library. This means
> >> they need a dependency like "SHA1 or newer" instead of "exactly this SHA1".
> > 
> > It is useful and simpler to work like this but could introduce some
> > bugs.
> 
> But that model is awfully successful and is used by all distributions I know,
> so I suspect it is not that dangerous (especially when you do your own QA).
> 

> > The "gimp" team has tested it with libpng 1.2.3 and maybe know that it
> > did not work with previous versions but if they do not have any crystal
> > ball they never know if newer versions will not break something.
> > In fact I doubt that the first version of gimp will work with the last
> > version of libpng.
> 
> But in the real world it is exactly like that: gimp will work with all libpng
> 1.2.3 and newer, only when libpng is updated to 2.0.0 you have to check that
> again. Of course there will be bugs in some combinations. But the advantage of
> being able to then only fix libpng and have the bug fixed in Gimp without
> having to change it is far greater than the possible problem you are describing
> here.
> 

> >>> It is just the same with aptitude on debian.
> >>> Each package know there dependency by themselves, does not contain there
> >>> dependencies, and do not need a bigger superpackage to tell them what
> >>> are there own dependencies.
> >>
> >> And this is a very good point for the "version x.yy-z *or newer*" argument,
> >> they are /never/ tied to the /exact/ x.yy-z version, as that would make the
> >> dependencies pretty much unusable. They use a "newer than x.yy-z" scheme.
> > 
> > It is an other feature that the one I need.
> > But it is a good idea.
> > 
> > Nothing prevent us to make a patch to add a new test in git status to
> > see if the current SHA1 in the libpng repository has the SHA1 of the
> > gitlink in the gimp in its ancestor.
> 
> To make that feature useful for others (e.g. at my dayjob) this would be
> necessary. And we would never want the exact SHA1 match, even though that
> information might be what others (like you) want.
> 

> > But if git handle this config file.
> > Update it on a "git add ../libpng && git commit"
> 
> I'm not sure an automatic update at "git commit" would be the right thing to
> do, as I think that should only happen after all tests have run successful,
> not at the time you commit it. But anyways, that could be done with a post
> commit hook. Or the test script can do it when it succeeded.
> 

> > And control the matching between the project and libraries on
> > "git status".
> 
> An extension to "git status" to display the dependencies that aren't met is
> a valid goal. What about starting with a script ("git depends"?) and then see
> what can go into status?
> 
> > I can not see the difference with a gitlink.
> 
> Then you can just use a config file for that, no? ;-)
> 

> Le lundi 01 août 2011 à 21:39 +0200, Jens Lehmann a écrit :
>> Am 30.07.2011 23:55, schrieb henri GEIST:
>>> I can not see the difference with a gitlink.
>>
>> Then you can just use a config file for that, no? ;-)
> 
> Off corse, I immediately start to work on it.

> Am 02.08.2011 14:19, schrieb henri GEIST:
> > Le lundi 01 ao??t 2011 ?? 21:39 +0200, Jens Lehmann a ??crit :
> >> Am 30.07.2011 23:55, schrieb henri GEIST:
> >>> I can not see the difference with a gitlink.
> >>
> >> Then you can just use a config file for that, no? ;-)
> > 
> > Off corse, I immediately start to work on it.
> 
> I'm looking forward to that!

> Hi Henri,
> 
> On Tue, Aug 02, 2011 at 08:42:24PM +0200, Jens Lehmann wrote:
> > Am 02.08.2011 14:19, schrieb henri GEIST:
> > > Le lundi 01 ao??t 2011 ?? 21:39 +0200, Jens Lehmann a ??crit :
> > >> Am 30.07.2011 23:55, schrieb henri GEIST:
> > >>> I can not see the difference with a gitlink.
> > >>
> > >> Then you can just use a config file for that, no? ;-)
> > > 
> > > Off corse, I immediately start to work on it.
> > 
> > I'm looking forward to that!
> 
> Before hacking away please share some design information about this.
> 

> Another thing:
> 
> It sounds like the workflow you want to achieve is similar to the one
> the android developers are using. They have a lot of submodules which
> need automatic updating.
> 
> The last time I checked they used repo (similar tool to submodules)
> together with gerrit.
> 
> AFAIR they wanted to switch to submodules and have gerrit automatically
> make the superproject commits for single submodule changes. Additionally
> if a change involves multiple submodules the developer should be able to
> tie them together using a superproject commit.
> 
> Have a look over there whether that workflow might help you?

> I plan to use a config file containing lines like
>
> "path_to_poited_repo   SHA1_of_intended_commit   URL_of_origin"
>
> the URL part will not be required.
>
> this file will be a list of pointer to other project.

> henri GEIST <henri.geist@flying-robots.com> writes:
> 
>> I plan to use a config file containing lines like
>>
>> "path_to_poited_repo   SHA1_of_intended_commit   URL_of_origin"
>>
>> the URL part will not be required.
>>
>> this file will be a list of pointer to other project.
> 
> I wasn't paying attention to this thread, but I have to ask "why" here.
> 
> The first two are what gitlink was designed to do in the superproject that
> ties multiple submodules together, and the last one is also supplied by
> the .gitmodules in that superproject. This seems to be adding the same
> information in a redundant way by saying "this version A0 of submodule A
> wants version B0 of submodule B and version C0 of submodule C" when the
> supermodule can say "the consistent view I record is to have version A0,
> B0 and C0 of submodules A, B and C, respectively".

> 1) To use me, you need another submodule "foo"
>
>    This is very helpful when you want to add the Gimp submodule and it
>    can tell you you'll need the libpng submodule too in your superproject
>    (but I'd vote to use the submodule name here, not the path as that
>    should be the superproject's decision).

> Jens Lehmann <Jens.Lehmann@web.de> writes:
> 
>> 1) To use me, you need another submodule "foo"
>>
>>    This is very helpful when you want to add the Gimp submodule and it
>>    can tell you you'll need the libpng submodule too in your superproject
>>    (but I'd vote to use the submodule name here, not the path as that
>>    should be the superproject's decision).
> 
> That is something you can add to .gitmodules in the superproject, no?
> E.g.
> 
> 	[submodule "Gimp"]
>         	url = http://some/where/
> 		path = gimp/
>         	depends = version 1.2.3 of "Glib"
> 
> 	[submodule "Glib"]
>         	url = http://some/where/else/
> 		path = libs/glib

>> a) Of this submodule "foo" I need at least that version because I won't
>>    compile/work with older versions of that. (this can be tightened to
>>    "exactly that version" to give henri the behavior he wants, but that
>>    should be policy, not mandatory)
> 
> The "loose collection of projects" approach like that has its uses, and it
> is called "repo". Why re-invent it? The behaviour Henri wants to specify
> the exact version is how git submodules work already, so I do not see
> there is anything to be done here.

>> That is all stuff the submodule knows better than the superproject.
> 
> Not necessarily. The version A0 of submodule A may depend on submodule B
> and may also know it must have at least version B0 of that submodule, but
> the superproject would know other constraints, e.g. the superproject
> itself also calls into submodule B and wants a newer version B1 of it.

> Am 03.08.2011 21:41, schrieb Junio C Hamano:
> > Jens Lehmann <Jens.Lehmann@web.de> writes:
> > 
> >> 1) To use me, you need another submodule "foo"
> >>
> >>    This is very helpful when you want to add the Gimp submodule and it
> >>    can tell you you'll need the libpng submodule too in your superproject
> >>    (but I'd vote to use the submodule name here, not the path as that
> >>    should be the superproject's decision).
> > 
> > That is something you can add to .gitmodules in the superproject, no?
> > E.g.
> > 
> > 	[submodule "Gimp"]
> >         	url = http://some/where/
> > 		path = gimp/
> >         	depends = version 1.2.3 of "Glib"
> > 
> > 	[submodule "Glib"]
> >         	url = http://some/where/else/
> > 		path = libs/glib
> 
> The "depends" information is not very useful inside the superproject,
> because when you already know that there you can simply commit version
> 1.2.3 of Glib together with Gimp instead of adding that information.
> That's what gitlinks are for, no?
> 
> But when you fetch a new version of Gimp into your submodule, it would be
> really nice if the superproject could be notified that the Gimp developers
> updated to 1.2.4 of Glib and inform you that an update of Glib might be
> appropriate. That could avoid having you to dig through compiler errors to
> find out that the new foobar() function from Glib 1.2.4 is needed (and if
> you need to pull in a bugfix in Glib, you might notice that *a lot* later
> when you forget to do that).
> 

> >> In addition to that, it can (but mustn't) specify any of the following:
> > 
> > I am guessing you meant "does not have to", instead of mustn't, here...
> 
> Sure, thanks for deciphering that.
> 
> >> a) Of this submodule "foo" I need at least that version because I won't
> >>    compile/work with older versions of that. (this can be tightened to
> >>    "exactly that version" to give henri the behavior he wants, but that
> >>    should be policy, not mandatory)
> > 
> > The "loose collection of projects" approach like that has its uses, and it
> > is called "repo". Why re-invent it? The behaviour Henri wants to specify
> > the exact version is how git submodules work already, so I do not see
> > there is anything to be done here.
> 
> Let me make this clear: this is not about changing how submodules are
> committed in a superproject. It is not about having a loose collection of
> projects, they stay tied together in a defined state by the superproject.
> 

> Henri wanted it a bit upside down: any submodule could request a certain
> version of another submodule somewhere else in the repo. And he wanted to
> use gitlinks from one submodule to another for that, which I - hopefully -
> convinced him was no good idea.
> 

> But I understand his need to have some kind of "version hint" from one
> submodule to another. Just that he wants to take the hint very serious,
> while I see it as means to communicate from the submodule maintainer to
> the superproject developers that another submodule might have to be
> updated too when they do that to his.
> 

> >> b) And if you don't know where to get it, use this url
> > 
> > Again that is the job of .gitmodules in the superproject.
> 
> Yes. But this idea is about how the url could get into the .gitmodules of
> the superproject in the first place. That can make it easier for the
> superproject's developer to import a submodule into his repo and much more
> important: it makes it possible to pull in submodule dependencies
> automatically e.g. when running "git submodule add --resolve-dependencies
> Gimp".

> >> That is all stuff the submodule knows better than the superproject.
> > 
> > Not necessarily. The version A0 of submodule A may depend on submodule B
> > and may also know it must have at least version B0 of that submodule, but
> > the superproject would know other constraints, e.g. the superproject
> > itself also calls into submodule B and wants a newer version B1 of it.
> 
> Right. That's what I tried to explain to Henri, the superproject ties it all
> together. But I also like his idea to add a way to communicate information
> from the submodule to the superproject, and give the superproject a choice
> if it wants to use it.
> 

> Le mercredi 03 août 2011 à 23:30 +0200, Jens Lehmann a écrit :
>> Am 03.08.2011 21:41, schrieb Junio C Hamano:
>>> Jens Lehmann <Jens.Lehmann@web.de> writes:
>> But when you fetch a new version of Gimp into your submodule, it would be
>> really nice if the superproject could be notified that the Gimp developers
>> updated to 1.2.4 of Glib and inform you that an update of Glib might be
>> appropriate. That could avoid having you to dig through compiler errors to
>> find out that the new foobar() function from Glib 1.2.4 is needed (and if
>> you need to pull in a bugfix in Glib, you might notice that *a lot* later
>> when you forget to do that).
>>
> 
> Exact, I am really happy to read this.
> And better do not bother to have the suproject.

>>>> In addition to that, it can (but mustn't) specify any of the following:
>>>
>>> I am guessing you meant "does not have to", instead of mustn't, here...
>>
>> Sure, thanks for deciphering that.
>>
>>>> a) Of this submodule "foo" I need at least that version because I won't
>>>>    compile/work with older versions of that. (this can be tightened to
>>>>    "exactly that version" to give henri the behavior he wants, but that
>>>>    should be policy, not mandatory)
>>>
>>> The "loose collection of projects" approach like that has its uses, and it
>>> is called "repo". Why re-invent it? The behaviour Henri wants to specify
>>> the exact version is how git submodules work already, so I do not see
>>> there is anything to be done here.
>>
>> Let me make this clear: this is not about changing how submodules are
>> committed in a superproject. It is not about having a loose collection of
>> projects, they stay tied together in a defined state by the superproject.
>>
> 
> Yes but for me, from when I started this this topic, it was all about
> having a loose collection of project with dependency references between
> them. And get rid of the superproject.
> It is my first and only goal.

>> Henri wanted it a bit upside down: any submodule could request a certain
>> version of another submodule somewhere else in the repo. And he wanted to
>> use gitlinks from one submodule to another for that, which I - hopefully -
>> convinced him was no good idea.
>>
> 
> You just convince me that submodules are more than I need and to make a
> lighter independent version of submodules which will never been followed
> by git commands.

>>>> b) And if you don't know where to get it, use this url
>>>
>>> Again that is the job of .gitmodules in the superproject.
>>
>> Yes. But this idea is about how the url could get into the .gitmodules of
>> the superproject in the first place. That can make it easier for the
>> superproject's developer to import a submodule into his repo and much more
>> important: it makes it possible to pull in submodule dependencies
>> automatically e.g. when running "git submodule add --resolve-dependencies
>> Gimp".
> 
> Only if you have a superproject.
> If not do the same thing from the gimp repository, now it contain all
> necessary infos to do the job.

>>>> That is all stuff the submodule knows better than the superproject.
>>>
>>> Not necessarily. The version A0 of submodule A may depend on submodule B
>>> and may also know it must have at least version B0 of that submodule, but
>>> the superproject would know other constraints, e.g. the superproject
>>> itself also calls into submodule B and wants a newer version B1 of it.
>>
>> Right. That's what I tried to explain to Henri, the superproject ties it all
>> together. But I also like his idea to add a way to communicate information
>> from the submodule to the superproject, and give the superproject a choice
>> if it wants to use it.
>>
> 
> yes but the superproject contain no code in your design.
> Then it will never need anything by itself.
> It is only a container which you will inform with data already known by
> the submodules I do not see any value to it.

> Am 04.08.2011 00:29, schrieb henri GEIST:
> > Le mercredi 03 août 2011 à 23:30 +0200, Jens Lehmann a écrit :
> >> Am 03.08.2011 21:41, schrieb Junio C Hamano:
> >>> Jens Lehmann <Jens.Lehmann@web.de> writes:
> >> But when you fetch a new version of Gimp into your submodule, it would be
> >> really nice if the superproject could be notified that the Gimp developers
> >> updated to 1.2.4 of Glib and inform you that an update of Glib might be
> >> appropriate. That could avoid having you to dig through compiler errors to
> >> find out that the new foobar() function from Glib 1.2.4 is needed (and if
> >> you need to pull in a bugfix in Glib, you might notice that *a lot* later
> >> when you forget to do that).
> >>
> > 
> > Exact, I am really happy to read this.
> > And better do not bother to have the suproject.
> 
> I don't get this, you can't get rid of the superproject.

> 
> > cd to gimp directory, type git status it can tell you every thing and
> > when your satisfied you just have to type make.
> > At this point the superproject have not any use. 
> 
> "git status" inside the submodule won't tell you anything about the
> dependencies,

> 
> >>>> In addition to that, it can (but mustn't) specify any of the following:
> >>>
> >>> I am guessing you meant "does not have to", instead of mustn't, here...
> >>
> >> Sure, thanks for deciphering that.
> >>
> >>>> a) Of this submodule "foo" I need at least that version because I won't
> >>>>    compile/work with older versions of that. (this can be tightened to
> >>>>    "exactly that version" to give henri the behavior he wants, but that
> >>>>    should be policy, not mandatory)
> >>>
> >>> The "loose collection of projects" approach like that has its uses, and it
> >>> is called "repo". Why re-invent it? The behaviour Henri wants to specify
> >>> the exact version is how git submodules work already, so I do not see
> >>> there is anything to be done here.
> >>
> >> Let me make this clear: this is not about changing how submodules are
> >> committed in a superproject. It is not about having a loose collection of
> >> projects, they stay tied together in a defined state by the superproject.
> >>
> > 
> > Yes but for me, from when I started this this topic, it was all about
> > having a loose collection of project with dependency references between
> > them. And get rid of the superproject.
> > It is my first and only goal.
> 
> But I fail to see how that would improve anything.
> 

> >> Henri wanted it a bit upside down: any submodule could request a certain
> >> version of another submodule somewhere else in the repo. And he wanted to
> >> use gitlinks from one submodule to another for that, which I - hopefully -
> >> convinced him was no good idea.
> >>
> > 
> > You just convince me that submodules are more than I need and to make a
> > lighter independent version of submodules which will never been followed
> > by git commands.
> 
> Submodules are what you need, but it's no use to implement dependencies by
> using gitlinks that point outside of their repositories.
> 

> >>>> b) And if you don't know where to get it, use this url
> >>>
> >>> Again that is the job of .gitmodules in the superproject.
> >>
> >> Yes. But this idea is about how the url could get into the .gitmodules of
> >> the superproject in the first place. That can make it easier for the
> >> superproject's developer to import a submodule into his repo and much more
> >> important: it makes it possible to pull in submodule dependencies
> >> automatically e.g. when running "git submodule add --resolve-dependencies
> >> Gimp".
> > 
> > Only if you have a superproject.
> > If not do the same thing from the gimp repository, now it contain all
> > necessary infos to do the job.
> 
> No, it doesn't. Apart from the Gimp project telling you what version it
> wants, you need to have a place to record the version that the user really
> used. And that won't work without a superproject.
> 

> >>>> That is all stuff the submodule knows better than the superproject.
> >>>
> >>> Not necessarily. The version A0 of submodule A may depend on submodule B
> >>> and may also know it must have at least version B0 of that submodule, but
> >>> the superproject would know other constraints, e.g. the superproject
> >>> itself also calls into submodule B and wants a newer version B1 of it.
> >>
> >> Right. That's what I tried to explain to Henri, the superproject ties it all
> >> together. But I also like his idea to add a way to communicate information
> >> from the submodule to the superproject, and give the superproject a choice
> >> if it wants to use it.
> >>
> > 
> > yes but the superproject contain no code in your design.
> > Then it will never need anything by itself.
> > It is only a container which you will inform with data already known by
> > the submodules I do not see any value to it.
> 
> But being the container that ties it all together is more than enough value.
> 

> Le mercredi 03 ao??t 2011 ?? 23:30 +0200, Jens Lehmann a ??crit :
> > Let me make this clear: this is not about changing how submodules are
> > committed in a superproject. It is not about having a loose collection of
> > projects, they stay tied together in a defined state by the superproject.
> > 
> 
> Yes but for me, from when I started this this topic, it was all about
> having a loose collection of project with dependency references between
> them. And get rid of the superproject.
> It is my first and only goal.

> > Henri wanted it a bit upside down: any submodule could request a certain
> > version of another submodule somewhere else in the repo. And he wanted to
> > use gitlinks from one submodule to another for that, which I - hopefully -
> > convinced him was no good idea.
> > 
> 
> You just convince me that submodules are more than I need and to make a
> lighter independent version of submodules which will never been followed
> by git commands.

> Hi,
> 
> On Thu, Aug 04, 2011 at 12:29:22AM +0200, henri GEIST wrote:
> > Le mercredi 03 ao??t 2011 ?? 23:30 +0200, Jens Lehmann a ??crit :
> > > Let me make this clear: this is not about changing how submodules are
> > > committed in a superproject. It is not about having a loose collection of
> > > projects, they stay tied together in a defined state by the superproject.
> > > 
> > 
> > Yes but for me, from when I started this this topic, it was all about
> > having a loose collection of project with dependency references between
> > them. And get rid of the superproject.
> > It is my first and only goal.
> 
> In that case maybe what you want is your own completely independent
> implementation of a tool which manages a collection of submodules.

> So if you really want to work on this I suggest
> you try to implement your solution outside of core git first. If you can
> prove that you can solve all the immanent problems we can discuss
> integrating it into git again.
> 

> > > Henri wanted it a bit upside down: any submodule could request a certain
> > > version of another submodule somewhere else in the repo. And he wanted to
> > > use gitlinks from one submodule to another for that, which I - hopefully -
> > > convinced him was no good idea.
> > > 
> > 
> > You just convince me that submodules are more than I need and to make a
> > lighter independent version of submodules which will never been followed
> > by git commands.
> 
> As described above this is probably the best way for you. Maybe you can
> prove that such a tool works but git's submodules implementation can
> currently not really assist you with your approach.

> Am 03.08.2011 19:11, schrieb Junio C Hamano:
> But the superproject is still the place to say: I know these versions of
> all submodules work together, so I commit their gitlinks here. But this
> scheme enables submodules to give hints to help the superproject's user.
> 
> > I also suspect that allowing each submodule to know and demand specific
> > versions of other submodules will lead to inconsistencies. Which version
> > of submodule C would you demand to have when submodule A wants version C0
> > and submodule B wants version C1 of it?
> 
> Right, in the discussion so far it seemed like henri seems to be the only
> user who is wanting an exact match, and he says he needs to see these
> inconsistencies. But I think he can modify the "version xxx or newer" to
> his needs without imposing these inconsistencies on users (like me) who
> don't want to see them.

> Hi,
> 
> On Wed, Aug 03, 2011 at 09:07:14PM +0200, Jens Lehmann wrote:
> > Am 03.08.2011 19:11, schrieb Junio C Hamano:
> > But the superproject is still the place to say: I know these versions of
> > all submodules work together, so I commit their gitlinks here. But this
> > scheme enables submodules to give hints to help the superproject's user.
> > 
> > > I also suspect that allowing each submodule to know and demand specific
> > > versions of other submodules will lead to inconsistencies. Which version
> > > of submodule C would you demand to have when submodule A wants version C0
> > > and submodule B wants version C1 of it?
> > 
> > Right, in the discussion so far it seemed like henri seems to be the only
> > user who is wanting an exact match, and he says he needs to see these
> > inconsistencies. But I think he can modify the "version xxx or newer" to
> > his needs without imposing these inconsistencies on users (like me) who
> > don't want to see them.
> 
> And I imagine if a submodule has such hints we could add a command say
> 
> 	git submodule resolve-dependencies
> 
> which could resolve such "I need a version newer than" hints given by a
> submodule to help the user to update a submodule in the superproject.
> 
> Disclaimer: I think we need to think about all the implications such a
> scheme introduces very carefully. The picture is still a bit blurry for
> me.
> 

> Am 03.08.2011 19:11, schrieb Junio C Hamano:
> > henri GEIST <henri.geist@flying-robots.com> writes:
> > 
> >> I plan to use a config file containing lines like
> >>
> >> "path_to_poited_repo   SHA1_of_intended_commit   URL_of_origin"
> >>
> >> the URL part will not be required.
> >>
> >> this file will be a list of pointer to other project.
> > 
> > I wasn't paying attention to this thread, but I have to ask "why" here.
> > 
> > The first two are what gitlink was designed to do in the superproject that
> > ties multiple submodules together, and the last one is also supplied by
> > the .gitmodules in that superproject. This seems to be adding the same
> > information in a redundant way by saying "this version A0 of submodule A
> > wants version B0 of submodule B and version C0 of submodule C" when the
> > supermodule can say "the consistent view I record is to have version A0,
> > B0 and C0 of submodules A, B and C, respectively".
> 
> During the discussion this evolved from a simple "I need that submodule
> with exactly this version" to something I believe is more generic and
> very useful for others. As I see it now a submodule should be able to say:
> 
> 1) To use me, you need another submodule "foo"
> 
>    This is very helpful when you want to add the Gimp submodule and it
>    can tell you you'll need the libpng submodule too in your superproject
>    (but I'd vote to use the submodule name here, not the path as that
>    should be the superproject's decision).
> 
> In addition to that, it can (but mustn't) specify any of the following:
> 
> a) Of this submodule "foo" I need at least that version because I won't
>    compile/work with older versions of that. (this can be tightened to
>    "exactly that version" to give henri the behavior he wants, but that
>    should be policy, not mandatory)
> 

>    Gimp could say it needs at least libpng 012345 because in that version
>    the function foobar() was added it now depends on. Normally this won't
>    be updated very often, but if people like henri use that to say "I'll
>    only promise to work well with that exact version, as that went through
>    extensive QA" they might change that on virtually every commit.
> 

> b) And if you don't know where to get it, use this url
> 
>    That can give the superproject a hint where it can clone that
>    repository from. That could be helpful for distributions to sort out
>    the dependencies of the packages they pull in.
> 
> That is all stuff the submodule knows better than the superproject. And
> that information can be used to *inform* the user about the submodule's
> needs, maybe using "git status --submodule-dependencies" will print:
> 
> # submodule "Gimp" requests a libpng 567890 or newer
> # submodule "foo" has missing dependency "bar"

> But the user can choose to ignore that (because he knows he has the png
> support disabled and he doesn't need the fancy help files from bar).
> 
> And maybe "git submodule add" learns an option to automatically add all
> the other submodules the new one depends on too (for that we would need
> the url).
> 

> > I also suspect that allowing each submodule to know and demand specific
> > versions of other submodules will lead to inconsistencies. Which version
> > of submodule C would you demand to have when submodule A wants version C0
> > and submodule B wants version C1 of it?
> 
> Right, in the discussion so far it seemed like henri seems to be the only
> user who is wanting an exact match, and he says he needs to see these
> inconsistencies.

> henri GEIST <henri.geist@flying-robots.com> writes:
> 
> > I plan to use a config file containing lines like
> >
> > "path_to_poited_repo   SHA1_of_intended_commit   URL_of_origin"
> >
> > the URL part will not be required.
> >
> > this file will be a list of pointer to other project.
> 
> I wasn't paying attention to this thread, but I have to ask "why" here.
> 
> The first two are what gitlink was designed to do in the superproject that
> ties multiple submodules together, and the last one is also supplied by
> the .gitmodules in that superproject.

> This seems to be adding the same information in a redundant way by saying
> "this version A0 of submodule A wants version B0 of submodule B and
> version C0 of submodule C" when the supermodule can say "the consistent
> view I record is to have version A0, B0 and C0 of submodules A, B and C,
> respectively".
> 

> I also suspect that allowing each submodule to know and demand specific
> versions of other submodules will lead to inconsistencies. Which version
> of submodule C would you demand to have when submodule A wants version C0
> and submodule B wants version C1 of it?
> 

> Let say a concret exemple
> 
> 3 different teams work on libtiff, libpng, and libjpeg they are totally
> unrelated.
> 
> One more team is working on the "gimp". And they need those 3 libs in
> specific versions not necessarily there heads.
> 
> One other unrelated team is working on "gqview" and need the same libs
> in other specifics versions (Why should they know what te gimp team
> does)
> 
> Neither "gimp" and "gqview" project will contain directory with those
> libs inside. They just depend on them.
> 
> And the last team work on the gnome project which need the "gimp" and
> "gqview". It will be this team witch have to care about having both
> "gimp" and "gqview" sharing the same libs version>
> And has well the gnome project will not contain "gqview" and "gimp" in
> its own tree.
> It will also depend on them.
> 
> It is just the same with aptitude on debian.
> Each package know there dependency by themselves, does not contain there
> dependencies, and do not need a bigger superpackage to tell them what
> are there own dependencies.

> Hi,
> 
> On Fri, Jul 29, 2011 at 11:39:37AM +0200, henri GEIST wrote:
> > Let say a concret exemple
> > 
> > 3 different teams work on libtiff, libpng, and libjpeg they are totally
> > unrelated.
> > 
> > One more team is working on the "gimp". And they need those 3 libs in
> > specific versions not necessarily there heads.
> > 
> > One other unrelated team is working on "gqview" and need the same libs
> > in other specifics versions (Why should they know what te gimp team
> > does)
> > 
> > Neither "gimp" and "gqview" project will contain directory with those
> > libs inside. They just depend on them.
> > 
> > And the last team work on the gnome project which need the "gimp" and
> > "gqview". It will be this team witch have to care about having both
> > "gimp" and "gqview" sharing the same libs version>
> > And has well the gnome project will not contain "gqview" and "gimp" in
> > its own tree.
> > It will also depend on them.
> > 
> > It is just the same with aptitude on debian.
> > Each package know there dependency by themselves, does not contain there
> > dependencies, and do not need a bigger superpackage to tell them what
> > are there own dependencies.
> 
> As Jens mentioned already in this example you have a
> 
>         somemodule A needs a version of lib C higher than X
> 	somemodule B needs a version of lib C higher than Y
> 
> relation. Which in the case of submodules is A points to X and B points
> to Y. Lets assume X is contained in Y. Since only the superproject knows
> about both A and B its the only instance that can resolve this conflict
> of dependence on C and can choose Y. In your example aptitude would be
> the superproject containing everything.
> 

> Adding the ability to point to git repositories outside of the worktree
> does not solve anything but rather creates more problems. Resolving such
> dependencies can not be achieved if only A knows that it needs version X
> and only B knows that it needs version Y.
> 

> henri GEIST <henri.geist@flying-robots.com> writes:
> 
>> We can obviously solve this by doing trees of submodules just reflecting
>> the trees of dependency but it create somme problems.
>>
>>   1. In project 4 I have 2 times project 1 and 3 times libraries 1 and 2
>>      And 2 times library 4.
>>   2. It is a wast of space.
>>   3. Different version of the same libraries or projects could be used.
>>   4. when linking object, multiples objects will export the same symbols
> 
> All of the above are something your build procedure needs to solve
> regardless, even if you are not using git submodules (or even if you are
> not using any SCM, for that matter).  If you want to (and you do want to)
> avoid duplication, version drift and associated issues of use of the
> common library 1 across project 1 and project 2, you would organize your
> source tree so that both of your (sub)projects to use the library code
> from a common location.
> 
> One possible working tree organization may look like this:
> 
> 	-+- lib1
>          +- project1/Makefile -- refers to ../lib1
>          +- project2/Makefile -- refers to ../lib1
> 

>> Anything/library_1
>> Anything/library_2
>> Anything/library_3
>> Anything/library_4
>> Anything/project_1 with a git submodule add ../library_1 ../library_2
>> Anything/project_2 with a git submodule add ../library_1 ../library_3
>> Anything/project_3 with a git submodule add ../library_2 ../library_4
>>                                             ../project_1
>> Anything/project_4 with a git submodule add ../library_4 ../project_1
>>                                             ../project_2 ../project_3

> The top-level superproject (what you called "Anything") binds project1,
> project2 and lib1 as its submodules, and it can say where you want your
> users to fetch these submodules in its .gitmodules, and at what version
> lib1 (and projects) to use in its tree objects as gitlinks.
> 

> what I really need is to do :
> 
> cd project_1
> git add ../library_1
> 
> then in case of modification in library_1
> 
> A git status in project_1 directory will say me :
> 
> modified:   ../library_1 (modified content)
> or
> modified:   ../library_1 (new commits)
> or even
> modified:   ../library_1 (new commits, modified content)
> 
> Just like it do for submodules with out "../" in the path.
> 
> And I really think the metadata to do so should be stored in project_1
> since it is him which depend on library_1 version "abcd1234..." and this
> information need to be self contained.
> May be in project_1/.git or project_1/.gitmodules
> 
> I do not see the point of having a third party project "Anything" Just
> to say to project_1 hey you need library_1.
> If it need it, it should already know it.

> Am 27.06.2011 20:40, schrieb henri GEIST:
> > what I really need is to do :
> > 
> > cd project_1
> > git add ../library_1
> > 
> > then in case of modification in library_1
> > 
> > A git status in project_1 directory will say me :
> > 
> > modified:   ../library_1 (modified content)
> > or
> > modified:   ../library_1 (new commits)
> > or even
> > modified:   ../library_1 (new commits, modified content)
> > 
> > Just like it do for submodules with out "../" in the path.
> > 
> > And I really think the metadata to do so should be stored in project_1
> > since it is him which depend on library_1 version "abcd1234..." and this
> > information need to be self contained.
> > May be in project_1/.git or project_1/.gitmodules
> > 
> > I do not see the point of having a third party project "Anything" Just
> > to say to project_1 hey you need library_1.
> > If it need it, it should already know it.
> 
> The point is it avoids ambiguity and cross-dependencies. If you have
> project_1 requesting the version "123456" of library_1 and project_2
> wants "abcdef", you'll always have one project disagreeing (= being
> dirty).

> On Tuesday, August 2, 2011, henri GEIST
> <henri.geist@flying-robots.com> wrote:
> > Le mardi 02 août 2011 à 00:12 +0200, Heiko Voigt a écrit :
> >> Adding the ability to point to git repositories outside of the
> worktree
> >> does not solve anything but rather creates more problems. Resolving
> such
> >> dependencies can not be achieved if only A knows that it needs
> version X
> >> and only B knows that it needs version Y.
> >>
> >
> > Why not it work perfect for me and for debian as well.
> 
> In the Debian analogy: A knows that it needs version X, B knows that
> it needs version Y, but A does not know that if it installs version X,
> B will break.  That's the job of the superproject (Aptitude).
>