threads / discuss / 7392

Subproject status

Subject: Subproject status

## tl;dr

8 messages between Mar 25, 2007 and Mar 27, 2007.

replies: 7people: 5as markdown or json

Daniel Barkalow· Mar 25, 2007, 17:37 UTC · lore

I remember that last time I checked, there were a number of designs for subprojects and at least a couple of implementations, but none that was complete to the point of being mergeable. Are there any subproject implementations available that haven't run into unsolveable problems? I'm presently only looking at stuff that totals to a reasonable single project size and rate of growth, so I'm not worried about the large-scale storage requirement issue.

	-Daniel
*This .sig left intentionally blank*
Jakub Narebski· Mar 25, 2007, 23:34 UTC · re: Daniel Barkalow · lore

Re: Subproject status

[Cc: Daniel Barkalow <barkalow@iabervon.org>, git@vger.kernel.org]
Daniel Barkalow wrote:
Show 7 quoted lines
> I remember that last time I checked, there were a number of designs for 
> subprojects and at least a couple of implementations, but none that was 
> complete to the point of being mergeable. Are there any subproject 
> implementations available that haven't run into unsolveable problems? I'm 
> presently only looking at stuff that totals to a reasonable single project 
> size and rate of growth, so I'm not worried about the large-scale storage 
> requirement issue.
Check out http://git.or.cz/gitwiki/SubprojectSupport

Could _please_ anyone who wrote some kind of submodule support, add info about it (links to archive, to repository, perhaps to working prototype) to this page? Thanks in advance!

-- 
Jakub Narebski
Warsaw, Poland
ShadeHawk on #git
Han-Wen Nienhuys· Mar 26, 2007, 02:11 UTC · re: Jakub Narebski · lore

Re: Subproject status

Jakub Narebski wrote:
Show 13 quoted lines
> [Cc: Daniel Barkalow <barkalow@iabervon.org>, git@vger.kernel.org]
> 
> Daniel Barkalow wrote:
> 
>> I remember that last time I checked, there were a number of designs for 
>> subprojects and at least a couple of implementations, but none that was 
>> complete to the point of being mergeable. Are there any subproject 
>> implementations available that haven't run into unsolveable problems? I'm 
>> presently only looking at stuff that totals to a reasonable single project 
>> size and rate of growth, so I'm not worried about the large-scale storage 
>> requirement issue.
> 
> Check out http://git.or.cz/gitwiki/SubprojectSupport
this points to
  http://www.kernel.org/git/?p=git/git.git;a=blob;hb=todo;f=Subpro.txt
which is a broken. It results in
1Reading blob failed.
Han-Wen
Han-Wen Nienhuys· Mar 26, 2007, 04:34 UTC · re: Jakub Narebski · lore

Re: Subproject status

Jakub Narebski wrote:
Show 13 quoted lines
> [Cc: Daniel Barkalow <barkalow@iabervon.org>, git@vger.kernel.org]
> 
> Daniel Barkalow wrote:
> 
>> I remember that last time I checked, there were a number of designs for 
>> subprojects and at least a couple of implementations, but none that was 
>> complete to the point of being mergeable. Are there any subproject 
>> implementations available that haven't run into unsolveable problems? I'm 
>> presently only looking at stuff that totals to a reasonable single project 
>> size and rate of growth, so I'm not worried about the large-scale storage 
>> requirement issue.
> 
> Check out http://git.or.cz/gitwiki/SubprojectSupport
this points to
  http://www.kernel.org/git/?p=git/git.git;a=blob;hb=todo;f=Subpro.txt
which is a broken. It results in
1Reading blob failed.
Han-Wen
Jakub Narebski· Mar 26, 2007, 08:47 UTC · re: Han-Wen Nienhuys · lore

Re: Subproject status

Han-Wen Nienhuys wrote:
> Jakub Narebski wrote:
Show 11 quoted lines
>> Check out http://git.or.cz/gitwiki/SubprojectSupport
> 
> this points to
> 
>   http://www.kernel.org/git/?p=git/git.git;a=blob;hb=todo;f=Subpro.txt
> 
> which is a broken. It results in
> 
> 
> 
> 1Reading blob failed.

Thanks. Subpro.txt was removed from todo, as it described old, abandoned idea about subproject support (the "mountpoint" in commit object one).

Fixed (the link, not gitweb: I don't know where this '1' comes from).
-- 
Jakub Narebski
Warsaw, Poland
ShadeHawk on #git
Martin Waitz· Mar 26, 2007, 09:39 UTC · re: Daniel Barkalow · lore

Re: Subproject status

hoi :)
On Sun, Mar 25, 2007 at 01:37:25PM -0400, Daniel Barkalow wrote:
Show 7 quoted lines
> I remember that last time I checked, there were a number of designs for 
> subprojects and at least a couple of implementations, but none that was 
> complete to the point of being mergeable. Are there any subproject 
> implementations available that haven't run into unsolveable problems? I'm 
> presently only looking at stuff that totals to a reasonable single project 
> size and rate of growth, so I'm not worried about the large-scale storage 
> requirement issue.

You can try to play with my prototype: git.admingilde.org/tali/git.git, branch module2 (to get an example of how to use it, look at the test script in t/t7500-submodule.sh).

The core operations should work, but not all the user interface is adapted to support submodules. E.g. git-status will not show if a submodule has dirty changes, it will only show submodule changes which are already commited to the submodule but not yet to the supermodule. But at least simple merges do work with submodules.

I abondoned this branch (no further work, only occasionally pulling in updates from upstream git.git) as it has scalability problems with large projects. At the moment I'm working on the module3 branch, which tries to address the scalability problem by separating the object store. This is still in the phase of early prototyping, just to see if it can work. But I am already at the point that I am confident to be able to finish it this way.

The objects created by the module2 and module3 branches are the same, module3 only moves those belonging to the submodule to another location. So If you start using module2 branch now you should be able to upgrade later. But to be extra sure, we should have some discussion about the object format here. (There is nothing new here, really: Just one more tree entry file mode which says that this is not a file or directory entry, but a submodule, represented by one commit.)

-- 
Martin Waitz
Daniel Barkalow· Mar 27, 2007, 00:04 UTC · re: Martin Waitz · lore

Re: Subproject status

On Mon, 26 Mar 2007, Martin Waitz wrote:
> You can try to play with my prototype:
> git.admingilde.org/tali/git.git, branch module2
> (to get an example of how to use it, look at the test script in
> t/t7500-submodule.sh).

I tried that yesterday, but I fail reading comprehension; I lost the "module2" bit by the time I actually fetched. I'll look into this.

Show 9 quoted lines
> The core operations should work, but not all the user interface is
> adapted to support submodules.  E.g. git-status will not show if a
> submodule has dirty changes, it will only show submodule changes which
> are already commited to the submodule but not yet to the supermodule.
> But at least simple merges do work with submodules.
> 
> I abondoned this branch (no further work, only occasionally pulling in
> updates from upstream git.git) as it has scalability problems with
> large projects.

That's what I remember. But it's only an issue with super-scale projects (i.e., where your subprojects are full-sized projects in their own right, and you've got a hundred of them), right? I'm working on the other end (factoring out the common bit from a bunch of similar small projects), so I should be fine with respect to scalability of the implementation.

Would you guess that a patch series to complete the module2 user interface adaptation would also apply to module3 and therefore be useful in the future?

Show 7 quoted lines
> The objects created by the module2 and module3 branches are the same,
> module3 only moves those belonging to the submodule to another location.
> So If you start using module2 branch now you should be able to upgrade
> later.  But to be extra sure, we should have some discussion about the
> object format here.  (There is nothing new here, really: Just one more
> tree entry file mode which says that this is not a file or directory
> entry, but a submodule, represented by one commit.)

So, I had the nutty idea that it would be convenient if I could make different files in a single directory come from different projects. But I can't think of a sane user interface, so I think that this isn't practical from that direction, so it's probably not worth worrying about from the data structure end, either. (Answer for the usecase: "ln -s make/Makefile Makefile; git add Makefile", and mock systems that don't handle symlinks).

But, just to be clear, the semantics of having a commit abcd at a path foo are, with respect to the tree this represents, that abcd:* appears at foo/*. Right?

Are there any standards we should discuss with respect to refs related to subprojects?

I've updated the wiki page with this information.
	-Daniel
*This .sig left intentionally blank*
David Lang· Mar 26, 2007, 23:46 UTC · re: Daniel Barkalow · lore

Re: Subproject status

On Mon, 26 Mar 2007, Daniel Barkalow wrote:
Show 9 quoted lines
>
> So, I had the nutty idea that it would be convenient if I could make
> different files in a single directory come from different projects. But I
> can't think of a sane user interface, so I think that this isn't
> practical from that direction, so it's probably not worth worrying about
> from the data structure end, either. (Answer for the usecase:
> "ln -s make/Makefile Makefile; git add Makefile", and mock systems that
> don't handle symlinks).
>

this would definantly be useful (managing files in /etc or /home are easy examples

for this sort of thing doing 'git add file -index project' would be a fair first-cut at a UI. later on how about creating a dummy .git directory that would record 'file X is part of project Y' so that you could get away with only specifying it once, and then either have a default, or have git add complain if there are multiple places to put things and you don't specify which one (and if you add fules to multiple projects at the same time then you could either prompt for additional checkin comments as needed, or use the same comment for all projects)

David Lang

← back to recent threads