Where can I find more information about submodules? I'm interested in both internal implementation details and how to use them.
- Dmitry
Johannes SchindelinRe: Submodules
Hi,
On Tue, 7 Aug 2007, Dmitry Kakurin wrote:
> Where can I find more information about submodules? > I'm interested in both internal implementation details and how to use them.
Most information is found in git-submodule.sh, although Documentation/git-submodule.txt has beginnings of a good documentation.
The basic idea is this:
A subdirectory contains a self-contained project.
Once that project is at a stage you want to have in the superproject, you can commit that submodule state, much like a subdirectory.
However, the history can progress in much coarser steps for the superproject. IOW an ideal case for msysgit.git, where we want to have a stable version which can be checked out with GitMe.
Also, find more about submodules here:
http://git.or.cz/gitwiki/SubprojectSupport?highlight=%28submodule%29
Ciao, Dscho
Re: Submodules
Yeah, I've already went thru these. But I want (much) more details. I can ask a bunch of random questions on this list, like:
* Why after 'submodule update' it becomes detached, what if I want it to stay on certain branch? * How do I control which branches are fetched? * What if I do 'commit -a' standing in /super, will /super/submodule commit as well? * What if I'm standing in /super/submodule and do 'commit -a'? Will super notice it in any way?
But I was hoping that it's all summarized somewhere already.
- Dmitry On 8/7/07, Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:
Show 27 quoted lines
> Hi, > > On Tue, 7 Aug 2007, Dmitry Kakurin wrote: > > > Where can I find more information about submodules? > > I'm interested in both internal implementation details and how to use them. > > Most information is found in git-submodule.sh, although > Documentation/git-submodule.txt has beginnings of a good documentation. > > The basic idea is this: > > A subdirectory contains a self-contained project. > > Once that project is at a stage you want to have in the superproject, you > can commit that submodule state, much like a subdirectory. > > However, the history can progress in much coarser steps for the > superproject. IOW an ideal case for msysgit.git, where we want to have a > stable version which can be checked out with GitMe. > > Also, find more about submodules here: > > http://git.or.cz/gitwiki/SubprojectSupport?highlight=%28submodule%29 > > Ciao, > Dscho
Re: Submodules
On Tue, Aug 07, 2007 at 04:10:38PM -0700, Dmitry Kakurin wrote:
Show 5 quoted lines
> Yeah, I've already went thru these. But I want (much) more details. > I can ask a bunch of random questions on this list, like: > > * Why after 'submodule update' it becomes detached, what if I want it > to stay on certain branch?
You'd need to have a way to specify which branch you want to have updated. There was some discussion about this some time ago and the general conclusion was that a detached HEAD was the best solution.
> * How do I control which branches are fetched?
Currently, you can't.
> * What if I do 'commit -a' standing in /super, will /super/submodule > commit as well?
No.
> * What if I'm standing in /super/submodule and do 'commit -a'? Will > super notice it in any way?
Any commit in the submodule will be seen as a "change" to the submodule from the perspective of the supermodule.
skimo
Johannes SchindelinRe: Submodules
Hi,
completely forgot: submodules do not need to be checked out. So, if there is a submodule you are not interested in working on, you can leave it.
Ciao, Dscho
Re: Submodules
This is exactly the level of details I'm talking about: * how come sumbodules are not initialized when I do a clone of super. I expect to be able to build super after I clone it. Is there a new (undocumented) flag to clone? * is it OK to *not* init a submodule? will super become unhappy? Can I do commits to super in this case? * why submodules should be listed in 2 places: in .submodules and in super/.git/config?
- Dmitry
P.S. There is an old saying "One fool can ask so many questions that a thousand wise man would not be able to answer" :-)
On 8/7/07, Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:
> completely forgot: submodules do not need to be checked out. So, if there > is a submodule you are not interested in working on, you can leave it.
Johannes SchindelinRe: Submodules
Hi,
On Tue, 7 Aug 2007, Dmitry Kakurin wrote:
> * how come sumbodules are not initialized when I do a clone of super. > I expect to be able to build super after I clone it. Is there a new > (undocumented) flag to clone?
There might be. But one of the reasons submodules are not just tree objects is that you should be able _not_ to check out all of them (think KDE).
> * why submodules should be listed in 2 places: in .submodules and in > super/.git/config?
Well, URLs can change. Therefore, .submodules has what is considered the current URL, but you can change it yourself in .git/config. (At least that is how I understood it.)
Ciao, Dscho
Re: Submodules
On Tue, Aug 07, 2007 at 04:31:57PM -0700, Dmitry Kakurin wrote:
> This is exactly the level of details I'm talking about: > * how come sumbodules are not initialized when I do a clone of super.
See you second question.
> I expect to be able to build super after I clone it. Is there a new > (undocumented) flag to clone?
Not (yet). Right now, you have to do
git submodule init git submodule update
after you clone to fetch and check-out all (first-level) submodules.
> * is it OK to *not* init a submodule? will super become unhappy? Can I > do commits to super in this case?
Yes. No. Yes. In fact, only the "git submodule" subcommand are affected by a "git submodule init". Doing a "git submodule update" will actually check out the submodules and from then on, the HEAD of this checked-out submodule will be considered the content of the submodule in the working tree of the supermodule.
> * why submodules should be listed in 2 places: in .submodules and in > super/.git/config?
It only has to be specified in .git/config. The value in .gitmodules (if present) is used by "git submodule init" as a default value for the one in .git/config.
skimo
Junio C HamanoRe: Submodules
There was a discussion about the semantics currently implemented, as a subset of what we might want to eventually have, and some recommended that we mention where things stand now in the release notes.
How about this?
diff --git a/Documentation/RelNotes-1.5.3.txt b/Documentation/RelNotes-1.5.3.txt index 21bb1fc..901f813 100644 --- a/Documentation/RelNotes-1.5.3.txt +++ b/Documentation/RelNotes-1.5.3.txt @@ -9,6 +9,19 @@ Updates since v1.5.2 * The submodule support has Porcelain layer. + Note that the current submodule support is minimal and this is + deliberately so. A design decision we made is that operations + at the supermodule level do not recurse into submodules by + default. The expectation is that later we would add a + mechanism to tell git which submodules the user is interested + in, and this information might be used to determine the + recursive behaviour of certain commands (e.g. "git checkout" + and "git diff"), but currently we haven't agreed on what that + mechanism should look like. Therefore, if you use submodules, + you would probably need "git submodule update" on the + submodules you care about after running a "git checkout" at + the supermodule level. + * There are a handful pack-objects changes to help you cope better with repositories with pathologically large blobs in them.
Johannes SchindelinRe: Submodules
Hi,
On Wed, 8 Aug 2007, Junio C Hamano wrote:
Show 13 quoted lines
> + Note that the current submodule support is minimal and this is > + deliberately so. A design decision we made is that operations > + at the supermodule level do not recurse into submodules by > + default. The expectation is that later we would add a > + mechanism to tell git which submodules the user is interested > + in, and this information might be used to determine the > + recursive behaviour of certain commands (e.g. "git checkout" > + and "git diff"), but currently we haven't agreed on what that > + mechanism should look like. Therefore, if you use submodules, > + you would probably need "git submodule update" on the > + submodules you care about after running a "git checkout" at > + the supermodule level. > +
Nice write up!
Is it true that you can run "update" without "init" first? (I haven't tried yet.)
Also, I realised that git is not really happy unless you install alternates pointing to the object stores of the submodules. Shouldn't we make this my default (for example in "init")?
Ciao, Dscho
Junio C HamanoRe: Submodules
Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
Show 26 quoted lines
> Hi, > > On Wed, 8 Aug 2007, Junio C Hamano wrote: > >> + Note that the current submodule support is minimal and this is >> + deliberately so. A design decision we made is that operations >> + at the supermodule level do not recurse into submodules by >> + default. The expectation is that later we would add a >> + mechanism to tell git which submodules the user is interested >> + in, and this information might be used to determine the >> + recursive behaviour of certain commands (e.g. "git checkout" >> + and "git diff"), but currently we haven't agreed on what that >> + mechanism should look like. Therefore, if you use submodules, >> + you would probably need "git submodule update" on the >> + submodules you care about after running a "git checkout" at >> + the supermodule level. >> + > > Nice write up! > > Is it true that you can run "update" without "init" first? (I haven't > tried yet.) > > Also, I realised that git is not really happy unless you install > alternates pointing to the object stores of the submodules. Shouldn't we > make this my default (for example in "init")?
Sorry, but I do not use submodules yet myself. The questions need to be answered, errors need to be corrected and omissions need to be added by people who have been more involved in that area of the system.
That's would be Sven and Lars, probably.
I also think we should have a paragraph of similar spirit about the work-tree stuff as the top-level bullet point, before the "New commands and options" list. I suspect Matthias and you are the prime candidates to do the honors?
Re: Submodules
On Wed, Aug 08, 2007 at 12:59:30PM -0700, Junio C Hamano wrote:
> Sorry, but I do not use submodules yet myself.
If you would ever start using submodules (for, say, gitgui), then I'm sure we'd have much more usable submodules support in no time.
skimo
Junio C HamanoRe: Submodules
Sven Verdoolaege <skimo@kotnet.org> writes:
Show 6 quoted lines
> On Wed, Aug 08, 2007 at 12:59:30PM -0700, Junio C Hamano wrote: >> Sorry, but I do not use submodules yet myself. > > If you would ever start using submodules (for, say, gitgui), > then I'm sure we'd have much more usable submodules support > in no time.
Heh, I do not have infinite time to spend on every little corner features, and have been hoping that other people who have more immediate need of submodule support to be the guinea pigs to encounter real-life issues first and come up with solutions to them, before I start using it.
So far, I think the plan has been working out alright ;-).
Johannes SchindelinRe: Submodules
Hi,
On Wed, 8 Aug 2007, Junio C Hamano wrote:
Show 16 quoted lines
> Sven Verdoolaege <skimo@kotnet.org> writes: > > > On Wed, Aug 08, 2007 at 12:59:30PM -0700, Junio C Hamano wrote: > >> Sorry, but I do not use submodules yet myself. > > > > If you would ever start using submodules (for, say, gitgui), > > then I'm sure we'd have much more usable submodules support > > in no time. > > Heh, I do not have infinite time to spend on every little corner > features, and have been hoping that other people who have more > immediate need of submodule support to be the guinea pigs to > encounter real-life issues first and come up with solutions to > them, before I start using it. > > So far, I think the plan has been working out alright ;-).
Sure. I already like submodules, and I think that msysgit.git using it can bring submodules along more painlessly for git.git users.
Ciao, Dscho
Junio C HamanoRe: Submodules
Ok, this is my second try.
- Incorporated Sven's suggestion to give an example of problematic situation on submodule after "git-checkout" at the supermodule level;
- Describe semantic clean-up of specifying GIT_DIR that is not strictly needed (because it is situated at the normal place relative to the working tree).
---
diff --git a/Documentation/RelNotes-1.5.3.txt b/Documentation/RelNotes-1.5.3.txt index 21bb1fc..bf7a341 100644 --- a/Documentation/RelNotes-1.5.3.txt +++ b/Documentation/RelNotes-1.5.3.txt @@ -9,6 +9,40 @@ Updates since v1.5.2 * The submodule support has Porcelain layer. + Note that the current submodule support is minimal and this is + deliberately so. A design decision we made is that operations + at the supermodule level do not recurse into submodules by + default. The expectation is that later we would add a + mechanism to tell git which submodules the user is interested + in, and this information might be used to determine the + recursive behaviour of certain commands (e.g. "git checkout" + and "git diff"), but currently we haven't agreed on what that + mechanism should look like. In particular, if you have any + submodules checked out, running a "git checkout" at the + supermodule level will not update these submodules. They will + therefore appear to be modified (to the state prior to the + checkout) to any subsequent git command, until they have been + updated explicitly using "git submodule update". + +* You can have a work tree not at the parent directory of .git + which is the repository. This is done with GIT_WORK_TREE + environment, or --work-tree option. + + This support slightly changes the semantics of having GIT_DIR + environment variable. The rule used to be that when it is + given, you are supposed to be at the top level of the working + tree. However, if it names a .git directory at the top of the + working tree, you can be in a subdirectory of that directory + and git notices you are in the subdirectory. E.g. in t/ + subdirectory of git.git project, + + $ cd t + $ GIT_DIR=../.git/ git ls-files -s + + will list the files in t/ (i.e. your current directory). It + used not to notice that you are in a subdirectory and listed + the files from the top-level of the working tree. + * There are a handful pack-objects changes to help you cope better with repositories with pathologically large blobs in them.
Re: Submodules
On Wednesday 2007, August 08, Johannes Schindelin wrote:
> Also, I realised that git is not really happy unless you install > alternates pointing to the object stores of the submodules. Shouldn't we > make this my default (for example in "init")?
I use submodules on a daily basis and haven't experienced this at all. Would you mind saying a little more about this problem?
Andy
-- Dr Andy Parkins, M Eng (hons), MIET andyparkins@gmail.com
Johannes SchindelinRe: Submodules
Hi,
On Wed, 8 Aug 2007, Andy Parkins wrote:
Show 8 quoted lines
> On Wednesday 2007, August 08, Johannes Schindelin wrote: > > > Also, I realised that git is not really happy unless you install > > alternates pointing to the object stores of the submodules. Shouldn't we > > make this my default (for example in "init")? > > I use submodules on a daily basis and haven't experienced this at all. > Would you mind saying a little more about this problem?
Not at all!
This is what I did. I cloned msysgit.git, which contains one submodule. To initialise that, I did "git submodule init" and "git submodule update". It went and cloned the submodule. Fine.
Then I committed in the submodule, a well-needed fix.
If I now go to the superproject again, and say "git submodule status", it will not find the newest commit, and complain that it cannot access that object.
Ciao, Dscho
Re: Submodules
On Wed, Aug 08, 2007 at 10:12:52PM +0100, Johannes Schindelin wrote:
Show 9 quoted lines
> This is what I did. I cloned msysgit.git, which contains one submodule. > To initialise that, I did "git submodule init" and "git submodule update". > It went and cloned the submodule. Fine. > > Then I committed in the submodule, a well-needed fix. > > If I now go to the superproject again, and say "git submodule status", it > will not find the newest commit, and complain that it cannot access that > object.
There must be something else you're not telling us. This procedure works just fine for me
bash-3.00$ git clone git://repo.or.cz/msysgit.git Initialized empty Git repository in /home/skimo/git/msysgit/.git/ remote: Generating pack... remote: Done counting 1857 objects. remote: Deltifying 1857 objects... remote: 100% (1857/1857) done Indexing 1857 objects... remote: Total 1857 (delta 475), reused 1857 (delta 475) 100% (1857/1857) done Resolving 475 deltas... 100% (475/475) done bash-3.00$ cd msysgit/ bash-3.00$ git submodule init Submodule 'git' (git://repo.or.cz/git/mingw/4msysgit.git/) registered for path 'git' bash-3.00$ git submodule update Initialized empty Git repository in /home/skimo/git/msysgit/git/.git/ remote: Generating pack... remote: Done counting 58074 objects. remote: Deltifying 58074 objects... remote: 100% (58074/58074) done Indexing 58074 objects... remote: Total 58074 (delta 40790), reused 55132 (delta 37853) 100% (58074/58074) done Resolving 40790 deltas... 100% (40790/40790) done Submodule path 'git': checked out 'f1e1dc5119bd9862bb4d2a975c8ca6362ea43af5' bash-3.00$ cd git/ bash-3.00$ touch a; git add a; git commit -m 'add a' Created commit 5318784: add a 0 files changed, 0 insertions(+), 0 deletions(-) create mode 100644 a bash-3.00$ cd .. bash-3.00$ git submodule status +5318784e6d492d988284a4d9aa4025496f2a3309 git (v1.5.3-rc2-690-g5318784)
skimo
Re: Submodules
On Wed, Aug 08, 2007 at 12:33:26PM -0700, Junio C Hamano wrote:
Show 13 quoted lines
> + Note that the current submodule support is minimal and this is > + deliberately so. A design decision we made is that operations > + at the supermodule level do not recurse into submodules by > + default. The expectation is that later we would add a > + mechanism to tell git which submodules the user is interested > + in, and this information might be used to determine the > + recursive behaviour of certain commands (e.g. "git checkout" > + and "git diff"), but currently we haven't agreed on what that > + mechanism should look like. Therefore, if you use submodules, > + you would probably need "git submodule update" on the > + submodules you care about after running a "git checkout" at > + the supermodule level. > +
Maybe you could be a bit more explicit about the possibly unexpected results of "git checkout". Perhaps something like the following instead of the last sentence:
In particular, if you have any submodules checked out,
running a "git checkout" at the supermodule level will
not update these submodules. They will therefore appear
to be modified (to the state prior to the checkout)
to any subsequent git command, until they have been
updated explicitly using "git submodule update".skimo