threads / discuss / 45884
Subject: Add an option to automatically submodule update on checkout
Ciro SantilliThis is a must if you are working with submodules, otherwise every git checkout requires a git submodule update, and you forget it, and things break, and you understand, and you go to stack overflow questions http://stackoverflow.com/questions/22328053/why-doesnt-git-checkout-automatically-do-git-submodule-update-recursive | http://stackoverflow.com/questions/4611512/is-there-a-way-to-make-git-pull-automatically-update-submodules and you give up and create aliases :-)
Randall S. BeckerRE: Add an option to automatically submodule update on checkout
On May 6, 2017 4:38 AM Ciro Santilli wrote:
> This is a must if you are working with submodules, otherwise every git checkout requires a git submodule update, > and you forget it, and things break, and you understand, and you go to stack overflow questions > http://stackoverflow.com/questions/22328053/why-doesnt-git-checkout-automatically-do-git-submodule-update-recursive > http://stackoverflow.com/questions/4611512/is-there-a-way-to-make-git-pull-automatically-update-submodules > and you give up and create aliases :-)
I rather like the concept of supporting --recurse-submodules. The complexity is that the branches in all submodules all have to have compatible semantics when doing the checkout, which is by no means guaranteed. In the scenario where you are including a submodule from a third-party (very common - see gnulib), the branches likely won't be there, so you have a high probability of having the command fail or produce the same results as currently exists if you allow the checkout even with problems (another option?). If you have control of everything, then this makes sense.
Cheers, Randall
Stefan BellerRe: Add an option to automatically submodule update on checkout
On Mon, May 8, 2017 at 7:42 AM, Randall S. Becker <rsbecker@nexbridge.com> wrote:
> On May 6, 2017 4:38 AM Ciro Santilli wrote: >> This is a must if you are working with submodules, otherwise every git checkout requires a git submodule update, >> and you forget it, and things break, and you understand, and you go to stack overflow questions >> http://stackoverflow.com/questions/22328053/why-doesnt-git-checkout-automatically-do-git-submodule-update-recursive >> http://stackoverflow.com/questions/4611512/is-there-a-way-to-make-git-pull-automatically-update-submodules >> and you give up and create aliases :-)
The upcoming release (2.13) will have "git checkout --recurse-submodules", which will checkout the submodules at the commit as recorded in the superproject.
I plan to add an option "submodule.recurse" (name is subject to bikeshedding), which would make the --recurse-submodules flag given by default for all commands that support the flag. (Currently cooking we have reset --recurse-submodules, already existing there is push/pull).
> I rather like the concept of supporting --recurse-submodules. The complexity is that the branches in all submodules all have to have compatible semantics when doing the checkout, which is by no means guaranteed. In the scenario where you are including a submodule from a third-party (very common - see gnulib), the branches likely won't be there, so you have a high probability of having the command fail or produce the same results as currently exists if you allow the checkout even with problems (another option?). If you have control of everything, then this makes sense.
I am trying to give the use case of having control over everything (or rather mixed) more thought as well, e.g. "checkout --recurse-submodules -b <name>" may want to create the branches in a subset of submodules as well.
Thanks, Stefan
> > Cheers, > Randall >
Randall S. BeckerRE: Add an option to automatically submodule update on checkout
On May 8, 2017 12:25 PM, Stefan Beller wrote:
>On Mon, May 8, 2017 at 7:42 AM, Randall S. Becker <rsbecker@nexbridge.com> wrote: >> On May 6, 2017 4:38 AM Ciro Santilli wrote: >>> This is a must if you are working with submodules, otherwise every >>> git checkout requires a git submodule update, and you forget it, and >>> things break, and you understand, and you go to stack overflow >>> questions >>> http://stackoverflow.com/questions/22328053/why-doesnt-git-checkout-a >>> utomatically-do-git-submodule-update-recursive >>> http://stackoverflow.com/questions/4611512/is-there-a-way-to-make-git >>> -pull-automatically-update-submodules >>> and you give up and create aliases :-)
> The upcoming release (2.13) will have "git checkout --recurse-submodules", which will checkout the submodules > at the commit as recorded in the superproject. > I plan to add an option "submodule.recurse" (name is subject to bikeshedding), which would make the --recurse-submodules > flag given by default for all commands that support the flag. (Currently cooking we have reset --recurse-submodules, already > existing there is push/pull).
Brilliant! 😊
>> I rather like the concept of supporting --recurse-submodules. The complexity is that the branches in all submodules all have to have compatible >>semantics when doing the checkout, which is by no means guaranteed. In the scenario where you are including a submodule from a third-party (very >>common - see gnulib), the branches likely won't be there, so you have a high probability of having the command fail or produce the same results as >>currently exists if you allow the checkout even with problems (another option?). If you have control of everything, then this makes sense.
>I am trying to give the use case of having control over everything (or rather mixed) more thought as well, e.g. "checkout --recurse-submodules -b ><name>" may want to create the branches in a subset of submodules as well.
I have to admit that I just assumed it would have to work that way this would not be particularly useful. However, in thinking about it, we might want to limit the depth of how far -b <name> takes effect. If the super module brings in submodules entirely within control of the development group, having -b <name> apply down to leaf submodules makes sense (in some policies). However, if some submodules span out to, say, gnulib, that might not make particular sense. Some downward limit might be appropriate. Perhaps, in the submodule ref, you might want to qualify it as <commit>:<ref> (but the impact of that is probably and admittedly pretty horrid). I hesitate to suggest a numeric limit, as that assumes that submodules are organized in a balanced tree - which is axiomatically unreasonable. Maybe something in .git/config, like
[branch "topic*"] submodules=a,b,c
But I suspect that would make things even more confusing.
Cheers, Randall
Stefan BellerRe: Add an option to automatically submodule update on checkout
On Mon, May 8, 2017 at 9:46 AM, Randall S. Becker <rsbecker@nexbridge.com> wrote:
> On May 8, 2017 12:25 PM, Stefan Beller wrote: >>On Mon, May 8, 2017 at 7:42 AM, Randall S. Becker <rsbecker@nexbridge.com> wrote: >>> On May 6, 2017 4:38 AM Ciro Santilli wrote: >>>> This is a must if you are working with submodules, otherwise every >>>> git checkout requires a git submodule update, and you forget it, and >>>> things break, and you understand, and you go to stack overflow >>>> questions >>>> http://stackoverflow.com/questions/22328053/why-doesnt-git-checkout-a >>>> utomatically-do-git-submodule-update-recursive >>>> http://stackoverflow.com/questions/4611512/is-there-a-way-to-make-git >>>> -pull-automatically-update-submodules >>>> and you give up and create aliases :-) > >> The upcoming release (2.13) will have "git checkout --recurse-submodules", which will checkout the submodules >> at the commit as recorded in the superproject. >> I plan to add an option "submodule.recurse" (name is subject to bikeshedding), which would make the --recurse-submodules >> flag given by default for all commands that support the flag. (Currently cooking we have reset --recurse-submodules, already >> existing there is push/pull). > > Brilliant! 😊 > >>> I rather like the concept of supporting --recurse-submodules. The complexity is that the branches in all submodules all have to have compatible >>semantics when doing the checkout, which is by no means guaranteed. In the scenario where you are including a submodule from a third-party (very >>common - see gnulib), the branches likely won't be there, so you have a high probability of having the command fail or produce the same results as >>currently exists if you allow the checkout even with problems (another option?). If you have control of everything, then this makes sense. > >>I am trying to give the use case of having control over everything (or rather mixed) more thought as well, e.g. "checkout --recurse-submodules -b ><name>" may want to create the branches in a subset of submodules as well. > > I have to admit that I just assumed it would have to work that way this would not be particularly useful. However, in thinking about it, we might want to limit the depth of how far -b <name> takes effect. If the super module brings in submodules entirely within control of the development group, having -b <name> apply down to leaf submodules makes sense (in some policies). However, if some submodules span out to, say, gnulib, that might not make particular sense. Some downward limit might be appropriate. Perhaps, in the submodule ref, you might want to qualify it as <commit>:<ref> (but the impact of that is probably and admittedly pretty horrid). I hesitate to suggest a numeric limit, as that assumes that submodules are organized in a balanced tree - which is axiomatically unreasonable. Maybe something in .git/config, like > > [branch "topic*"] > submodules=a,b,c > > But I suspect that would make things even more confusing.
I thought about having yet-another-flag in the .gitmodules file, which states if the submodule is extern or internal.
[submodule "gnulib"]
path=./gnulib
external = true # implies no branch for checkout -b --recurse-submodulesI think there are a couple more situations where such "external" submodules are treated differently, so maybe we'd want to think carefully about the actual name as different workflows would want to have different features for an internal/external submodule.
Thanks, Stefan
Randall S. BeckerRE: Add an option to automatically submodule update on checkout
On May 8, 2017 12:55 PM, Stefan Beller wrote:
>On Mon, May 8, 2017 at 9:46 AM, Randall S. Becker <rsbecker@nexbridge.com> wrote: >> On May 8, 2017 12:25 PM, Stefan Beller wrote: >>>On Mon, May 8, 2017 at 7:42 AM, Randall S. Becker <rsbecker@nexbridge.com> wrote: >>>> On May 6, 2017 4:38 AM Ciro Santilli wrote: >>>>> This is a must if you are working with submodules, otherwise every >>>>> git checkout requires a git submodule update, and you forget it, >>>>> and things break, and you understand, and you go to stack overflow >>>>> questions >>>>> http://stackoverflow.com/questions/22328053/why-doesnt-git-checkout >>>>> -a utomatically-do-git-submodule-update-recursive >>>>> http://stackoverflow.com/questions/4611512/is-there-a-way-to-make-g >>>>> it -pull-automatically-update-submodules >>>>> and you give up and create aliases :-) >> >>> The upcoming release (2.13) will have "git checkout >>> --recurse-submodules", which will checkout the submodules at the commit as recorded in the superproject. >>> I plan to add an option "submodule.recurse" (name is subject to >>> bikeshedding), which would make the --recurse-submodules flag given >>> by default for all commands that support the flag. (Currently cooking we have reset --recurse-submodules, already existing there is push/pull). >> >> Brilliant! 😊 >> >>>> I rather like the concept of supporting --recurse-submodules. The complexity is that the branches in all submodules all have to have compatible >>>semantics when doing the checkout, which is by no means guaranteed. In the scenario where you are including a submodule from a third-party (very >>>common - see gnulib), the branches likely won't be there, so you have a high probability of having the command fail or produce the same results as >>>currently exists if you allow the checkout even with problems (another option?). If you have control of everything, then this makes sense. >> >>>I am trying to give the use case of having control over everything (or rather mixed) more thought as well, e.g. "checkout --recurse-submodules -b >><name>" may want to create the branches in a subset of submodules as well. >> >> I have to admit that I just assumed it would have to work that way >> this would not be particularly useful. However, in thinking about it, >> we might want to limit the depth of how far -b <name> takes effect. If >> the super module brings in submodules entirely within control of the >> development group, having -b <name> apply down to leaf submodules >> makes sense (in some policies). However, if some submodules span out >> to, say, gnulib, that might not make particular sense. Some downward >> limit might be appropriate. Perhaps, in the submodule ref, you might >> want to qualify it as <commit>:<ref> (but the impact of that is >> probably and admittedly pretty horrid). I hesitate to suggest a >> numeric limit, as that assumes that submodules are organized in a >> balanced tree - which is axiomatically unreasonable. Maybe something >> in .git/config, like >> >> [branch "topic*"] >> submodules=a,b,c >> >> But I suspect that would make things even more confusing.
>I thought about having yet-another-flag in the .gitmodules file, which states if the submodule is extern or internal.
>[submodule "gnulib"] > path=./gnulib > external = true # implies no branch for checkout -b --recurse-submodules
>I think there are a couple more situations where such "external" submodules are treated differently, so maybe we'd want to think carefully about the >actual name as different workflows would want to have different features for an internal/external submodule.
I didn't want to open up that one, but yes. That makes sense. However, I don't like overloading what "external" means or might mean in the future. Would you consider a distinct Boolean for that, like inherit-branch=true?
Cheers, Randall
Re: Add an option to automatically submodule update on checkout
On 05/08, Randall S. Becker wrote:
> On May 8, 2017 12:55 PM, Stefan Beller wrote: > >On Mon, May 8, 2017 at 9:46 AM, Randall S. Becker <rsbecker@nexbridge.com> wrote: > >> On May 8, 2017 12:25 PM, Stefan Beller wrote: > >>>On Mon, May 8, 2017 at 7:42 AM, Randall S. Becker <rsbecker@nexbridge.com> wrote: > >>>> On May 6, 2017 4:38 AM Ciro Santilli wrote: > >>>>> This is a must if you are working with submodules, otherwise every > >>>>> git checkout requires a git submodule update, and you forget it, > >>>>> and things break, and you understand, and you go to stack overflow > >>>>> questions > >>>>> http://stackoverflow.com/questions/22328053/why-doesnt-git-checkout > >>>>> -a utomatically-do-git-submodule-update-recursive > >>>>> http://stackoverflow.com/questions/4611512/is-there-a-way-to-make-g > >>>>> it -pull-automatically-update-submodules > >>>>> and you give up and create aliases :-) > >> > >>> The upcoming release (2.13) will have "git checkout > >>> --recurse-submodules", which will checkout the submodules at the commit as recorded in the superproject. > >>> I plan to add an option "submodule.recurse" (name is subject to > >>> bikeshedding), which would make the --recurse-submodules flag given > >>> by default for all commands that support the flag. (Currently cooking we have reset --recurse-submodules, already existing there is push/pull). > >> > >> Brilliant! 😊 > >> > >>>> I rather like the concept of supporting --recurse-submodules. The complexity is that the branches in all submodules all have to have compatible >>>semantics when doing the checkout, which is by no means guaranteed. In the scenario where you are including a submodule from a third-party (very >>>common - see gnulib), the branches likely won't be there, so you have a high probability of having the command fail or produce the same results as >>>currently exists if you allow the checkout even with problems (another option?). If you have control of everything, then this makes sense. > >> > >>>I am trying to give the use case of having control over everything (or rather mixed) more thought as well, e.g. "checkout --recurse-submodules -b >><name>" may want to create the branches in a subset of submodules as well. > >> > >> I have to admit that I just assumed it would have to work that way > >> this would not be particularly useful. However, in thinking about it, > >> we might want to limit the depth of how far -b <name> takes effect. If > >> the super module brings in submodules entirely within control of the > >> development group, having -b <name> apply down to leaf submodules > >> makes sense (in some policies). However, if some submodules span out > >> to, say, gnulib, that might not make particular sense. Some downward > >> limit might be appropriate. Perhaps, in the submodule ref, you might > >> want to qualify it as <commit>:<ref> (but the impact of that is > >> probably and admittedly pretty horrid). I hesitate to suggest a > >> numeric limit, as that assumes that submodules are organized in a > >> balanced tree - which is axiomatically unreasonable. Maybe something > >> in .git/config, like > >> > >> [branch "topic*"] > >> submodules=a,b,c > >> > >> But I suspect that would make things even more confusing. > > >I thought about having yet-another-flag in the .gitmodules file, which states if the submodule is extern or internal. > > >[submodule "gnulib"] > > path=./gnulib > > external = true # implies no branch for checkout -b --recurse-submodules > > >I think there are a couple more situations where such "external" submodules are treated differently, so maybe we'd want to think carefully about the >actual name as different workflows would want to have different features for an internal/external submodule. > > I didn't want to open up that one, but yes. That makes sense. However, I don't like overloading what "external" means or might mean in the future. Would you consider a distinct Boolean for that, like inherit-branch=true?
Something like that kind of already exists. The 'branch' field. Internal repos would most likely use the '.' value to indicate that the submodules should track the superproject's branch. While a value of say 'foo' would indicate that the submodule should always be on branch 'foo'; this could be used for external repositories.
> > Cheers, > Randall >
-- Brandon Williams
Stefan BellerRe: Add an option to automatically submodule update on checkout
On Mon, May 8, 2017 at 10:08 AM, Brandon Williams <bmwill@google.com> wrote:
>> >> >[submodule "gnulib"] >> > path=./gnulib >> > external = true # implies no branch for checkout -b --recurse-submodules >> >> >I think there are a couple more situations where such "external" submodules are treated differently, so maybe we'd want to think carefully about the >actual name as different workflows would want to have different features for an internal/external submodule. >> >> I didn't want to open up that one, but yes. That makes sense. However, I don't like overloading what "external" means or might mean in the future. Would you consider a distinct Boolean for that, like inherit-branch=true? > > Something like that kind of already exists. The 'branch' field. > Internal repos would most likely use the '.' value to indicate that the > submodules should track the superproject's branch. While a value of say > 'foo' would indicate that the submodule should always be on branch > 'foo'; this could be used for external repositories.
so for external repos you'd keep the branch unset, such that you strictly checkout the sha1 object into a detached HEAD.
Makes sense.
Thanks, Stefan
Junio C HamanoRe: Add an option to automatically submodule update on checkout
"Randall S. Becker" <rsbecker@nexbridge.com> writes:
> I have to admit that I just assumed it would have to work that way > this would not be particularly useful. However, in thinking about > it, we might want to limit the depth of how far -b <name> takes > effect. If the super module brings in submodules entirely within > control of the development group, having -b <name> apply down to > leaf submodules makes sense (in some policies). However, if some > submodules span out to, say, gnulib, that might not make > particular sense.
I do not see a strong reason to avoid your own branches in "other people's project" like this.
The submodule's upstream may be a project you have no control over, but the repository you have locally is under your total control and you can use any branch names to suit the need of your project as the whole (i.e. the superproject and submodules bound to it).
The fact that local branch names are under your control and for your own use is true even when you are not using submodules, by the way.
Randall S. BeckerRE: Add an option to automatically submodule update on checkout
On May 8, 2017 10:58 PM, Junio C Hamano wrote:
>"Randall S. Becker" <rsbecker@nexbridge.com> writes: >> I have to admit that I just assumed it would have to work that way >> this would not be particularly useful. However, in thinking about it, >> we might want to limit the depth of how far -b <name> takes effect. If >> the super module brings in submodules entirely within control of the >> development group, having -b <name> apply down to leaf submodules >> makes sense (in some policies). However, if some submodules span out >> to, say, gnulib, that might not make particular sense.
>I do not see a strong reason to avoid your own branches in "other people's
project" like this.
>The submodule's upstream may be a project you have no control over, but the
repository you have locally is under your total control and you can use >any branch names to suit the need of your project as the whole (i.e. the superproject and submodules bound to it).
>The fact that local branch names are under your control and for your own
use is true even when you are not using submodules, by the way.
I agree with the technical aspects of this, but doing a checkout -b into something like gnulib will pin the code you are using in that submodule to whatever commit was referenced when you did the checkout. Example: In a situation like that, I would want gnulib to stay on 'master'. It is my opinion, FWIW, that this is a matter of policy or standards within the organization using git that we should not be imposing one way or another. In the current state of affairs (2.12.x), when I checkout, I make sure that people are aware of which branch each submodule is on because git won't go into the submodules - I'm fine with imposing that as a policy at present because it takes positive action by the developers and I keep the master branch on my own repositories locked down and it's obvious when they are accidentally on it. But we're talking changing this so that checkout branches can apply recursively. This changes the policy requirements so that people have to further act to undo what git will do by default on recursion. The policy will be at a high level the same (i.e., always make sure you know what branch you are on in submodules), but the implementation of it will need to be different (i.e., after you checkout recursive, go into each submodule and undo what git just did by checking out the default branch on some submodules ___ ___ ___, which depends on which super repository they are using, is onerous for me to manage, and for my developers to remember to do).
With Respect, Randall
Re: Add an option to automatically submodule update on checkout
On 05/08, Stefan Beller wrote:
> On Mon, May 8, 2017 at 7:42 AM, Randall S. Becker > <rsbecker@nexbridge.com> wrote: > > On May 6, 2017 4:38 AM Ciro Santilli wrote: > >> This is a must if you are working with submodules, otherwise every > >> git checkout requires a git submodule update, and you forget it, > >> and things break, and you understand, and you go to stack overflow > >> questions > >> http://stackoverflow.com/questions/22328053/why-doesnt-git-checkout-automatically-do-git-submodule-update-recursive > >> http://stackoverflow.com/questions/4611512/is-there-a-way-to-make-git-pull-automatically-update-submodules > >> and you give up and create aliases :-) > > The upcoming release (2.13) will have "git checkout > --recurse-submodules", which will checkout the submodules at the > commit as recorded in the superproject. > > I plan to add an option "submodule.recurse" (name is subject to > bikeshedding), which would make the --recurse-submodules flag given by > default for all commands that support the flag. (Currently cooking we > have reset --recurse-submodules, already existing there is push/pull).
Well pull not so much...it'll do a recursive fetch but not a recursive merge/rebase. That is something on the docket to get done in the next couple months though.
> > > I rather like the concept of supporting --recurse-submodules. The > > complexity is that the branches in all submodules all have to have > > compatible semantics when doing the checkout, which is by no means > > guaranteed. In the scenario where you are including a submodule from > > a third-party (very common - see gnulib), the branches likely won't > > be there, so you have a high probability of having the command fail > > or produce the same results as currently exists if you allow the > > checkout even with problems (another option?). If you have control > > of everything, then this makes sense. > > I am trying to give the use case of having control over everything (or > rather mixed) more thought as well, e.g. "checkout > --recurse-submodules -b <name>" may want to create the branches in a > subset of submodules as well. > > Thanks, Stefan > > > > > Cheers, Randall > >
-- Brandon Williams