# Re: [PATCH v5 0/2] submodule update: add --remote for submodule's upstream changes 67 messages from 2012-11-29 to 2012-12-21. Participants: W. Trevor King, Phil Hord, Jens Lehmann, Junio C Hamano, Michael J Gruber, Marc Branchaud, wking@tremily.us, Heiko Voigt. Thread: https://gitlist.dev/t/32245 ## W. Trevor King, 2012-11-29 19:13 Subject: Re: [PATCH v5 0/2] submodule update: add --remote for submodule's upstream changes Message-ID: <20121129191326.GC27409@odin.tremily.us> URL: https://gitlist.dev/e/20121129191326.GC27409%40odin.tremily.us In-Reply-To: ``` On Thu, Nov 29, 2012 at 01:29:12PM -0500, Phil Hord wrote: > On Fri, Nov 23, 2012 at 12:54 PM, W. Trevor King wrote: > > [snip initial thoughts leading to the update --remote v5] > > I was thinking the same thing, but reading this whole thread a couple of > weeks late. Thanks for noticing. > > Moreover, I think that 'git submodule update --pull' is also the wrong way > to spell this action. Maybe you are misled from the outset by your > current workflow: Did you see my v5 (add --remote) series? > For that reason, I don't like the --pull switch since it implies a > fetch, but I will not always want to do a fetch. $ git submodule update --remote --no-fetch will not fetch the submodule remotes. > I don't know which remote I should be tracking, though. I suppose > it is 'origin' for now, but maybe it is just whatever > $superproject's HEAD's remote-tracking branch indicates. With the --remote series, I always use "origin" because that's what `submodule add` should be setting up. If people want to change that up by hand, we may need a submodule..remote configuration option. > I am not sure I want the gitlinks in superproject to update automatically > in the index, but I definitely do not want to automatically create a commit > for them. Commits are dissabled by default (see my recent --commit RFC for how they would be enabled). > But I really don't want to figure out how to handle submodule > collisions during a merge (or rebase!) of my superproject with changes that > someone else auto-committed in his local $superproject as he and I > arbitrarily floated up the upstream independently. There is nothing but > loathing down that path. This is true. I'm not sure how gitlink collisions are currently handled… Trevor -- This email may be signed or encrypted with GnuPG (http://www.gnupg.org). For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy ``` ## Phil Hord, 2012-11-30 01:11 Subject: Re: [PATCH v5 0/2] submodule update: add --remote for submodule's upstream changes Message-ID: URL: https://gitlist.dev/e/CABURp0piLAG%2BhEsav-uro%2Bnq9ZRZ9CFFjVG8VKYk3ZtYvRi8%3DA%40mail.gmail.com In-Reply-To: <20121129191326.GC27409@odin.tremily.us> ``` On Thu, Nov 29, 2012 at 2:13 PM, W. Trevor King wrote: > > On Thu, Nov 29, 2012 at 01:29:12PM -0500, Phil Hord wrote: >> On Fri, Nov 23, 2012 at 12:54 PM, W. Trevor King wrote: >> > [snip initial thoughts leading to the update --remote v5] >> >> I was thinking the same thing, but reading this whole thread a couple of >> weeks late. Thanks for noticing. >> >> Moreover, I think that 'git submodule update --pull' is also the wrong way >> to spell this action. Maybe you are misled from the outset by your >> current workflow: > > Did you see my v5 (add --remote) series? Eventually, I did. Sorry for the out-of-order replies. >> For that reason, I don't like the --pull switch since it implies a >> fetch, but I will not always want to do a fetch. > > $ git submodule update --remote --no-fetch > > will not fetch the submodule remotes. This seems precisely backwards to me. Why not use $ git submodule update --remote --fetch to do your "default" behavior instead? I suppose I am arguing against the tide of the dominant workflow, but the fetch-by-default idea needlessly conflates two primitive operations: "float" and "fetch". >> I don't know which remote I should be tracking, though. I suppose >> it is 'origin' for now, but maybe it is just whatever >> $superproject's HEAD's remote-tracking branch indicates. > > With the --remote series, I always use "origin" because that's what > `submodule add` should be setting up. If people want to change that > up by hand, we may need a submodule..remote configuration > option. I've always felt that the "origin" defaults are broken and are simply being ignored because most users do not trip over them. But ISTR that submodule commands use the remote indicated by the superproject's current remote-tracking configuration, with a fallback to 'origin' if there is none. Sort of a "best effort" algorithm, I think. Am I remembering that wrong? >> I am not sure I want the gitlinks in superproject to update automatically >> in the index, but I definitely do not want to automatically create a commit >> for them. > > Commits are dissabled by default (see my recent --commit RFC for how > they would be enabled). > >> But I really don't want to figure out how to handle submodule >> collisions during a merge (or rebase!) of my superproject with changes that >> someone else auto-committed in his local $superproject as he and I >> arbitrarily floated up the upstream independently. There is nothing but >> loathing down that path. > > This is true. I'm not sure how gitlink collisions are currently > handled… They've always been trouble for me. But it may be that I am ignorant. Phil ``` ## W. Trevor King, 2012-11-30 03:27 Subject: Re: [PATCH v5 0/2] submodule update: add --remote for submodule's upstream changes Message-ID: <20121130032719.GE29257@odin.tremily.us> URL: https://gitlist.dev/e/20121130032719.GE29257%40odin.tremily.us In-Reply-To: ``` On Thu, Nov 29, 2012 at 08:11:20PM -0500, Phil Hord wrote: > On Thu, Nov 29, 2012 at 2:13 PM, W. Trevor King wrote: > > On Thu, Nov 29, 2012 at 01:29:12PM -0500, Phil Hord wrote: > >> For that reason, I don't like the --pull switch since it implies a > >> fetch, but I will not always want to do a fetch. > > > > $ git submodule update --remote --no-fetch > > > > will not fetch the submodule remotes. > > This seems precisely backwards to me. Why not use > > $ git submodule update --remote --fetch > > to do your "default" behavior instead? I suppose I am arguing > against the tide of the dominant workflow, but the fetch-by-default > idea needlessly conflates two primitive operations: "float" and > "fetch". Because --no-fetch is the existing option, and if it ain't broke… ;). > >> I don't know which remote I should be tracking, though. I suppose > >> it is 'origin' for now, but maybe it is just whatever > >> $superproject's HEAD's remote-tracking branch indicates. > > > > With the --remote series, I always use "origin" because that's what > > `submodule add` should be setting up. If people want to change that > > up by hand, we may need a submodule..remote configuration > > option. > > I've always felt that the "origin" defaults are broken and are simply > being ignored because most users do not trip over them. But ISTR that > submodule commands use the remote indicated by the superproject's > current remote-tracking configuration, with a fallback to 'origin' if > there is none. Sort of a "best effort" algorithm, I think. Am I > remembering that wrong? The current code uses a bare "git-fetch". I'm not sure what that defaults to if you're on a detached head. If it bothers you, I'm fine adding the submodule..remote option in v6. > >> I am not sure I want the gitlinks in superproject to update automatically > >> in the index, but I definitely do not want to automatically create a commit > >> for them. > > > > Commits are dissabled by default (see my recent --commit RFC for how > > they would be enabled). > > > >> But I really don't want to figure out how to handle submodule > >> collisions during a merge (or rebase!) of my superproject with changes that > >> someone else auto-committed in his local $superproject as he and I > >> arbitrarily floated up the upstream independently. There is nothing but > >> loathing down that path. > > > > This is true. I'm not sure how gitlink collisions are currently > > handled… > > They've always been trouble for me. But it may be that I am ignorant. I haven't dealt with any gitlink merges, but I think that supporting easy gitlink merges is orthogonal to this --remote option. For simple cases like "autocommitted submodule floats", one of the conflicting gitlinks will be an ancestor of the other, so it should be easy to automate that merge. Cheers, Trevor -- This email may be signed or encrypted with GnuPG (http://www.gnupg.org). For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy ``` ## W. Trevor King, 2012-11-30 17:53 Subject: [RFC] remove/deprecate 'submodule init' and 'sync' Message-ID: <20121130175309.GA718@odin.tremily.us> URL: https://gitlist.dev/e/20121130175309.GA718%40odin.tremily.us In-Reply-To: <20121130032719.GE29257@odin.tremily.us> ``` On Wed, Nov 28, 2012 at 12:19:04AM +0100, Jens Lehmann wrote: > Am 26.11.2012 22:00, schrieb W. Trevor King: > > From: "W. Trevor King" > > > > This allows users to override the .gitmodules value with a > > per-repository value. > > Your intentions makes lots of sense, but your patch does more than > that. Copying the branch setting into .git/config sets the initial > branch setting into stone. That makes it impossible to have a branch > "foo" in the superproject using a branch "bar" in a submodule and > another superproject branch "frotz" using branch "nitfol" for the > same submodule. You should use the branch setting from .git/config > if present and fall back to the branch setting from .gitmodules if > not, which would enable the user to have her own setting if she > doesn't like what upstream provides but would still enable others > to follow different submodule branches in different superproject > branches. I've mulling this over, and when I started coding support for submodule..remote, I had an idea. On Thu, Nov 29, 2012 at 10:27:19PM -0500, W. Trevor King wrote: > On Thu, Nov 29, 2012 at 08:11:20PM -0500, Phil Hord wrote: > > I've always felt that the "origin" defaults are broken and are simply > > being ignored because most users do not trip over them. But ISTR that > > submodule commands use the remote indicated by the superproject's > > current remote-tracking configuration, with a fallback to 'origin' if > > there is none. Sort of a "best effort" algorithm, I think. Am I > > remembering that wrong? > > The current code uses a bare "git-fetch". I'm not sure what that > defaults to if you're on a detached head. If it bothers you, I'm fine > adding the submodule..remote option in v6. In my v5 patch, I check for submodule..remote first in the usual `git config` files. If I don't find what I'm looking for I fall back on .gitmodules (basically Jens' suggestion). However, my initial copying-to-.git/config approach was mostly done to mimic existing configuration handling in git-submodule.sh. Since I agree with Jens on configuration precendence, and I now had two options to read (.branch and .remote), I thought I'd pull the logic out into its own function (code included at the end). While I was shifting the existing submodule config handling over to my new function, I noticed that with this logic, `submodule init` doesn't really do anything important anymore. Likewise for `submodule sync`, which seems to be quite similar to `init`. What to do about this? `init` has been around for a while, so we can't just remove it (maybe in 2.0?). Leaving it in place is not really a problem though, it just means that the user is locking in the current .gitmodules configuration (as Jens pointed out with respect to .branch). I may be way off base here, as I'm fairly new to submodules in general and these two commands in particular, but I thought I'd float the idea. Cheers, Trevor --- # # Print a submodule configuration setting # # $1 = submodule name # $2 = option name # $3 = default value # # Checks in the usual git-config places first (for overrides), # otherwise it falls back on .gitmodules. This allows you to # distribute project-wide defaults in .gitmodules, while still # customizing individual repositories if necessary. If the option is # not in .gitmodules either, print a default value. # get_submodule_config() { name="$1" option="$2" default="$3" value=$(git config submodule."$name"."$option") if test -z "$value" then value=$(git config -f .gitmodules submodule."$name"."$option") fi printf '%s' "${value:-$default}" } -- This email may be signed or encrypted with GnuPG (http://www.gnupg.org). For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy ``` ## W. Trevor King, 2012-11-30 18:17 Subject: Re: [RFC] remove/deprecate 'submodule init' and 'sync' Message-ID: <20121130181709.GA967@odin.tremily.us> URL: https://gitlist.dev/e/20121130181709.GA967%40odin.tremily.us In-Reply-To: <20121130175309.GA718@odin.tremily.us> ``` On Fri, Nov 30, 2012 at 12:53:09PM -0500, W. Trevor King wrote: > Likewise for `submodule sync`, which seems to be > quite similar to `init`. Ah, I'd remove the part of `sync` that touches the superproject's .git/config, but keep the part that stores the superproject-reorded URL in the submodule's config: url=$(get_submodule_config "$name" url) up_path=$(get_up_path "$sm_path") url=$(resolve_relative_url "$url" "$up_path") && if test -n "$url" then if test -e "$sm_path"/.git then ( clear_local_git_env cd "$sm_path" remote=$(get_default_remote) git config remote."$remote".url "$url" ) fi fi I should probably also tweak sync to do similar things with submodule..branch and .remote as part of my `--update remote` series. Cheers, Trevor -- This email may be signed or encrypted with GnuPG (http://www.gnupg.org). For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy ``` ## Phil Hord, 2012-11-30 23:52 Subject: Re: [RFC] remove/deprecate 'submodule init' and 'sync' Message-ID: URL: https://gitlist.dev/e/CABURp0qNBcFnxbvhn7PsKWLUOsTiK4u5vx-%3D6cG3JQHw9aUeHA%40mail.gmail.com In-Reply-To: <20121130175309.GA718@odin.tremily.us> ``` On Fri, Nov 30, 2012 at 12:53 PM, W. Trevor King wrote: > On Wed, Nov 28, 2012 at 12:19:04AM +0100, Jens Lehmann wrote: >> Am 26.11.2012 22:00, schrieb W. Trevor King: >> > From: "W. Trevor King" >> > >> > This allows users to override the .gitmodules value with a >> > per-repository value. >> >> Your intentions makes lots of sense, but your patch does more than >> that. Copying the branch setting into .git/config sets the initial >> branch setting into stone. That makes it impossible to have a branch >> "foo" in the superproject using a branch "bar" in a submodule and >> another superproject branch "frotz" using branch "nitfol" for the >> same submodule. You should use the branch setting from .git/config >> if present and fall back to the branch setting from .gitmodules if >> not, which would enable the user to have her own setting if she >> doesn't like what upstream provides but would still enable others >> to follow different submodule branches in different superproject >> branches. > > I've mulling this over, and when I started coding support for > submodule..remote, I had an idea. > > On Thu, Nov 29, 2012 at 10:27:19PM -0500, W. Trevor King wrote: >> On Thu, Nov 29, 2012 at 08:11:20PM -0500, Phil Hord wrote: >> > I've always felt that the "origin" defaults are broken and are simply >> > being ignored because most users do not trip over them. But ISTR that >> > submodule commands use the remote indicated by the superproject's >> > current remote-tracking configuration, with a fallback to 'origin' if >> > there is none. Sort of a "best effort" algorithm, I think. Am I >> > remembering that wrong? >> >> The current code uses a bare "git-fetch". I'm not sure what that >> defaults to if you're on a detached head. If it bothers you, I'm fine >> adding the submodule..remote option in v6. > > In my v5 patch, I check for submodule..remote first in the usual > `git config` files. If I don't find what I'm looking for I fall back > on .gitmodules (basically Jens' suggestion). However, my initial > copying-to-.git/config approach was mostly done to mimic existing > configuration handling in git-submodule.sh. Since I agree with Jens > on configuration precendence, and I now had two options to read > (.branch and .remote), I thought I'd pull the logic out into its own > function (code included at the end). While I was shifting the > existing submodule config handling over to my new function, I noticed > that with this logic, `submodule init` doesn't really do anything > important anymore. If I never 'submodule init' a submodule, it does not get visited by 'git submodule foreach', among others. I think some people use this behavior explicitly. On the other hand, I've also notice that a submodule which I have removed does not get de-inited later one. It causes my 'git submodule foreach' to emit errors. :-( Phil > Likewise for `submodule sync`, which seems to be > quite similar to `init`. > > What to do about this? `init` has been around for a while, so we > can't just remove it (maybe in 2.0?). Leaving it in place is not > really a problem though, it just means that the user is locking in the > current .gitmodules configuration (as Jens pointed out with respect to > .branch). > > I may be way off base here, as I'm fairly new to submodules in general > and these two commands in particular, but I thought I'd float the > idea. > > Cheers, > Trevor > > --- > # > # Print a submodule configuration setting > # > # $1 = submodule name > # $2 = option name > # $3 = default value > # > # Checks in the usual git-config places first (for overrides), > # otherwise it falls back on .gitmodules. This allows you to > # distribute project-wide defaults in .gitmodules, while still > # customizing individual repositories if necessary. If the option is > # not in .gitmodules either, print a default value. > # > get_submodule_config() > { > name="$1" > option="$2" > default="$3" > value=$(git config submodule."$name"."$option") > if test -z "$value" > then > value=$(git config -f .gitmodules submodule."$name"."$option") > fi > printf '%s' "${value:-$default}" > } > > -- > This email may be signed or encrypted with GnuPG (http://www.gnupg.org). > For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy ``` ## W. Trevor King, 2012-12-01 12:48 Subject: Re: [RFC] remove/deprecate 'submodule init' and 'sync' Message-ID: <20121201124842.GA32291@odin.tremily.us> URL: https://gitlist.dev/e/20121201124842.GA32291%40odin.tremily.us In-Reply-To: ``` On Fri, Nov 30, 2012 at 06:52:22PM -0500, Phil Hord wrote: > If I never 'submodule init' a submodule, it does not get visited by > 'git submodule foreach', among others. I think some people use this > behavior explicitly. This is something I'll fix while working up a trial patch. Currently cmd_update calls module_clone if the /.git does not exist. This should probably happen in each command (in a wrapper around module_list?). It's possible that module_list itself would need some work, but I haven't absorbed its implementation yet [1]. Trevor [1]: I read Perl by sounding out each letter ;). -- This email may be signed or encrypted with GnuPG (http://www.gnupg.org). For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy ``` ## Jens Lehmann, 2012-12-01 15:38 Subject: Re: [RFC] remove/deprecate 'submodule init' and 'sync' Message-ID: <50BA245A.5000702@web.de> URL: https://gitlist.dev/e/50BA245A.5000702%40web.de In-Reply-To: <20121130175309.GA718@odin.tremily.us> ``` Am 30.11.2012 18:53, schrieb W. Trevor King: > In my v5 patch, I check for submodule..remote first in the usual > `git config` files. If I don't find what I'm looking for I fall back > on .gitmodules (basically Jens' suggestion). However, my initial > copying-to-.git/config approach was mostly done to mimic existing > configuration handling in git-submodule.sh. Since I agree with Jens > on configuration precendence, and I now had two options to read > (.branch and .remote), I thought I'd pull the logic out into its own > function (code included at the end). While I was shifting the > existing submodule config handling over to my new function, I noticed > that with this logic, `submodule init` doesn't really do anything > important anymore. Likewise for `submodule sync`, which seems to be > quite similar to `init`. You need to handle the 'url' setting differently. While I think the 'update' setting should not be copied into .git/config at all (because it makes it impossible for upstream to change that later without the user copying that himself as 'sync' doesn't do that) the 'url' setting in .git/config has two important implications: 1) It tells the submodule commands that the user wants to have that submodule populated (which is done in a subsequent "update" after "init" copied the url there). 2) It can be used to follow moving upstreams (think of checking out an earlier commit before the upstream was moved, you won't be able to clone it from there without having the new setting persist). And which repository you follow is a matter of trust, so the extra "git submodule sync" in that case is a good thing to have. So I believe 'url' is the only setting that should be copied into .git/config while all the others shouldn't. > What to do about this? `init` has been around for a while, so we > can't just remove it (maybe in 2.0?). Leaving it in place is not > really a problem though, it just means that the user is locking in the > current .gitmodules configuration (as Jens pointed out with respect to > .branch). We still need those commands to set and update the "url" setting. > --- > # > # Print a submodule configuration setting > # > # $1 = submodule name > # $2 = option name > # $3 = default value > # > # Checks in the usual git-config places first (for overrides), > # otherwise it falls back on .gitmodules. This allows you to > # distribute project-wide defaults in .gitmodules, while still > # customizing individual repositories if necessary. If the option is > # not in .gitmodules either, print a default value. > # > get_submodule_config() > { > name="$1" > option="$2" > default="$3" > value=$(git config submodule."$name"."$option") > if test -z "$value" > then > value=$(git config -f .gitmodules submodule."$name"."$option") > fi > printf '%s' "${value:-$default}" > } Something like that makes sense. You can use it for the settings you add first and we can then reuse that for 'update' in a separate patch later. ``` ## Jens Lehmann, 2012-12-01 15:42 Subject: Re: [RFC] remove/deprecate 'submodule init' and 'sync' Message-ID: <50BA257F.9070607@web.de> URL: https://gitlist.dev/e/50BA257F.9070607%40web.de In-Reply-To: <20121201124842.GA32291@odin.tremily.us> ``` Am 01.12.2012 13:48, schrieb W. Trevor King: > On Fri, Nov 30, 2012 at 06:52:22PM -0500, Phil Hord wrote: >> If I never 'submodule init' a submodule, it does not get visited by 'git submodule foreach', among others. I think some people use this behavior explicitly. > > This is something I'll fix while working up a trial patch. Currently cmd_update calls module_clone if the /.git does not exist. This should probably happen in each command (in a wrapper around module_list?). It's possible that module_list itself would need some work, but I haven't absorbed its implementation yet [1]. Please do not fix it, this is a feature. "update" is the only command where that should happen (and only if "url" is set in .git/config, as I explained in my other mail). So everything should be fine here. ``` ## Jens Lehmann, 2012-12-01 15:56 Subject: Re: [RFC] remove/deprecate 'submodule init' and 'sync' Message-ID: <50BA2892.7060706@web.de> URL: https://gitlist.dev/e/50BA2892.7060706%40web.de In-Reply-To: ``` Am 01.12.2012 00:52, schrieb Phil Hord: > If I never 'submodule init' a submodule, it does not get visited by > 'git submodule foreach', among others. I think some people use this > behavior explicitly. > > On the other hand, I've also notice that a submodule which I have > removed does not get de-inited later one. It causes my 'git submodule > foreach' to emit errors. :-( I'm currently hacking on "git submodule deinit" which removes the 'url' setting from git/config. This should do the trick for you, right? Just removing that submodule automagically would not work that well, as it would deinitialize a submodule when you switch to a branch where it isn't present and you'd have to reinitialize it when you come back. ``` ## W. Trevor King, 2012-12-01 16:30 Subject: Re: [RFC] remove/deprecate 'submodule init' and 'sync' Message-ID: <20121201163004.GB4823@odin.tremily.us> URL: https://gitlist.dev/e/20121201163004.GB4823%40odin.tremily.us In-Reply-To: <50BA257F.9070607@web.de> ``` On Sat, Dec 01, 2012 at 04:38:02PM +0100, Jens Lehmann wrote: > Am 30.11.2012 18:53, schrieb W. Trevor King: > > In my v5 patch, I check for submodule..remote first in the usual > > `git config` files. If I don't find what I'm looking for I fall back > > on .gitmodules (basically Jens' suggestion). However, my initial > > copying-to-.git/config approach was mostly done to mimic existing > > configuration handling in git-submodule.sh. Since I agree with Jens > > on configuration precendence, and I now had two options to read > > (.branch and .remote), I thought I'd pull the logic out into its own > > function (code included at the end). While I was shifting the > > existing submodule config handling over to my new function, I noticed > > that with this logic, `submodule init` doesn't really do anything > > important anymore. Likewise for `submodule sync`, which seems to be > > quite similar to `init`. > > You need to handle the 'url' setting differently. While I think the > 'update' setting should not be copied into .git/config at all > (because it makes it impossible for upstream to change that later > without the user copying that himself as 'sync' doesn't do that) the > 'url' setting in .git/config has two important implications: > > 1) It tells the submodule commands that the user wants to have that > submodule populated (which is done in a subsequent "update" after > "init" copied the url there). Good point, but this should depend on submodule..update; having it as a side effect of a local submodule..url makes no sense. Perhaps `submodule init` should be reduced to just wrap: $ git config submodule..update checkout where the default update configuration would be 'none'. > 2) It can be used to follow moving upstreams (think of checking out > an earlier commit before the upstream was moved, you won't be able > to clone it from there without having the new setting persist). > And which repository you follow is a matter of trust, so the extra > "git submodule sync" in that case is a good thing to have. > > So I believe 'url' is the only setting that should be copied into > .git/config while all the others shouldn't. If you want to override the old repository location for an old commit, setting submodule..url makes sense. My rewritten `sync` updates the local submodule..url in the superproject if the configuration option is already set [1]. Perhaps a `sync --local` invocation should forcibly populate the local submodule..url to make this workflow easier. Bundling sugar for this special case should not happen under an extra command called `init`. > > [snip get_submodule_config()] > > Something like that makes sense. You can use it for the settings you add > first and we can then reuse that for 'update' in a separate patch later. I'm currently working out the details independently against v1.8.0. This will be a fairly major shift, so I think it should stay independent of `update --remote`. The `update --remote` stuff should be easy to adjust/rebase if the `init` removal/adjustment develops into something acceptable. Cheers, Trevor -- This email may be signed or encrypted with GnuPG (http://www.gnupg.org). For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy ``` ## W. Trevor King, 2012-12-01 16:37 Subject: Re: [RFC] remove/deprecate 'submodule init' and 'sync' Message-ID: <20121201163714.GC4823@odin.tremily.us> URL: https://gitlist.dev/e/20121201163714.GC4823%40odin.tremily.us In-Reply-To: <50BA2892.7060706@web.de> ``` On Sat, Dec 01, 2012 at 04:56:02PM +0100, Jens Lehmann wrote: > Am 01.12.2012 00:52, schrieb Phil Hord: > > If I never 'submodule init' a submodule, it does not get visited by > > 'git submodule foreach', among others. I think some people use this > > behavior explicitly. > > > > On the other hand, I've also notice that a submodule which I have > > removed does not get de-inited later one. It causes my 'git submodule > > foreach' to emit errors. :-( > > I'm currently hacking on "git submodule deinit" which removes the 'url' > setting from git/config. This should do the trick for you, right? > > Just removing that submodule automagically would not work that well, as > it would deinitialize a submodule when you switch to a branch where it > isn't present and you'd have to reinitialize it when you come back. I think this is another case where we should be looping through submodules based on the revision-specific .gitmodules content, and querying the local config only to determine if the user wants to update them (to drop into them with foreach, etc.). -- This email may be signed or encrypted with GnuPG (http://www.gnupg.org). For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy ``` ## Jens Lehmann, 2012-12-01 16:45 Subject: [PATCH] submodule: add 'deinit' command Message-ID: <50BA3412.60309@web.de> URL: https://gitlist.dev/e/50BA3412.60309%40web.de In-Reply-To: <50BA2892.7060706@web.de> ``` With "git submodule init" the user is able to tell git he cares about one or more submodules and wants to have it populated on the next call to "git submodule update". But currently there is no easy way he could tell git he does not care about a submodule anymore and wants to get rid of his local work tree (except he knows a lot about submodule internals and removes the "submodule.$name.url" setting from .git/config himself). Help those users by providing a 'deinit' command. This removes the url setting from .git/config either for the given submodule(s) or for all those which have been initialized if none were given. Complain only when for a submodule given on the command line the url setting can't be found in .git/config. Add tests and link the man pages of "git submodule deinit" and "git rm" to assist the user in deciding whether removing or unregistering the submodule is the right thing to do for him. Signed-off-by: Jens Lehmann --- Am 01.12.2012 16:56, schrieb Jens Lehmann: > Am 01.12.2012 00:52, schrieb Phil Hord: >> If I never 'submodule init' a submodule, it does not get visited by >> 'git submodule foreach', among others. I think some people use this >> behavior explicitly. >> >> On the other hand, I've also notice that a submodule which I have >> removed does not get de-inited later one. It causes my 'git submodule >> foreach' to emit errors. :-( > > I'm currently hacking on "git submodule deinit" which removes the 'url' > setting from git/config. This should do the trick for you, right? And here we go ... Documentation/git-rm.txt | 4 ++++ Documentation/git-submodule.txt | 11 +++++++++ git-submodule.sh | 50 ++++++++++++++++++++++++++++++++++++++++- t/t7400-submodule-basic.sh | 11 +++++++++ 4 files changed, 75 insertions(+), 1 deletion(-) diff --git a/Documentation/git-rm.txt b/Documentation/git-rm.txt index 262436b..ec42bf5 100644 --- a/Documentation/git-rm.txt +++ b/Documentation/git-rm.txt @@ -149,6 +149,10 @@ files that aren't ignored are present in the submodules work tree. Ignored files are deemed expendable and won't stop a submodule's work tree from being removed. +If you only want to remove the local checkout of a submodule from your +work tree without committing that use `git submodule deinit` instead +(see linkgit:git-submodule[1]). + EXAMPLES -------- `git rm Documentation/\*.txt`:: diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt index b1de3ba..fba77f6 100644 --- a/Documentation/git-submodule.txt +++ b/Documentation/git-submodule.txt @@ -13,6 +13,7 @@ SYNOPSIS [--reference ] [--] [] 'git submodule' [--quiet] status [--cached] [--recursive] [--] [...] 'git submodule' [--quiet] init [--] [...] +'git submodule' [--quiet] deinit [--] [...] 'git submodule' [--quiet] update [--init] [-N|--no-fetch] [--rebase] [--reference ] [--merge] [--recursive] [--] [...] 'git submodule' [--quiet] summary [--cached|--files] [(-n|--summary-limit) ] @@ -134,6 +135,16 @@ init:: the explicit 'init' step if you do not intend to customize any submodule locations. +deinit:: + Unregister the submodules, i.e. remove the `submodule.$name.url` + setting from .git/config. Further calls to `git submodule update`, + `git submodule foreach` and `git submodule sync` will skip any + unregistered submodules until they are initialized again, so use + this command if you don't want to have a local checkout of the + submodule in your work tree anymore. If you really want to remove + a submodule from the repository and commit that use + linkgit:git-rm[1] instead. + update:: Update the registered submodules, i.e. clone missing submodules and checkout the commit specified in the index of the containing repository. diff --git a/git-submodule.sh b/git-submodule.sh index 2365149..4059a2e 100755 --- a/git-submodule.sh +++ b/git-submodule.sh @@ -8,6 +8,7 @@ dashless=$(basename "$0" | sed -e 's/-/ /') USAGE="[--quiet] add [-b ] [-f|--force] [--name ] [--reference ] [--] [] or: $dashless [--quiet] status [--cached] [--recursive] [--] [...] or: $dashless [--quiet] init [--] [...] + or: $dashless [--quiet] deinit [--] [...] or: $dashless [--quiet] update [--init] [-N|--no-fetch] [-f|--force] [--rebase] [--reference ] [--merge] [--recursive] [--] [...] or: $dashless [--quiet] summary [--cached|--files] [--summary-limit ] [commit] [--] [...] or: $dashless [--quiet] foreach [--recursive] @@ -516,6 +517,53 @@ cmd_init() } # +# Unregister submodules from .git/config +# +# $@ = requested paths (default to all) +# +cmd_deinit() +{ + # parse $args after "submodule ... init". + while test $# -ne 0 + do + case "$1" in + -q|--quiet) + GIT_QUIET=1 + ;; + --) + shift + break + ;; + -*) + usage + ;; + *) + break + ;; + esac + shift + done + + module_list "$@" | + while read mode sha1 stage sm_path + do + die_if_unmatched "$mode" + name=$(module_name "$sm_path") || exit + url=$(git config submodule."$name".url) + if test -z "$url" + then + # Only mention uninitialized submodules when its + # path have been specified + test "$#" != "0" && + say "$(eval_gettext "No url found for submodule path '\$sm_path' in .git/config")" + continue + fi + git config --unset submodule."$name".url && + say "$(eval_gettext "Submodule '\$name' (\$url) unregistered")" + done +} + +# # Update each submodule path to correct revision, using clone and checkout as needed # # $@ = requested paths (default to all) @@ -1108,7 +1156,7 @@ cmd_sync() while test $# != 0 && test -z "$command" do case "$1" in - add | foreach | init | update | status | summary | sync) + add | foreach | init | deinit | update | status | summary | sync) command=$1 ;; -q|--quiet) diff --git a/t/t7400-submodule-basic.sh b/t/t7400-submodule-basic.sh index de7d453..803bda7 100755 --- a/t/t7400-submodule-basic.sh +++ b/t/t7400-submodule-basic.sh @@ -756,4 +756,15 @@ test_expect_success 'submodule add with an existing name fails unless forced' ' ) ' +test_expect_success 'submodule deinit should unregister submodule url from .git/config' ' + url=$(git config submodule.example.url) && + git submodule deinit && + test -z "$(git config submodule.example.url)" +' + +test_expect_success 'submodule deinit complains only when explicitly used on an uninitialized submodule' ' + git submodule deinit && + test_must_fail git submodule deinit example +' + test_done -- 1.7.11.7 ``` ## W. Trevor King, 2012-12-01 16:54 Subject: Re: [RFC] remove/deprecate 'submodule init' and 'sync' Message-ID: <20121201165404.GD4823@odin.tremily.us> URL: https://gitlist.dev/e/20121201165404.GD4823%40odin.tremily.us In-Reply-To: <20121130175309.GA718@odin.tremily.us> ``` I'm currently stuck with adding a commit-less existing repository as a submodule (which happens in t7400-submodule-basic.sh, ../bar/a/b/c works with relative local path): $ mkdir -p super/sub $ cd super $ git init $ (cd sub && git init) $ git submodule add ./ sub $ git status # On branch master # # Initial commit # # Changes to be committed: # (use "git rm --cached ..." to unstage) # # new file: .gitmodules # What I'm missing is a gitlink form sub for 'Subproject commit 00000...' or some such. When the subproject has an actual commit, things work as expected: $ mkdir -p super/sub $ cd super $ git init $ (cd sub && git init && echo line-1 > file && git add file && git commit -m file) $ git submodule add ./ sub $ git status # On branch master # # Initial commit # # Changes to be committed: # (use "git rm --cached ..." to unstage) # # new file: .gitmodules # new file: sub # This means that module_list isn't aware of the empty submodule, when the user has just explicitly added it. Fixing this would seem to need either 'Subproject commit 00000...' as I suggested earlier, or an adjustment to module_list that also spits out submodules that are in .gitmodules but not in the index. Cheers, Trevor -- This email may be signed or encrypted with GnuPG (http://www.gnupg.org). For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy ``` ## Jens Lehmann, 2012-12-01 17:25 Subject: Re: [RFC] remove/deprecate 'submodule init' and 'sync' Message-ID: <50BA3D7D.8040707@web.de> URL: https://gitlist.dev/e/50BA3D7D.8040707%40web.de In-Reply-To: <20121201163004.GB4823@odin.tremily.us> ``` Am 01.12.2012 17:30, schrieb W. Trevor King: > On Sat, Dec 01, 2012 at 04:38:02PM +0100, Jens Lehmann wrote: >> You need to handle the 'url' setting differently. While I think the 'update' setting should not be copied into .git/config at all (because it makes it impossible for upstream to change that later without the user copying that himself as 'sync' doesn't do that) the 'url' setting in .git/config has two important implications: >> >> 1) It tells the submodule commands that the user wants to have that submodule populated (which is done in a subsequent "update" after "init" copied the url there). > > Good point, but this should depend on submodule..update; having it as a side effect of a local submodule..url makes no sense. Sorry, but that makes tons of sense: url controls if the submodule is to be populated and from where, update controls how (and can even veto populating it if set to "none"). We /could/ do it differently, but I can't see why we should (and risk severe compatibility issues). > Perhaps `submodule init` should be reduced to just wrap: > > $ git config submodule..update checkout > > where the default update configuration would be 'none'. > >> 2) It can be used to follow moving upstreams (think of checking out an earlier commit before the upstream was moved, you won't be able to clone it from there without having the new setting persist). And which repository you follow is a matter of trust, so the extra "git submodule sync" in that case is a good thing to have. >> >> So I believe 'url' is the only setting that should be copied into .git/config while all the others shouldn't. > > If you want to override the old repository location for an old commit, setting submodule..url makes sense. My rewritten `sync` updates the local submodule..url in the superproject if the configuration option is already set [1]. Perhaps a `sync --local` invocation should forcibly populate the local submodule..url to make this workflow easier. Bundling sugar for this special case should not happen under an extra command called `init`. What real world problems do we have with the current init/sync that this approach would solve? >>> [snip get_submodule_config()] >> >> Something like that makes sense. You can use it for the settings you add first and we can then reuse that for 'update' in a separate patch later. > > I'm currently working out the details independently against v1.8.0. This will be a fairly major shift, so I think it should stay independent of `update --remote`. The `update --remote` stuff should be easy to adjust/rebase if the `init` removal/adjustment develops into something acceptable. I totally agree. Let's get the `update --remote` stuff ready first. ``` ## W. Trevor King, 2012-12-01 17:49 Subject: Re: [RFC] remove/deprecate 'submodule init' and 'sync' Message-ID: <20121201174920.GE4823@odin.tremily.us> URL: https://gitlist.dev/e/20121201174920.GE4823%40odin.tremily.us In-Reply-To: <50BA3D7D.8040707@web.de> ``` On Sat, Dec 01, 2012 at 06:25:17PM +0100, Jens Lehmann wrote: > Am 01.12.2012 17:30, schrieb W. Trevor King: > > On Sat, Dec 01, 2012 at 04:38:02PM +0100, Jens Lehmann wrote: > > > 1) It tells the submodule commands that the user wants to have > > > that submodule populated (which is done in a subsequent "update" > > > after "init" copied the url there). > > > > Good point, but this should depend on submodule..update; > > having it as a side effect of a local submodule..url makes > > no sense. > > Sorry, but that makes tons of sense: url controls if the submodule > is to be populated and from where, update controls how (and can even > veto populating it if set to "none"). We /could/ do it differently, > but I can't see why we should (and risk severe compatibility issues). I think removing `init` will cause some compatibility issues anyway, so I was re-imaging how you do it. I don't think update='none' and "don't populate my submodule" are distinct ideas, while a locally configured url="somwhere" and "please populate my submodule" are (with the blank-url case defaulting to the superproject itself). > > > 2) It can be used to follow moving upstreams (think of checking > > > out an earlier commit before the upstream was moved, you won't > > > be able to clone it from there without having the new setting > > > persist). And which repository you follow is a matter of trust, > > > so the extra "git submodule sync" in that case is a good thing > > > to have. > > > > > > So I believe 'url' is the only setting that should be copied > > > into .git/config while all the others shouldn't. > > > > If you want to override the old repository location for an old > > commit, setting submodule..url makes sense. My rewritten > > `sync` updates the local submodule..url in the superproject > > if the configuration option is already set [1]. Perhaps a `sync > > --local` invocation should forcibly populate the local > > submodule..url to make this workflow easier. Bundling sugar > > for this special case should not happen under an extra command > > called `init`. > > What real world problems do we have with the current init/sync that > this approach would solve? I don't have any, but in my `update --remote` series I'm adding two new config options that are handled differently (define in .gitmodules, override in superproject .git/config) than existing submodules options. I'm trying to avoid confusing users by standardizing on the more flexible method, which avoids copying stuff into the superproject's .git/config, and under which the current `init` functionality doesn't make much sense. > > > > [snip get_submodule_config()] > > > > > > Something like that makes sense. You can use it for the settings > > > you add first and we can then reuse that for 'update' in a > > > separate patch later. > > > > I'm currently working out the details independently against > > v1.8.0. This will be a fairly major shift, so I think it should > > stay independent of `update --remote`. The `update --remote` > > stuff should be easy to adjust/rebase if the `init` > > removal/adjustment develops into something acceptable. > > I totally agree. Let's get the `update --remote` stuff ready first. Ok, but we'll have the possible confusion about option setting that I mention above. Still, it's good to minimize the number of irons in the fire, and an `init` removal will probably not get in until 2.0 anyway. If other people are fine with the different initialization paths, I'll put the init-removal on hold for now. Cheers, Trevor -- This email may be signed or encrypted with GnuPG (http://www.gnupg.org). For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy ``` ## Jens Lehmann, 2012-12-01 18:04 Subject: Re: [RFC] remove/deprecate 'submodule init' and 'sync' Message-ID: <50BA4695.7030008@web.de> URL: https://gitlist.dev/e/50BA4695.7030008%40web.de In-Reply-To: <20121201174920.GE4823@odin.tremily.us> ``` Am 01.12.2012 18:49, schrieb W. Trevor King: > I think removing `init` will cause some compatibility issues anyway, > so I was re-imaging how you do it. I don't think update='none' and > "don't populate my submodule" are distinct ideas, while a locally > configured url="somwhere" and "please populate my submodule" are (with > the blank-url case defaulting to the superproject itself). Why would we want to remove "init"? It still has to copy the "url" setting (and it would be a compatibility nightmare if we would change that, imagine different git versions used on the same work tree). >> What real world problems do we have with the current init/sync that >> this approach would solve? > > I don't have any, but in my `update --remote` series I'm adding two > new config options that are handled differently (define in > .gitmodules, override in superproject .git/config) than existing > submodules options. No, they're not. They are just handled differently than "url" and "update", but will behave just like "fetchRecurseSubmodules" and "ignore" do since day one. And as I explained in another mail I think "url" is special and "update" should be change to behave like the other two some day. ``` ## W. Trevor King, 2012-12-01 18:16 Subject: Re: [RFC] remove/deprecate 'submodule init' and 'sync' Message-ID: <20121201181643.GF4823@odin.tremily.us> URL: https://gitlist.dev/e/20121201181643.GF4823%40odin.tremily.us In-Reply-To: <50BA4695.7030008@web.de> ``` On Sat, Dec 01, 2012 at 07:04:05PM +0100, Jens Lehmann wrote: > Am 01.12.2012 18:49, schrieb W. Trevor King: > > I think removing `init` will cause some compatibility issues anyway, > > so I was re-imaging how you do it. I don't think update='none' and > > "don't populate my submodule" are distinct ideas, while a locally > > configured url="somwhere" and "please populate my submodule" are (with > > the blank-url case defaulting to the superproject itself). > > Why would we want to remove "init"? It still has to copy the "url" > setting (and it would be a compatibility nightmare if we would change > that, imagine different git versions used on the same work tree). In my init-less rewrite, it doesn't have to copy the url setting. People using older versions of Git would need to run `init` using their old version. Having the url defined in .git/config won't break my init-less submodule commands, it just means that the value in .gitmodules will be masked. > >> What real world problems do we have with the current init/sync that > >> this approach would solve? > > > > I don't have any, but in my `update --remote` series I'm adding two > > new config options that are handled differently (define in > > .gitmodules, override in superproject .git/config) than existing > > submodules options. > > No, they're not. They are just handled differently than "url" and > "update", but will behave just like "fetchRecurseSubmodules" and > "ignore" do since day one. And as I explained in another mail I > think "url" is special and "update" should be change to behave like > the other two some day. I somehow missed those earlier. Thanks for correcting my tunnel vision. This makes me much happier about postponing the init-removal. Cheers, Trevor -- This email may be signed or encrypted with GnuPG (http://www.gnupg.org). For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy ``` ## Junio C Hamano, 2012-12-02 02:00 Subject: Re: [PATCH] submodule: add 'deinit' command Message-ID: <7vy5hhmcwp.fsf@alter.siamese.dyndns.org> URL: https://gitlist.dev/e/7vy5hhmcwp.fsf%40alter.siamese.dyndns.org In-Reply-To: <50BA3412.60309@web.de> ``` Jens Lehmann writes: > With "git submodule init" the user is able to tell git he cares about one > or more submodules and wants to have it populated on the next call to "git > submodule update". But currently there is no easy way he could tell git he > does not care about a submodule anymore and wants to get rid of his local > work tree (except he knows a lot about submodule internals and removes the > "submodule.$name.url" setting from .git/config himself). > > Help those users by providing a 'deinit' command. This removes the url > setting from .git/config either for the given submodule(s) or for all > those which have been initialized if none were given. Complain only when > for a submodule given on the command line the url setting can't be found > in .git/config. > > Add tests and link the man pages of "git submodule deinit" and "git rm" to > assist the user in deciding whether removing or unregistering the submodule > is the right thing to do for him. > > Signed-off-by: Jens Lehmann > --- I fully agree with your analysis on the reason why the "url" element is special and has to be copied to $GIT_DIR/config, but when you deinit (or uninit) a submodule to say you are no longer interested in it and do not want it populated in the context of the superproject, I am not sure if removing only submodule.$name.url (so that when you later decide to "init" it again, you will keep the values for submodule.$name.update and other things from the previous life) is the sane thing to do, or it is better to remove submodule.$name.* altogether as if an earlier "init" has never happened. Would it be worth analyzing the pros-and-cons here? > Am 01.12.2012 16:56, schrieb Jens Lehmann: >> Am 01.12.2012 00:52, schrieb Phil Hord: >>> If I never 'submodule init' a submodule, it does not get visited by >>> 'git submodule foreach', among others. I think some people use this >>> behavior explicitly. >>> >>> On the other hand, I've also notice that a submodule which I have >>> removed does not get de-inited later one. It causes my 'git submodule >>> foreach' to emit errors. :-( >> >> I'm currently hacking on "git submodule deinit" which removes the 'url' >> setting from git/config. This should do the trick for you, right? > > And here we go ... > > > Documentation/git-rm.txt | 4 ++++ > Documentation/git-submodule.txt | 11 +++++++++ > git-submodule.sh | 50 ++++++++++++++++++++++++++++++++++++++++- > t/t7400-submodule-basic.sh | 11 +++++++++ > 4 files changed, 75 insertions(+), 1 deletion(-) > > diff --git a/Documentation/git-rm.txt b/Documentation/git-rm.txt > index 262436b..ec42bf5 100644 > --- a/Documentation/git-rm.txt > +++ b/Documentation/git-rm.txt > @@ -149,6 +149,10 @@ files that aren't ignored are present in the submodules work tree. > Ignored files are deemed expendable and won't stop a submodule's work > tree from being removed. > > +If you only want to remove the local checkout of a submodule from your > +work tree without committing that use `git submodule deinit` instead > +(see linkgit:git-submodule[1]). > + > EXAMPLES > -------- > `git rm Documentation/\*.txt`:: > diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt > index b1de3ba..fba77f6 100644 > --- a/Documentation/git-submodule.txt > +++ b/Documentation/git-submodule.txt > @@ -13,6 +13,7 @@ SYNOPSIS > [--reference ] [--] [] > 'git submodule' [--quiet] status [--cached] [--recursive] [--] [...] > 'git submodule' [--quiet] init [--] [...] > +'git submodule' [--quiet] deinit [--] [...] > 'git submodule' [--quiet] update [--init] [-N|--no-fetch] [--rebase] > [--reference ] [--merge] [--recursive] [--] [...] > 'git submodule' [--quiet] summary [--cached|--files] [(-n|--summary-limit) ] > @@ -134,6 +135,16 @@ init:: > the explicit 'init' step if you do not intend to customize > any submodule locations. > > +deinit:: > + Unregister the submodules, i.e. remove the `submodule.$name.url` > + setting from .git/config. Further calls to `git submodule update`, > + `git submodule foreach` and `git submodule sync` will skip any > + unregistered submodules until they are initialized again, so use > + this command if you don't want to have a local checkout of the > + submodule in your work tree anymore. If you really want to remove > + a submodule from the repository and commit that use > + linkgit:git-rm[1] instead. > + > update:: > Update the registered submodules, i.e. clone missing submodules and > checkout the commit specified in the index of the containing repository. > diff --git a/git-submodule.sh b/git-submodule.sh > index 2365149..4059a2e 100755 > --- a/git-submodule.sh > +++ b/git-submodule.sh > @@ -8,6 +8,7 @@ dashless=$(basename "$0" | sed -e 's/-/ /') > USAGE="[--quiet] add [-b ] [-f|--force] [--name ] [--reference ] [--] [] > or: $dashless [--quiet] status [--cached] [--recursive] [--] [...] > or: $dashless [--quiet] init [--] [...] > + or: $dashless [--quiet] deinit [--] [...] > or: $dashless [--quiet] update [--init] [-N|--no-fetch] [-f|--force] [--rebase] [--reference ] [--merge] [--recursive] [--] [...] > or: $dashless [--quiet] summary [--cached|--files] [--summary-limit ] [commit] [--] [...] > or: $dashless [--quiet] foreach [--recursive] > @@ -516,6 +517,53 @@ cmd_init() > } > > # > +# Unregister submodules from .git/config > +# > +# $@ = requested paths (default to all) > +# > +cmd_deinit() > +{ > + # parse $args after "submodule ... init". > + while test $# -ne 0 > + do > + case "$1" in > + -q|--quiet) > + GIT_QUIET=1 > + ;; > + --) > + shift > + break > + ;; > + -*) > + usage > + ;; > + *) > + break > + ;; > + esac > + shift > + done > + > + module_list "$@" | > + while read mode sha1 stage sm_path > + do > + die_if_unmatched "$mode" > + name=$(module_name "$sm_path") || exit > + url=$(git config submodule."$name".url) > + if test -z "$url" > + then > + # Only mention uninitialized submodules when its > + # path have been specified > + test "$#" != "0" && > + say "$(eval_gettext "No url found for submodule path '\$sm_path' in .git/config")" > + continue > + fi > + git config --unset submodule."$name".url && > + say "$(eval_gettext "Submodule '\$name' (\$url) unregistered")" > + done > +} > + > +# > # Update each submodule path to correct revision, using clone and checkout as needed > # > # $@ = requested paths (default to all) > @@ -1108,7 +1156,7 @@ cmd_sync() > while test $# != 0 && test -z "$command" > do > case "$1" in > - add | foreach | init | update | status | summary | sync) > + add | foreach | init | deinit | update | status | summary | sync) > command=$1 > ;; > -q|--quiet) > diff --git a/t/t7400-submodule-basic.sh b/t/t7400-submodule-basic.sh > index de7d453..803bda7 100755 > --- a/t/t7400-submodule-basic.sh > +++ b/t/t7400-submodule-basic.sh > @@ -756,4 +756,15 @@ test_expect_success 'submodule add with an existing name fails unless forced' ' > ) > ' > > +test_expect_success 'submodule deinit should unregister submodule url from .git/config' ' > + url=$(git config submodule.example.url) && > + git submodule deinit && > + test -z "$(git config submodule.example.url)" > +' > + > +test_expect_success 'submodule deinit complains only when explicitly used on an uninitialized submodule' ' > + git submodule deinit && > + test_must_fail git submodule deinit example > +' > + > test_done ``` ## W. Trevor King, 2012-12-02 03:17 Subject: [PATCH v6 0/4] submodule update: add --remote for submodule's upstream changes Message-ID: URL: https://gitlist.dev/e/cover.1354417618.git.wking%40tremily.us In-Reply-To: <20121130032719.GE29257@odin.tremily.us> ``` From: "W. Trevor King" On Thu, Nov 29, 2012 at 10:27:19PM -0500, W. Trevor King wrote: > On Thu, Nov 29, 2012 at 08:11:20PM -0500, Phil Hord wrote: > > I've always felt that the "origin" defaults are broken and are simply > > being ignored because most users do not trip over them. But ISTR that > > submodule commands use the remote indicated by the superproject's > > current remote-tracking configuration, with a fallback to 'origin' if > > there is none. Sort of a "best effort" algorithm, I think. Am I > > remembering that wrong? > > The current code uses a bare "git-fetch". I'm not sure what that > defaults to if you're on a detached head. If it bothers you, I'm fine > adding the submodule..remote option in v6. Here it is. Now the remote defaults to $(get_default_remote) with an optional override via submodule..remote. Changes since v5: * New patch 1 for easy config variable setup. * Minor tweaks and rewordings in patches 2 and 3 (v5 patches 1 and 2). * New patch 4 adding submodule..remote. I'm fine with squashing patches 1, 2, and 4 together, if people prefer a more compact series. W. Trevor King (4): submodule: add get_submodule_config helper funtion submodule update: add --remote for submodule's upstream changes submodule add: If --branch is given, record it in .gitmodules submodule update: add submodule..remote config option Documentation/config.txt | 8 ++++- Documentation/git-submodule.txt | 27 ++++++++++++++- Documentation/gitmodules.txt | 5 +++ git-submodule.sh | 74 ++++++++++++++++++++++++++++++++++++++--- t/t7400-submodule-basic.sh | 1 + t/t7406-submodule-update.sh | 49 +++++++++++++++++++++++++++ 6 files changed, 158 insertions(+), 6 deletions(-) -- 1.8.0.4.gf74b0fc.dirty ``` ## W. Trevor King, 2012-12-02 03:17 Subject: [PATCH v6 1/4] submodule: add get_submodule_config helper funtion Message-ID: <436a73a8fdc8f0695aa597d53483d4c4bae16ebb.1354417618.git.wking@tremily.us> URL: https://gitlist.dev/e/436a73a8fdc8f0695aa597d53483d4c4bae16ebb.1354417618.git.wking%40tremily.us In-Reply-To: ``` From: "W. Trevor King" Several submodule configuration variables (e.g. fetchRecurseSubmodules) are read from .gitmodules with local overrides from the usual git config files. This shell function mimics that logic to help initialize configuration variables in git-submodule.sh. Signed-off-by: W. Trevor King --- git-submodule.sh | 27 +++++++++++++++++++++++++++ 1 file changed, 27 insertions(+) diff --git a/git-submodule.sh b/git-submodule.sh index ab6b110..97ce5e4 100755 --- a/git-submodule.sh +++ b/git-submodule.sh @@ -152,6 +152,33 @@ die_if_unmatched () } # +# Print a submodule configuration setting +# +# $1 = submodule name +# $2 = option name +# $3 = default value +# +# Checks in the usual git-config places first (for overrides), +# otherwise it falls back on .gitmodules. This allows you to +# distribute project-wide defaults in .gitmodules, while still +# customizing individual repositories if necessary. If the option is +# not in .gitmodules either, print a default value. +# +get_submodule_config() +{ + name="$1" + option="$2" + default="$3" + value=$(git config submodule."$name"."$option") + if test -z "$value" + then + value=$(git config -f .gitmodules submodule."$name"."$option") + fi + printf '%s' "${value:-$default}" +} + + +# # Map submodule path to submodule name # # $1 = path -- 1.8.0.4.gf74b0fc.dirty ``` ## W. Trevor King, 2012-12-02 03:17 Subject: [PATCH v6 2/4] submodule update: add --remote for submodule's upstream changes Message-ID: URL: https://gitlist.dev/e/ec5d0235322619aff6c1c64b0a346efb0e4d0a32.1354417618.git.wking%40tremily.us In-Reply-To: ``` From: "W. Trevor King" The current `update` command incorporates the superproject's gitlinked SHA-1 ($sha1) into the submodule HEAD ($subsha1). Depending on the options you use, it may checkout $sha1, rebase the $subsha1 onto $sha1, or merge $sha1 into $subsha1. This helps you keep up with changes in the upstream superproject. However, it's also useful to stay up to date with changes in the upstream subproject. Previous workflows for incorporating such changes include the ungainly: $ git submodule foreach 'git checkout $(git config --file $toplevel/.gitmodules submodule.$name.branch) && git pull' With this patch, all of the useful functionality for incorporating superproject changes can be reused to incorporate upstream subproject updates. When you specify --remote, the target $sha1 is replaced with a $sha1 of the submodule's origin/master tracking branch. If you want to merge a different tracking branch, you can configure the `submodule..branch` option in `.gitmodules`. You can override the `.gitmodules` configuration setting for a particular superproject by configuring the option in that superproject's default configuration (using the usual configuration hierarchy, e.g. `.git/config`, `~/.gitconfig`, etc.). Previous use of submodule..branch ======================================= Because we're adding a new configuration option, it's a good idea to check if anyone else is already using the option. The foreach-pull example above was described by Ævar in commit f030c96d8643fa0a1a9b2bd9c2f36a77721fb61f Author: Ævar Arnfjörð Bjarmason Date: Fri May 21 16:10:10 2010 +0000 git-submodule foreach: Add $toplevel variable Gerrit uses the same interpretation for the setting, but because Gerrit has direct access to the subproject repositories, it updates the superproject repositories automatically when a subproject changes. Gerrit also accepts the special value '.', which it expands into the superproject's branch name. Although the --remote functionality is using `submodule..branch` slightly differently, the effect is the same. The foreach-pull example uses the option to record the name of the local branch to checkout before pulls. The tracking branch to be pulled is recorded in `.git/modules//config`, which was initialized by the module clone during `submodule add` or `submodule init`. Because the branch name stored in `submodule..branch` was likely the same as the branch name used during the initial `submodule add`, the same branch will be pulled in each workflow. Implementation details ====================== In order to ensure a current tracking branch state, `update --remote` fetches the submodule's remote repository before calculating the SHA-1. However, I didn't change the logic guarding the existing fetch: if test -z "$nofetch" then # Run fetch only if $sha1 isn't present or it # is not reachable from a ref. (clear_local_git_env; cd "$path" && ( (rev=$(git rev-list -n 1 $sha1 --not --all 2>/dev/null) && test -z "$rev") || git-fetch)) || die "$(eval_gettext "Unable to fetch in submodule path '\$path'")" fi There will not be a double-fetch, because the new $sha1 determined after the `--remote` triggered fetch should always exist in the repository. If it doesn't, it's because some racy process removed it from the submodule's repository and we *should* be re-fetching. Signed-off-by: W. Trevor King --- Documentation/config.txt | 7 ++++++- Documentation/git-submodule.txt | 25 ++++++++++++++++++++++++- Documentation/gitmodules.txt | 5 +++++ git-submodule.sh | 22 +++++++++++++++++++++- t/t7406-submodule-update.sh | 31 +++++++++++++++++++++++++++++++ 5 files changed, 87 insertions(+), 3 deletions(-) diff --git a/Documentation/config.txt b/Documentation/config.txt index 11f320b..6f4663c 100644 --- a/Documentation/config.txt +++ b/Documentation/config.txt @@ -1998,7 +1998,12 @@ submodule..update:: for a submodule. These variables are initially populated by 'git submodule init'; edit them to override the URL and other values found in the `.gitmodules` file. See - linkgit:git-submodule[1] and linkgit:gitmodules[5] for details. + +submodule..branch:: + The remote branch name for a submodule, used by `git submodule + update --remote`. Set this option to override the value found in + the `.gitmodules` file. See linkgit:git-submodule[1] and + linkgit:gitmodules[5] for details. submodule..fetchRecurseSubmodules:: This option can be used to control recursive fetching of this diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt index b4683bb..72dd52f 100644 --- a/Documentation/git-submodule.txt +++ b/Documentation/git-submodule.txt @@ -13,7 +13,7 @@ SYNOPSIS [--reference ] [--] [] 'git submodule' [--quiet] status [--cached] [--recursive] [--] [...] 'git submodule' [--quiet] init [--] [...] -'git submodule' [--quiet] update [--init] [-N|--no-fetch] [--rebase] +'git submodule' [--quiet] update [--init] [--remote] [-N|--no-fetch] [--rebase] [--reference ] [--merge] [--recursive] [--] [...] 'git submodule' [--quiet] summary [--cached|--files] [(-n|--summary-limit) ] [commit] [--] [...] @@ -236,6 +236,29 @@ OPTIONS (the default). This limit only applies to modified submodules. The size is always limited to 1 for added/deleted/typechanged submodules. +--remote:: + This option is only valid for the update command. Instead of using + the superproject's recorded SHA-1 to update the submodule, use the + status of the submodule's remote tracking branch. The remote used + is branch's remote (`branch..remote`), defaulting to `origin`. + The remote branch used defaults to `master`, but the branch name may + be overridden by setting the `submodule..branch` option in + either `.gitmodules` or `.git/config` (with `.git/config` taking + precedence). ++ +This works for any of the supported update procedures (`--checkout`, +`--rebase`, etc.). The only change is the source of the target SHA-1. +For example, `submodule update --remote --merge` will merge upstream +submodule changes into the submodules, while `submodule update +--merge` will merge superproject gitlink changes into the submodules. ++ +In order to ensure a current tracking branch state, `update --remote` +fetches the submodule's remote repository before calculating the +SHA-1. This makes `submodule update --remote --merge` similar to +running `git pull` in the submodule. If you don't want to fetch (for +something closer to `git merge`), you should use `submodule update +--remote --no-fetch --merge`. + -N:: --no-fetch:: This option is only valid for the update command. diff --git a/Documentation/gitmodules.txt b/Documentation/gitmodules.txt index 4effd78..4004fa6 100644 --- a/Documentation/gitmodules.txt +++ b/Documentation/gitmodules.txt @@ -47,6 +47,11 @@ submodule..update:: This config option is overridden if 'git submodule update' is given the '--merge', '--rebase' or '--checkout' options. +submodule..branch:: + A remote branch name for tracking updates in the upstream submodule. + If the option is not specified, it defaults to 'master'. See the + `--remote` documentation in linkgit:git-submodule[1] for details. + submodule..fetchRecurseSubmodules:: This option can be used to control recursive fetching of this submodule. If this option is also present in the submodules entry in diff --git a/git-submodule.sh b/git-submodule.sh index 97ce5e4..104b5de 100755 --- a/git-submodule.sh +++ b/git-submodule.sh @@ -8,7 +8,8 @@ dashless=$(basename "$0" | sed -e 's/-/ /') USAGE="[--quiet] add [-b branch] [-f|--force] [--reference ] [--] [] or: $dashless [--quiet] status [--cached] [--recursive] [--] [...] or: $dashless [--quiet] init [--] [...] - or: $dashless [--quiet] update [--init] [-N|--no-fetch] [-f|--force] [--rebase] [--reference ] [--merge] [--recursive] [--] [...] + or: $dashless [--quiet] update [--init] [--remote] [-N|--no-fetch] [-f|--force] [--rebase] [--reference ] [--merge] [--recursive] [--] [...] +ges or: $dashless [--quiet] summary [--cached|--files] [--summary-limit ] [commit] [--] [...] or: $dashless [--quiet] foreach [--recursive] or: $dashless [--quiet] sync [--] [...]" @@ -26,6 +27,7 @@ cached= recursive= init= files= +remote= nofetch= update= prefix= @@ -536,6 +538,9 @@ cmd_update() -i|--init) init=1 ;; + --remote) + remote=1 + ;; -N|--no-fetch) nofetch=1 ;; @@ -596,6 +601,7 @@ cmd_update() fi name=$(module_name "$sm_path") || exit url=$(git config submodule."$name".url) + branch=$(get_submodule_config "$name" branch master) if ! test -z "$update" then update_module=$update @@ -630,6 +636,20 @@ Maybe you want to use 'update --init'?")" die "$(eval_gettext "Unable to find current revision in submodule path '\$sm_path'")" fi + if test -n "$remote" + then + if test -z "$nofetch" + then + # Fetch remote before determining tracking $sha1 + (clear_local_git_env; cd "$sm_path" && git-fetch) || + die "$(eval_gettext "Unable to fetch in submodule path '\$sm_path'")" + fi + remote_name=$(get_default_remote) + sha1=$(clear_local_git_env; cd "$sm_path" && + git rev-parse --verify "${remote_name}/${branch}") || + die "$(eval_gettext "Unable to find current ${remote_name}/${branch} revision in submodule path '\$sm_path'")" + fi + if test "$subsha1" != "$sha1" -o -n "$force" then subforce=$force diff --git a/t/t7406-submodule-update.sh b/t/t7406-submodule-update.sh index 1542653..a567834 100755 --- a/t/t7406-submodule-update.sh +++ b/t/t7406-submodule-update.sh @@ -135,6 +135,37 @@ test_expect_success 'submodule update --force forcibly checks out submodules' ' ) ' +test_expect_success 'submodule update --remote should fetch upstream changes' ' + (cd submodule && + echo line4 >> file && + git add file && + test_tick && + git commit -m "upstream line4" + ) && + (cd super && + git submodule update --remote --force submodule && + cd submodule && + test "$(git log -1 --oneline)" = "$(GIT_DIR=../../submodule/.git git log -1 --oneline)" + ) +' + +test_expect_success 'local config should override .gitmodules branch' ' + (cd submodule && + git checkout -b test-branch && + echo line5 >> file && + git add file && + test_tick && + git commit -m "upstream line5" && + git checkout master + ) && + (cd super && + git config submodule.submodule.branch test-branch && + git submodule update --remote --force submodule && + cd submodule && + test "$(git log -1 --oneline)" = "$(GIT_DIR=../../submodule/.git git log -1 --oneline test-branch)" + ) +' + test_expect_success 'submodule update --rebase staying on master' ' (cd super/submodule && git checkout master -- 1.8.0.4.gf74b0fc.dirty ``` ## W. Trevor King, 2012-12-02 03:17 Subject: [PATCH v6 3/4] submodule add: If --branch is given, record it in .gitmodules Message-ID: URL: https://gitlist.dev/e/be4777f670198aedae24c3974fddd575fc734c0c.1354417619.git.wking%40tremily.us In-Reply-To: ``` From: "W. Trevor King" This allows you to easily record a submodule..branch option in .gitmodules when you add a new submodule. With this patch, $ git submodule add -b [] $ git config -f .gitmodules submodule..branch reduces to $ git submodule add -b [] This means that future calls to $ git submodule update --remote ... will get updates from the same branch that you used to initialize the submodule, which is usually what you want. Signed-off-by: W. Trevor King --- Documentation/git-submodule.txt | 2 ++ git-submodule.sh | 4 ++++ t/t7400-submodule-basic.sh | 1 + 3 files changed, 7 insertions(+) diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt index 72dd52f..988bba9 100644 --- a/Documentation/git-submodule.txt +++ b/Documentation/git-submodule.txt @@ -208,6 +208,8 @@ OPTIONS -b:: --branch:: Branch of repository to add as submodule. + The name of the branch is recorded as `submodule..branch` in + `.gitmodules` for `update --remote`. -f:: --force:: diff --git a/git-submodule.sh b/git-submodule.sh index 104b5de..27b02fe 100755 --- a/git-submodule.sh +++ b/git-submodule.sh @@ -395,6 +395,10 @@ Use -f if you really want to add it." >&2 git config -f .gitmodules submodule."$sm_path".path "$sm_path" && git config -f .gitmodules submodule."$sm_path".url "$repo" && + if test -n "$branch" + then + git config -f .gitmodules submodule."$sm_path".branch "$branch" + fi && git add --force .gitmodules || die "$(eval_gettext "Failed to register submodule '\$sm_path'")" } diff --git a/t/t7400-submodule-basic.sh b/t/t7400-submodule-basic.sh index 5397037..90e2915 100755 --- a/t/t7400-submodule-basic.sh +++ b/t/t7400-submodule-basic.sh @@ -133,6 +133,7 @@ test_expect_success 'submodule add --branch' ' ( cd addtest && git submodule add -b initial "$submodurl" submod-branch && + test "initial" = "$(git config -f .gitmodules submodule.submod-branch.branch)" && git submodule init ) && -- 1.8.0.4.gf74b0fc.dirty ``` ## W. Trevor King, 2012-12-02 03:17 Subject: [PATCH v6 4/4] submodule update: add submodule..remote config option Message-ID: URL: https://gitlist.dev/e/b9635b844051e681af1c80447fedac5ee77280f7.1354417619.git.wking%40tremily.us In-Reply-To: ``` From: "W. Trevor King" Don't force the user to clone from the tracked repository (branch..remote) or `origin`. By setting submodule..remote in .gitmodules or the usual git config files, you can easily point a submodule at a different remote when using `submodule update --remote`. The configured remote name is also used in `submodule sync` to determine which remote..url is updated with the submodule's origin URL. Signed-off-by: W. Trevor King --- Documentation/config.txt | 7 ++++--- Documentation/git-submodule.txt | 10 +++++----- git-submodule.sh | 27 +++++++++++++++++++++------ t/t7406-submodule-update.sh | 18 ++++++++++++++++++ 4 files changed, 48 insertions(+), 14 deletions(-) diff --git a/Documentation/config.txt b/Documentation/config.txt index 6f4663c..c54b9b4 100644 --- a/Documentation/config.txt +++ b/Documentation/config.txt @@ -1999,10 +1999,11 @@ submodule..update:: by 'git submodule init'; edit them to override the URL and other values found in the `.gitmodules` file. See +submodule..remote:: submodule..branch:: - The remote branch name for a submodule, used by `git submodule - update --remote`. Set this option to override the value found in - the `.gitmodules` file. See linkgit:git-submodule[1] and + The remote repository and branch names for a submodule, used by `git + submodule update --remote`. Set these options to override the value + found in the `.gitmodules` file. See linkgit:git-submodule[1] and linkgit:gitmodules[5] for details. submodule..fetchRecurseSubmodules:: diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt index 988bba9..1d8d5f1 100644 --- a/Documentation/git-submodule.txt +++ b/Documentation/git-submodule.txt @@ -242,11 +242,11 @@ OPTIONS This option is only valid for the update command. Instead of using the superproject's recorded SHA-1 to update the submodule, use the status of the submodule's remote tracking branch. The remote used - is branch's remote (`branch..remote`), defaulting to `origin`. - The remote branch used defaults to `master`, but the branch name may - be overridden by setting the `submodule..branch` option in - either `.gitmodules` or `.git/config` (with `.git/config` taking - precedence). + is branch's remote (`branch..remote`, defaulting to `origin`), + and the remote branch used defaults to `master`, but either may be + overridden by setting the `submodule..remote` or + `submodule..branch` option in `.gitmodules` or `.git/config` + (with `.git/config` taking precedence). + This works for any of the supported update procedures (`--checkout`, `--rebase`, etc.). The only change is the source of the target SHA-1. diff --git a/git-submodule.sh b/git-submodule.sh index 27b02fe..3e39e29 100755 --- a/git-submodule.sh +++ b/git-submodule.sh @@ -179,6 +179,21 @@ get_submodule_config() printf '%s' "${value:-$default}" } +# +# Print the name of a submodule's configured remote +# +# $1 = submodule name +# +get_submodule_remote() +{ + name="$1" + remote=$(get_submodule_config "$name" remote) + if test -z "$remote" + then + remote=$(get_default_remote) + fi + printf '%s' "${remote}" +} # # Map submodule path to submodule name @@ -605,6 +620,7 @@ cmd_update() fi name=$(module_name "$sm_path") || exit url=$(git config submodule."$name".url) + remote_name=$(get_submodule_remote "$name") branch=$(get_submodule_config "$name" branch master) if ! test -z "$update" then @@ -645,10 +661,9 @@ Maybe you want to use 'update --init'?")" if test -z "$nofetch" then # Fetch remote before determining tracking $sha1 - (clear_local_git_env; cd "$sm_path" && git-fetch) || - die "$(eval_gettext "Unable to fetch in submodule path '\$sm_path'")" + (clear_local_git_env; cd "$sm_path" && git-fetch "$remote_name") || + die "$(eval_gettext "Unable to fetch '\$remote_name' in submodule path '\$sm_path'")" fi - remote_name=$(get_default_remote) sha1=$(clear_local_git_env; cd "$sm_path" && git rev-parse --verify "${remote_name}/${branch}") || die "$(eval_gettext "Unable to find current ${remote_name}/${branch} revision in submodule path '\$sm_path'")" @@ -669,8 +684,8 @@ Maybe you want to use 'update --init'?")" # is not reachable from a ref. (clear_local_git_env; cd "$sm_path" && ( (rev=$(git rev-list -n 1 $sha1 --not --all 2>/dev/null) && - test -z "$rev") || git-fetch)) || - die "$(eval_gettext "Unable to fetch in submodule path '\$sm_path'")" + test -z "$rev") || git-fetch "$remote_name")) || + die "$(eval_gettext "Unable to fetch '\$remote_name' in submodule path '\$sm_path'")" fi # Is this something we just cloned? @@ -1110,7 +1125,7 @@ cmd_sync() ( clear_local_git_env cd "$sm_path" - remote=$(get_default_remote) + remote=$(get_submodule_remote "$name") git config remote."$remote".url "$sub_origin_url" ) fi diff --git a/t/t7406-submodule-update.sh b/t/t7406-submodule-update.sh index a567834..86c85f8 100755 --- a/t/t7406-submodule-update.sh +++ b/t/t7406-submodule-update.sh @@ -149,6 +149,24 @@ test_expect_success 'submodule update --remote should fetch upstream changes' ' ) ' +test_expect_success 'local config should override .gitmodules remote' ' + (cd submodule && + echo line5-master >> file && + git add file && + test_tick && + git commit -m "upstream line5-master" + ) && + (cd super/submodule && + git remote rename origin test-remote + ) && + (cd super && + git config submodule.submodule.remote test-remote && + git submodule update --remote --force submodule && + cd submodule && + test "$(git log -1 --oneline)" = "$(GIT_DIR=../../submodule/.git git log -1 --oneline)" + ) +' + test_expect_success 'local config should override .gitmodules branch' ' (cd submodule && git checkout -b test-branch && -- 1.8.0.4.gf74b0fc.dirty ``` ## W. Trevor King, 2012-12-02 19:09 Subject: Re: [RFC] remove/deprecate 'submodule init' and 'sync' Message-ID: <20121202190929.GG9401@odin.tremily.us> URL: https://gitlist.dev/e/20121202190929.GG9401%40odin.tremily.us In-Reply-To: <7vy5hhmcwp.fsf@alter.siamese.dyndns.org> ``` Before I get into the details, I'd like to point out that I actually understand the purpose of `submodule init` now ;). To avoid further confusion, my current one-line command summaries would be: init: mark a submodule as active for future submodule operation deinit: mark a submodule as inactive for future submodule operation sync: update remote..origin in submodules to reflect changes in .gitmodules or the superproject's remote URL. I don't think we disagree on that, we just don't agree on how to implement it. Currently, Git uses submodule..url in the superproject's local configuration as a marker for submodule activation. This is not (as far as I know) discussed in the docs, which is why I initially missunderstood the purpose of `init` to be “setup the superproject's local configuration so we don't have to keep resolving the submodules URL relative to the superproject's upstream URL”. With the proposed `deinit` docs, this role for the submodule..url is mentioned, but not in a place where casual users will be able to easily connect it to the purpose of `init`. I floated using submodule..update (with 'none' for inactive and anything else for active) as an alternative marker: On Sat, Dec 01, 2012 at 01:16:43PM -0500, W. Trevor King wrote: > On Sat, Dec 01, 2012 at 07:04:05PM +0100, Jens Lehmann wrote: > > Am 01.12.2012 18:49, schrieb W. Trevor King: > > > I think removing `init` will cause some compatibility issues anyway, > > > so I was re-imaging how you do it. I don't think update='none' and > > > "don't populate my submodule" are distinct ideas, while a locally > > > configured url="somwhere" and "please populate my submodule" are (with > > > the blank-url case defaulting to the superproject itself). > > > > Why would we want to remove "init"? It still has to copy the "url" > > setting (and it would be a compatibility nightmare if we would change > > that, imagine different git versions used on the same work tree). > > In my init-less rewrite, it doesn't have to copy the url setting. > People using older versions of Git would need to run `init` using > their old version. Having the url defined in .git/config won't break > my init-less submodule commands, it just means that the value in > .gitmodules will be masked. but that doesn't seem to be going over very well. Junio may have been weighing in obliquely with: On Sat, Dec 01, 2012 at 06:00:06PM -0800, Junio C Hamano wrote: > Jens Lehmann writes: > > [snip v1 deinit commit message] > > I fully agree with your analysis on the reason why the "url" element > is special and has to be copied to $GIT_DIR/config, but when you > deinit (or uninit) a submodule to say you are no longer interested > in it and do not want it populated in the context of the > superproject, I am not sure if removing only submodule.$name.url (so > that when you later decide to "init" it again, you will keep the > values for submodule.$name.update and other things from the previous > life) is the sane thing to do, or it is better to remove > submodule.$name.* altogether as if an earlier "init" has never > happened. Would it be worth analyzing the pros-and-cons here? Let me take another stab at presenting my argument in favor of a different activity marker. Proposal: Add a new boolean option, submodule..active, to explicitly mark submodules as active (with “active” defined as “to be returned by module_list()”). Strip down `init` (and the --init part of `update --init`) to just setting this option to true. `deinit` only sets this option to false (but a `deinit --clean` could remove the whole submodule. section). With this in place, extracting URLs for submodule operations be similar to the extraction of other variables (.gitmodules defaults with superproject-local .git/config overrides). This also makes it easier to track maintenance updates in .gitmodules-defined URLs, because you aren't forced to bake overrides into your local .git/config The upgrade path from earlier versions of Git is easy: if submodule..active is unset, use the presence of submodule..url to determine its initial value. In the case where you check out an earlier superproject commit which is missing a particular submodule (or remove a submodule without deinit-ing), the presense of an active setting in .git/config should not cause an error, which they currently seem to: On Sat, Dec 01, 2012 at 11:37:14AM -0500, W. Trevor King wrote: > On Sat, Dec 01, 2012 at 04:56:02PM +0100, Jens Lehmann wrote: > > Am 01.12.2012 00:52, schrieb Phil Hord: > > > If I never 'submodule init' a submodule, it does not get visited by > > > 'git submodule foreach', among others. I think some people use this > > > behavior explicitly. > > > > > > On the other hand, I've also notice that a submodule which I have > > > removed does not get de-inited later one. It causes my 'git submodule > > > foreach' to emit errors. :-( > > > > I'm currently hacking on "git submodule deinit" which removes the 'url' > > setting from git/config. This should do the trick for you, right? > > > > Just removing that submodule automagically would not work that well, as > > it would deinitialize a submodule when you switch to a branch where it > > isn't present and you'd have to reinitialize it when you come back. > > I think this is another case where we should be looping through > submodules based on the revision-specific .gitmodules content, and > querying the local config only to determine if the user wants to > update them (to drop into them with foreach, etc.). Thoughts? Trevor -- This email may be signed or encrypted with GnuPG (http://www.gnupg.org). For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy ``` ## Jens Lehmann, 2012-12-02 19:32 Subject: Re: [PATCH v5 0/2] submodule update: add --remote for submodule's upstream changes Message-ID: <50BBACC3.1070504@web.de> URL: https://gitlist.dev/e/50BBACC3.1070504%40web.de In-Reply-To: <20121130032719.GE29257@odin.tremily.us> ``` Am 30.11.2012 04:27, schrieb W. Trevor King: > On Thu, Nov 29, 2012 at 08:11:20PM -0500, Phil Hord wrote: >> On Thu, Nov 29, 2012 at 2:13 PM, W. Trevor King wrote: >>> On Thu, Nov 29, 2012 at 01:29:12PM -0500, Phil Hord wrote: >>>> But I really don't want to figure out how to handle submodule >>>> collisions during a merge (or rebase!) of my superproject with changes that >>>> someone else auto-committed in his local $superproject as he and I >>>> arbitrarily floated up the upstream independently. There is nothing but >>>> loathing down that path. >>> >>> This is true. I'm not sure how gitlink collisions are currently >>> handled… >> >> They've always been trouble for me. But it may be that I am ignorant. > > I haven't dealt with any gitlink merges, but I think that supporting > easy gitlink merges is orthogonal to this --remote option. For simple > cases like "autocommitted submodule floats", one of the conflicting > gitlinks will be an ancestor of the other, so it should be easy to > automate that merge. Submodule merges where one submodule commit is the ancestor of the other are already resolved automatically in recent git. So Phil's example will just work as long as only fast-forward merges are needed. ``` ## Jens Lehmann, 2012-12-02 19:55 Subject: Re: [PATCH] submodule: add 'deinit' command Message-ID: <50BBB22A.7050901@web.de> URL: https://gitlist.dev/e/50BBB22A.7050901%40web.de In-Reply-To: <7vy5hhmcwp.fsf@alter.siamese.dyndns.org> ``` Am 02.12.2012 03:00, schrieb Junio C Hamano: > Jens Lehmann writes: > >> With "git submodule init" the user is able to tell git he cares about one >> or more submodules and wants to have it populated on the next call to "git >> submodule update". But currently there is no easy way he could tell git he >> does not care about a submodule anymore and wants to get rid of his local >> work tree (except he knows a lot about submodule internals and removes the >> "submodule.$name.url" setting from .git/config himself). >> >> Help those users by providing a 'deinit' command. This removes the url >> setting from .git/config either for the given submodule(s) or for all >> those which have been initialized if none were given. Complain only when >> for a submodule given on the command line the url setting can't be found >> in .git/config. >> >> Add tests and link the man pages of "git submodule deinit" and "git rm" to >> assist the user in deciding whether removing or unregistering the submodule >> is the right thing to do for him. >> >> Signed-off-by: Jens Lehmann >> --- > > I fully agree with your analysis on the reason why the "url" element > is special and has to be copied to $GIT_DIR/config, but when you > deinit (or uninit) a submodule to say you are no longer interested > in it and do not want it populated in the context of the > superproject, I am not sure if removing only submodule.$name.url (so > that when you later decide to "init" it again, you will keep the > values for submodule.$name.update and other things from the previous > life) is the sane thing to do, or it is better to remove > submodule.$name.* altogether as if an earlier "init" has never > happened. Would it be worth analyzing the pros-and-cons here? Sure. I was worried about throwing away other settings the user might have set in the submodule.$name section and the first reflex was to protect them. But thinking about that again I noticed we are already throwing away a possibly user customized "url" setting, so we already remove a possibly customized setting. Maybe the principle of least surprise is better followed when we nuke the whole section, as it might surprise the user more to have a setting resurrected he customized in the last life cycle of the submodule than seeing that after an deinit followed by an init all former customizations are consistently gone. So I tend to think now that removing the whole section would be the better solution here. Opinions by other submodule users? ``` ## Jens Lehmann, 2012-12-02 20:29 Subject: Re: [RFC] remove/deprecate 'submodule init' and 'sync' Message-ID: <50BBBA29.2000106@web.de> URL: https://gitlist.dev/e/50BBBA29.2000106%40web.de In-Reply-To: <20121202190929.GG9401@odin.tremily.us> ``` Am 02.12.2012 20:09, schrieb W. Trevor King: > Before I get into the details, I'd like to point out that I actually > understand the purpose of `submodule init` now ;). To avoid further > confusion, my current one-line command summaries would be: > > init: mark a submodule as active for future submodule operation > deinit: mark a submodule as inactive for future submodule operation > sync: update remote..origin in submodules to reflect changes > in .gitmodules or the superproject's remote URL. > > I don't think we disagree on that, we just don't agree on how to > implement it. Nope, it is already implemented and you are arguing to change the current implementation. To quote from another mail: Am 01.12.2012 18:49, schrieb W. Trevor King: > On Sat, Dec 01, 2012 at 06:25:17PM +0100, Jens Lehmann wrote: >> What real world problems do we have with the current init/sync that >> this approach would solve? > > I don't have any, ... We don't want to change working code and cause compatibility issues just because we /could/ do things differently, no? ``` ## W. Trevor King, 2012-12-02 21:11 Subject: Re: [RFC] remove/deprecate 'submodule init' and 'sync' Message-ID: <20121202211159.GA12429@odin.tremily.us> URL: https://gitlist.dev/e/20121202211159.GA12429%40odin.tremily.us In-Reply-To: <50BBBA29.2000106@web.de> ``` From: "W. Trevor King" To: Jens Lehmann , Junio C Hamano Cc: Phil Hord , Git , Heiko Voigt , Jeff King , Shawn Pearce , Nahor Bcc: Subject: Re: [RFC] remove/deprecate 'submodule init' and 'sync' Reply-To: In-Reply-To: <50BBBA29.2000106@web.de> <50BBB22A.7050901@web.de> <20121202190929.GG9401@odin.tremily.us> OpenPGP: id=39A2F3FA2AB17E5D8764F388FC29BDCDF15F5BE8; url=http://tremily.us/pubkey.txt On Sun, Dec 02, 2012 at 09:29:29PM +0100, Jens Lehmann wrote: > Am 02.12.2012 20:09, schrieb W. Trevor King: > > Before I get into the details, I'd like to point out that I actually > > understand the purpose of `submodule init` now ;). To avoid further > > confusion, my current one-line command summaries would be: > > > > init: mark a submodule as active for future submodule operation > > deinit: mark a submodule as inactive for future submodule operation > > sync: update remote..origin in submodules to reflect changes > > in .gitmodules or the superproject's remote URL. > > > > I don't think we disagree on that, we just don't agree on how to > > implement it. > > Nope, it is already implemented and you are arguing to change the > current implementation. Agreed. > To quote from another mail: > > Am 01.12.2012 18:49, schrieb W. Trevor King: > > On Sat, Dec 01, 2012 at 06:25:17PM +0100, Jens Lehmann wrote: > >> What real world problems do we have with the current init/sync that > >> this approach would solve? > > > > I don't have any, ... > > We don't want to change working code and cause compatibility issues > just because we /could/ do things differently, no? In principle, yes, but in this case I think changing the implementation does not risk much in the way of compatibility issues (it only hurts users who rely on `submodule init` setting submodule..url for reasons of their own. A few of the existing tests explictly check the url setting, so perhaps there are a number of users who do require this side effect? I think this risk is outweighed by the benefits of having a clearer activation option. For example: On Sun, Dec 02, 2012 at 08:55:22PM +0100, Jens Lehmann wrote: > Sure. I was worried about throwing away other settings the user > might have set in the submodule.$name section and the first reflex > was to protect them. But thinking about that again I noticed we are > already throwing away a possibly user customized "url" setting, so > we already remove a possibly customized setting. With submodule..active, there's nothing customized that you'd have to nuke on deinit (except 'active' iteself, which the user is explicitly asking for). Cheers, Trevor -- This email may be signed or encrypted with GnuPG (http://www.gnupg.org). For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy ``` ## Junio C Hamano, 2012-12-03 07:58 Subject: Re: [PATCH] submodule: add 'deinit' command Message-ID: <7vhao31s9e.fsf@alter.siamese.dyndns.org> URL: https://gitlist.dev/e/7vhao31s9e.fsf%40alter.siamese.dyndns.org In-Reply-To: <50BBB22A.7050901@web.de> ``` Jens Lehmann writes: > Maybe the principle of least surprise is better followed when we > nuke the whole section, as it might surprise the user more to have > a setting resurrected he customized in the last life cycle of the > submodule than seeing that after an deinit followed by an init all > former customizations are consistently gone. So I tend to think now > that removing the whole section would be the better solution here. I tend to agree; I suspect that a "deinit" would be mostly done either to (1) correct mistakes the user made during a recent "init" and perhaps "sync"; or (2) tell Git that the user has finished woing with this particular submodule and does not intend to use it for quite a while. For both (1) and (2), I think it would be easier to users if we gave them a clean slate, the same state as the one the user who never had ran "init" on it would be in. A user in situation (1) is asking for a clean slate, and a user in situation (2) is better served if he does not have to worry about leftover entries in $GIT_DIR/config he has long forgotten from many months ago (during which time the way the project uses the particular submodule may well have changed) giving non-standard experience different from what other project participants would get. If there were a sane workflow where it makes sense to frequently run "deinit" followed by some operation followed by "init", it may be helpful to have an option to keep the other customization. And one consideration when implementing that "deinit --keep-customization" option might be to introduce the submodule.$name.activated boolean; that way, the operation can keep the customized upstream URL. In any case, it needs to be shown that such a workflow exists in the first place to justify "deinit --keep-customization". I think the default should be to remove the submodule.$name section. ``` ## W. Trevor King, 2012-12-03 15:38 Subject: Re: [RFC] remove/deprecate 'submodule init' and 'sync' Message-ID: <20121203153855.GA14981@odin.tremily.us> URL: https://gitlist.dev/e/20121203153855.GA14981%40odin.tremily.us In-Reply-To: <20121202211159.GA12429@odin.tremily.us> ``` On Sun, Dec 02, 2012 at 04:11:59PM -0500, W. Trevor King wrote: > On Sun, Dec 02, 2012 at 09:29:29PM +0100, Jens Lehmann wrote: > > Am 01.12.2012 18:49, schrieb W. Trevor King: > > > On Sat, Dec 01, 2012 at 06:25:17PM +0100, Jens Lehmann wrote: > > >> What real world problems do we have with the current init/sync that > > >> this approach would solve? > > > > > > I don't have any, ... > > > > We don't want to change working code and cause compatibility issues > > just because we /could/ do things differently, no? > > In principle, yes, but in this case I think changing the > implementation does not risk much in the way of compatibility issues > (it only hurts users who rely on `submodule init` setting > submodule..url for reasons of their own. A few of the existing > tests explictly check the url setting, so perhaps there are a number > of users who do require this side effect? > > I think this risk is outweighed by the benefits of having a clearer > activation option. For anyone interested in an implementation of my submodule..active proposal, I've posted an initial version: git://github.com/wking/git.git wtk/submodule.name.active I can re-post it here as a PATCH series, but I don't think we're at the level of patch-specific feedback yet. I'm currently pretty happy with it except for the last commit: HACK work around missing index entry for existing empty submodules To solve that cleanly, I'd need a solution to the commit-less existing repository problem which I mentioned earlier: On Sat, Dec 01, 2012 at 11:54:04AM -0500, W. Trevor King wrote: > I'm currently stuck with adding a commit-less existing repository as a > submodule (which happens in t7400-submodule-basic.sh, ../bar/a/b/c > works with relative local path): > > $ mkdir -p super/sub > $ cd super > $ git init > $ (cd sub && git init) > $ git submodule add ./ sub > $ git status > # On branch master > # > # Initial commit > # > # Changes to be committed: > # (use "git rm --cached ..." to unstage) > # > # new file: .gitmodules > # > > What I'm missing is a gitlink form sub for 'Subproject commit > 00000...' or some such. When the subproject has an actual commit, > things work as expected: > > $ mkdir -p super/sub > $ cd super > $ git init > $ (cd sub && git init && echo line-1 > file && git add file && git commit -m file) > $ git submodule add ./ sub > $ git status > # On branch master > # > # Initial commit > # > # Changes to be committed: > # (use "git rm --cached ..." to unstage) > # > # new file: .gitmodules > # new file: sub > # > > This means that module_list isn't aware of the empty submodule, when > the user has just explicitly added it. Fixing this would seem to need > either 'Subproject commit 00000...' as I suggested earlier, or an > adjustment to module_list that also spits out submodules that are in > .gitmodules but not in the index. Other than that, I think all the changes in the test suite are logically sound and unlikely to cause problems with existing usage. Cheers, Trevor -- This email may be signed or encrypted with GnuPG (http://www.gnupg.org). For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy ``` ## Junio C Hamano, 2012-12-03 16:46 Subject: Re: [PATCH v6 2/4] submodule update: add --remote for submodule's upstream changes Message-ID: <7v8v9ft761.fsf@alter.siamese.dyndns.org> URL: https://gitlist.dev/e/7v8v9ft761.fsf%40alter.siamese.dyndns.org In-Reply-To: ``` "W. Trevor King" writes: > From: "W. Trevor King" > > The current `update` command incorporates the superproject's gitlinked > SHA-1 ($sha1) into the submodule HEAD ($subsha1). Depending on the > options you use, it may checkout $sha1, rebase the $subsha1 onto > $sha1, or merge $sha1 into $subsha1. This helps you keep up with > changes in the upstream superproject. > > However, it's also useful to stay up to date with changes in the > upstream subproject. Previous workflows for incorporating such > changes include the ungainly: > > $ git submodule foreach 'git checkout $(git config --file $toplevel/.gitmodules submodule.$name.branch) && git pull' > > With this patch, all of the useful functionality for incorporating > superproject changes can be reused to incorporate upstream subproject > updates. When you specify --remote, the target $sha1 is replaced with > a $sha1 of the submodule's origin/master tracking branch. If you want > to merge a different tracking branch, you can configure the > `submodule..branch` option in `.gitmodules`. You can override > the `.gitmodules` configuration setting for a particular superproject > by configuring the option in that superproject's default configuration > (using the usual configuration hierarchy, e.g. `.git/config`, > `~/.gitconfig`, etc.). > > Previous use of submodule..branch > ======================================= > > Because we're adding a new configuration option, it's a good idea to > check if anyone else is already using the option. The foreach-pull > example above was described by Ævar in > > commit f030c96d8643fa0a1a9b2bd9c2f36a77721fb61f > Author: Ævar Arnfjörð Bjarmason > Date: Fri May 21 16:10:10 2010 +0000 > > git-submodule foreach: Add $toplevel variable > > Gerrit uses the same interpretation for the setting, but because > Gerrit has direct access to the subproject repositories, it updates > the superproject repositories automatically when a subproject changes. > Gerrit also accepts the special value '.', which it expands into the > superproject's branch name. > > Although the --remote functionality is using `submodule..branch` > slightly differently, the effect is the same. The foreach-pull > example uses the option to record the name of the local branch to > checkout before pulls. The tracking branch to be pulled is recorded > in `.git/modules//config`, which was initialized by the module > clone during `submodule add` or `submodule init`. Because the branch > name stored in `submodule..branch` was likely the same as the > branch name used during the initial `submodule add`, the same branch > will be pulled in each workflow. > > Implementation details > ====================== > > In order to ensure a current tracking branch state, `update --remote` > fetches the submodule's remote repository before calculating the > SHA-1. However, I didn't change the logic guarding the existing fetch: > > if test -z "$nofetch" > then > # Run fetch only if $sha1 isn't present or it > # is not reachable from a ref. > (clear_local_git_env; cd "$path" && > ( (rev=$(git rev-list -n 1 $sha1 --not --all 2>/dev/null) && > test -z "$rev") || git-fetch)) || > die "$(eval_gettext "Unable to fetch in submodule path '\$path'")" > fi > > There will not be a double-fetch, because the new $sha1 determined > after the `--remote` triggered fetch should always exist in the > repository. If it doesn't, it's because some racy process removed it > from the submodule's repository and we *should* be re-fetching. As you hinted in the first paragraph, you could flip between merge, rebase, and detach with a command line option when running the "update" subcommand, but I would imagine that the expected use pattern is that for a particular project, you would choose one mode and consistently stick to that mode. To make it easier, the user can set submodule.$name.update once and run "update" without having to give any flags. And this is about adding another mode to the "update" subcommand where the HEAD is not detached, nor merged, nor rebased, but is set to follow whatever commit a remote branch points at. Shouldn't the patch add a way for the user to set a configuration variable to signal that this new mode is always used when "update" is run without a command line flag? As the user has to configure submodule.$name.branch in order to use this mode anyway, I have a feeling that taking that as a signal, and ignoring submodule.$name.update altogether, might be a simpler interface from the end user's point of view. That is, (1) if you are not interested in the submodule $name, you do not "init" it; you "init" it for all of the following. (2) if you want to have the tree state as recorded in the superproject, you do "update" without any option to make the HEAD of the submodule detached at the commit the superproject's tree records; (3) if you want to follow the upstream project of the submodule, you set submodule.$name.branch to the branch you want to follow, and you do "update"---submodule.$name.update is ignored and you will make the HEAD of the submodule detached at the tip of the branch at the remote (using remote-tracking branch); (4) if you want to --merge or --rebase, you give them from the command line, or use submodule.$name.update. I may be oversimplifying a bit, but a separate submodule.$name.remote feels very wrong; if it were a new token "remote" that can be set as the value of submodule.$name.update (in addition to existing "none", "rebase" and "merge"), it might be a bit more understandable, though. How does this compare with the floating submodules Heiko has been working on? ``` ## W. Trevor King, 2012-12-03 18:15 Subject: Re: [PATCH v6 2/4] submodule update: add --remote for submodule's upstream changes Message-ID: <20121203181519.GC14981@odin.tremily.us> URL: https://gitlist.dev/e/20121203181519.GC14981%40odin.tremily.us In-Reply-To: <7v8v9ft761.fsf@alter.siamese.dyndns.org> ``` On Mon, Dec 03, 2012 at 08:46:46AM -0800, Junio C Hamano wrote: > As you hinted in the first paragraph, you could flip between merge, > rebase, and detach with a command line option when running the > "update" subcommand, but I would imagine that the expected use > pattern is that for a particular project, you would choose one mode > and consistently stick to that mode. To make it easier, the user > can set submodule.$name.update once and run "update" without having > to give any flags. Sure. > And this is about adding another mode to the "update" subcommand > where the HEAD is not detached, nor merged, nor rebased, but is set > to follow whatever commit a remote branch points at. This is about adding another suite of modes. Currently you can rebase/merge/checkout the superproject-recorded $sha1. I'm adding the ability to rebase/merge/checkout a submodule-upstream branch. I discuss this explicitly in Documentation/git-submodule.txt when describing --remote. > Shouldn't the patch add a way for the user to set a configuration > variable to signal that this new mode is always used when "update" > is run without a command line flag? How about a new submodule..update-source with (which can be either superproject-gitlink or submodule-upstream)? Or to be a bit simpler and less explicit, a submodule..update-remote boolean? For lack of a better name, I'll call this submodule.. below. > As the user has to configure submodule.$name.branch in order to use > this mode anyway, I have a feeling that taking that as a signal, and > ignoring submodule.$name.update altogether, might be a simpler > interface from the end user's point of view. That is, As I mention earlier, submodule..update is still important. I think it's good to add a new submodule.. config and a --no-remote option (to override a configured submodule..). This way a user that generally updates via the superproject's gitlink can still configure a branch to update from when they use --remote. > (1) if you are not interested in the submodule $name, you do not > "init" it; you "init" it for all of the following. > > (2) if you want to have the tree state as recorded in the > superproject, you do "update" without any option to make the > HEAD of the submodule detached at the commit the superproject's > tree records; > > (3) if you want to follow the upstream project of the submodule, > you set submodule.$name.branch to the branch you want to > follow, and you do "update"---submodule.$name.update is ignored > and you will make the HEAD of the submodule detached at the tip > of the branch at the remote (using remote-tracking branch); > > (4) if you want to --merge or --rebase, you give them from the > command line, or use submodule.$name.update. But what if your whant to merge the upstream project into a currently checked out submodule branch? Or rebase a currently detached head against the upstream branch? > I may be oversimplifying a bit, but a separate > submodule.$name.remote feels very wrong; I use submodule..remote in patch 4 to specify the name of the superproject's remote (for when get_default_remote doesn't give the value you want), but I think you're referring to the potential submodule.. and the presense of the --remote option. > How does this compare with the floating submodules Heiko has been > working on? Heiko's older hv/floating_submodules also uses submodule..branch (with a similar interpretation). There's also a --branch option to `update` for command-line overrides (which I don't have, perhaps I should add them?). He reverts to the original behavior in the presense of submodule..branch with `update --checkout`, or when submodule..branch=HEAD. He also fetches all remotes, while I fetch just the explicitly configured submodule..remote falling back on $(get_default_remote. His submodule..branch is the full local ref for the branch (e.g. 'origin/master'), while mine is just the remote branch (e.g. 'master'). I split the remote (e.g. 'origin') into submodule..remote in patch 4 so you can explicitly fetch just that remote (and not all the remotes you may have configured for that submodule). For reasons that I don't understand, he only supports the `checkout` update logic for remote branches. Heiko's newer hv/floating_submodules_draft builds on my earlier v4 --local-branch option, but he uses his own processing logic. He pulls the existing 'update to $sha1' logic out into a new handle_on_demand_update() and uses the stored submodule..branch as the name of a local submodule branch to check out when tracking. Then he pulls that local branches default upstream (configured in .git/modules//config) with --ff-only. Cheers, Trevor -- This email may be signed or encrypted with GnuPG (http://www.gnupg.org). For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy ``` ## W. Trevor King, 2012-12-03 18:38 Subject: Re: [PATCH v6 2/4] submodule update: add --remote for submodule's upstream changes Message-ID: <20121203183802.GD14981@odin.tremily.us> URL: https://gitlist.dev/e/20121203183802.GD14981%40odin.tremily.us In-Reply-To: <20121203181519.GC14981@odin.tremily.us> ``` As an example to make this clearer: $ cat .gitmodules [submodule "sub1"] path = sub1 url = git://example.com/sub1.git remote = remote1 branch = branch1 update-source = submodule-upstream update = rebase [submodule "sub2"] ... Means that `git submodule update sub1` will fetch remote1 and rebase the current sub1 checkout against the tip of remote1/branch1. The git://example.com/sub1.git URL is not actually used during this update. Presumably the user setup remote1 intentionally in the submodule, and wants to use the URL they've configured there. Perhaps I need to ammend my submodule update: add submodule..remote config option patch (#4) to adjust the remote that has it's URL changed by `sync`? I may also want to append some form of the following commit (from my submodule..active proposal): submodule add: configure existing submodule url if not set [1] Cheers, Trevor [1]: https://github.com/wking/git/commit/b045c16cffe6eb86c157a6c7397166a46e147442 -- This email may be signed or encrypted with GnuPG (http://www.gnupg.org). For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy ``` ## Junio C Hamano, 2012-12-03 19:30 Subject: Re: [PATCH v6 1/4] submodule: add get_submodule_config helper funtion Message-ID: <7vehj7q6gr.fsf@alter.siamese.dyndns.org> URL: https://gitlist.dev/e/7vehj7q6gr.fsf%40alter.siamese.dyndns.org In-Reply-To: <436a73a8fdc8f0695aa597d53483d4c4bae16ebb.1354417618.git.wking@tremily.us> ``` "W. Trevor King" writes: > From: "W. Trevor King" > > Several submodule configuration variables > (e.g. fetchRecurseSubmodules) are read from .gitmodules with local > overrides from the usual git config files. This shell function mimics > that logic to help initialize configuration variables in > git-submodule.sh. > > Signed-off-by: W. Trevor King > --- > git-submodule.sh | 27 +++++++++++++++++++++++++++ > 1 file changed, 27 insertions(+) > > diff --git a/git-submodule.sh b/git-submodule.sh > index ab6b110..97ce5e4 100755 > --- a/git-submodule.sh > +++ b/git-submodule.sh > @@ -152,6 +152,33 @@ die_if_unmatched () > } > > # > +# Print a submodule configuration setting > +# > +# $1 = submodule name > +# $2 = option name > +# $3 = default value > +# > +# Checks in the usual git-config places first (for overrides), > +# otherwise it falls back on .gitmodules. This allows you to > +# distribute project-wide defaults in .gitmodules, while still > +# customizing individual repositories if necessary. If the option is > +# not in .gitmodules either, print a default value. > +# > +get_submodule_config() > +{ style (see CodingGuidelines): get_submodule_config () { > + name="$1" > + option="$2" > + default="$3" > + value=$(git config submodule."$name"."$option") This will get unwieldy quickly once we have submodule.$name.$var that takes a boolean option, as there are different ways to spell boolean and "git config --bool" is the way to ask for canonicalized "true" or "false". If we never query any boolean via this helper function, it is obviously not an issue, though. > + if test -z "$value" > + then > + value=$(git config -f .gitmodules submodule."$name"."$option") > + fi > + printf '%s' "${value:-$default}" > +} > + > + > +# > # Map submodule path to submodule name > # > # $1 = path ``` ## Junio C Hamano, 2012-12-03 20:29 Subject: Re: [PATCH v6 2/4] submodule update: add --remote for submodule's upstream changes Message-ID: <7vr4n6q3qm.fsf@alter.siamese.dyndns.org> URL: https://gitlist.dev/e/7vr4n6q3qm.fsf%40alter.siamese.dyndns.org In-Reply-To: <20121203183802.GD14981@odin.tremily.us> ``` "W. Trevor King" writes: > As an example to make this clearer: > > $ cat .gitmodules > [submodule "sub1"] > path = sub1 > url = git://example.com/sub1.git > remote = remote1 > branch = branch1 > update-source = submodule-upstream > update = rebase > [submodule "sub2"] > ... Maybe it is just me but that "remote = remote1" sticks out like a sore thumb. If you are showing the .gitmodules file to be shared as hints to project participants, why does it even need to have both URL and remote1? If remote1 points at some other repository, the recipient of this .gitmodules file would not have any clue where it is. If remote1 points at the same repository as the URL, why should it be there in the first place? The superproject is in no business to force what local remote name each participant would call in their submodule checkout, and more importantly, there is no _need_ to do so. We could extend that reasoning to the branch name (which is also a local matter, at least technically), but this is a lot more justifiable. If the upstream of the superproject is the same organization as the upstream of the submodule project, which is often the case when a large project is organized as a forest of submodules bound at the top-level with a superproject, the superproject commit on a particular superproject branch may want any update necessary to complete the superproject made to submodules on specific branches at the central meeting place. The superproject's Milestone22 branch may want to bind commits that is on submodule's Milestone22 branch. While a participant locally *can* create M22 branch in the submodule and set it to build upon Milestone22 branch taken from the central repository, most people don't. They use the same branch names between local and remote (i.e. refs/heads/*:refs/remotes/origin/* to keep the remote-tracking branches under the same name, and the local branch $any builds upon the corresponding remote-tracking branch refs/remotes/origin/$any. Most importantly, the work done on local branch $any is pushed out to refs/heads/$any at the remote of the submodule). Because of how people use "push" to push $any branch to the branch of the same name $any at the central meeting place, and because the upstream wants participants to use a particular branch name in the submodule at the central meeting place, the set-up ends up dictating what local branch name should be used. But I do not see any reason to require or even suggest any local nickname that is to be used to call the remote. It really is a local matter. Why should .gitmodules have "remote = ..." line? On the other hand, if you meant the above as an excerpt from $GIT_DIR/config, it also does not make sense. At that point, the participant own the file and updating url to point at whatever different repository without changing the remote name is sufficient. It looks way over-engineered for unclear/dubious benefit. ``` ## W. Trevor King, 2012-12-04 00:17 Subject: Re: [PATCH v6 1/4] submodule: add get_submodule_config helper funtion Message-ID: <20121204001717.GA17375@odin.tremily.us> URL: https://gitlist.dev/e/20121204001717.GA17375%40odin.tremily.us In-Reply-To: <7vr4n6q3qm.fsf@alter.siamese.dyndns.org> ``` On Mon, Dec 03, 2012 at 11:30:12AM -0800, Junio C Hamano wrote: > > +get_submodule_config() > > +{ > > style (see CodingGuidelines): > > get_submodule_config () { Will fix. I was generally just copying the surrounding code. > > + name="$1" > > + option="$2" > > + default="$3" > > + value=$(git config submodule."$name"."$option") > > This will get unwieldy quickly once we have submodule.$name.$var > that takes a boolean option, as there are different ways to spell > boolean and "git config --bool" is the way to ask for canonicalized > "true" or "false". > > If we never query any boolean via this helper function, it is > obviously not an issue, though. We do in my submodule..active branch, and I adjusted the function in submodule: add submodule..active [1] to add additional options passed through to `git config`. You do have to pick a default to use the extra options though. If that becomes a problem, I'd suggest extending git config itself to add a file above or below the usual series of files. Then get_submodule_config could be git config --bottom-file .gitmodules submodule."$name"."$option" or something, without needing a separate shell function. On Mon, Dec 03, 2012 at 12:29:05PM -0800, Junio C Hamano wrote: > "W. Trevor King" writes: > > > As an example to make this clearer: > > > > $ cat .gitmodules > > [submodule "sub1"] > > path = sub1 > > url = git://example.com/sub1.git > > remote = remote1 > > branch = branch1 > > update-source = submodule-upstream > > update = rebase > > [submodule "sub2"] > > ... > > Maybe it is just me but that "remote = remote1" sticks out like a > sore thumb. > > If you are showing the .gitmodules file to be shared as hints to > project participants, why does it even need to have both URL and > remote1? The remote name will probably only ever get configured locally in .git/config. I put it in (as a separate patch) mostly because Phil suggested something like it: On Thu, Nov 29, 2012 at 10:27:19PM -0500, W. Trevor King wrote: > On Thu, Nov 29, 2012 at 08:11:20PM -0500, Phil Hord wrote: > > I've always felt that the "origin" defaults are broken and are simply > > being ignored because most users do not trip over them. But ISTR that > > submodule commands use the remote indicated by the superproject's > > current remote-tracking configuration, with a fallback to 'origin' if > > there is none. Sort of a "best effort" algorithm, I think. Am I > > remembering that wrong? > > The current code uses a bare "git-fetch". I'm not sure what that > defaults to if you're on a detached head. If it bothers you, I'm fine > adding the submodule..remote option in v6. and I hadn't heard any comments against it. I'm not really attached to that patch though, so feel free to leave it out (unless Phil chimes in with stronger motivation?). On Mon, Dec 03, 2012 at 12:29:05PM -0800, Junio C Hamano wrote: > But I do not see any reason to require or even suggest any local > nickname that is to be used to call the remote. It really is a > local matter. Why should .gitmodules have "remote = ..." line? The idea for configuring it at all probably goes something like “I don't like where upstream (origin) is taking this submodule. I want to follow *my* upstream, but I've called it something besides origin. Look, a submodule..remote option! Now I don't have to rename my-remote→origin→original-remote.” I don't think this will come up all that often. > On the other hand, if you meant the above as an excerpt from > $GIT_DIR/config, it also does not make sense. At that point, the > participant own the file and updating url to point at whatever > different repository without changing the remote name is sufficient. Unless they still want to keep an the origin remote to track the original submodule upstream. Maybe they'll want to switch back to following that remote later. As I hinted at above, if they have remotes `alice`, `bob`, etc., it's easier to flip between them by configuring submodule..remote $ git config submodule.submod.remote alice than it is to reconfigure the submodule's origin: $ cd submod $ git remote rename origin charlie $ git remote rename alice origin > It looks way over-engineered for unclear/dubious benefit. I'm not going to push for submodule..remote. Drop at will. Cheers, Trevor [1]: https://github.com/wking/git/commit/fbe2d8419902700a6b0b40defaa5801811b887f7#L0R288 -- This email may be signed or encrypted with GnuPG (http://www.gnupg.org). For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy ``` ## Jens Lehmann, 2012-12-04 21:48 Subject: [PATCH v2] submodule: add 'deinit' command Message-ID: <50BE6FB9.70301@web.de> URL: https://gitlist.dev/e/50BE6FB9.70301%40web.de In-Reply-To: <7vhao31s9e.fsf@alter.siamese.dyndns.org> ``` With "git submodule init" the user is able to tell git he cares about one or more submodules and wants to have it populated on the next call to "git submodule update". But currently there is no easy way he could tell git he does not care about a submodule anymore and wants to get rid of his local work tree (except he knows a lot about submodule internals and removes the "submodule.$name.url" setting from .git/config himself). Help those users by providing a 'deinit' command. This removes the whole submodule. section from .git/config either for the given submodule(s) or for all those which have been initialized if none were given. Complain only when for a submodule given on the command line the url setting can't be found in .git/config. Add tests and link the man pages of "git submodule deinit" and "git rm" to assist the user in deciding whether removing or unregistering the submodule is the right thing to do for him. Signed-off-by: Jens Lehmann --- Am 03.12.2012 08:58, schrieb Junio C Hamano: > Jens Lehmann writes: > >> Maybe the principle of least surprise is better followed when we >> nuke the whole section, as it might surprise the user more to have >> a setting resurrected he customized in the last life cycle of the >> submodule than seeing that after an deinit followed by an init all >> former customizations are consistently gone. So I tend to think now >> that removing the whole section would be the better solution here. > > I tend to agree; I suspect that a "deinit" would be mostly done > either to > > (1) correct mistakes the user made during a recent "init" and > perhaps "sync"; or > > (2) tell Git that the user has finished woing with this particular > submodule and does not intend to use it for quite a while. > > For both (1) and (2), I think it would be easier to users if we gave > them a clean slate, the same state as the one the user who never had > ran "init" on it would be in. A user in situation (1) is asking for > a clean slate, and a user in situation (2) is better served if he > does not have to worry about leftover entries in $GIT_DIR/config he > has long forgotten from many months ago (during which time the way > the project uses the particular submodule may well have changed) > giving non-standard experience different from what other project > participants would get. Changes in v2: - Remove the whole submodule section instead of only removing the "url" setting and explain why we do that in a comment - Reworded commit message and git-submodule.txt to reflect that - Extend the test to check that a custom settings are removed Documentation/git-rm.txt | 4 ++++ Documentation/git-submodule.txt | 12 ++++++++++ git-submodule.sh | 52 ++++++++++++++++++++++++++++++++++++++++- t/t7400-submodule-basic.sh | 12 ++++++++++ 4 files changed, 79 insertions(+), 1 deletion(-) diff --git a/Documentation/git-rm.txt b/Documentation/git-rm.txt index 262436b..ec42bf5 100644 --- a/Documentation/git-rm.txt +++ b/Documentation/git-rm.txt @@ -149,6 +149,10 @@ files that aren't ignored are present in the submodules work tree. Ignored files are deemed expendable and won't stop a submodule's work tree from being removed. +If you only want to remove the local checkout of a submodule from your +work tree without committing that use `git submodule deinit` instead +(see linkgit:git-submodule[1]). + EXAMPLES -------- `git rm Documentation/\*.txt`:: diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt index b1de3ba..08b55a7 100644 --- a/Documentation/git-submodule.txt +++ b/Documentation/git-submodule.txt @@ -13,6 +13,7 @@ SYNOPSIS [--reference ] [--] [] 'git submodule' [--quiet] status [--cached] [--recursive] [--] [...] 'git submodule' [--quiet] init [--] [...] +'git submodule' [--quiet] deinit [--] [...] 'git submodule' [--quiet] update [--init] [-N|--no-fetch] [--rebase] [--reference ] [--merge] [--recursive] [--] [...] 'git submodule' [--quiet] summary [--cached|--files] [(-n|--summary-limit) ] @@ -134,6 +135,17 @@ init:: the explicit 'init' step if you do not intend to customize any submodule locations. +deinit:: + Unregister the submodules, i.e. remove the whole `submodule.$name` + section from .git/config. Further calls to `git submodule update`, + `git submodule foreach` and `git submodule sync` will skip any + unregistered submodules until they are initialized again, so use + this command if you don't want to have a local checkout of the + submodule in your work tree anymore (but note that this command + does not remove the submodule work tree). If you really want to + remove a submodule from the repository and commit that use + linkgit:git-rm[1] instead. + update:: Update the registered submodules, i.e. clone missing submodules and checkout the commit specified in the index of the containing repository. diff --git a/git-submodule.sh b/git-submodule.sh index 2365149..3f558ed 100755 --- a/git-submodule.sh +++ b/git-submodule.sh @@ -8,6 +8,7 @@ dashless=$(basename "$0" | sed -e 's/-/ /') USAGE="[--quiet] add [-b ] [-f|--force] [--name ] [--reference ] [--] [] or: $dashless [--quiet] status [--cached] [--recursive] [--] [...] or: $dashless [--quiet] init [--] [...] + or: $dashless [--quiet] deinit [--] [...] or: $dashless [--quiet] update [--init] [-N|--no-fetch] [-f|--force] [--rebase] [--reference ] [--merge] [--recursive] [--] [...] or: $dashless [--quiet] summary [--cached|--files] [--summary-limit ] [commit] [--] [...] or: $dashless [--quiet] foreach [--recursive] @@ -516,6 +517,55 @@ cmd_init() } # +# Unregister submodules from .git/config +# +# $@ = requested paths (default to all) +# +cmd_deinit() +{ + # parse $args after "submodule ... init". + while test $# -ne 0 + do + case "$1" in + -q|--quiet) + GIT_QUIET=1 + ;; + --) + shift + break + ;; + -*) + usage + ;; + *) + break + ;; + esac + shift + done + + module_list "$@" | + while read mode sha1 stage sm_path + do + die_if_unmatched "$mode" + name=$(module_name "$sm_path") || exit + url=$(git config submodule."$name".url) + if test -z "$url" + then + # Only mention uninitialized submodules when its + # path have been specified + test "$#" != "0" && + say "$(eval_gettext "No url found for submodule path '\$sm_path' in .git/config")" + continue + fi + # Remove the whole section so we have a clean state when the user + # later decides to init this submodule again + git config --remove-section submodule."$name" && + say "$(eval_gettext "Submodule '\$name' (\$url) unregistered")" + done +} + +# # Update each submodule path to correct revision, using clone and checkout as needed # # $@ = requested paths (default to all) @@ -1108,7 +1158,7 @@ cmd_sync() while test $# != 0 && test -z "$command" do case "$1" in - add | foreach | init | update | status | summary | sync) + add | foreach | init | deinit | update | status | summary | sync) command=$1 ;; -q|--quiet) diff --git a/t/t7400-submodule-basic.sh b/t/t7400-submodule-basic.sh index de7d453..ee4f0ab 100755 --- a/t/t7400-submodule-basic.sh +++ b/t/t7400-submodule-basic.sh @@ -756,4 +756,16 @@ test_expect_success 'submodule add with an existing name fails unless forced' ' ) ' +test_expect_success 'submodule deinit should remove the whole submodule section from .git/config' ' + git config submodule.example.foo bar && + git submodule deinit && + test -z "$(git config submodule.example.url)" && + test -z "$(git config submodule.example.foo)" +' + +test_expect_success 'submodule deinit complains only when explicitly used on an uninitialized submodule' ' + git submodule deinit && + test_must_fail git submodule deinit example +' + test_done -- 1.8.0.1.348.gc64da69 ``` ## Junio C Hamano, 2012-12-04 23:06 Subject: Re: [PATCH v2] submodule: add 'deinit' command Message-ID: <7v1uf5mn74.fsf@alter.siamese.dyndns.org> URL: https://gitlist.dev/e/7v1uf5mn74.fsf%40alter.siamese.dyndns.org In-Reply-To: <50BE6FB9.70301@web.de> ``` Jens Lehmann writes: > +If you only want to remove the local checkout of a submodule from your > +work tree without committing that use `git submodule deinit` instead > +(see linkgit:git-submodule[1]). I'll add a comma between "without commiting that" and "use X instead"; it will read better, I think. > +test_expect_success 'submodule deinit should remove the whole submodule section from .git/config' ' > + git config submodule.example.foo bar && > + git submodule deinit && > + test -z "$(git config submodule.example.url)" && > + test -z "$(git config submodule.example.foo)" > +' This is sufficient, but it might be cleaner to see if git config --get-regexp "^submodule\.example\." results in empty. Does not make much difference to warrant a re-roll. > +test_expect_success 'submodule deinit complains only when explicitly used on an uninitialized submodule' ' > + git submodule deinit && > + test_must_fail git submodule deinit example > +' > + > test_done Thanks; will queue. ``` ## W. Trevor King, 2012-12-11 18:58 Subject: [PATCH v7 0/3] submodule update: add --remote for submodule's upstream changes Message-ID: URL: https://gitlist.dev/e/cover.1355251862.git.wking%40tremily.us In-Reply-To: <20121204001717.GA17375@odin.tremily.us> ``` From: "W. Trevor King" I see that this series has dropped out of "what's cooking?". Hopefully this reroll gets it back in ;). Changes since v6 (both in response to Junio's comments): * Fix style in get_submodule_config definition. * Drop the submodule..remote config option (v6's patch 4). W. Trevor King (3): submodule: add get_submodule_config helper funtion submodule update: add --remote for submodule's upstream changes submodule add: If --branch is given, record it in .gitmodules Documentation/config.txt | 7 +++++- Documentation/git-submodule.txt | 27 ++++++++++++++++++++- Documentation/gitmodules.txt | 5 ++++ git-submodule.sh | 52 ++++++++++++++++++++++++++++++++++++++++- t/t7400-submodule-basic.sh | 1 + t/t7406-submodule-update.sh | 31 ++++++++++++++++++++++++ 6 files changed, 120 insertions(+), 3 deletions(-) -- 1.8.0 ``` ## W. Trevor King, 2012-12-11 18:58 Subject: [PATCH v7 1/3] submodule: add get_submodule_config helper funtion Message-ID: <81a253f4d88cd2f3febedc9f24754c4390723a46.1355251862.git.wking@tremily.us> URL: https://gitlist.dev/e/81a253f4d88cd2f3febedc9f24754c4390723a46.1355251862.git.wking%40tremily.us In-Reply-To: ``` From: "W. Trevor King" Several submodule configuration variables (e.g. fetchRecurseSubmodules) are read from .gitmodules with local overrides from the usual git config files. This shell function mimics that logic to help initialize configuration variables in git-submodule.sh. Signed-off-by: W. Trevor King --- git-submodule.sh | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+) diff --git a/git-submodule.sh b/git-submodule.sh index ab6b110..f969f28 100755 --- a/git-submodule.sh +++ b/git-submodule.sh @@ -152,6 +152,32 @@ die_if_unmatched () } # +# Print a submodule configuration setting +# +# $1 = submodule name +# $2 = option name +# $3 = default value +# +# Checks in the usual git-config places first (for overrides), +# otherwise it falls back on .gitmodules. This allows you to +# distribute project-wide defaults in .gitmodules, while still +# customizing individual repositories if necessary. If the option is +# not in .gitmodules either, print a default value. +# +get_submodule_config () { + name="$1" + option="$2" + default="$3" + value=$(git config submodule."$name"."$option") + if test -z "$value" + then + value=$(git config -f .gitmodules submodule."$name"."$option") + fi + printf '%s' "${value:-$default}" +} + + +# # Map submodule path to submodule name # # $1 = path -- 1.8.0 ``` ## W. Trevor King, 2012-12-11 18:58 Subject: [PATCH v7 2/3] submodule update: add --remote for submodule's upstream changes Message-ID: <64d109da03c521303ad87b8370bf09ab28a5c09f.1355251862.git.wking@tremily.us> URL: https://gitlist.dev/e/64d109da03c521303ad87b8370bf09ab28a5c09f.1355251862.git.wking%40tremily.us In-Reply-To: ``` From: "W. Trevor King" The current `update` command incorporates the superproject's gitlinked SHA-1 ($sha1) into the submodule HEAD ($subsha1). Depending on the options you use, it may checkout $sha1, rebase the $subsha1 onto $sha1, or merge $sha1 into $subsha1. This helps you keep up with changes in the upstream superproject. However, it's also useful to stay up to date with changes in the upstream subproject. Previous workflows for incorporating such changes include the ungainly: $ git submodule foreach 'git checkout $(git config --file $toplevel/.gitmodules submodule.$name.branch) && git pull' With this patch, all of the useful functionality for incorporating superproject changes can be reused to incorporate upstream subproject updates. When you specify --remote, the target $sha1 is replaced with a $sha1 of the submodule's origin/master tracking branch. If you want to merge a different tracking branch, you can configure the `submodule..branch` option in `.gitmodules`. You can override the `.gitmodules` configuration setting for a particular superproject by configuring the option in that superproject's default configuration (using the usual configuration hierarchy, e.g. `.git/config`, `~/.gitconfig`, etc.). Previous use of submodule..branch ======================================= Because we're adding a new configuration option, it's a good idea to check if anyone else is already using the option. The foreach-pull example above was described by Ævar in commit f030c96d8643fa0a1a9b2bd9c2f36a77721fb61f Author: Ævar Arnfjörð Bjarmason Date: Fri May 21 16:10:10 2010 +0000 git-submodule foreach: Add $toplevel variable Gerrit uses the same interpretation for the setting, but because Gerrit has direct access to the subproject repositories, it updates the superproject repositories automatically when a subproject changes. Gerrit also accepts the special value '.', which it expands into the superproject's branch name. Although the --remote functionality is using `submodule..branch` slightly differently, the effect is the same. The foreach-pull example uses the option to record the name of the local branch to checkout before pulls. The tracking branch to be pulled is recorded in `.git/modules//config`, which was initialized by the module clone during `submodule add` or `submodule init`. Because the branch name stored in `submodule..branch` was likely the same as the branch name used during the initial `submodule add`, the same branch will be pulled in each workflow. Implementation details ====================== In order to ensure a current tracking branch state, `update --remote` fetches the submodule's remote repository before calculating the SHA-1. However, I didn't change the logic guarding the existing fetch: if test -z "$nofetch" then # Run fetch only if $sha1 isn't present or it # is not reachable from a ref. (clear_local_git_env; cd "$path" && ( (rev=$(git rev-list -n 1 $sha1 --not --all 2>/dev/null) && test -z "$rev") || git-fetch)) || die "$(eval_gettext "Unable to fetch in submodule path '\$path'")" fi There will not be a double-fetch, because the new $sha1 determined after the `--remote` triggered fetch should always exist in the repository. If it doesn't, it's because some racy process removed it from the submodule's repository and we *should* be re-fetching. Signed-off-by: W. Trevor King --- Documentation/config.txt | 7 ++++++- Documentation/git-submodule.txt | 25 ++++++++++++++++++++++++- Documentation/gitmodules.txt | 5 +++++ git-submodule.sh | 22 +++++++++++++++++++++- t/t7406-submodule-update.sh | 31 +++++++++++++++++++++++++++++++ 5 files changed, 87 insertions(+), 3 deletions(-) diff --git a/Documentation/config.txt b/Documentation/config.txt index 11f320b..6f4663c 100644 --- a/Documentation/config.txt +++ b/Documentation/config.txt @@ -1998,7 +1998,12 @@ submodule..update:: for a submodule. These variables are initially populated by 'git submodule init'; edit them to override the URL and other values found in the `.gitmodules` file. See - linkgit:git-submodule[1] and linkgit:gitmodules[5] for details. + +submodule..branch:: + The remote branch name for a submodule, used by `git submodule + update --remote`. Set this option to override the value found in + the `.gitmodules` file. See linkgit:git-submodule[1] and + linkgit:gitmodules[5] for details. submodule..fetchRecurseSubmodules:: This option can be used to control recursive fetching of this diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt index b4683bb..72dd52f 100644 --- a/Documentation/git-submodule.txt +++ b/Documentation/git-submodule.txt @@ -13,7 +13,7 @@ SYNOPSIS [--reference ] [--] [] 'git submodule' [--quiet] status [--cached] [--recursive] [--] [...] 'git submodule' [--quiet] init [--] [...] -'git submodule' [--quiet] update [--init] [-N|--no-fetch] [--rebase] +'git submodule' [--quiet] update [--init] [--remote] [-N|--no-fetch] [--rebase] [--reference ] [--merge] [--recursive] [--] [...] 'git submodule' [--quiet] summary [--cached|--files] [(-n|--summary-limit) ] [commit] [--] [...] @@ -236,6 +236,29 @@ OPTIONS (the default). This limit only applies to modified submodules. The size is always limited to 1 for added/deleted/typechanged submodules. +--remote:: + This option is only valid for the update command. Instead of using + the superproject's recorded SHA-1 to update the submodule, use the + status of the submodule's remote tracking branch. The remote used + is branch's remote (`branch..remote`), defaulting to `origin`. + The remote branch used defaults to `master`, but the branch name may + be overridden by setting the `submodule..branch` option in + either `.gitmodules` or `.git/config` (with `.git/config` taking + precedence). ++ +This works for any of the supported update procedures (`--checkout`, +`--rebase`, etc.). The only change is the source of the target SHA-1. +For example, `submodule update --remote --merge` will merge upstream +submodule changes into the submodules, while `submodule update +--merge` will merge superproject gitlink changes into the submodules. ++ +In order to ensure a current tracking branch state, `update --remote` +fetches the submodule's remote repository before calculating the +SHA-1. This makes `submodule update --remote --merge` similar to +running `git pull` in the submodule. If you don't want to fetch (for +something closer to `git merge`), you should use `submodule update +--remote --no-fetch --merge`. + -N:: --no-fetch:: This option is only valid for the update command. diff --git a/Documentation/gitmodules.txt b/Documentation/gitmodules.txt index 4effd78..4004fa6 100644 --- a/Documentation/gitmodules.txt +++ b/Documentation/gitmodules.txt @@ -47,6 +47,11 @@ submodule..update:: This config option is overridden if 'git submodule update' is given the '--merge', '--rebase' or '--checkout' options. +submodule..branch:: + A remote branch name for tracking updates in the upstream submodule. + If the option is not specified, it defaults to 'master'. See the + `--remote` documentation in linkgit:git-submodule[1] for details. + submodule..fetchRecurseSubmodules:: This option can be used to control recursive fetching of this submodule. If this option is also present in the submodules entry in diff --git a/git-submodule.sh b/git-submodule.sh index f969f28..1395079 100755 --- a/git-submodule.sh +++ b/git-submodule.sh @@ -8,7 +8,8 @@ dashless=$(basename "$0" | sed -e 's/-/ /') USAGE="[--quiet] add [-b branch] [-f|--force] [--reference ] [--] [] or: $dashless [--quiet] status [--cached] [--recursive] [--] [...] or: $dashless [--quiet] init [--] [...] - or: $dashless [--quiet] update [--init] [-N|--no-fetch] [-f|--force] [--rebase] [--reference ] [--merge] [--recursive] [--] [...] + or: $dashless [--quiet] update [--init] [--remote] [-N|--no-fetch] [-f|--force] [--rebase] [--reference ] [--merge] [--recursive] [--] [...] +ges or: $dashless [--quiet] summary [--cached|--files] [--summary-limit ] [commit] [--] [...] or: $dashless [--quiet] foreach [--recursive] or: $dashless [--quiet] sync [--] [...]" @@ -26,6 +27,7 @@ cached= recursive= init= files= +remote= nofetch= update= prefix= @@ -535,6 +537,9 @@ cmd_update() -i|--init) init=1 ;; + --remote) + remote=1 + ;; -N|--no-fetch) nofetch=1 ;; @@ -595,6 +600,7 @@ cmd_update() fi name=$(module_name "$sm_path") || exit url=$(git config submodule."$name".url) + branch=$(get_submodule_config "$name" branch master) if ! test -z "$update" then update_module=$update @@ -629,6 +635,20 @@ Maybe you want to use 'update --init'?")" die "$(eval_gettext "Unable to find current revision in submodule path '\$sm_path'")" fi + if test -n "$remote" + then + if test -z "$nofetch" + then + # Fetch remote before determining tracking $sha1 + (clear_local_git_env; cd "$sm_path" && git-fetch) || + die "$(eval_gettext "Unable to fetch in submodule path '\$sm_path'")" + fi + remote_name=$(get_default_remote) + sha1=$(clear_local_git_env; cd "$sm_path" && + git rev-parse --verify "${remote_name}/${branch}") || + die "$(eval_gettext "Unable to find current ${remote_name}/${branch} revision in submodule path '\$sm_path'")" + fi + if test "$subsha1" != "$sha1" -o -n "$force" then subforce=$force diff --git a/t/t7406-submodule-update.sh b/t/t7406-submodule-update.sh index 1542653..a567834 100755 --- a/t/t7406-submodule-update.sh +++ b/t/t7406-submodule-update.sh @@ -135,6 +135,37 @@ test_expect_success 'submodule update --force forcibly checks out submodules' ' ) ' +test_expect_success 'submodule update --remote should fetch upstream changes' ' + (cd submodule && + echo line4 >> file && + git add file && + test_tick && + git commit -m "upstream line4" + ) && + (cd super && + git submodule update --remote --force submodule && + cd submodule && + test "$(git log -1 --oneline)" = "$(GIT_DIR=../../submodule/.git git log -1 --oneline)" + ) +' + +test_expect_success 'local config should override .gitmodules branch' ' + (cd submodule && + git checkout -b test-branch && + echo line5 >> file && + git add file && + test_tick && + git commit -m "upstream line5" && + git checkout master + ) && + (cd super && + git config submodule.submodule.branch test-branch && + git submodule update --remote --force submodule && + cd submodule && + test "$(git log -1 --oneline)" = "$(GIT_DIR=../../submodule/.git git log -1 --oneline test-branch)" + ) +' + test_expect_success 'submodule update --rebase staying on master' ' (cd super/submodule && git checkout master -- 1.8.0 ``` ## W. Trevor King, 2012-12-11 18:58 Subject: [PATCH v7 3/3] submodule add: If --branch is given, record it in .gitmodules Message-ID: <0b2b49bb7337459502d46b814e350ec857cedfaa.1355251862.git.wking@tremily.us> URL: https://gitlist.dev/e/0b2b49bb7337459502d46b814e350ec857cedfaa.1355251862.git.wking%40tremily.us In-Reply-To: ``` From: "W. Trevor King" This allows you to easily record a submodule..branch option in .gitmodules when you add a new submodule. With this patch, $ git submodule add -b [] $ git config -f .gitmodules submodule..branch reduces to $ git submodule add -b [] This means that future calls to $ git submodule update --remote ... will get updates from the same branch that you used to initialize the submodule, which is usually what you want. Signed-off-by: W. Trevor King --- Documentation/git-submodule.txt | 2 ++ git-submodule.sh | 4 ++++ t/t7400-submodule-basic.sh | 1 + 3 files changed, 7 insertions(+) diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt index 72dd52f..988bba9 100644 --- a/Documentation/git-submodule.txt +++ b/Documentation/git-submodule.txt @@ -208,6 +208,8 @@ OPTIONS -b:: --branch:: Branch of repository to add as submodule. + The name of the branch is recorded as `submodule..branch` in + `.gitmodules` for `update --remote`. -f:: --force:: diff --git a/git-submodule.sh b/git-submodule.sh index 1395079..9f3f437 100755 --- a/git-submodule.sh +++ b/git-submodule.sh @@ -394,6 +394,10 @@ Use -f if you really want to add it." >&2 git config -f .gitmodules submodule."$sm_path".path "$sm_path" && git config -f .gitmodules submodule."$sm_path".url "$repo" && + if test -n "$branch" + then + git config -f .gitmodules submodule."$sm_path".branch "$branch" + fi && git add --force .gitmodules || die "$(eval_gettext "Failed to register submodule '\$sm_path'")" } diff --git a/t/t7400-submodule-basic.sh b/t/t7400-submodule-basic.sh index 5397037..90e2915 100755 --- a/t/t7400-submodule-basic.sh +++ b/t/t7400-submodule-basic.sh @@ -133,6 +133,7 @@ test_expect_success 'submodule add --branch' ' ( cd addtest && git submodule add -b initial "$submodurl" submod-branch && + test "initial" = "$(git config -f .gitmodules submodule.submod-branch.branch)" && git submodule init ) && -- 1.8.0 ``` ## Junio C Hamano, 2012-12-12 05:42 Subject: Re: [PATCH v7 0/3] submodule update: add --remote for submodule's upstream changes Message-ID: <7vtxrr6d2f.fsf@alter.siamese.dyndns.org> URL: https://gitlist.dev/e/7vtxrr6d2f.fsf%40alter.siamese.dyndns.org In-Reply-To: ``` What branch did you base this series on? The preimage of git-submodule.sh in [2/3] does not seem to match anything I have (I could wiggle the patch, but in general I would rather prefer not having to). ``` ## Michael J Gruber, 2012-12-12 15:08 Subject: Re: [PATCH v2] submodule: add 'deinit' command Message-ID: <50C89DF3.20303@drmicha.warpmail.net> URL: https://gitlist.dev/e/50C89DF3.20303%40drmicha.warpmail.net In-Reply-To: <50BE6FB9.70301@web.de> ``` Jens Lehmann venit, vidit, dixit 04.12.2012 22:48: > With "git submodule init" the user is able to tell git he cares about one > or more submodules and wants to have it populated on the next call to "git > submodule update". But currently there is no easy way he could tell git he > does not care about a submodule anymore and wants to get rid of his local > work tree (except he knows a lot about submodule internals and removes the > "submodule.$name.url" setting from .git/config himself). > > Help those users by providing a 'deinit' command. This removes the whole > submodule. section from .git/config either for the given > submodule(s) or for all those which have been initialized if none were > given. Complain only when for a submodule given on the command line the > url setting can't be found in .git/config. Whoaaa, so why not have "git rm" remove everything unless I specify a file to be removed? I know I'm exaggerating a bit, but defaulting to "--all" for a destructive operation seems to be a bit harsh, especially when the command is targeted at "those" users that you mention. > Add tests and link the man pages of "git submodule deinit" and "git rm" > to assist the user in deciding whether removing or unregistering the > submodule is the right thing to do for him. > > Signed-off-by: Jens Lehmann > --- > > Am 03.12.2012 08:58, schrieb Junio C Hamano: >> Jens Lehmann writes: >> >>> Maybe the principle of least surprise is better followed when we >>> nuke the whole section, as it might surprise the user more to have >>> a setting resurrected he customized in the last life cycle of the >>> submodule than seeing that after an deinit followed by an init all >>> former customizations are consistently gone. So I tend to think now >>> that removing the whole section would be the better solution here. >> >> I tend to agree; I suspect that a "deinit" would be mostly done >> either to >> >> (1) correct mistakes the user made during a recent "init" and >> perhaps "sync"; or >> >> (2) tell Git that the user has finished woing with this particular >> submodule and does not intend to use it for quite a while. >> >> For both (1) and (2), I think it would be easier to users if we gave >> them a clean slate, the same state as the one the user who never had >> ran "init" on it would be in. A user in situation (1) is asking for >> a clean slate, and a user in situation (2) is better served if he >> does not have to worry about leftover entries in $GIT_DIR/config he >> has long forgotten from many months ago (during which time the way >> the project uses the particular submodule may well have changed) >> giving non-standard experience different from what other project >> participants would get. > > Changes in v2: > - Remove the whole submodule section instead of only removing the > "url" setting and explain why we do that in a comment > - Reworded commit message and git-submodule.txt to reflect that > - Extend the test to check that a custom settings are removed > > > Documentation/git-rm.txt | 4 ++++ > Documentation/git-submodule.txt | 12 ++++++++++ > git-submodule.sh | 52 ++++++++++++++++++++++++++++++++++++++++- > t/t7400-submodule-basic.sh | 12 ++++++++++ > 4 files changed, 79 insertions(+), 1 deletion(-) > > diff --git a/Documentation/git-rm.txt b/Documentation/git-rm.txt > index 262436b..ec42bf5 100644 > --- a/Documentation/git-rm.txt > +++ b/Documentation/git-rm.txt > @@ -149,6 +149,10 @@ files that aren't ignored are present in the submodules work tree. > Ignored files are deemed expendable and won't stop a submodule's work > tree from being removed. > > +If you only want to remove the local checkout of a submodule from your > +work tree without committing that use `git submodule deinit` instead > +(see linkgit:git-submodule[1]). > + > EXAMPLES > -------- > `git rm Documentation/\*.txt`:: > diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt > index b1de3ba..08b55a7 100644 > --- a/Documentation/git-submodule.txt > +++ b/Documentation/git-submodule.txt > @@ -13,6 +13,7 @@ SYNOPSIS > [--reference ] [--] [] > 'git submodule' [--quiet] status [--cached] [--recursive] [--] [...] > 'git submodule' [--quiet] init [--] [...] > +'git submodule' [--quiet] deinit [--] [...] > 'git submodule' [--quiet] update [--init] [-N|--no-fetch] [--rebase] > [--reference ] [--merge] [--recursive] [--] [...] > 'git submodule' [--quiet] summary [--cached|--files] [(-n|--summary-limit) ] > @@ -134,6 +135,17 @@ init:: > the explicit 'init' step if you do not intend to customize > any submodule locations. > > +deinit:: > + Unregister the submodules, i.e. remove the whole `submodule.$name` > + section from .git/config. Further calls to `git submodule update`, > + `git submodule foreach` and `git submodule sync` will skip any > + unregistered submodules until they are initialized again, so use > + this command if you don't want to have a local checkout of the > + submodule in your work tree anymore (but note that this command > + does not remove the submodule work tree). If you really want to > + remove a submodule from the repository and commit that use > + linkgit:git-rm[1] instead. > + > update:: > Update the registered submodules, i.e. clone missing submodules and > checkout the commit specified in the index of the containing repository. > diff --git a/git-submodule.sh b/git-submodule.sh > index 2365149..3f558ed 100755 > --- a/git-submodule.sh > +++ b/git-submodule.sh > @@ -8,6 +8,7 @@ dashless=$(basename "$0" | sed -e 's/-/ /') > USAGE="[--quiet] add [-b ] [-f|--force] [--name ] [--reference ] [--] [] > or: $dashless [--quiet] status [--cached] [--recursive] [--] [...] > or: $dashless [--quiet] init [--] [...] > + or: $dashless [--quiet] deinit [--] [...] > or: $dashless [--quiet] update [--init] [-N|--no-fetch] [-f|--force] [--rebase] [--reference ] [--merge] [--recursive] [--] [...] > or: $dashless [--quiet] summary [--cached|--files] [--summary-limit ] [commit] [--] [...] > or: $dashless [--quiet] foreach [--recursive] > @@ -516,6 +517,55 @@ cmd_init() > } > > # > +# Unregister submodules from .git/config > +# > +# $@ = requested paths (default to all) > +# > +cmd_deinit() > +{ > + # parse $args after "submodule ... init". > + while test $# -ne 0 > + do > + case "$1" in > + -q|--quiet) > + GIT_QUIET=1 > + ;; > + --) > + shift > + break > + ;; > + -*) > + usage > + ;; > + *) > + break > + ;; > + esac > + shift > + done > + > + module_list "$@" | > + while read mode sha1 stage sm_path > + do > + die_if_unmatched "$mode" > + name=$(module_name "$sm_path") || exit > + url=$(git config submodule."$name".url) > + if test -z "$url" > + then > + # Only mention uninitialized submodules when its > + # path have been specified > + test "$#" != "0" && > + say "$(eval_gettext "No url found for submodule path '\$sm_path' in .git/config")" > + continue > + fi > + # Remove the whole section so we have a clean state when the user > + # later decides to init this submodule again > + git config --remove-section submodule."$name" && > + say "$(eval_gettext "Submodule '\$name' (\$url) unregistered")" > + done > +} > + > +# > # Update each submodule path to correct revision, using clone and checkout as needed > # > # $@ = requested paths (default to all) > @@ -1108,7 +1158,7 @@ cmd_sync() > while test $# != 0 && test -z "$command" > do > case "$1" in > - add | foreach | init | update | status | summary | sync) > + add | foreach | init | deinit | update | status | summary | sync) > command=$1 > ;; > -q|--quiet) > diff --git a/t/t7400-submodule-basic.sh b/t/t7400-submodule-basic.sh > index de7d453..ee4f0ab 100755 > --- a/t/t7400-submodule-basic.sh > +++ b/t/t7400-submodule-basic.sh > @@ -756,4 +756,16 @@ test_expect_success 'submodule add with an existing name fails unless forced' ' > ) > ' > > +test_expect_success 'submodule deinit should remove the whole submodule section from .git/config' ' > + git config submodule.example.foo bar && > + git submodule deinit && > + test -z "$(git config submodule.example.url)" && > + test -z "$(git config submodule.example.foo)" > +' > + > +test_expect_success 'submodule deinit complains only when explicitly used on an uninitialized submodule' ' > + git submodule deinit && > + test_must_fail git submodule deinit example > +' > + > test_done > ``` ## W. Trevor King, 2012-12-12 15:24 Subject: Re: [PATCH v7 0/3] submodule update: add --remote for submodule's upstream changes Message-ID: <20121212152437.GB5157@odin.tremily.us> URL: https://gitlist.dev/e/20121212152437.GB5157%40odin.tremily.us In-Reply-To: <7vtxrr6d2f.fsf@alter.siamese.dyndns.org> ``` On Tue, Dec 11, 2012 at 09:42:48PM -0800, Junio C Hamano wrote: > What branch did you base this series on? Every version of this series has been based on v1.8.0. > The preimage of git-submodule.sh in [2/3] does not seem to match > anything I have (I could wiggle the patch, but in general I would > rather prefer not having to). From patch 1/3: diff --git a/git-submodule.sh b/git-submodule.sh index ab6b110..f969f28 100755 And from patch 2/3: diff --git a/git-submodule.sh b/git-submodule.sh index f969f28..1395079 100755 ab6b110 is in v1.8.0: $ git ls-tree v1.8.0 git-submodule.sh 100755 blob ab6b1107b6090494f192f361471ed5748ffa7dc1 git-submodule.sh I can reroll if necessary, but I'm not sure what I've done wrong… Cheers, Trevor ``` ## Jens Lehmann, 2012-12-12 17:22 Subject: Re: [PATCH v2] submodule: add 'deinit' command Message-ID: <50C8BD6B.9010702@web.de> URL: https://gitlist.dev/e/50C8BD6B.9010702%40web.de In-Reply-To: <50C89DF3.20303@drmicha.warpmail.net> ``` Am 12.12.2012 16:08, schrieb Michael J Gruber: > Jens Lehmann venit, vidit, dixit 04.12.2012 22:48: >> With "git submodule init" the user is able to tell git he cares about one >> or more submodules and wants to have it populated on the next call to "git >> submodule update". But currently there is no easy way he could tell git he >> does not care about a submodule anymore and wants to get rid of his local >> work tree (except he knows a lot about submodule internals and removes the >> "submodule.$name.url" setting from .git/config himself). >> >> Help those users by providing a 'deinit' command. This removes the whole >> submodule. section from .git/config either for the given >> submodule(s) or for all those which have been initialized if none were >> given. Complain only when for a submodule given on the command line the >> url setting can't be found in .git/config. > > Whoaaa, so why not have "git rm" remove everything unless I specify a > file to be removed? Because "git add" doesn't add any file in that case either? > I know I'm exaggerating a bit, but defaulting to "--all" for a > destructive operation seems to be a bit harsh, especially when the > command is targeted at "those" users that you mention. All other submodule commands (except add, which only operates on a single submodule to be) iterate over all submodules if none were explicitly given on the command line. So I made deinit just behave like all the others - and especially init - do. But if people really are surprised by being consistent here I won't argue against adding such a "--all" option, but currently I'm not convinced it is worth it. Especially as I suspect the number of submodule users having customized those in .git/config is not that high ... ``` ## Phil Hord, 2012-12-12 17:43 Subject: Re: [PATCH v7 2/3] submodule update: add --remote for submodule's upstream changes Message-ID: URL: https://gitlist.dev/e/CABURp0oLmSjiZAOJxEzwSmL%2BjimpVj8DcDi-odPTzCpCcyH8yA%40mail.gmail.com In-Reply-To: <64d109da03c521303ad87b8370bf09ab28a5c09f.1355251862.git.wking@tremily.us> ``` Thanks for looking after this. On Tue, Dec 11, 2012 at 1:58 PM, W. Trevor King wrote: > From: "W. Trevor King" > > The current `update` command incorporates the superproject's gitlinked > SHA-1 ($sha1) into the submodule HEAD ($subsha1). Depending on the > options you use, it may checkout $sha1, rebase the $subsha1 onto > $sha1, or merge $sha1 into $subsha1. This helps you keep up with > changes in the upstream superproject. > > However, it's also useful to stay up to date with changes in the > upstream subproject. Previous workflows for incorporating such > changes include the ungainly: > > $ git submodule foreach 'git checkout $(git config --file $toplevel/.gitmodules submodule.$name.branch) && git pull' > > With this patch, all of the useful functionality for incorporating > superproject changes can be reused to incorporate upstream subproject > updates. When you specify --remote, the target $sha1 is replaced with > a $sha1 of the submodule's origin/master tracking branch. If you want > to merge a different tracking branch, you can configure the > `submodule..branch` option in `.gitmodules`. You can override > the `.gitmodules` configuration setting for a particular superproject > by configuring the option in that superproject's default configuration > (using the usual configuration hierarchy, e.g. `.git/config`, > `~/.gitconfig`, etc.). > > Previous use of submodule..branch > ======================================= > > Because we're adding a new configuration option, it's a good idea to > check if anyone else is already using the option. The foreach-pull > example above was described by Ævar in > > commit f030c96d8643fa0a1a9b2bd9c2f36a77721fb61f > Author: Ævar Arnfjörð Bjarmason > Date: Fri May 21 16:10:10 2010 +0000 > > git-submodule foreach: Add $toplevel variable > > Gerrit uses the same interpretation for the setting, but because > Gerrit has direct access to the subproject repositories, it updates > the superproject repositories automatically when a subproject changes. > Gerrit also accepts the special value '.', which it expands into the > superproject's branch name. > > Although the --remote functionality is using `submodule..branch` > slightly differently, the effect is the same. The foreach-pull > example uses the option to record the name of the local branch to > checkout before pulls. The tracking branch to be pulled is recorded > in `.git/modules//config`, which was initialized by the module > clone during `submodule add` or `submodule init`. Because the branch > name stored in `submodule..branch` was likely the same as the > branch name used during the initial `submodule add`, the same branch > will be pulled in each workflow. > > Implementation details > ====================== > > In order to ensure a current tracking branch state, `update --remote` > fetches the submodule's remote repository before calculating the > SHA-1. However, I didn't change the logic guarding the existing fetch: > > if test -z "$nofetch" > then > # Run fetch only if $sha1 isn't present or it > # is not reachable from a ref. > (clear_local_git_env; cd "$path" && > ( (rev=$(git rev-list -n 1 $sha1 --not --all 2>/dev/null) && > test -z "$rev") || git-fetch)) || > die "$(eval_gettext "Unable to fetch in submodule path '\$path'")" > fi > > There will not be a double-fetch, because the new $sha1 determined > after the `--remote` triggered fetch should always exist in the > repository. If it doesn't, it's because some racy process removed it > from the submodule's repository and we *should* be re-fetching. > > Signed-off-by: W. Trevor King > --- > Documentation/config.txt | 7 ++++++- > Documentation/git-submodule.txt | 25 ++++++++++++++++++++++++- > Documentation/gitmodules.txt | 5 +++++ > git-submodule.sh | 22 +++++++++++++++++++++- > t/t7406-submodule-update.sh | 31 +++++++++++++++++++++++++++++++ > 5 files changed, 87 insertions(+), 3 deletions(-) > > diff --git a/Documentation/config.txt b/Documentation/config.txt > index 11f320b..6f4663c 100644 > --- a/Documentation/config.txt > +++ b/Documentation/config.txt > @@ -1998,7 +1998,12 @@ submodule..update:: > for a submodule. These variables are initially populated > by 'git submodule init'; edit them to override the > URL and other values found in the `.gitmodules` file. See > - linkgit:git-submodule[1] and linkgit:gitmodules[5] for details. > + > +submodule..branch:: > + The remote branch name for a submodule, used by `git submodule > + update --remote`. Set this option to override the value found in > + the `.gitmodules` file. See linkgit:git-submodule[1] and > + linkgit:gitmodules[5] for details. > > submodule..fetchRecurseSubmodules:: > This option can be used to control recursive fetching of this > diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt > index b4683bb..72dd52f 100644 > --- a/Documentation/git-submodule.txt > +++ b/Documentation/git-submodule.txt > @@ -13,7 +13,7 @@ SYNOPSIS > [--reference ] [--] [] > 'git submodule' [--quiet] status [--cached] [--recursive] [--] [...] > 'git submodule' [--quiet] init [--] [...] > -'git submodule' [--quiet] update [--init] [-N|--no-fetch] [--rebase] > +'git submodule' [--quiet] update [--init] [--remote] [-N|--no-fetch] [--rebase] > [--reference ] [--merge] [--recursive] [--] [...] > 'git submodule' [--quiet] summary [--cached|--files] [(-n|--summary-limit) ] > [commit] [--] [...] > @@ -236,6 +236,29 @@ OPTIONS > (the default). This limit only applies to modified submodules. The > size is always limited to 1 for added/deleted/typechanged submodules. > > +--remote:: > + This option is only valid for the update command. Instead of using > + the superproject's recorded SHA-1 to update the submodule, use the > + status of the submodule's remote tracking branch. The remote used > + is branch's remote (`branch..remote`), defaulting to `origin`. > + The remote branch used defaults to `master`, but the branch name may > + be overridden by setting the `submodule..branch` option in > + either `.gitmodules` or `.git/config` (with `.git/config` taking > + precedence). > ++ > +This works for any of the supported update procedures (`--checkout`, > +`--rebase`, etc.). The only change is the source of the target SHA-1. > +For example, `submodule update --remote --merge` will merge upstream > +submodule changes into the submodules, while `submodule update > +--merge` will merge superproject gitlink changes into the submodules. > ++ > +In order to ensure a current tracking branch state, `update --remote` > +fetches the submodule's remote repository before calculating the > +SHA-1. This makes `submodule update --remote --merge` similar to > +running `git pull` in the submodule. If you don't want to fetch (for > +something closer to `git merge`), you should use `submodule update > +--remote --no-fetch --merge`. I assume the same can be said for 'submodue update --remote --rebase', right? I wonder if this can be made merge/rebase-agnostic. Is it still true if I word it like this?: In order to ensure a current tracking branch state, `update --remote` fetches the submodule's remote repository before calculating the SHA-1. If you don't want to fetch, you should use `submodule update --remote --no-fetch`. > + > -N:: > --no-fetch:: > This option is only valid for the update command. > diff --git a/Documentation/gitmodules.txt b/Documentation/gitmodules.txt > index 4effd78..4004fa6 100644 > --- a/Documentation/gitmodules.txt > +++ b/Documentation/gitmodules.txt > @@ -47,6 +47,11 @@ submodule..update:: > This config option is overridden if 'git submodule update' is given > the '--merge', '--rebase' or '--checkout' options. > > +submodule..branch:: > + A remote branch name for tracking updates in the upstream submodule. > + If the option is not specified, it defaults to 'master'. See the > + `--remote` documentation in linkgit:git-submodule[1] for details. > + > submodule..fetchRecurseSubmodules:: > This option can be used to control recursive fetching of this > submodule. If this option is also present in the submodules entry in > diff --git a/git-submodule.sh b/git-submodule.sh > index f969f28..1395079 100755 > --- a/git-submodule.sh > +++ b/git-submodule.sh > @@ -8,7 +8,8 @@ dashless=$(basename "$0" | sed -e 's/-/ /') > USAGE="[--quiet] add [-b branch] [-f|--force] [--reference ] [--] [] > or: $dashless [--quiet] status [--cached] [--recursive] [--] [...] > or: $dashless [--quiet] init [--] [...] > - or: $dashless [--quiet] update [--init] [-N|--no-fetch] [-f|--force] [--rebase] [--reference ] [--merge] [--recursive] [--] [...] > + or: $dashless [--quiet] update [--init] [--remote] [-N|--no-fetch] [-f|--force] [--rebase] [--reference ] [--merge] [--recursive] [--] [...] > +ges I think there's an unintentionally added line here with "ges". > or: $dashless [--quiet] summary [--cached|--files] [--summary-limit ] [commit] [--] [...] > or: $dashless [--quiet] foreach [--recursive] > or: $dashless [--quiet] sync [--] [...]" > @@ -26,6 +27,7 @@ cached= > recursive= > init= > files= > +remote= > nofetch= > update= > prefix= > @@ -535,6 +537,9 @@ cmd_update() > -i|--init) > init=1 > ;; > + --remote) > + remote=1 > + ;; > -N|--no-fetch) > nofetch=1 > ;; > @@ -595,6 +600,7 @@ cmd_update() > fi > name=$(module_name "$sm_path") || exit > url=$(git config submodule."$name".url) > + branch=$(get_submodule_config "$name" branch master) > if ! test -z "$update" > then > update_module=$update > @@ -629,6 +635,20 @@ Maybe you want to use 'update --init'?")" > die "$(eval_gettext "Unable to find current revision in submodule path '\$sm_path'")" > fi > > + if test -n "$remote" > + then > + if test -z "$nofetch" > + then > + # Fetch remote before determining tracking $sha1 > + (clear_local_git_env; cd "$sm_path" && git-fetch) || You should 'git fetch $remote_name' here, and of course, initialize remote_name before this. But how can we know the remote_name in the first place? Is it safe to assume the submodule remote names will match those in the superproject? > + die "$(eval_gettext "Unable to fetch in submodule path '\$sm_path'")" > + fi > + remote_name=$(get_default_remote) This get_default_remote finds the remote for the remote-tracking branch for HEAD in the superproject. It is possible that HEAD != $branch, so we have very few clues to go on here to get a more reasonable answer, so I do not have any good suggestions to improve this. One option would be to find the remote given for submodule."$branch".merge, but this would suppose there is some remote-tracking branch configured in the submodule, and that is not likely to be the case. > + sha1=$(clear_local_git_env; cd "$sm_path" && > + git rev-parse --verify "${remote_name}/${branch}") || This does assume the submodule remote names will match those in the superproject. Is this safe? > + die "$(eval_gettext "Unable to find current ${remote_name}/${branch} revision in submodule path '\$sm_path'")" > + fi > + > if test "$subsha1" != "$sha1" -o -n "$force" > then > subforce=$force > diff --git a/t/t7406-submodule-update.sh b/t/t7406-submodule-update.sh > index 1542653..a567834 100755 > --- a/t/t7406-submodule-update.sh > +++ b/t/t7406-submodule-update.sh > @@ -135,6 +135,37 @@ test_expect_success 'submodule update --force forcibly checks out submodules' ' > ) > ' > > +test_expect_success 'submodule update --remote should fetch upstream changes' ' > + (cd submodule && > + echo line4 >> file && > + git add file && > + test_tick && > + git commit -m "upstream line4" > + ) && > + (cd super && > + git submodule update --remote --force submodule && > + cd submodule && > + test "$(git log -1 --oneline)" = "$(GIT_DIR=../../submodule/.git git log -1 --oneline)" > + ) > +' > + > +test_expect_success 'local config should override .gitmodules branch' ' > + (cd submodule && > + git checkout -b test-branch && > + echo line5 >> file && > + git add file && > + test_tick && > + git commit -m "upstream line5" && > + git checkout master > + ) && > + (cd super && > + git config submodule.submodule.branch test-branch && > + git submodule update --remote --force submodule && > + cd submodule && > + test "$(git log -1 --oneline)" = "$(GIT_DIR=../../submodule/.git git log -1 --oneline test-branch)" > + ) > +' > + > test_expect_success 'submodule update --rebase staying on master' ' > (cd super/submodule && > git checkout master > -- > 1.8.0 > ``` ## Junio C Hamano, 2012-12-12 18:19 Subject: Re: [PATCH v7 0/3] submodule update: add --remote for submodule's upstream changes Message-ID: <7vzk1j3zgr.fsf@alter.siamese.dyndns.org> URL: https://gitlist.dev/e/7vzk1j3zgr.fsf%40alter.siamese.dyndns.org In-Reply-To: <20121212152437.GB5157@odin.tremily.us> ``` "W. Trevor King" writes: > On Tue, Dec 11, 2012 at 09:42:48PM -0800, Junio C Hamano wrote: >> What branch did you base this series on? > > Every version of this series has been based on v1.8.0. Thanks. There were quite a few changes to git-submodule.sh since then to 'master' and I had to either wiggle the patch or know which exact one 1/3 needs to be applied to in order to allow 2/3 to apply (try applying these three to 'master' yourself---you will likely to see that 2/3 will stop with conflicts). In any case, I ended up applying them by editing the patches, and I should have a good copy in 'pu'. Please double check the result. Thanks. ``` ## Junio C Hamano, 2012-12-12 19:32 Subject: Re: [PATCH v2] submodule: add 'deinit' command Message-ID: <7vr4mv3w2x.fsf@alter.siamese.dyndns.org> URL: https://gitlist.dev/e/7vr4mv3w2x.fsf%40alter.siamese.dyndns.org In-Reply-To: <50C8BD6B.9010702@web.de> ``` Jens Lehmann writes: > Especially as I suspect the number of submodule users having > customized those in .git/config is not that high ... I thought the point of "deinit" was to say "I am not interested in having a checkout of these submodules in my working tree anymore". The user could do "rm -fr submodule && mkdir submodule" to remove it locally and keep "diff" and "status" from noticing the removal, but the primary reason the user needs an explicit "deinit" is because many subcommands of "git submodule" are documented to operate on all submodules that have been "init"ed when given no explicit submodule names [*1*]. Your "deinit" is documented not to actually remove the submodule checkout, but that very much goes against my intuition. What is the justification behind that choice? "We'll remove the configuration, you remove the tree yourself" will invite the mistake of running "git rm" on it, which you wanted to avoid with the addition to the "git rm" documentation, no? [Footnote] *1* In reality, the code looks at presense of .git in the submodule path to decide if it has been "init"ed (cf. cmd_update), but this implementation of "deinit" does not seem to cause that .git to be removed, leaving the submodule in "init"ed state from these other command's perspective. ``` ## Junio C Hamano, 2012-12-12 19:54 Subject: Re: [PATCH v7 2/3] submodule update: add --remote for submodule's upstream changes Message-ID: <7v7gon3v2r.fsf@alter.siamese.dyndns.org> URL: https://gitlist.dev/e/7v7gon3v2r.fsf%40alter.siamese.dyndns.org In-Reply-To: ``` Phil Hord writes: >> + if test -n "$remote" >> + then >> + if test -z "$nofetch" >> + then >> + # Fetch remote before determining tracking $sha1 >> + (clear_local_git_env; cd "$sm_path" && git-fetch) || > > You should 'git fetch $remote_name' here, and of course, initialize > remote_name before this. But how can we know the remote_name in the > first place? Is it safe to assume the submodule remote names will > match those in the superproject? > >> + die "$(eval_gettext "Unable to fetch in submodule path '\$sm_path'")" >> + fi >> + remote_name=$(get_default_remote) > > This get_default_remote finds the remote for the remote-tracking > branch for HEAD in the superproject. It is possible that HEAD != > $branch, so we have very few clues to go on here to get a more > reasonable answer, so I do not have any good suggestions to improve > this. > > One option would be to find the remote given for > submodule."$branch".merge, but this would suppose there is some > remote-tracking branch configured in the submodule, and that is not > likely to be the case. > >> + sha1=$(clear_local_git_env; cd "$sm_path" && >> + git rev-parse --verify "${remote_name}/${branch}") || > > This does assume the submodule remote names will match those in the > superproject. Is this safe? All good points. Thanks for reviewing. ``` ## Jens Lehmann, 2012-12-12 22:25 Subject: Re: [PATCH v2] submodule: add 'deinit' command Message-ID: <50C90469.8080303@web.de> URL: https://gitlist.dev/e/50C90469.8080303%40web.de In-Reply-To: <7vr4mv3w2x.fsf@alter.siamese.dyndns.org> ``` Am 12.12.2012 20:32, schrieb Junio C Hamano: > Jens Lehmann writes: > >> Especially as I suspect the number of submodule users having >> customized those in .git/config is not that high ... > > I thought the point of "deinit" was to say "I am not interested in > having a checkout of these submodules in my working tree anymore". Yes. (But I'm not sure users expect that command to also remove the work tree) > The user could do "rm -fr submodule && mkdir submodule" to remove it > locally and keep "diff" and "status" from noticing the removal, but > the primary reason the user needs an explicit "deinit" is because > many subcommands of "git submodule" are documented to operate on all > submodules that have been "init"ed when given no explicit submodule > names [*1*]. The real reason we need deinit is that the next run of "submodule update" will otherwise happily recreate the submodule checkout you just removed as long as it finds the url setting in .git/config. > Your "deinit" is documented not to actually remove the submodule > checkout, but that very much goes against my intuition. What is the > justification behind that choice? I thought it should match what "submodule init" does, which is to do nothing to the work tree until the next "submodule update" is run. (But I agree that analogy is somewhat flawed until we teach "update" to remove a deinitialized submodule - or maybe teach it the --deinit option which could do both). On the other hand with current git submodule work trees always stay around anyway until you remove them by hand (e.g. when you switch to a branch that doesn't have it), so I'm not sure what would surprise people more here. So I just left the work tree unchanged. > "We'll remove the configuration, > you remove the tree yourself" will invite the mistake of running > "git rm" on it, which you wanted to avoid with the addition to the > "git rm" documentation, no? I think that'll happen only if git would remind them that they still have a populated work tree, which I believe it shouldn't. > [Footnote] > > *1* In reality, the code looks at presense of .git in the submodule > path to decide if it has been "init"ed (cf. cmd_update), but this > implementation of "deinit" does not seem to cause that .git to be > removed, leaving the submodule in "init"ed state from these other > command's perspective. Nope, cmd_update() checks first if the url is found in .git/config and skips the submodule if not. I rechecked and only "summary" and "foreach" still recurse into a deinitialized submodule, which they shouldn't. But a quick test shows that "git status" and "git diff" also still inspect a deinitialized submodule, so there's some work left to do to handle the case where the work tree is not removed. So unless people agree that deinit should also remove the work tree I'll prepare some patches teaching all git commands to consistently ignore deinitialized submodules. Opinions? ``` ## Junio C Hamano, 2012-12-12 22:34 Subject: Re: [PATCH v2] submodule: add 'deinit' command Message-ID: <7vlid23nnc.fsf@alter.siamese.dyndns.org> URL: https://gitlist.dev/e/7vlid23nnc.fsf%40alter.siamese.dyndns.org In-Reply-To: <50C90469.8080303@web.de> ``` Jens Lehmann writes: > So unless people agree that deinit should also remove the work > tree I'll prepare some patches teaching all git commands to > consistently ignore deinitialized submodules. Opinions? While I agree that consistency is good, "deinit" that does not remove the working tree of the submodule the user explicitly said he no longer wants to have the checkout for is a bug, and I think these two are orthogonal issues. In other words, "Ignore deinitialized submodules even when an earlier bug in deinit failed to remove the working tree" is a robustness issue for the other recursing commands, not an excuse for "deinit" to have such a bug in the first place, I think. ``` ## W. Trevor King, 2012-12-12 22:44 Subject: Re: [PATCH v7 0/3] submodule update: add --remote for submodule's upstream changes Message-ID: <20121212224425.GA7729@odin.tremily.us> URL: https://gitlist.dev/e/20121212224425.GA7729%40odin.tremily.us In-Reply-To: <7vzk1j3zgr.fsf@alter.siamese.dyndns.org> ``` On Wed, Dec 12, 2012 at 10:19:32AM -0800, Junio C Hamano wrote: > In any case, I ended up applying them by editing the patches, and I > should have a good copy in 'pu'. Please double check the result. Your 'pu' branch looks good to me. Most of the differences with my initial patch are due to irrelevant context lines. I would change patch 3 (commit 2f507f9a in 'pu') to use git config -f .gitmodules submodule."$sm_name".branch "$branch" ^ instead of git config -f .gitmodules submodule."$sm_path".branch "$branch" ^ to match the nearby changes from 73b0898d. Trevor -- This email may be signed or encrypted with GnuPG (http://www.gnupg.org). For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy ``` ## W. Trevor King, 2012-12-12 23:02 Subject: Re: [PATCH v7 2/3] submodule update: add --remote for submodule's upstream changes Message-ID: <20121212230217.GB7729@odin.tremily.us> URL: https://gitlist.dev/e/20121212230217.GB7729%40odin.tremily.us In-Reply-To: ``` On Wed, Dec 12, 2012 at 12:43:23PM -0500, Phil Hord wrote: > On Tue, Dec 11, 2012 at 1:58 PM, W. Trevor King wrote: > > diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt > > … > > +--remote:: > > [snip some --remote documentation] > > +In order to ensure a current tracking branch state, `update --remote` > > +fetches the submodule's remote repository before calculating the > > +SHA-1. This makes `submodule update --remote --merge` similar to > > +running `git pull` in the submodule. If you don't want to fetch (for > > +something closer to `git merge`), you should use `submodule update > > +--remote --no-fetch --merge`. > > I assume the same can be said for 'submodue update --remote --rebase', > right? Yes. > I wonder if this can be made merge/rebase-agnostic. Is it still > true if I word it like this?: > > In order to ensure a current tracking branch state, `update --remote` > fetches the submodule's remote repository before calculating the > SHA-1. If you don't want to fetch, you should use `submodule update > --remote --no-fetch`. Works for me. Will change in v8 (which I'll base on 'master'). > > diff --git a/git-submodule.sh b/git-submodule.sh > > index f969f28..1395079 100755 > > --- a/git-submodule.sh > > +++ b/git-submodule.sh > > @@ -8,7 +8,8 @@ dashless=$(basename "$0" | sed -e 's/-/ /') > > USAGE="[--quiet] add [-b branch] [-f|--force] [--reference ] [--] [] > > or: $dashless [--quiet] status [--cached] [--recursive] [--] [...] > > or: $dashless [--quiet] init [--] [...] > > - or: $dashless [--quiet] update [--init] [-N|--no-fetch] [-f|--force] [--rebase] [--reference ] [--merge] [--recursive] [--] [...] > > + or: $dashless [--quiet] update [--init] [--remote] [-N|--no-fetch] [-f|--force] [--rebase] [--reference ] [--merge] [--recursive] [--] [...] > > +ges > > I think there's an unintentionally added line here with "ges". That is embarrassing :p. Will fix in v8. > > + if test -n "$remote" > > + then > > + if test -z "$nofetch" > > + then > > + # Fetch remote before determining tracking $sha1 > > + (clear_local_git_env; cd "$sm_path" && git-fetch) || > > You should 'git fetch $remote_name' here, and of course, initialize > remote_name before this. But how can we know the remote_name in the > first place? Is it safe to assume the submodule remote names will > match those in the superproject? The other git-fetch call from git-submodule.sh is also bare (i.e. no specified remote). When the remote needs to be specified, other portions of git-submodule.sh use $(get_default_remote), which is (I think) what the user should expect. v6 of this series had a configurable remote name, but Junio wasn't keen on the additional configuration option. I don't really mind either way. > > > + die "$(eval_gettext "Unable to fetch in submodule path '\$sm_path'")" > > + fi > > + remote_name=$(get_default_remote) > > This get_default_remote finds the remote for the remote-tracking > branch for HEAD in the superproject. It is possible that HEAD != > $branch, so we have very few clues to go on here to get a more > reasonable answer, so I do not have any good suggestions to improve > this. For detached HEADs, get_default_remote should fall back to 'origin', which seems sane. If the user wants a different default, they've likely checkout out a branch in the submodule, setup that branch's remote, and will be using --merge or --rebase. If anyone expects users who will be using detached heads to *want* to specify a different remote than 'origin', that would be a good argument for reinstating my submodule..remote patch from v6. > > + sha1=$(clear_local_git_env; cd "$sm_path" && > > + git rev-parse --verify "${remote_name}/${branch}") || > > This does assume the submodule remote names will match those in the > superproject. Is this safe? Another good catch. I should be calling get_default_remote after cd-ing into the submodule. Will change in v8. Thanks for the feedback :) Trevor -- This email may be signed or encrypted with GnuPG (http://www.gnupg.org). For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy ``` ## W. Trevor King, 2012-12-12 23:09 Subject: Re: [PATCH v2] submodule: add 'deinit' command Message-ID: <20121212230926.GC7729@odin.tremily.us> URL: https://gitlist.dev/e/20121212230926.GC7729%40odin.tremily.us In-Reply-To: <7vlid23nnc.fsf@alter.siamese.dyndns.org> ``` On Wed, Dec 12, 2012 at 02:34:47PM -0800, Junio C Hamano wrote: > Jens Lehmann writes: > > > So unless people agree that deinit should also remove the work > > tree I'll prepare some patches teaching all git commands to > > consistently ignore deinitialized submodules. Opinions? > > While I agree that consistency is good, "deinit" that does not > remove the working tree of the submodule the user explicitly said he > no longer wants to have the checkout for is a bug, and I think these > two are orthogonal issues. Should `deinit` remove the submodule checkout, replace it with the original gitlink, and clear the .git/config information then? That would restore the user to the state they'd be in if they were never interested in the submodule. Trevor -- This email may be signed or encrypted with GnuPG (http://www.gnupg.org). For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy ``` ## Junio C Hamano, 2012-12-12 23:35 Subject: Re: [PATCH v2] submodule: add 'deinit' command Message-ID: <7vsj7a268w.fsf@alter.siamese.dyndns.org> URL: https://gitlist.dev/e/7vsj7a268w.fsf%40alter.siamese.dyndns.org In-Reply-To: <20121212230926.GC7729@odin.tremily.us> ``` "W. Trevor King" writes: > Should `deinit` remove the submodule checkout, replace it with the > original gitlink, and clear the .git/config information then? That > would restore the user to the state they'd be in if they were never > interested in the submodule. AFAIU, "restore the user to the state" is the goal. I am not sure what you meant by "replace it with the original gitlink", though. A checkout with a submodule that the user is not interested in would have an empty directory at that path, no? ``` ## W. Trevor King, 2012-12-13 00:28 Subject: Re: [PATCH v2] submodule: add 'deinit' command Message-ID: <20121213002805.GA8380@odin.tremily.us> URL: https://gitlist.dev/e/20121213002805.GA8380%40odin.tremily.us In-Reply-To: <7vsj7a268w.fsf@alter.siamese.dyndns.org> ``` On Wed, Dec 12, 2012 at 03:35:59PM -0800, Junio C Hamano wrote: > "W. Trevor King" writes: > > > Should `deinit` remove the submodule checkout, replace it with the > > original gitlink, and clear the .git/config information then? That > > would restore the user to the state they'd be in if they were never > > interested in the submodule. > > AFAIU, "restore the user to the state" is the goal. I am not sure > what you meant by "replace it with the original gitlink", though. A > checkout with a submodule that the user is not interested in would > have an empty directory at that path, no? Ah yes, the gitlink is only in the index. Sorry for the noise. -- This email may be signed or encrypted with GnuPG (http://www.gnupg.org). For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy ``` ## Marc Branchaud, 2012-12-13 15:47 Subject: Re: [PATCH v2] submodule: add 'deinit' command Message-ID: <50C9F87E.6090902@xiplink.com> URL: https://gitlist.dev/e/50C9F87E.6090902%40xiplink.com In-Reply-To: <50C90469.8080303@web.de> ``` On 12-12-12 05:25 PM, Jens Lehmann wrote: > > So unless people agree that deinit should also remove the work > tree I'll prepare some patches teaching all git commands to > consistently ignore deinitialized submodules. Opinions? I agree with Trevor's suggestion that deinit should restore the user to the state he would be in if he were never interested in the submodule. So clean up .git/config and remove the work tree. (Maybe just issue a warning instead if the submodule's work tree is dirty.) Also, given that semantic, I agree with Michael that a bare "git submodule deinit" should *not* deinitialize all the submodules. It should require a "--all" for that. The bare command should just print a usage summary. M. ``` ## wking@tremily.us, 2012-12-19 16:03 Subject: [PATCH v8 0/3] submodule update: add --remote for submodule's upstream changes Message-ID: URL: https://gitlist.dev/e/cover.1355932282.git.wking%40tremily.us In-Reply-To: <20121212230217.GB7729@odin.tremily.us> ``` From: "W. Trevor King" Comments on v7 seem to have petered out, so here's v8. Changes since v7: * Series based on gitster/master instead of v1.8.0. * In Documentation/config.txt, restored trailing line of submodule..update documentation, which I had accidentally removed in v7. * In Documentation/git-submodule.txt, make --no-fetch example in the --remote description more general, following Phil's suggestion. * In git-submodule.sh: * Remove accidental "ges" line. * Use the submodule's default remote to determine which tracking branch to fetch. In v7 I'd been using the superproject's default remote. * In cmd_add(), use sm_name instead of sm_path to store the --branch option (catching up with 73b0898). W. Trevor King (3): submodule: add get_submodule_config helper funtion submodule update: add --remote for submodule's upstream changes submodule add: If --branch is given, record it in .gitmodules Documentation/config.txt | 6 +++++ Documentation/git-submodule.txt | 25 +++++++++++++++++++- Documentation/gitmodules.txt | 5 ++++ git-submodule.sh | 51 ++++++++++++++++++++++++++++++++++++++++- t/t7400-submodule-basic.sh | 1 + t/t7406-submodule-update.sh | 31 +++++++++++++++++++++++++ 6 files changed, 117 insertions(+), 2 deletions(-) -- 1.8.0 ``` ## wking@tremily.us, 2012-12-19 16:03 Subject: [PATCH v8 1/3] submodule: add get_submodule_config helper funtion Message-ID: <3377beb925bc209d90058493b74d174db1b7aa50.1355932282.git.wking@tremily.us> URL: https://gitlist.dev/e/3377beb925bc209d90058493b74d174db1b7aa50.1355932282.git.wking%40tremily.us In-Reply-To: ``` From: "W. Trevor King" Several submodule configuration variables (e.g. fetchRecurseSubmodules) are read from .gitmodules with local overrides from the usual git config files. This shell function mimics that logic to help initialize configuration variables in git-submodule.sh. Signed-off-by: W. Trevor King --- git-submodule.sh | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+) diff --git a/git-submodule.sh b/git-submodule.sh index 2365149..263a60c 100755 --- a/git-submodule.sh +++ b/git-submodule.sh @@ -153,6 +153,32 @@ die_if_unmatched () } # +# Print a submodule configuration setting +# +# $1 = submodule name +# $2 = option name +# $3 = default value +# +# Checks in the usual git-config places first (for overrides), +# otherwise it falls back on .gitmodules. This allows you to +# distribute project-wide defaults in .gitmodules, while still +# customizing individual repositories if necessary. If the option is +# not in .gitmodules either, print a default value. +# +get_submodule_config () { + name="$1" + option="$2" + default="$3" + value=$(git config submodule."$name"."$option") + if test -z "$value" + then + value=$(git config -f .gitmodules submodule."$name"."$option") + fi + printf '%s' "${value:-$default}" +} + + +# # Map submodule path to submodule name # # $1 = path -- 1.8.0 ``` ## wking@tremily.us, 2012-12-19 16:03 Subject: [PATCH v8 2/3] submodule update: add --remote for submodule's upstream changes Message-ID: <6f5822c998ad01146727214941b4b52a5b3680dd.1355932282.git.wking@tremily.us> URL: https://gitlist.dev/e/6f5822c998ad01146727214941b4b52a5b3680dd.1355932282.git.wking%40tremily.us In-Reply-To: ``` From: "W. Trevor King" The current `update` command incorporates the superproject's gitlinked SHA-1 ($sha1) into the submodule HEAD ($subsha1). Depending on the options you use, it may checkout $sha1, rebase the $subsha1 onto $sha1, or merge $sha1 into $subsha1. This helps you keep up with changes in the upstream superproject. However, it's also useful to stay up to date with changes in the upstream subproject. Previous workflows for incorporating such changes include the ungainly: $ git submodule foreach 'git checkout $(git config --file $toplevel/.gitmodules submodule.$name.branch) && git pull' With this patch, all of the useful functionality for incorporating superproject changes can be reused to incorporate upstream subproject updates. When you specify --remote, the target $sha1 is replaced with a $sha1 of the submodule's origin/master tracking branch. If you want to merge a different tracking branch, you can configure the `submodule..branch` option in `.gitmodules`. You can override the `.gitmodules` configuration setting for a particular superproject by configuring the option in that superproject's default configuration (using the usual configuration hierarchy, e.g. `.git/config`, `~/.gitconfig`, etc.). Previous use of submodule..branch ======================================= Because we're adding a new configuration option, it's a good idea to check if anyone else is already using the option. The foreach-pull example above was described by Ævar in commit f030c96d8643fa0a1a9b2bd9c2f36a77721fb61f Author: Ævar Arnfjörð Bjarmason Date: Fri May 21 16:10:10 2010 +0000 git-submodule foreach: Add $toplevel variable Gerrit uses the same interpretation for the setting, but because Gerrit has direct access to the subproject repositories, it updates the superproject repositories automatically when a subproject changes. Gerrit also accepts the special value '.', which it expands into the superproject's branch name. Although the --remote functionality is using `submodule..branch` slightly differently, the effect is the same. The foreach-pull example uses the option to record the name of the local branch to checkout before pulls. The tracking branch to be pulled is recorded in `.git/modules//config`, which was initialized by the module clone during `submodule add` or `submodule init`. Because the branch name stored in `submodule..branch` was likely the same as the branch name used during the initial `submodule add`, the same branch will be pulled in each workflow. Implementation details ====================== In order to ensure a current tracking branch state, `update --remote` fetches the submodule's remote repository before calculating the SHA-1. However, I didn't change the logic guarding the existing fetch: if test -z "$nofetch" then # Run fetch only if $sha1 isn't present or it # is not reachable from a ref. (clear_local_git_env; cd "$path" && ( (rev=$(git rev-list -n 1 $sha1 --not --all 2>/dev/null) && test -z "$rev") || git-fetch)) || die "$(eval_gettext "Unable to fetch in submodule path '\$path'")" fi There will not be a double-fetch, because the new $sha1 determined after the `--remote` triggered fetch should always exist in the repository. If it doesn't, it's because some racy process removed it from the submodule's repository and we *should* be re-fetching. Signed-off-by: W. Trevor King --- Documentation/config.txt | 6 ++++++ Documentation/git-submodule.txt | 23 ++++++++++++++++++++++- Documentation/gitmodules.txt | 5 +++++ git-submodule.sh | 21 ++++++++++++++++++++- t/t7406-submodule-update.sh | 31 +++++++++++++++++++++++++++++++ 5 files changed, 84 insertions(+), 2 deletions(-) diff --git a/Documentation/config.txt b/Documentation/config.txt index bf8f911..7976a6b 100644 --- a/Documentation/config.txt +++ b/Documentation/config.txt @@ -1995,6 +1995,12 @@ submodule..update:: URL and other values found in the `.gitmodules` file. See linkgit:git-submodule[1] and linkgit:gitmodules[5] for details. +submodule..branch:: + The remote branch name for a submodule, used by `git submodule + update --remote`. Set this option to override the value found in + the `.gitmodules` file. See linkgit:git-submodule[1] and + linkgit:gitmodules[5] for details. + submodule..fetchRecurseSubmodules:: This option can be used to control recursive fetching of this submodule. It can be overridden by using the --[no-]recurse-submodules diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt index b1de3ba..8bf173a 100644 --- a/Documentation/git-submodule.txt +++ b/Documentation/git-submodule.txt @@ -13,7 +13,7 @@ SYNOPSIS [--reference ] [--] [] 'git submodule' [--quiet] status [--cached] [--recursive] [--] [...] 'git submodule' [--quiet] init [--] [...] -'git submodule' [--quiet] update [--init] [-N|--no-fetch] [--rebase] +'git submodule' [--quiet] update [--init] [--remote] [-N|--no-fetch] [--rebase] [--reference ] [--merge] [--recursive] [--] [...] 'git submodule' [--quiet] summary [--cached|--files] [(-n|--summary-limit) ] [commit] [--] [...] @@ -236,6 +236,27 @@ OPTIONS (the default). This limit only applies to modified submodules. The size is always limited to 1 for added/deleted/typechanged submodules. +--remote:: + This option is only valid for the update command. Instead of using + the superproject's recorded SHA-1 to update the submodule, use the + status of the submodule's remote tracking branch. The remote used + is branch's remote (`branch..remote`), defaulting to `origin`. + The remote branch used defaults to `master`, but the branch name may + be overridden by setting the `submodule..branch` option in + either `.gitmodules` or `.git/config` (with `.git/config` taking + precedence). ++ +This works for any of the supported update procedures (`--checkout`, +`--rebase`, etc.). The only change is the source of the target SHA-1. +For example, `submodule update --remote --merge` will merge upstream +submodule changes into the submodules, while `submodule update +--merge` will merge superproject gitlink changes into the submodules. ++ +In order to ensure a current tracking branch state, `update --remote` +fetches the submodule's remote repository before calculating the +SHA-1. If you don't want to fetch, you should use `submodule update +--remote --no-fetch`. + -N:: --no-fetch:: This option is only valid for the update command. diff --git a/Documentation/gitmodules.txt b/Documentation/gitmodules.txt index ab3e91c..52d7ae4 100644 --- a/Documentation/gitmodules.txt +++ b/Documentation/gitmodules.txt @@ -49,6 +49,11 @@ submodule..update:: This config option is overridden if 'git submodule update' is given the '--merge', '--rebase' or '--checkout' options. +submodule..branch:: + A remote branch name for tracking updates in the upstream submodule. + If the option is not specified, it defaults to 'master'. See the + `--remote` documentation in linkgit:git-submodule[1] for details. + submodule..fetchRecurseSubmodules:: This option can be used to control recursive fetching of this submodule. If this option is also present in the submodules entry in diff --git a/git-submodule.sh b/git-submodule.sh index 263a60c..6ae51c6 100755 --- a/git-submodule.sh +++ b/git-submodule.sh @@ -8,7 +8,7 @@ dashless=$(basename "$0" | sed -e 's/-/ /') USAGE="[--quiet] add [-b ] [-f|--force] [--name ] [--reference ] [--] [] or: $dashless [--quiet] status [--cached] [--recursive] [--] [...] or: $dashless [--quiet] init [--] [...] - or: $dashless [--quiet] update [--init] [-N|--no-fetch] [-f|--force] [--rebase] [--reference ] [--merge] [--recursive] [--] [...] + or: $dashless [--quiet] update [--init] [--remote] [-N|--no-fetch] [-f|--force] [--rebase] [--reference ] [--merge] [--recursive] [--] [...] or: $dashless [--quiet] summary [--cached|--files] [--summary-limit ] [commit] [--] [...] or: $dashless [--quiet] foreach [--recursive] or: $dashless [--quiet] sync [--recursive] [--] [...]" @@ -26,6 +26,7 @@ cached= recursive= init= files= +remote= nofetch= update= prefix= @@ -559,6 +560,9 @@ cmd_update() -i|--init) init=1 ;; + --remote) + remote=1 + ;; -N|--no-fetch) nofetch=1 ;; @@ -619,6 +623,7 @@ cmd_update() fi name=$(module_name "$sm_path") || exit url=$(git config submodule."$name".url) + branch=$(get_submodule_config "$name" branch master) if ! test -z "$update" then update_module=$update @@ -653,6 +658,20 @@ Maybe you want to use 'update --init'?")" die "$(eval_gettext "Unable to find current revision in submodule path '\$sm_path'")" fi + if test -n "$remote" + then + if test -z "$nofetch" + then + # Fetch remote before determining tracking $sha1 + (clear_local_git_env; cd "$sm_path" && git-fetch) || + die "$(eval_gettext "Unable to fetch in submodule path '\$sm_path'")" + fi + remote_name=$(clear_local_git_env; cd "$sm_path" && get_default_remote) + sha1=$(clear_local_git_env; cd "$sm_path" && + git rev-parse --verify "${remote_name}/${branch}") || + die "$(eval_gettext "Unable to find current ${remote_name}/${branch} revision in submodule path '\$sm_path'")" + fi + if test "$subsha1" != "$sha1" -o -n "$force" then subforce=$force diff --git a/t/t7406-submodule-update.sh b/t/t7406-submodule-update.sh index feaec6c..4975ec0 100755 --- a/t/t7406-submodule-update.sh +++ b/t/t7406-submodule-update.sh @@ -135,6 +135,37 @@ test_expect_success 'submodule update --force forcibly checks out submodules' ' ) ' +test_expect_success 'submodule update --remote should fetch upstream changes' ' + (cd submodule && + echo line4 >> file && + git add file && + test_tick && + git commit -m "upstream line4" + ) && + (cd super && + git submodule update --remote --force submodule && + cd submodule && + test "$(git log -1 --oneline)" = "$(GIT_DIR=../../submodule/.git git log -1 --oneline)" + ) +' + +test_expect_success 'local config should override .gitmodules branch' ' + (cd submodule && + git checkout -b test-branch && + echo line5 >> file && + git add file && + test_tick && + git commit -m "upstream line5" && + git checkout master + ) && + (cd super && + git config submodule.submodule.branch test-branch && + git submodule update --remote --force submodule && + cd submodule && + test "$(git log -1 --oneline)" = "$(GIT_DIR=../../submodule/.git git log -1 --oneline test-branch)" + ) +' + test_expect_success 'submodule update --rebase staying on master' ' (cd super/submodule && git checkout master -- 1.8.0 ``` ## wking@tremily.us, 2012-12-19 16:03 Subject: [PATCH v8 3/3] submodule add: If --branch is given, record it in .gitmodules Message-ID: <2c8df95b9f4cc9ceb9d6d96d5deac22320541dd9.1355932282.git.wking@tremily.us> URL: https://gitlist.dev/e/2c8df95b9f4cc9ceb9d6d96d5deac22320541dd9.1355932282.git.wking%40tremily.us In-Reply-To: ``` From: "W. Trevor King" This allows you to easily record a submodule..branch option in .gitmodules when you add a new submodule. With this patch, $ git submodule add -b [] $ git config -f .gitmodules submodule..branch reduces to $ git submodule add -b [] This means that future calls to $ git submodule update --remote ... will get updates from the same branch that you used to initialize the submodule, which is usually what you want. Signed-off-by: W. Trevor King --- Documentation/git-submodule.txt | 2 ++ git-submodule.sh | 4 ++++ t/t7400-submodule-basic.sh | 1 + 3 files changed, 7 insertions(+) diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt index 8bf173a..b1996f1 100644 --- a/Documentation/git-submodule.txt +++ b/Documentation/git-submodule.txt @@ -208,6 +208,8 @@ OPTIONS -b:: --branch:: Branch of repository to add as submodule. + The name of the branch is recorded as `submodule..branch` in + `.gitmodules` for `update --remote`. -f:: --force:: diff --git a/git-submodule.sh b/git-submodule.sh index 6ae51c6..22ec5b6 100755 --- a/git-submodule.sh +++ b/git-submodule.sh @@ -417,6 +417,10 @@ Use -f if you really want to add it." >&2 git config -f .gitmodules submodule."$sm_name".path "$sm_path" && git config -f .gitmodules submodule."$sm_name".url "$repo" && + if test -n "$branch" + then + git config -f .gitmodules submodule."$sm_name".branch "$branch" + fi && git add --force .gitmodules || die "$(eval_gettext "Failed to register submodule '\$sm_path'")" } diff --git a/t/t7400-submodule-basic.sh b/t/t7400-submodule-basic.sh index de7d453..2683cba 100755 --- a/t/t7400-submodule-basic.sh +++ b/t/t7400-submodule-basic.sh @@ -133,6 +133,7 @@ test_expect_success 'submodule add --branch' ' ( cd addtest && git submodule add -b initial "$submodurl" submod-branch && + test "initial" = "$(git config -f .gitmodules submodule.submod-branch.branch)" && git submodule init ) && -- 1.8.0 ``` ## Junio C Hamano, 2012-12-19 17:43 Subject: Re: [PATCH v8 3/3] submodule add: If --branch is given, record it in .gitmodules Message-ID: <7vhani9bul.fsf@alter.siamese.dyndns.org> URL: https://gitlist.dev/e/7vhani9bul.fsf%40alter.siamese.dyndns.org In-Reply-To: <2c8df95b9f4cc9ceb9d6d96d5deac22320541dd9.1355932282.git.wking@tremily.us> ``` wking@tremily.us writes: > From: "W. Trevor King" > > This allows you to easily record a submodule..branch option in > .gitmodules when you add a new submodule. With this patch, > > $ git submodule add -b [] > $ git config -f .gitmodules submodule..branch > > reduces to > > $ git submodule add -b [] > > This means that future calls to > > $ git submodule update --remote ... > > will get updates from the same branch that you used to initialize the > submodule, which is usually what you want. I agree that it would usually be what you want when you are using the --remote option. Will replace the previous round with this. Thanks. ``` ## Heiko Voigt, 2012-12-21 08:18 Subject: Re: [PATCH v8 0/3] submodule update: add --remote for submodule's upstream changes Message-ID: <20121221081803.GA560@book.hvoigt.net> URL: https://gitlist.dev/e/20121221081803.GA560%40book.hvoigt.net In-Reply-To: ``` Hi, On Wed, Dec 19, 2012 at 11:03:30AM -0500, wking@tremily.us wrote: > From: "W. Trevor King" > > Comments on v7 seem to have petered out, so here's v8. Changes since > v7: > > * Series based on gitster/master instead of v1.8.0. > * In Documentation/config.txt, restored trailing line of > submodule..update documentation, which I had accidentally > removed in v7. > * In Documentation/git-submodule.txt, make --no-fetch example in the > --remote description more general, following Phil's suggestion. > * In git-submodule.sh: > * Remove accidental "ges" line. > * Use the submodule's default remote to determine which tracking > branch to fetch. In v7 I'd been using the superproject's default > remote. > * In cmd_add(), use sm_name instead of sm_path to store the --branch > option (catching up with 73b0898). Sorry, I was not able to follow the discussion that closely lately but I like the outcome. For me there is nothing to change or add functionality wise. Thanks. Cheers Heiko ``` ## Heiko Voigt, 2012-12-21 08:20 Subject: Re: [PATCH v8 1/3] submodule: add get_submodule_config helper funtion Message-ID: <20121221082033.GB560@book.hvoigt.net> URL: https://gitlist.dev/e/20121221082033.GB560%40book.hvoigt.net In-Reply-To: <3377beb925bc209d90058493b74d174db1b7aa50.1355932282.git.wking@tremily.us> ``` On Wed, Dec 19, 2012 at 11:03:31AM -0500, wking@tremily.us wrote: > diff --git a/git-submodule.sh b/git-submodule.sh > index 2365149..263a60c 100755 > --- a/git-submodule.sh > +++ b/git-submodule.sh > @@ -153,6 +153,32 @@ die_if_unmatched () [...] > +get_submodule_config () { > + name="$1" > + option="$2" > + default="$3" > + value=$(git config submodule."$name"."$option") > + if test -z "$value" > + then > + value=$(git config -f .gitmodules submodule."$name"."$option") > + fi > + printf '%s' "${value:-$default}" > +} > + > + > +# Minor nit: For all other functions we only have one newline as separator. Cheers Heiko ``` ## W. Trevor King, 2012-12-21 11:04 Subject: Re: [PATCH v8 1/3] submodule: add get_submodule_config helper funtion Message-ID: <20121221110425.GE16849@odin.tremily.us> URL: https://gitlist.dev/e/20121221110425.GE16849%40odin.tremily.us In-Reply-To: <20121221082033.GB560@book.hvoigt.net> ``` On Fri, Dec 21, 2012 at 09:20:33AM +0100, Heiko Voigt wrote: > On Wed, Dec 19, 2012 at 11:03:31AM -0500, wking@tremily.us wrote: > > diff --git a/git-submodule.sh b/git-submodule.sh > > index 2365149..263a60c 100755 > > --- a/git-submodule.sh > > +++ b/git-submodule.sh > > @@ -153,6 +153,32 @@ die_if_unmatched () > [...] > > +get_submodule_config () { > > + name="$1" > > + option="$2" > > + default="$3" > > + value=$(git config submodule."$name"."$option") > > + if test -z "$value" > > + then > > + value=$(git config -f .gitmodules submodule."$name"."$option") > > + fi > > + printf '%s' "${value:-$default}" > > +} > > + > > + > > +# > > Minor nit: For all other functions we only have one newline as > separator. I'm fine rerolling this, or Junio can make the change on my behalf, or it can be left as is ;). Cheers, Trevor -- This email may be signed or encrypted with GnuPG (http://www.gnupg.org). For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy ```