git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: Tight submodule bindings

From
W. Trevor King <wking@tremily.us>
Date
Jan 13, 2014, 20:07 UTC
Message-ID
<20140113200724.GP10613@odin.tremily.us>
In-Reply-To
<52D44081.7070504@web.de>
On Mon, Jan 13, 2014 at 08:37:37PM +0100, Jens Lehmann wrote:
Show 12 quoted lines
> Am 12.01.2014 02:08, schrieb W. Trevor King:
> > For folks who treat the submodule as a black box (and do no local
> > development), switchable trees are all they care about.  They can
> > easily checkout (or not, with deinit), the submodule tree at a
> > gitlinked hash, and everything is nice and reproducible.  The fact
> > that 'submod' is stored as a commit object and not a tree, is just
> > a convenient marker for optional
> > init/deinit/remote-update-integration functionality.
> 
> But there are users (like me) who do not treat submodules as
> black boxes and nonetheless do development in them with update
> set to checkout (after creating a feature branch of course ;-).

I'm still not clear on how this works for you ;). Can you sketch out an example shell history showing how you use checkout updates to do this?

Show 10 quoted lines
> > When you checkout a submodule for the first time, Git should take
> > the default information from .gitmodules and file it away in the
> > submodule's appropriate out-of-tree config locations.
> 
> I disagree, that only makes sense for the URL setting (and this
> currently only happens with the update setting, which I intend to
> change). Everything else should be taken from .gitmodules unless
> the user wants to override it. The only setting I'm not so sure
> about is the local branch setting, as that might have to propagate
> into the submodule.

I think copying into the submodule's out-of-tree config is the way to go, because users won't always be driving the submodule from the superproject. If the settings are in the submodule's out-of-tree config, everything will be consistent betwee stuff run from the superproject and stuff run from submodule itself. It also allows us to use familiar configuration commands inside the submodule, and have those automatically mapped back into the .gitmodules file for us.

Show 8 quoted lines
> > In fact, I think life is easier for everyone if this is the
> > default, and we add a new option (submodule.<name>.sync = false)
> > that says “don't overwrite optional settings in my submodule's
> > out-of-tree config on checkout” for for folks who want to opt out.
> > Don't worry, this is not going to clobber people, because we'll be
> > syncing the other way too.
> 
> Yet another flag to make peoples life easier? I don't think so ;-)

I'm fine if there is no opt-out, and the syncing is mandatory, but I imagine that folks who want a local (unshared, not in .gitmodules) URL would complain.

Show 13 quoted lines
> > Purely local metadata
> > ---------------------
> > 
> > Some metadata does not make sense in the superproject tree.  For
> > example, whether a submodule is interesting enough to checkout
> > (init/deinit) or whether you want to auto-sync optional metadata
> > .gitmodules defaults.  This metadata should live in the
> > superproject's out-of-tree config, and should not be stored in the
> > in-tree .gitmodules.
> 
> Not always. It makes a lot of sense to let upstream mark a
> submodule as "too big and you won't need it anyway" in the
> .gitmodules file.
Good.  Then there's no need for this special class of settings.
Show 10 quoted lines
> > Since you *will* want to share the upstream URL, I proposed using
> > an explicit submodule.<name>.active setting to store the “do I
> > care” information [2], instead of overloading submodule.<name>.url
> > (I'd auto-sync the .gitmodule's submodule.<name>.url with the
> > subproject's remote.origin.url unless the user opted out of
> > .gitmodules syncing).
> 
> That is wrong as it would break horribly when you check out an old
> commit with a now dead submodule URL and that gets automatically
> synced.

If you've already checked out the submodule with a current URL, you should already have the old commit locally, and Git will use it without trying to re-fetch from the broken old URL.

Show 15 quoted lines
> > Subsequent checkouts
> > --------------------
> > 
> > Now that we have strict linking between the submodule state (both
> > in-tree and out-of-tree configs) and the superproject tree (gitlink
> > and .gitmodules), changing between superproject branches is really
> > easy:
> > 
> > 1. Make sure the working tree is not dirty.  If it is, ask the user to
> >    either add-and-commit or stash, and then die to let them do so.
> 
> This condition is too hard, relax that to "a trivial merge can
> switch from current state to target state" and make it behave just
> like branch switching in the superproject. After all submodules
> should behave as much as possible like content of the superproject.
Sounds good to me.
Show 9 quoted lines
> >  Clobber submodule
> >         configs and local branches at will (modulo
> >         submodule.<name>.sync), because any submodule configs that
> >         the user wanted to keep should have been added to the
> >         superproject branch earlier (or stashed).
> 
> I don't think I like this part, but I admit I do not fully
> understand what you mean here. Clobbering stuff the user did doesn't
> sound very nice.

It's fine because we forced them to commit or stash any (not trivially mergable) changes before starting the checkout command.

Show 12 quoted lines
> > Summary
> > -------
> > 
> > New .gitmodules options:
> > 
> > * submodule.<name>.local-branch, store the submodule's HEAD, must
> >   stay in sync for checkouts.
> 
> I'm still not convinced that the current branch setting couldn't be
> extended to carry that information, but no objections against
> configuring such a branch. But what do you mean with "must stay in
> sync for checkouts"?

That checkout-inducing commands should die if the .gitmodule's local-branch and the submodule's HEAD don't match.

Show 6 quoted lines
> > New .git/config options:
> > 
> > * submodule.<name>.active, for init/deinit.
> 
> I understand an option for automatic init (autoinit), but not for
> automatic deinit. Is the latter really useful?

This isn't auto-anything. This is just “I think the submodule is interesting, please turn it on” (i.e. I ran “git submodule init <submod>”).

Only active submodules should get all the syncing, setup, and teardown logic that goes along with submodule checkout. Inactive submodules are ignored.

Show 9 quoted lines
> > I see no reason to add --recurse-submodule flags to 'git checkout'
> > (and merge, …).  Anything that happens post-clone should recurse
> > through submodules automatically, and use the
> > submodule.<name>.active setting to decide when recursion is
> > desired.
> 
> Backwards compatibility and testing. Let's first implement that and
> provide a config option to enable it for real world testing, and
> then let's discuss changing the default later.
Ok.
Show 9 quoted lines
> > The current 'git submodule update --remote' would just be:
> > 
> >   $ git submodule foreach 'git pull'
> > 
> > because all of the local-branch checkouts would have already been
> > handled.
> 
> Nope, that does different things to submodules where "branch" isn't
> configured, right?

It does the same thing. Without submodule.<name>.branch configured, you just integrate the subproject's master.

Show 10 quoted lines
> >  Similarly, a global push would be just:
> > 
> >   $ git submodule foreach 'git push'
> 
> What's wrong with:
> 
> $ git push --recurse-submodules=on-demand
> 
> And it'll push the superproject at the same time. Extra points for
> already being implemented ;-)

That's a strong argument ;). I still don't think the new-in-1.7.4 UI change will add value. The new-in-1.7.7 --recurse-submodules=check would still be useful.

Show 9 quoted lines
> > I'd be happy to mock this up in shell, but only if anyone else
> > would be interested enough to review the implementation ;).  Then
> > I'll look into integrating the preferred model (this tightly bound
> > proposal, or v3's looser bindings, or <your idea here>) in C,
> > building on Jens and Jonathan's work.
> 
> The update modes (cleaning removed submodules and creating new ones)
> are better handled in my recursive checkout series. But I believe we
> can at least prototype the branch handling in shell.

I'll prototype it, and keep trying to convince you about the syncing ;). I think the main arguments for syncing are:

* No divergent configs between superproject-initiated actions and
  submodule-initiated actions.
* No work clobbered, or accidentally uncommitted, due to syncing
  submodule -> superproject before checkout-inducing commands.

Cheers, Trevor

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy
Previous: Jens LehmannNext: Junio C Hamano
Message 24 of 40 in “git-submodule.sh: Support 'checkout' as a valid update command”
  1. 1/2 git-submodule.sh: Support 'checkout' as a valid update commandFrancesco Pretto, Jan 5, 2014
  2. 2/2 Introduce git submodule attached updateFrancesco Pretto, Jan 5, 2014
  3. Francesco PrettoJan 5, 2014
  4. Heiko VoigtJan 5, 2014
  5. Francesco PrettoJan 5, 2014
  6. Heiko VoigtJan 6, 2014
  7. Francesco PrettoJan 6, 2014
  8. David EngsterJan 6, 2014
  9. W. Trevor KingJan 7, 2014
  10. W. Trevor KingJan 7, 2014
  11. Preferred local submodule branches (was: Introduce git submodule attached update)W. Trevor King, Jan 7, 2014
  12. W. Trevor KingJan 7, 2014
  13. W. Trevor KingJan 8, 2014
  14. W. Trevor KingJan 8, 2014
  15. 0/4 Preferred local submodule branchesW. Trevor King, Jan 9, 2014
  16. 1/4 submodule: Add helpers for configurable local branchesW. Trevor King, Jan 9, 2014
  17. 2/4 submodule: Teach 'update' to preserve local branchesW. Trevor King, Jan 9, 2014
  18. 3/4 submodule: Teach 'add' about a configurable local-branchW. Trevor King, Jan 9, 2014
  19. Francesco PrettoJan 15, 2014
  20. W. Trevor KingJan 15, 2014
  21. 4/4 submodule: Add a new 'checkout' commandW. Trevor King, Jan 9, 2014
  22. Tight submodule bindings (was: Preferred local submodule branches)W. Trevor King, Jan 12, 2014
  23. Jens LehmannJan 13, 2014
  24. W. Trevor KingJan 13, 2014
  25. Junio C HamanoJan 13, 2014
  26. W. Trevor KingJan 14, 2014
  27. Heiko VoigtJan 7, 2014
  28. W. Trevor KingJan 7, 2014
  29. Junio C HamanoJan 7, 2014
  30. Francesco PrettoJan 7, 2014
  31. Junio C HamanoJan 7, 2014
  32. Francesco PrettoJan 7, 2014
  33. Francesco PrettoJan 5, 2014
  34. Heiko VoigtJan 6, 2014
  35. W. Trevor KingJan 6, 2014
  36. Heiko VoigtJan 5, 2014
  37. W. Trevor KingJan 5, 2014
  38. Junio C HamanoJan 6, 2014
  39. Junio C HamanoJan 6, 2014
  40. Francesco PrettoJan 6, 2014

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.