git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [RFC/PATCH] add update to branch support for "floating submodules"

From
Phil Hord <phil.hord@gmail.com>
Date
Jan 30, 2012, 21:15 UTC
Message-ID
<CABURp0pDoS1wgJ+Fs3XFX=A_EuR4Gzi4mHLiQP+-icT_d3J+WQ@mail.gmail.com>
In-Reply-To
<4EE7BEF5.6050205@web.de>
I lost my grip on this thread over the holidays...
On Tue, Dec 13, 2011 at 4:09 PM, Jens Lehmann <Jens.Lehmann@web.de> wrote:
Show 74 quoted lines
> Am 13.12.2011 15:17, schrieb Phil Hord:
>> On Mon, Dec 12, 2011 at 2:36 PM, Junio C Hamano <gitster@pobox.com> wrote:
>> [...]
>>> Distro package dependency tracking is a poor analogy for many reasons, but
>>> I'll only touch a few.
>> [...]
>>> Naively, one might think that two branches, branch-1.0 and branch-2.0, can
>>> be defined in the repository of L, tell somebody (like "superproject that
>>> covers all the packages in the distro") that A wants branch-1.0 and B
>>> wants branch-2.0 of L respectively, to emulate this, but if one thinks
>>> further, one would realize that it is insufficient. For one thing, it is
>>> unclear what should happen when both A and B are asked to be checked out,
>>> but more importantly, in dependency requirements on real distro packaging,
>>> the application C could say "I want v1.0 API but v1.4 is broken and not
>>> compatible with me", which won't fit on the two-branches model. A
>>> workaround to add more branches to L could be devised but any workaround
>>> cannot be a good solution that allows a random application C among 47
>>> others to dictate how the branch structure of L project should look like.
>>>
>>> Fortunately, the dependency management is a solved problem by distro
>>> package management and build systems, and they do so without using
>>> anything from submodules. There is no point reinventing these logic in git
>>> submodules and emulating poorly.
>>>
>>> The only remotely plausible analogy around distro packaging would be a
>>> superproject full of all the packages in a distro as its submodules, and
>>> records exact versions of each and every package that goes on a release
>>> CD (or DVD). In that case, you do want to have a central registry that
>>> records what exact version of each package is used to cut the CD and the
>>> mother of all modules superproject could be one way to implement it. But
>>> that is not an example of floating, but is a direct opposite.
>>>
>>> This exchange convinced me further that anybody who wishes to use
>>> "floating" is better off either by doing one or both of the following:
>>>
>>>  - using "exact" but not updating religiously, as the interdepency
>>>   requirement in their project is not strict; or
>>>
>>>  - not using submodules at all, but merely keeping these unrelated A, B, C
>>>   and L as standalone repositories next to each other in the directory
>>>   structure.
>>
>> My interdependency requirements are not so cut-and-dry.  We use
>> submodules to isolate controlled regions of code.  We may need to
>> share our project with a contractor who is allowed to see code
>> pertaining to "vendorA" but not that for "vendorB" or "VendorN".  But
>> our in-house developers want to have all the vendor code in one place
>> for convenient integration. Submodules do this nicely for us.  We can
>> give the contractor just the main modules and the VendorA modules and
>> he'll be fine.  In-house devs get all the submodules (using the
>> vendor-ALL superproject).
>>
>> But this necessarily means there is too much coupling for comfort
>> between our submodules.   For example, when an API changes in the main
>> submodule, each of the vendor submodules is affected because they each
>> implement that API in a custom method.  Some of those vendor modules
>> belong to different people.  Submodule synchronization becomes a real
>> chore.
>
> Hmm, maybe having vendor-specific branches in the superproject would
> help here. But that is hard to tell without knowing more details about
> your setup. But I suspect your vendor-ALL superproject is exactly the
> right spot to deal with these kind of problems (and if that isn't easy
> that might be a result of the difficulty of the problem you are trying
> to solve here, keeping different vendors in sync with your API ;-).
>
>> Floating would help, I think.  Instead I do this:
>>
>>   git pull origin topic_foo && git submodule foreach 'git pull origin topic_foo'
>>
>>   git submodule foreach 'git push origin topic_foo' && git push origin topic_foo
>
> This sounds to me like you would need the "--recurse-submodules" option
> implemented for "git pull" and "git push", no?
Only if I have nested submodules, but yes, we do use --recurs* in our scripts.
> And I miss to see how
> floating would help when the tips of some submodules are not ready to
> work with other submodules tips ...

By project policy, for any branch, all submodules' tips of the same-named branch should be interoperable. The CI server looks after this, as much as he can.

I think of branch names as sticky notes (extra-lightweight tags, sometimes). We have linear history in many of our vendor submodules, but multiple "branches" indicate where each superproject branch has presumably finished integration.

Show 6 quoted lines
>> But not all my developers are git-gurus yet, and they sometimes mess
>> up their ad hoc scripts or miss important changes they forgot to push
>> in one submodule or another.
>
> Sure, even though current git should help you some by showing changes
> in the submodules.
Real newbies may not even remember to use 'git status' strategically.
Show 7 quoted lines
>>  Or worse, their pull or push fails and
>> they can't see the problem for all the noise.  So they email it to me.
>
> We circumvent that by not pulling, but fetching and merging in the
> submodule first and after that in the superproject. You have much more
> control about what is going wrong where (and can have more
> git-experienced people help with - or even do - the merges).

I do that, too, and I wish I didn't have to. I wish I could safely and sanely recover from a conflicted "git pull --recurse-submodules" pull from the superproject. That is, I wish doing so were as straightforward as recovering from the same condition would be if all my code were in one repository instead of in submodules.

Which is the gist -- I wish submodules did not make git more complicated than it already is.

Phil
Previous: Jens LehmannNext: Jens Lehmann
Message 20 of 27 in “add update to branch support for "floating submodules"”
  1. add update to branch support for "floating submodules"Heiko Voigt, Nov 9, 2011
  2. Junio C HamanoNov 9, 2011
  3. Heiko VoigtNov 29, 2011
  4. Leif GruenwoldtDec 10, 2011
  5. Jonathan NiederDec 10, 2011
  6. Junio C HamanoDec 10, 2011
  7. Leif GruenwoldtDec 10, 2011
  8. Andreas T.AuerDec 12, 2011
  9. Leif GruenwoldtDec 12, 2011
  10. Andreas T.AuerDec 12, 2011
  11. Leif GruenwoldtDec 12, 2011
  12. Jens LehmannDec 12, 2011
  13. Phil HordDec 12, 2011
  14. Marc BranchaudDec 13, 2011
  15. Jens LehmannDec 13, 2011
  16. Marc BranchaudDec 13, 2011
  17. Junio C HamanoDec 12, 2011
  18. Phil HordDec 13, 2011
  19. Jens LehmannDec 13, 2011
  20. Phil HordJan 30, 2012
  21. Jens LehmannJan 31, 2012
  22. Phil HordJan 31, 2012
  23. Jens LehmannFeb 1, 2012
  24. Phil HordFeb 6, 2012
  25. Jens LehmannFeb 6, 2012
  26. Brandon CaseyDec 13, 2011
  27. Gioele BarabucciDec 10, 2011

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.