git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [RFD/PATCH] submodule doc: describe where we can configure them

From
Stefan Beller <sbeller@google.com>
Date
May 4, 2016, 20:50 UTC
Message-ID
<CAGZ79kZs5zLAUdi53D=pTyFZF__73ZbW4o28C2hOuTV7_NfaDQ@mail.gmail.com>
In-Reply-To
<20160504150143.GA8575@sandbox>
On Wed, May 4, 2016 at 8:01 AM, Heiko Voigt <hvoigt@hvoigt.net> wrote:
Show 16 quoted lines
> On Tue, May 03, 2016 at 05:59:58PM -0700, Stefan Beller wrote:
>> On Tue, May 3, 2016 at 4:56 PM, Jonathan Nieder <jrnieder@gmail.com> wrote:
>> > Stefan Beller wrote:
>> >
>> >> This is similar to the gitignore document, but doesn't mirror
>> >> the current situation. It is rather meant to start a discussion for
>> >> the right approach for mirroring repositories with submodules.
>> >
>> > Ooh.
>>
>> Thanks for writing such a detailed answer. :)
>
> BTW, here is a pointer to the discussion (and what I wrote down) about
> this from back in 2014:
>
> https://github.com/jlehmann/git-submod-enhancements/wiki/Ideas#special-ref-overriding-gitmodules-values

Thanks for pointing at the prior discussion! Although not much happened since then (code wise)?

Show 43 quoted lines
>
>> > To fix this, we could allow additional .gitmodules settings to be put
>> > in another ref (perhaps something like "refs/repository/config" to allow
>> > sharing additional repository-specific configuration in other files
>> > within the same tree --- e.g., branch descriptions).  The semantics:
>> >
>> > * If there is a gitmodules file in refs/repository/config in the
>> >   repository I clone, then the submodule settings from it are stored
>> >   locally somewhere that overrides .gitmodules.  Perhaps
>> >   .git/info/<remotename>/gitmodules?
>> >
>> > * Later fetches from the remote would also update this gitmodules
>> >   file.
>> >
>> > * Settings from this gitmodules file can be overridden locally
>> >   using 'git config' until an explicit "git submodule sync" to
>> >   override the local configuration.
>> >
>> > What do you think?
>> >
>> > If two different remotes provide conflicting values for a setting
>> > in their gitmodules files, git would error out and ask the user
>> > to intervene with a tie-breaking "git config" setting.
>>
>> Let's look at an example with C mirroring from B, who mirrors from A.
>>
>> The user who clones the superproject from C may want to obtain submodules
>> from either C or B or A. All this can be configured in
>> the refs/repository/config value of C, but in case it is not configured in C,
>> it may fall back to the same branch from B. When and how would B get
>> that branch?
>
> I think B has to setup that branch on its own when it starts to mirror
> A and uses different submodule urls or other configs.
>
> Jonathan you suggested to copy the content from a remote to
> .git/info/<remotename>/gitmodules locally. How would one get it to the
> remote side? It seems to me as if we would need to implement additional
> infrastructure to do this. Would it not be simpler if we just kept it on
> a ref on the local side as well? We already have the infrastructure to
> read those values from a ref. We only would need to add something to
> write them. Then a simple push, which could be aliased in form of a
> git-submodule subcommand, suffices to get the values to the remote.
That is good idea!
Show 11 quoted lines
>
> That also solves issues when people clone from their working copy.
>
> I would like to think a little bit further about the conflict situation
> when two remotes are providing values. Configuring this looks to me like
> a nightmare for users. Maybe there is some sort of elegant solution?
> E.g. like we use the values from remote A during a fetch from A, the
> ones from B during a fetch from B and no values from a special ref in
> case there is no remote operation involved. Since the main goal is to
> support forking of submodules isn't there always a remote operation
> involved?
Here is what I imagine
When B mirrors from A, B sets up this special ref for its repository,
e.g. refs/meta/submodule-B and have a symbolic ref pointing at that.
(e.g. SUBMODULE_CONFIG pointing at refs/meta/submodule-B,
which has a worktree which contains a .gitmodules files which
sets up
  "submodule.baz.url = http://B/baz"
  "submodule.relativeBase = http://A"

That way anyone cloning from B would get the superproject and the submodule baz from B while the rest of the submodules are found at A.

When C mirrors from A, they add another branch refs/meta/submodule-C, which can either be a fork of refs/meta/submodule-B with some changes on top of it or it can add a reference to refs/meta/submodule-B, i.e. the configuration would be:

  "submodule.baseConfig = refs/meta/submodule-B"
  "submodule.foo.url = ssh://C/foo"
and SUBMODULE_CONFIG would point at refs/meta/submodule-C.
When cloning from C, the user would get
 * the superproject from C
 * submodule foo from C
 * submodule baz from B
 * all other submodules from A

By the inheriting property of the branch of B there are no conflicting values. C could just overwrite submodule.baseConfig for example.

Show 13 quoted lines
>
> My suggested scheme above does not solve the currently quite typical use
> case where you might 'git fetch' without submodules first and then do
> the submodule fetches during a 'git submodule update'. On the other hand
> in the 'ideal future world' where submodules behave like "normal files" the
> fetch will be done during the superproject fetch so in that case we
> could solve such conflicts.
>
> The main thing which we could keep in mind is that we only allow certain
> values in such special refs. E.g. only the ones needed to support the
> fork workflow. BTW, do we actually need to change other values than the
> URL? Addtionally we ignore other values that are more related to the
> overall project structure. E.g. like submodule.<name>.ignore.

Maybe we want to have a dedicated protocol field, eventually. A,B,C may have different standards on what they use by default. e.g. Use ssh at kernel.org, but http in a corporate mirror, because http is the only protocol not blocked by firewall. So I could imagine that a complete mirror of submodules with relative URLs wants to only replace ssh by http.

Show 6 quoted lines
>
> Ok after writing this it really feels like special casing a lot. I would
> not really call it elegant. At the same time limiting these special refs
> to one special use case (forking) might help us to keep the user
> interface[1] simpler and conflict free in the long run. Not sure. What
> do you think?
>
> Cheers Heiko
>
> [1] Which is not the simplest already.
Previous: Heiko VoigtNext: Heiko Voigt
Message 5 of 13 in “submodule doc: describe where we can configure them”
  1. submodule doc: describe where we can configure themStefan Beller, May 3, 2016
  2. Jonathan NiederMay 3, 2016
  3. Stefan BellerMay 4, 2016
  4. Heiko VoigtMay 4, 2016
  5. Stefan BellerMay 4, 2016
  6. Heiko VoigtMay 8, 2016
  7. Stefan BellerMay 9, 2016
  8. Heiko VoigtMay 11, 2016
  9. Junio C HamanoMay 4, 2016
  10. Heiko VoigtMay 8, 2016
  11. Junio C HamanoMay 9, 2016
  12. Heiko VoigtMay 11, 2016
  13. Junio C HamanoMay 4, 2016

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.