Re: nested submodules detection w/o .gitmodules file

From: Emily Shaffer <nasamuffin@google.com>
Date: Jan 23, 2023, 20:12 UTC
Message-ID: <CAJoAoZkjfOQwkeQqzQY5qDo7Md5QWSz1pOTBQKHL5KwNu2VoDg@mail.gmail.com>
In-Reply-To: <1716310675.20230122233403@inbox.ru>

On Sun, Jan 22, 2023 at 12:45 PM Andry <andry@inbox.ru> wrote:

Show 13 quoted lines

>
> Hello Git,
>
> I have a pretty long investigation has been started from usage 3dparty projects related directly or indirectly to the git submodules:
>
> `svn externals replacement` : https://github.com/chronoxor/gil/issues/6
> `svn complete replacement for externals` : https://github.com/dirk-thomas/vcstool/issues/243
>
> And stumbled on this discussion:
>
> `nested submodules detection w/o .gitmodules file` : https://github.com/gitextensions/gitextensions/discussions/10644 (https://github.com/gitextensions/gitextensions/issues/10642)
>
> The main question here is that, could the git have has submodules without `.submodules` file?

There is a little nuance here. Git can have nested repositories in a few different ways; submodules are just one of them. A submodule is the combination of a gitlink object in the object graph *and* a corresponding entry in the .gitmodules file. It's certainly possible to embed a nested repository in other ways, such as by ignoring the .gitmodules file, but then your nested repository is no longer a submodule, and operations which recurse over submodules will not consider that nested repository. The reason is that we need to understand where to clone the submodule from - that information isn't contained in the superproject's repository URL, and it can't be contained in the gitlink directly (which is in essence just a commit object, but one that exists in the nested repository, not in the superproject repository). If we didn't have a way of writing down the submodule's remote URL and version controlling it, we wouldn't have a way to populate the submodules when a user is cloning.

Show 21 quoted lines

>
> If no, then all side projects which utilizes it's own input file for the externals may subsequentially fail:
>
> https://github.com/chronoxor/gil
> https://github.com/dirk-thomas/vcstool
> https://github.com/ingydotnet/git-subrepo
>
> If yes, then other projects which does rely on the `.submodules` would have not actual or even invalid state:
>
> https://github.com/gitextensions/gitextensions
>
> Or even the github itself:
>
> `Zip archive to include submodule` : https://github.com/dear-github/dear-github/issues/214
>
> (`[PATCH] archive: add –recurse-submodules to git-archive command` : https://git.github.io/rev_news/2022/11/30/edition-93/, https://lore.kernel.org/git/pull.1359.git.git.1665597148042.gitgitgadget@gmail.com/)
>
> Mine point here is that:
>
> Git database is a primary storage. The `.gitmodules` file is not a primary storage, so can be in not an actual or desync state with the database.
> And any application or a 3dparty project must read the database directly.

The .gitmodules exists to help at clone time; it's possible, as I think you're pointing out, to have some intermediate state locally. But this file is what needs to be the source of truth for putting together the repository on a new machine for the first time.

Show 9 quoted lines

>
> But another problem here is that the git still does not have a stable API for that.
> For example, a submodule can be declared directly from the `.git/config` file in a working copy:
>
> https://git-scm.com/docs/git-submodule#Documentation/git-submodule.txt-init--ltpathgt82308203
> https://git-scm.com/docs/gitsubmodules#_active_submodules
>
> So, who is right and what is wrong here?
>