Re: [wishlist] support of cloning recursively from non-bare submodule hierarchies?

From: Stefan Beller <sbeller@google.com>
Date: Dec 13, 2018, 21:59 UTC
Message-ID: <CAGZ79kYMULBBDBt6Sx___m37TNkruYmpKto5ZGG7u7X1jHuKig@mail.gmail.com>
In-Reply-To: <20181213171917.GC4633@hopa.kiewit.dartmouth.edu>

On Thu, Dec 13, 2018 at 9:19 AM Yaroslav Halchenko <yoh@onerussian.com> wrote:

Show 23 quoted lines

>
> Example - on http://datasets.datalad.org we have a few hundred datasets
> organized into a hierarchy as git submodules.  Each  git submodules carries its
> own .git/ directory so they are "self sufficient" and we could readily assess
> their sizes, and "cut the tree" at any level without looking for the
> supermodule somewhere high up in the tree.
>
> .gitmodules typically has relative paths for the url and path for the
> submodules there, the form which I think we chose because it used to work (I
> could be utterly wrong! but I think it was done in an informed fashion)
> for git clone --recursive:
>
>         $> curl http://datasets.datalad.org/labs/gobbini/famface/.gitmodules
>         [submodule "data"]
>                 path = data
>                 url = ./data
>
> and possibly outside:
>
>         $> curl http://datasets.datalad.org/labs/gobbini/famface/data/.gitmodules
>         [submodule "scripts/mridefacer"]
>                 path = scripts/mridefacer
>                 url = https://github.com/yarikoptic/mridefacer

So far so good.

Show 5 quoted lines

> But unfortunately git doesn't even consider such (valid AFAIK) situation
> while cloning where url has to have .git suffix but repository is not bare and
> a relative "data" path (or "./data" url) is referring to the worktree.
>
>         $> git clone --recursive http://datasets.datalad.org/labs/gobbini/famface/.git

[..]

>         Submodule 'data' (http://datasets.datalad.org/labs/gobbini/famface/.git/data) registered for path 'data'

and here it goes wrong, and you would have expected to see .../gobbini/famface/data, eliding the .git ?

I just checked and this did not work neither in v2.18.0 nor v2.0.0 of Git, so it is either a real old regression in submodules, or something else. Is it possible that the clone worked once without the additional .git in the superproject URL?

> on the server I use the "smart HTTP" git backend, but not sure if that is the one to blame, since
> I do not see in the logs any attempt to get the /data from not under .git/:

If we want to strip off "/.git" of urls to make submodules work, we'd want to look at builtin/submodule--helper.c::compute_submodule_clone_url that was recently introduced.

I wonder if we'd just want to cut off the "/.git" and assume the submodule is there in the worktree. Or if we need to see if the submodule was absorbed into .git/modules/<name> on the remote side. (But if the submodule is checked out both would work)