Re: bug: submodule update fails to fetch

From: Sergei Golubchik <vuvova@gmail.com>
Date: Jun 22, 2023, 16:39 UTC
Message-ID: <ZJR5SPDj4Wt_gmRO@pweza>
In-Reply-To: <ZJRHlJvE4BMue1/Z@nand.local>

Hi, Taylor,

On Jun 22, Taylor Blau wrote:

Show 31 quoted lines

> On Thu, Jun 22, 2023 at 01:09:07PM +0200, Sergei Golubchik wrote:
> > Hi,
> >
> > Sometimes (my local repository has lots of branches) after switching
> > branches
> >
> >   git submodule update --init --recursive
> >
> > fails with something like
> >
> >   fatal: transport 'file' not allowed
> >   fatal: Fetched in submodule path 'wsrep-lib', but it did not contain e238c0d240c2557229b0523a4a032f3cf8b41639. Direct fetching of that commit failed.
> >
> > the submodule transport is not 'file' (it's https) and the direct
> > fetching of the commit actually works:
> >
> >   cd wsrep-lib
> >   git fetch origin e238c0d240c2557229b0523a4a032f3cf8b41639
> >   git checkout e238c0d240c2557229b0523a4a032f3cf8b41639
> >   cd ..
> >
> > after that
> >
> >   git submodule update --init --recursive
> >
> > succeeds.
> 
> It makes sense that after manually fetching the desired tip that the
> submodule update goes through OK, because there is nothing to do (the
> checked-out state matches what's in .gitmodules), so we don't have to
> use any transport mechanism.

Right. I just used it to show that git thinks the submodule was updated correctly and doesn't try to do anything after that.

> I recently changed the submodule update rules to disallow file-based
> submodules when not directly executed by the user. See a1d4f67c12
> (transport: make `protocol.file.allow` be "user" by default, 2022-07-29)
> for more of the details there.

Yes, I've seen it. That submodule shouldn't be affected:

  $ git remote -v
  origin  https://github.com/codership/wsrep-lib.git (fetch)
  origin  https://github.com/codership/wsrep-lib.git (push)

so I wouldn't want to circumvent your fix and allow the file transport that we aren't using.

Show 8 quoted lines

> But in the short-term, I am curious why we are complaining about needing
> to use the file transport when you claim that the submodule actually
> needs the HTTPS transport.
> 
> Are you able to share a copy of your repository, and/or its .gitmodules
> file, and your repository-local .gitconfig, as well? Do you have some
> `url.<base>.insteadOf` value configured elsewhere that would be
> rewriting those paths for you?

No insteadOf. Let me try to... Okay, here's the bug. In submodule--helper.c, fetch_in_submodule() function, there're lines: --------------------------- 2211 if (oid) { 2212 char *hex = oid_to_hex(oid); 2213 char *remote = get_default_remote(); 2214 2215 strvec_pushl(&cp.args, remote, hex, NULL); 2216 free(remote); 2217 } ---------------------------

this get_default_remote() appears to be getting the default remote name for the main repository and then uses it to fetch from the submodule.

It happens that my default remote isn't "origin" (long story), it's "github", but in the submodule it's of course "origin", there's no "github" remote there. As a result, `git submodule update` runs the command

  git fetch github ${commit_hash}

in the submodule, and that's interpreted as 'file' transport.

To repeat this you need a repository where the default remote isn't "origin" and a submodule where the commit cannot be fetched by simply `git fetch` and needs a direct fetch.

Hope this helps.

Regards, Sergei