Re: Mirror repositories for submodules

From: Jeff King <peff@peff.net>
Date: Jun 4, 2026, 06:16 UTC
Message-ID: <20260604061605.GA3194609@coredump.intra.peff.net>
In-Reply-To: <d64e7f31-4e00-478c-ab31-b671242865fb@hogyros.de>

On Thu, Jun 04, 2026 at 02:11:38PM +0900, Simon Richter wrote:

Show 21 quoted lines

> Cloning from our server will, depending on what upstream uses, either a
> relative URL (which will go to our server, but we have little control over
> what the name part of the repository base URL is going to be), or an
> absolute URL that instructs clients to pull from another place, which
> conflicts with our goal to have a self-contained archive.
> 
> The idea posited earlier, to have a "repository identity" that remains the
> same across forks and clones, is somewhat appealing, but the best idea I can
> come up with is generating some kind of repository UUID, and adding a
> symlink -- not a great design because it pollutes outside the repo:
> 
>     $ mkdir myproject
>     $ cd myproject
>     $ git init
>     $ ls -l ..
>     lrwxrwxrwx 1 simon simon   9 Jun  4 14:05
> 12345678-9abc-def0-1234-56789abcdef0.git -> myproject
>     drwxrwxr-x 2 simon simon  40 Jun  4 14:04 myproject
> 
> On the other hand, this can be used to construct a stable relative submodule
> URL.

Here's a thought experiment. What if you put the UUID into a URL, like:

  repoid://123456789.git

Then your in-repo .gitconfig would point to that repo id and be consistent. Of course you need some way to tell Git how to retrieve repoid:// URLs. You could do so with a custom remote helper (git-remote-repoid), but presumably that helper is eventually going to end up going over one of the normal Git protocols.

So we just need to tell Git how to resolve repo id URLs into concrete URLs. And indeed, we have url.*.insteadOf to do rewriting already. So for example, you can add a submodule but convert it into a uuid like this:

  $ git submodule add https://github.com/git/git.git
  $ git config -f .gitmodules submodule.git.url
  https://github.com/git/git.git
  $ git config -f .gitmodules submodule.git.url repoid://123456789.git
  $ git commit -am 'add submodule with magic repoid'

Now if somebody else comes along and clones it naively, the repo uuid is not useful to git by itself:

  $ git clone --recurse-submodules repo
  Submodule 'git' (repoid://123456789.git) registered for path 'git'
  Cloning into '/home/peff/tmp/repo/git'...
  fatal: transport 'repoid' not allowed
  fatal: clone of 'repoid://123456789.git' into submodule path '/home/peff/tmp/repo/git' failed

But imagine that "somehow" they have learned that 123456789.git can be found at some URL. You can do this:

  git -c url.https://github.com/git/git.git.insteadOf=repoid://123456789.git \
      clone --recurse-submodules repo.git

which would clone from the original URL. Or you could even imagine that they have a cache of repositories named by uuid, and then:

  git -c url.https://my/cache/.insteadOf=repoid:// ...

would rewrite all repoid://'s automatically.

The use of "-c" here is mostly for illustration. It is a per-command config, so when you later try to update the submodule, you'd run into the same problem. Probably you'd want to stuff your mapping into on-disk config (either ~/.gitconfig, or if you have a lot of them, perhaps some file included from there).

It would be nice if you could use "git clone -c" (note "-c" as an option to "clone", not to "git") to set a permanent per-repo config variable. But sadly the URL rewriting happens in the submodule repository, not the parent. So it has to be a per-user setting.

Now, all of that said, do we still need uuids at all? If the canonical submodule name is https://github.com/git/git.git, then anybody can just rewrite that locally in the same way using url.*.insteadOf config. And I think this is a pretty standard way of using submodules. E.g., you might rewrite https:// into ssh:// if you prefer that protocol. Or point to a local server if it's faster for you.

Which makes me wonder if I am missing something about the original request that started this thread. But it sounds to me like it is just asking for the existing URL-rewriting feature.

-Peff