Re: How to mirror and augment a git repository

From: Jeff King <peff@peff.net>
Date: Mar 6, 2023, 08:08 UTC
Message-ID: <ZAWfkkmrPUqNmrtu@coredump.intra.peff.net>
In-Reply-To: <87ilfgemkb.fsf@sebyte.me>

On Sat, Mar 04, 2023 at 12:19:16PM +0000, Sebastian Tennant wrote:

Show 5 quoted lines

>  Augmented mirror:
> 
>    $ git clone --mirror <upstream> upstream
>    $ cd upstream
>    $ git remote update  # regular cron job

The problem here is that your refspec for "origin" in the mirror will be "+refs/*:refs/*". So it claims to have responsibility for the whole refs namespace. And because of the "+", it will happily overwrite local refs when fetching, including branches pushed up by the client. You noticed it most with "prune", because that deletes local branches not present in upstream repo. But a similar problem would happen if both a client and the upstream had a branch named "foo".

Show 7 quoted lines

> I've tried running the augmented mirror as a plain bare repo, i.e.
> 
>    $ git config --unset remote.origin.fetch
>    $ git config --unset remote.origin.mirror
> 
> but then the cron job (git remote update) is no longer sufficient in
> making all upstream activity available downstream.

Right. If you drop the fetch refspec entirely, then it will only fetch HEAD during your cron jobs, which is not what you want. You want a refspec that tells Git to fetch everything, but you need to divide up the "refs/" namespace into local stuff and mirrored stuff.

You could use the normal "+refs/heads/*:refs/remotes/origin/*" refspec, but it's awkward for the clients to access "refs/remotes/" on the mirror.

So you probably want to keep the upstream branches in "refs/heads/", but mark a special part of the namespace. Like:

  cd augmented-mirror
  git config remote.origin.fetch '+refs/heads/*:refs/heads/upstream/*'

And then "git fetch" will pull all of the remote branches into the "upstream/" namespace. And it's safe to prune, because it will only delete branches in refs/heads/upstream/ (and you may want to just "git fetch --prune" as you fetch via cron, which is a little more efficient than a separate "git remote prune").

Clients can name their branches whatever they like, as long as they don't start with "upstream/", and that won't interfere with the mirror.

-Peff