Re: Synchronous replication on push

From: Konstantin Ryabitsev <konstantin@linuxfoundation.org>
Date: Nov 4, 2024, 14:40 UTC
Message-ID: <20241104-real-marmoset-of-success-9a6c8e@meerkat>
In-Reply-To: <20241104133544.A04D760A95@jupiter.mumble.net>

On Mon, Nov 04, 2024 at 01:35:44PM +0000, Taylor R Campbell wrote:

Show 10 quoted lines

> > > I'm asking about how to configure a _single_ frontend remote, from the
> > > perspective of developers who are pushing from their development
> > > workstations, so that it replicates to one or many backend stores.
> > > This is, for example, the usage model of Github's proprietary
> > > implementation.
> > 
> > I don't think there's built-in functionality for this and I'm not sure
> > that it can be done without additional software.
> 
> I'm happy to write some additional software.

Alternatively, you can take a look at grokmirror, which is what kernel.org uses: https://pypi.org/project/grokmirror/

It's pull-based instead of push-based, for several reasons:

1. We replicate to multiple worldwide frontends, and we expect that some of
   them may be unreachable at the time when we attempt a push
2. This allows us to propagate repository deletes
3. This allows us to propagate details like descriptions and authors

Grokmirror also has a listener daemon that can trigger a pull, so it's possible to have near-instantaneous replication by notifying the remote node that a repository has been updated and should be pulled.

-K