Re: Synchronous replication on push

From: Taylor R Campbell <git@campbell.mumble.net>
Date: Nov 4, 2024, 15:50 UTC
Message-ID: <20241104155041.235E260AB3@jupiter.mumble.net>
In-Reply-To: <20241104-real-marmoset-of-success-9a6c8e@meerkat>

Show 17 quoted lines

> Date: Mon, 4 Nov 2024 09:40:06 -0500
> From: Konstantin Ryabitsev <konstantin@linuxfoundation.org>
> 
> Alternatively, you can take a look at grokmirror, which is what kernel.org
> uses:
> https://pypi.org/project/grokmirror/
> 
> It's pull-based instead of push-based, for several reasons:
> 
> 1. We replicate to multiple worldwide frontends, and we expect that some of
>    them may be unreachable at the time when we attempt a push
> 2. This allows us to propagate repository deletes
> 3. This allows us to propagate details like descriptions and authors
> 
> Grokmirror also has a listener daemon that can trigger a pull, so it's
> possible to have near-instantaneous replication by notifying the remote node
> that a repository has been updated and should be pulled.

Thanks, that looks useful, but it's not quite what I'm looking for.

Part of the goal is essentially the same (qualitative) types of service guarantee that Github advertises:[*] once the user's `git push' command has succeeded with nonzero exit status, the objects and ref updates have been written to multiple backing stores so it would take a failure of a quorum of those backing stores to lose the data.

In particular, a backend may reject an update, and when this happens (in the multi-backend case, by enough backends that no quorum is reached), the user who ran `git push' needs to know that it failed so they don't, e.g., delete their branch, run git gc, and go on their merry way having silently lost data.

[*] https://github.blog/engineering/infrastructure/stretching-spokes/