Re: Prefetch maintenance might lead to thundering herd issues

From: Derrick Stolee <stolee@gmail.com>
Date: Sep 20, 2024, 19:20 UTC
Message-ID: <b67c7b6b-f11b-4306-b3ba-c440e8cf7cab@gmail.com>
In-Reply-To: <CAG=Um+3C1JMKoW+j03nF6uS2DjcJnKD7+0PSaiXGB-td9qyc8Q@mail.gmail.com>

On 9/20/24 11:15 AM, Shubham Kanodia wrote:

Show 12 quoted lines

> The current git-maintenance prefetch command allows daily, weekly, and
> hourly schedules.
> However, in repositories with many active developers, timing `fetch`
> to happen at the same wall clock time can lead to a thundering herd
> problem for the backend server.
> 
> Ideally, these fetch requests would be spaced out by a small amount,
> so as not to lead to a co-ordinated load on the server.
> 
> I don't see a way to configure this at the moment in a cross-platform
> way at the moment. Perhaps it might be sensible to offset the cron
> schedules slightly for users so as to not cause such an issue?

This is already handled by using a random minute of the hour, as implemented in 9b43399057 (maintenance: use random minute in cron scheduler, 2023-08-10), for example. There are similar uses for systemd, launchctl, and schtasks schedulers (look for uses of the get_random_minute() method).

Are you noticing that this isn't working as expected?

Thanks, -Stolee