From: Derrick Stolee <stolee@gmail.com>
Date: Fri, 20 Sep 2024 19:20:47 GMT
Subject: Re: Prefetch maintenance might lead to thundering herd issues
Message-ID: <b67c7b6b-f11b-4306-b3ba-c440e8cf7cab@gmail.com>
In-Reply-To: <CAG=Um+3C1JMKoW+j03nF6uS2DjcJnKD7+0PSaiXGB-td9qyc8Q@mail.gmail.com>

On 9/20/24 11:15 AM, Shubham Kanodia wrote:
> The current git-maintenance prefetch command allows daily, weekly, and
> hourly schedules.
> However, in repositories with many active developers, timing `fetch`
> to happen at the same wall clock time can lead to a thundering herd
> problem for the backend server.
> 
> Ideally, these fetch requests would be spaced out by a small amount,
> so as not to lead to a co-ordinated load on the server.
> 
> I don't see a way to configure this at the moment in a cross-platform
> way at the moment. Perhaps it might be sensible to offset the cron
> schedules slightly for users so as to not cause such an issue?

This is already handled by using a random minute of the hour, as
implemented in 9b43399057 (maintenance: use random minute in cron
scheduler, 2023-08-10), for example. There are similar uses for
systemd, launchctl, and schtasks schedulers (look for uses of
the get_random_minute() method).

Are you noticing that this isn't working as expected?

Thanks,
-Stolee