git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Prefetch maintenance might lead to thundering herd issues

From
Shubham Kanodia <shubham.kanodia10@gmail.com>
Date
Sep 20, 2024, 15:15 UTC
Message-ID
<CAG=Um+3C1JMKoW+j03nF6uS2DjcJnKD7+0PSaiXGB-td9qyc8Q@mail.gmail.com>

The current git-maintenance prefetch command allows daily, weekly, and hourly schedules. However, in repositories with many active developers, timing `fetch` to happen at the same wall clock time can lead to a thundering herd problem for the backend server.

Ideally, these fetch requests would be spaced out by a small amount, so as not to lead to a co-ordinated load on the server.

I don't see a way to configure this at the moment in a cross-platform way at the moment. Perhaps it might be sensible to offset the cron schedules slightly for users so as to not cause such an issue?

This might come at a small cost of predictability of the run, but that should be okay as predictability isn't the primary goal for maintenance.

Next: Derrick Stolee
Message 1 of 4 in “Prefetch maintenance might lead to thundering herd issues”
  1. Shubham KanodiaSep 20, 2024
  2. Derrick StoleeSep 20, 2024
  3. Shubham KanodiaSep 22, 2024
  4. Derrick StoleeSep 23, 2024

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.