git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 1/6] maintenance: add get_random_minute()

From
Derrick Stolee via GitGitGadget <gitgitgadget@gmail.com>
Date
Aug 7, 2023, 18:51 UTC
Message-ID
<fefdaa9457948ee5302e7cbfaae250e0b589d752.1691434300.git.gitgitgadget@gmail.com>
In-Reply-To
<pull.1567.git.1691434300.gitgitgadget@gmail.com>
From: Derrick Stolee <derrickstolee@github.com>

When we initially created background maintenance -- with its hourly, daily, and weekly schedules -- we considered the effects of all clients launching fetches to the server every hour on the hour. The worry of DDoSing server hosts was noted, but left as something we would consider for a future update.

As background maintenance has gained more adoption over the past three years, our worries about DDoSing the big Git hosts has been unfounded. Those systems, especially those serving public repositories, are already resilient to thundering herds of much smaller scale.

However, sometimes organizations spin up specific custom server infrastructure either in addition to or on top of their Git host. Some of these technologies are built for a different range of scale, and can hit concurrency limits sooner. Organizations with such custom infrastructures are more likely to recommend tools like `scalar` which furthers their adoption of background maintenance.

To help solve for this, create get_random_minute() as a method to help Git select a random minute when creating schedules in the future. The integrations with this method do not yet exist, but will follow in future changes.

One thing that is important for testability is that we notice when we are under a test scenario and return a predictable result. The schedules themselves are not checked for this value, but at least one launchctl test checks that we do not unnecessarily reboot the schedule if it has not changed from a previous version.

Signed-off-by: Derrick Stolee <derrickstolee@github.com>
---
 builtin/gc.c | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)
diff --git a/builtin/gc.c b/builtin/gc.c
index f3942188a61..66a972bc292 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -1708,6 +1708,23 @@ static int get_schedule_cmd(const char **cmd, int *is_available)
 	return 1;
 }
 
+MAYBE_UNUSED
+static int get_random_minute(void)
+{
+	static int random_initialized = 0;
+
+	/* Use a static value when under tests. */
+	if (!getenv("GIT_TEST_MAINTENANCE_SCHEDULER"))
+		return 13;
+
+	if (!random_initialized) {
+		srand((unsigned int)getpid());
+		random_initialized = 1;
+	}
+
+	return rand() % 60;
+}
+
 static int is_launchctl_available(void)
 {
 	const char *cmd = "launchctl";
-- 
gitgitgadget
Previous: Taylor BlauNext: Taylor Blau
Message 4 of 47 in “maintenance: schedule maintenance on a random minute”
  1. 0/6 maintenance: schedule maintenance on a random minuteDerrick Stolee via GitGitGadget, Aug 7, 2023
  2. 2/6 maintenance: use random minute in launchctl schedulerDerrick Stolee via GitGitGadget, Aug 7, 2023
  3. Taylor BlauAug 7, 2023
  4. 1/6 maintenance: add get_random_minute()Derrick Stolee via GitGitGadget, Aug 7, 2023
  5. Taylor BlauAug 7, 2023
  6. Junio C HamanoAug 7, 2023
  7. Junio C HamanoAug 8, 2023
  8. Taylor BlauAug 8, 2023
  9. Junio C HamanoAug 8, 2023
  10. Junio C HamanoAug 8, 2023
  11. Taylor BlauAug 8, 2023
  12. Derrick StoleeAug 8, 2023
  13. Taylor BlauAug 8, 2023
  14. Derrick StoleeAug 9, 2023
  15. Junio C HamanoAug 9, 2023
  16. Taylor BlauAug 9, 2023
  17. 3/6 maintenance: use random minute in Windows schedulerDerrick Stolee via GitGitGadget, Aug 7, 2023
  18. 4/6 maintenance: use random minute in cron schedulerDerrick Stolee via GitGitGadget, Aug 7, 2023
  19. 5/6 maintenance: swap method locationsDerrick Stolee via GitGitGadget, Aug 7, 2023
  20. Taylor BlauAug 7, 2023
  21. 6/6 maintenance: use random minute in systemd schedulerDerrick Stolee via GitGitGadget, Aug 7, 2023
  22. Taylor BlauAug 7, 2023
  23. Derrick StoleeAug 8, 2023
  24. Taylor BlauAug 8, 2023
  25. Phillip WoodAug 8, 2023
  26. Derrick StoleeAug 8, 2023
  27. Derrick StoleeAug 8, 2023
  28. Phillip WoodAug 9, 2023
  29. Phillip WoodAug 8, 2023
  30. Phillip WoodAug 8, 2023
  31. Derrick StoleeAug 8, 2023
  32. Derrick StoleeAug 8, 2023
  33. Phillip WoodAug 9, 2023
  34. 0/8 maintenance: schedule maintenance on a random minuteDerrick Stolee via GitGitGadget, Aug 10, 2023
  35. 1/8 maintenance: add get_random_minute()Derrick Stolee via GitGitGadget, Aug 10, 2023
  36. Taylor BlauAug 10, 2023
  37. 2/8 maintenance: use random minute in launchctl schedulerDerrick Stolee via GitGitGadget, Aug 10, 2023
  38. 4/8 maintenance: use random minute in cron schedulerDerrick Stolee via GitGitGadget, Aug 10, 2023
  39. 3/8 maintenance: use random minute in Windows schedulerDerrick Stolee via GitGitGadget, Aug 10, 2023
  40. 5/8 maintenance: swap method locationsDerrick Stolee via GitGitGadget, Aug 10, 2023
  41. 7/8 maintenance: fix systemd schedule overlapsDerrick Stolee via GitGitGadget, Aug 10, 2023
  42. Junio C HamanoAug 10, 2023
  43. Phillip WoodAug 14, 2023
  44. 6/8 maintenance: use random minute in systemd schedulerDerrick Stolee via GitGitGadget, Aug 10, 2023
  45. Phillip WoodAug 14, 2023
  46. 8/8 maintenance: update schedule before configDerrick Stolee via GitGitGadget, Aug 10, 2023
  47. Phillip WoodAug 14, 2023

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.