git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v2 2/3] t/lib-httpd: make http-429 first-request check atomic

From
Michael Montalbo via GitGitGadget <gitgitgadget@gmail.com>
Date
Jul 10, 2026, 17:30 UTC
Message-ID
<5f56f32a74b3d900148f02901bcd104927c5e088.1783704657.git.gitgitgadget@gmail.com>
In-Reply-To
<pull.2171.v2.git.1783704657.gitgitgadget@gmail.com>
From: Michael Montalbo <mmontalbo@gmail.com>

http-429.sh records "already returned 429 once" with a "test -f" followed by a "touch" of a shared state file. That check-then-act is not atomic: Apache can run this CGI for several requests at once, and two of them can both pass the "test -f" before either "touch"es, so both treat themselves as the first request. The retry flow that drives this endpoint is mostly sequential, so this has not been seen to fail, but the race is latent.

Decide whether this is the first request with a single atomic mkdir, which fails if the directory already exists, so exactly one of any concurrent requests is rate-limited and the rest are forwarded.

Skipping state for "permanent" is required for correctness, not just an optimization. The marker tells a later or concurrent request that a 429 has already been served, so that it forwards to git-http-backend instead of rate-limiting. Since "permanent" must return 429 to every request, that marker must never become visible to another such request.

The original did not achieve this by staying stateless: its "touch" of the marker ran unconditionally, and the "permanent" case removed it afterward with "rm -f". That create-then-remove leaves a window in which a concurrent "permanent" request sees the marker and is forwarded. It is the same class of check-then-act race this patch removes from the first-request check, latent for the same reason: the flow is mostly sequential. This version fuses the check and the mark into one atomic mkdir and, rather than recreate the pattern as mkdir-then-rmdir, skips the mkdir for "permanent" with a "!= permanent" guard. No marker is ever created, so there is no window and every "permanent" request rate-limits.

There is no accompanying regression test. The check and the set are adjacent commands with no external step in between to synchronize on, so the overlap cannot be forced deterministically, only reproduced probabilistically; the fix is preventive.

Signed-off-by: Michael Montalbo <mmontalbo@gmail.com>
---
 t/lib-httpd/http-429.sh | 28 +++++++++++++++++-----------
 1 file changed, 17 insertions(+), 11 deletions(-)
diff --git a/t/lib-httpd/http-429.sh b/t/lib-httpd/http-429.sh
index c97b16145b..9746ec67ae 100644
--- a/t/lib-httpd/http-429.sh
+++ b/t/lib-httpd/http-429.sh
@@ -26,14 +26,24 @@ repo_path="${remaining#*/}"  # Get rest (repo path)
 # The repo name is the first component before any "/"
 repo_name="${repo_path%%/*}"
 
-# Use current directory (HTTPD_ROOT_PATH) for state file
-# Create a safe filename from test_context, retry_after and repo_name
-# This ensures all requests for the same test context share the same state file
+# Use current directory (HTTPD_ROOT_PATH) for state.
+# Create a safe name from test_context, retry_after and repo_name so that all
+# requests for the same test context share the same state.
 safe_name=$(echo "${test_context}-${retry_after}-${repo_name}" | tr '/' '_' | tr -cd 'a-zA-Z0-9_-')
-state_file="http-429-state-${safe_name}"
+state="http-429-state-${safe_name}"
 
-# Check if this is the first call (no state file exists)
-if test -f "$state_file"
+# This endpoint returns 429 to the first request and forwards later ones to
+# git-http-backend, so the retry succeeds. Apache can run this CGI for several
+# requests at once, so a single atomic "mkdir" elects that first request: the
+# one whose mkdir succeeds returns 429 and leaves the directory behind as the
+# "already rate-limited" marker; every later request finds the directory (mkdir
+# fails) and is forwarded.
+#
+# "permanent" is the exception: it must return 429 to every request and never
+# succeed, so it skips the mkdir and records no state. A leftover directory
+# would make its own later requests find the marker and be forwarded, which is
+# exactly what "permanent" must not do.
+if test "$retry_after" != permanent && ! mkdir "$state" 2>/dev/null
 then
 	# Already returned 429 once, forward to git-http-backend
 	# Set PATH_INFO to just the repo path (without retry-after value)
@@ -52,9 +62,6 @@ then
 	exec "$GIT_EXEC_PATH/git-http-backend"
 fi
 
-# Mark that we've returned 429
-touch "$state_file"
-
 # Output HTTP 429 response
 printf "Status: 429 Too Many Requests\r\n"
 
@@ -67,8 +74,7 @@ case "$retry_after" in
 		printf "Retry-After: invalid-format-123abc\r\n"
 		;;
 	permanent)
-		# Always return 429, don't set state file for success
-		rm -f "$state_file"
+		# Always return 429
 		printf "Retry-After: 1\r\n"
 		printf "Content-Type: text/plain\r\n"
 		printf "\r\n"
-- 
gitgitgadget
Previous: Michael MontalboNext: Michael Montalbo via GitGitGadget
Message 15 of 43 in “t/lib-httpd: make CGI test helpers concurrency-safe”
  1. 0/3 t/lib-httpd: make CGI test helpers concurrency-safeMichael Montalbo via GitGitGadget, Jul 8, 2026
  2. 1/3 t/lib-httpd: fix apply-one-time-script race under concurrent requestsMichael Montalbo via GitGitGadget, Jul 8, 2026
  3. Junio C HamanoJul 8, 2026
  4. Michael MontalboJul 9, 2026
  5. 2/3 t/lib-httpd: make http-429 first-request check atomicMichael Montalbo via GitGitGadget, Jul 8, 2026
  6. Junio C HamanoJul 8, 2026
  7. Junio C HamanoJul 8, 2026
  8. Michael MontalboJul 9, 2026
  9. 3/3 t/README: document writing concurrency-safe helpersMichael Montalbo via GitGitGadget, Jul 8, 2026
  10. Junio C HamanoJul 8, 2026
  11. 0/3 t/lib-httpd: make CGI test helpers concurrency-safeMichael Montalbo via GitGitGadget, Jul 10, 2026
  12. 1/3 t/lib-httpd: fix apply-one-time-script race under concurrent requestsMichael Montalbo via GitGitGadget, Jul 10, 2026
  13. Patrick SteinhardtAug 4, 2026
  14. Michael MontalboAug 7, 2026
  15. 2/3 t/lib-httpd: make http-429 first-request check atomicMichael Montalbo via GitGitGadget, Jul 10, 2026
  16. 3/3 t/README: document writing concurrency-safe helpersMichael Montalbo via GitGitGadget, Jul 10, 2026
  17. Patrick SteinhardtAug 4, 2026
  18. Michael MontalboAug 7, 2026
  19. Patrick SteinhardtAug 10, 2026
  20. Michael MontalboAug 2, 2026
  21. Junio C HamanoAug 3, 2026
  22. 0/3 t/lib-httpd: make CGI test helpers concurrency-safeMichael Montalbo via GitGitGadget, Aug 13, 2026
  23. 1/3 t/lib-httpd: fix apply-one-time-script race under concurrent requestsMichael Montalbo via GitGitGadget, Aug 13, 2026
  24. 2/3 t/lib-httpd: make http-429 first-request check atomicMichael Montalbo via GitGitGadget, Aug 13, 2026
  25. Patrick SteinhardtAug 31, 2026
  26. Junio C HamanoAug 31, 2026
  27. Michael MontalboAug 31, 2026
  28. Michael MontalboAug 31, 2026
  29. 3/3 t/lib-httpd: document writing concurrency-safe CGI helpersMichael Montalbo via GitGitGadget, Aug 13, 2026
  30. Patrick SteinhardtAug 31, 2026
  31. Junio C HamanoAug 26, 2026
  32. Patrick SteinhardtAug 31, 2026
  33. 0/3 t/lib-httpd: make CGI test helpers concurrency-safeMichael Montalbo via GitGitGadget, Sep 1, 2026
  34. 1/3 t/lib-httpd: fix apply-one-time-script race under concurrent requestsMichael Montalbo via GitGitGadget, Sep 1, 2026
  35. 2/3 t/lib-httpd: make http-429 first-request check atomicMichael Montalbo via GitGitGadget, Sep 1, 2026
  36. 3/3 t/lib-httpd: document writing concurrency-safe CGI helpersMichael Montalbo via GitGitGadget, Sep 1, 2026
  37. Patrick SteinhardtSep 1, 2026
  38. Michael MontalboSep 1, 2026
  39. 0/3 t/lib-httpd: make CGI test helpers concurrency-safeMichael Montalbo via GitGitGadget, Sep 1, 2026
  40. 1/3 t/lib-httpd: fix apply-one-time-script race under concurrent requestsMichael Montalbo via GitGitGadget, Sep 1, 2026
  41. 2/3 t/lib-httpd: make http-429 first-request check atomicMichael Montalbo via GitGitGadget, Sep 1, 2026
  42. 3/3 t/lib-httpd: document writing concurrency-safe CGI helpersMichael Montalbo via GitGitGadget, Sep 1, 2026
  43. Patrick SteinhardtSep 3, 2026

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.