git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH/RFC v2 4/4] remote-curl: main test case for the OS command line overflow

From
Ivan Todoroski <grnch@gmx.net>
Date
Mar 27, 2012, 06:28 UTC
Message-ID
<4F715DF6.7080401@gmx.net>
In-Reply-To
<4F715CF7.5070903@gmx.net>

This is main test case for the original problem that triggered this patch series. We create a repo with 50k tags and then test whether git-clone over the smart HTTP protocol succeeds.

Note that we construct the repo in a slightly different way than the original script used to reproduce the problem. This is because the original script just created 50k tags all pointing to the same commit, so if there was a bug where remote-curl.c was not passing all the refs to fetch-pack we wouldn't know. The clone would succed even if only one tag was passed, because all the other tags were pointing at the same SHA and would be considered present.

Instead we create a repo with 50k independent (dangling) commits and then tag each of those commits with a unique tag. This way if one of the tags is not given to fetch-pack, later stages of the clone would complain about it.

This allows us to test both that the command line overflow was fixed, as
well as that it was fixed in a way that doesn't leave out any of the
refs.
---
 t/t5551-http-fetch.sh |   32 ++++++++++++++++++++++++++++++++
 1 file changed, 32 insertions(+)
diff --git a/t/t5551-http-fetch.sh b/t/t5551-http-fetch.sh
index 26d355725f..fb970bda22 100755
--- a/t/t5551-http-fetch.sh
+++ b/t/t5551-http-fetch.sh
@@ -109,5 +109,37 @@ test_expect_success 'follow redirects (302)' '
 	git clone $HTTPD_URL/smart-redir-temp/repo.git --quiet repo-t
 '
 
+
+test -n "$GIT_TEST_LONG" && test_set_prereq EXPENSIVE
+
+test_expect_success EXPENSIVE 'create 50,000 tags in the repo' '
+	(
+	cd "$HTTPD_DOCUMENT_ROOT_PATH/repo.git" &&
+	N=50000 &&
+	for ((i=1; i<=$N; i++)); do
+		echo "commit refs/heads/too-many-refs"
+		echo "mark :$i"
+		echo "committer git <git@example.com> $i +0000"
+		echo "data 0"
+		echo "M 644 inline bla.txt"
+		echo "data 4"
+		echo "bla"
+		# make every commit dangling by always
+		# rewinding the branch after each commit
+		echo "reset refs/heads/too-many-refs"
+		echo "from :1"
+	done | git fast-import --export-marks=marks &&
+
+	# now assign tags to all the dangling commits we created above
+	export tag=refs/tags/artificially-long-tag-name-to-more-easily-demonstrate-the-problem &&
+	perl -nle '\''/:(.+) (.+)/; print "$2 $ENV{tag}-$1"'\'' < marks >> packed-refs
+	)
+'
+
+test_expect_success EXPENSIVE 'clone the 50,000 tag repo to check OS command line overflow' '
+	git clone $HTTPD_URL/smart/repo.git too-many-refs 2> too-many-refs.err &&
+	test_line_count = 0 too-many-refs.err
+'
+
 stop_httpd
 test_done
-- 
1.7.9.5.4.g4f508
Previous: Ivan TodoroskiNext: Junio C Hamano
Message 53 of 54 in “Clone fails on a repo with too many heads/tags”
  1. Ivan TodoroskiMar 18, 2012
  2. Ivan TodoroskiMar 18, 2012
  3. Nguyen Thai Ngoc DuyMar 18, 2012
  4. Jakub NarebskiMar 18, 2012
  5. Jeff KingMar 18, 2012
  6. Jakub NarebskiMar 18, 2012
  7. Jeff KingMar 19, 2012
  8. Nguyen Thai Ngoc DuyMar 19, 2012
  9. Jeff KingMar 19, 2012
  10. Ivan TodoroskiMar 19, 2012
  11. Nguyen Thai Ngoc DuyMar 19, 2012
  12. Jeff KingMar 19, 2012
  13. Ivan TodoroskiMar 21, 2012
  14. Shawn PearceMar 21, 2012
  15. Jeff KingMar 21, 2012
  16. Jakub NarebskiMar 21, 2012
  17. Ivan TodoroskiMar 21, 2012
  18. Jeff KingMar 21, 2012
  19. Ivan TodoroskiMar 24, 2012
  20. Jeff KingMar 25, 2012
  21. Jeff KingMar 25, 2012
  22. Ivan TodoroskiMar 25, 2012
  23. Ivan TodoroskiMar 25, 2012
  24. Jeff KingMar 26, 2012
  25. Ivan TodoroskiMar 27, 2012
  26. Ivan TodoroskiMar 25, 2012
  27. 1/2 fetch-pack: new option to read refs from stdinIvan Todoroski, Mar 24, 2012
  28. Jeff KingMar 25, 2012
  29. Ivan TodoroskiMar 25, 2012
  30. Ivan TodoroskiMar 25, 2012
  31. Ivan TodoroskiMar 25, 2012
  32. Jeff KingMar 26, 2012
  33. Ivan TodoroskiMar 26, 2012
  34. Jeff KingMar 26, 2012
  35. 2/2 remote-curl: send the refs to fetch-pack on stdinIvan Todoroski, Mar 24, 2012
  36. Jeff KingMar 25, 2012
  37. Ivan TodoroskiMar 25, 2012
  38. Jeff KingMar 26, 2012
  39. 0/4 Fix fetch-pack command line overflow during cloneIvan Todoroski, Mar 27, 2012
  40. 1/4 fetch-pack: new --stdin option to read refs from stdinIvan Todoroski, Mar 27, 2012
  41. Junio C HamanoMar 27, 2012
  42. Ivan TodoroskiMar 27, 2012
  43. Junio C HamanoMar 27, 2012
  44. Ivan TodoroskiMar 27, 2012
  45. 2/4 remote-curl: send the refs to fetch-pack on stdinIvan Todoroski, Mar 27, 2012
  46. Junio C HamanoMar 27, 2012
  47. Ivan TodoroskiMar 27, 2012
  48. 3/4 fetch-pack: test cases for the new --stdin optionIvan Todoroski, Mar 27, 2012
  49. Junio C HamanoMar 27, 2012
  50. Ivan TodoroskiMar 27, 2012
  51. Junio C HamanoMar 27, 2012
  52. Ivan TodoroskiMar 28, 2012
  53. 4/4 remote-curl: main test case for the OS command line overflowIvan Todoroski, Mar 27, 2012
  54. Junio C HamanoMar 27, 2012

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.