git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 06/10] traverse_trees(): respect max_allowed_tree_depth

From
Jeff King <peff@peff.net>
Date
Aug 31, 2023, 06:21 UTC
Message-ID
<20230831062140.GF3185325@coredump.intra.peff.net>
In-Reply-To
<20230831061735.GA2702156@coredump.intra.peff.net>

The tree-walk.c code walks trees recursively, and may run out of stack space. The easiest way to see this is with git-archive; on my 64-bit Linux system it runs out of stack trying to generate a tarfile with a tree depth of 13,772.

I've picked 4100 as the depth for our "big" test. I ran it with a much higher value to confirm that we do get a segfault without this patch. But really anything over 4096 is sufficient for its stated purpose, which is to find out if our default limit of 4096 is low enough to prevent segfaults on all platforms. Keeping it small saves us time on the test setup.

The tree-walk code that's touched here underlies unpack_trees(), so this protects any programs which use it, not just git-archive (but archive is easy to test, and was what alerted me to this issue in a real-world case).

Signed-off-by: Jeff King <peff@peff.net>
---
 t/t6700-tree-depth.sh | 66 +++++++++++++++++++++++++++++++++++++++++++
 tree-walk.c           |  4 +++
 2 files changed, 70 insertions(+)
 create mode 100755 t/t6700-tree-depth.sh
diff --git a/t/t6700-tree-depth.sh b/t/t6700-tree-depth.sh
new file mode 100755
index 0000000000..d4d17db2ae
--- /dev/null
+++ b/t/t6700-tree-depth.sh
@@ -0,0 +1,66 @@
+#!/bin/sh
+
+test_description='handling of deep trees in various commands'
+. ./test-lib.sh
+
+# We'll test against two depths here: a small one that will let us check the
+# behavior of the config setting easily, and a large one that should be
+# forbidden by default. Testing the default depth will let us know whether our
+# default is enough to prevent segfaults on systems that run the tests.
+small_depth=50
+big_depth=4100
+
+small_ok="-c core.maxtreedepth=$small_depth"
+small_no="-c core.maxtreedepth=$((small_depth-1))"
+
+# usage: mkdeep <name> <depth>
+#   Create a tag <name> containing a file whose path has depth <depth>.
+#
+# We'll use fast-import here for two reasons:
+#
+#   1. It's faster than creating $big_depth tree objects.
+#
+#   2. As we tighten tree limits, it's more likely to allow large sizes
+#      than trying to stuff a deep path into the index.
+mkdeep () {
+	{
+		echo "commit refs/tags/$1" &&
+		echo "committer foo <foo@example.com> 1234 -0000" &&
+		echo "data <<EOF" &&
+		echo "the commit message" &&
+		echo "EOF" &&
+
+		printf 'M 100644 inline ' &&
+		i=0 &&
+		while test $i -lt $2
+		do
+			printf 'a/'
+			i=$((i+1))
+		done &&
+		echo "file" &&
+
+		echo "data <<EOF" &&
+		echo "the file contents" &&
+		echo "EOF" &&
+		echo
+	} | git fast-import
+}
+
+test_expect_success 'create small tree' '
+	mkdeep small $small_depth
+'
+
+test_expect_success 'create big tree' '
+	mkdeep big $big_depth
+'
+
+test_expect_success 'limit recursion of git-archive' '
+	git $small_ok archive small >/dev/null &&
+	test_must_fail git $small_no archive small >/dev/null
+'
+
+test_expect_success 'default limit for git-archive fails gracefully' '
+	test_must_fail git archive big >/dev/null
+'
+
+test_done
diff --git a/tree-walk.c b/tree-walk.c
index 4efd0fc391..b517792ba2 100644
--- a/tree-walk.c
+++ b/tree-walk.c
@@ -9,6 +9,7 @@
 #include "tree.h"
 #include "pathspec.h"
 #include "json-writer.h"
+#include "environment.h"
 
 static const char *get_mode(const char *str, unsigned int *modep)
 {
@@ -449,6 +450,9 @@ int traverse_trees(struct index_state *istate,
 	int interesting = 1;
 	char *traverse_path;
 
+	if (traverse_trees_cur_depth > max_allowed_tree_depth)
+		return error("exceeded maximum allowed tree depth");
+
 	traverse_trees_count++;
 	traverse_trees_cur_depth++;
 
-- 
2.42.0.561.gaa987ecc69
Previous: Jeff KingNext: Jeff King
Message 7 of 16 in “tree name and depth limits”
  1. 0/10 tree name and depth limitsJeff King, Aug 31, 2023
  2. 01/10 tree-walk: reduce stack size for recursive functionsJeff King, Aug 31, 2023
  3. 02/10 tree-walk: drop MAX_TRAVERSE_TREES macroJeff King, Aug 31, 2023
  4. 03/10 tree-walk: rename "error" variableJeff King, Aug 31, 2023
  5. 04/10 fsck: detect very large tree pathnamesJeff King, Aug 31, 2023
  6. 05/10 add core.maxTreeDepth configJeff King, Aug 31, 2023
  7. 06/10 traverse_trees(): respect max_allowed_tree_depthJeff King, Aug 31, 2023
  8. 07/10 read_tree(): respect max_allowed_tree_depthJeff King, Aug 31, 2023
  9. 08/10 list-objects: respect max_allowed_tree_depthJeff King, Aug 31, 2023
  10. 09/10 tree-diff: respect max_allowed_tree_depthJeff King, Aug 31, 2023
  11. 10/10 lower core.maxTreeDepth default to 2048Jeff King, Aug 31, 2023
  12. Oswald BuddenhagenAug 31, 2023
  13. Jeff KingAug 31, 2023
  14. Junio C HamanoAug 31, 2023
  15. rsbecker@nexbridge.comAug 31, 2023
  16. Jeff KingAug 31, 2023

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.