git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 07/13] test-lib: create test data with "git init --no-template" (almost)

From
Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Date
Dec 12, 2021, 20:13 UTC
Message-ID
<patch-07.13-9402fb23b40-20211212T201308Z-avarab@gmail.com>
In-Reply-To
<cover-00.13-00000000000-20211212T201308Z-avarab@gmail.com>

Change "t/test-lib.sh" to set the newly added "GIT_NO_TEMPLATE_DIR=true" option in the environment, which makes any "git init" invocation take the equivalent of a "--no-template" option, unless explicitly overridden by config or CLI option.

Well, "almost". We're also adding a test-only "GIT_TEST_BARE_TEMPLATE=true", when it's set we'll interpret "GIT_NO_TEMPLATE_DIR=true" as meaning no template, except for empty "hooks" and "branches" directories.

This is because we've added various implicit dependencies on having these created by "git init". In subsequent commits we'll address some of those. More on that below.

The code being added here in "builtin/init-db.c" doesn't require careful review, since this lazy_mkdir_strbuf_or_die_setlen() function and the "no_template" condition will be gone in a few commits as we fix up a few tests to create their own "hooks" and "branches" directories.

This change reduces the I/O the test suite generates by quite a bit. Before this running it with --debug (so that trash is retained) results in ~770MB of trash* (according to "du -shc"), after it's reduced to around 590M, so almost 1/4 bytes we wrote were the same sample hooks and other repetitive data. The number of files created went down from ~151k to ~110k, which around the same reduction of (more than) 1/4.

This will also make our tests more reliable as we're now forced to check whether our software works with any arbitrary --template that may be in use in the wild, as opposed to our relatively "fat" current default.

On the "more on that below", these are the directories we're creating (only in the test suite) for now:

- "hooks": Removing it will be relatively simple, mostly a matter of a
  few tests needing a "mkdir .git/hooks", or the tests assume that
  they'll need to explicitly disable .git/hooks.
- "branches": Similarly trivial. The last attempt to get rid of it was
  aborted in c8a58ac5a52 (Revert "Don't create the $GIT_DIR/branches
  directory on init", 2009-10-31), but only a couple of tests files
  depend on it.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 builtin/init-db.c | 41 ++++++++++++++++++++++++++++++++++++++++-
 t/t0001-init.sh   | 19 ++++++++-----------
 t/test-lib.sh     |  6 ++++--
 wrap-for-bin.sh   |  4 +---
 4 files changed, 53 insertions(+), 17 deletions(-)
diff --git a/builtin/init-db.c b/builtin/init-db.c
index fcf538193c8..8dddb47bdc4 100644
--- a/builtin/init-db.c
+++ b/builtin/init-db.c
@@ -93,10 +93,36 @@ static void copy_templates_1(struct strbuf *path, struct strbuf *template_path,
 	}
 }
 
+static void lazy_mkdir_strbuf_or_die_setlen(struct strbuf *path, size_t oldlen,
+					    const char *dir)
+{
+	strbuf_addstr(path, dir);
+	if (mkdir(path->buf, 0777) < 0) {
+		int saved_errno = errno;
+		struct stat st;
+
+		/*
+		 * Unfortunately there's no EEXIST_{DIR,FILE}, and
+		 * we'd like to pass these only if the path is already
+		 * what we want it to be, not if it's a normal.
+		 */
+		if (lstat(path->buf, &st))
+			die_errno(_("cannot stat '%s'"), path->buf);
+		else if (S_ISDIR(st.st_mode))
+			goto cleanup;
+
+		errno = saved_errno;
+		die_errno(_("cannot mkdir '%s'"), path->buf);
+	}
+cleanup:
+	strbuf_setlen(path, oldlen);
+}
+
 static void copy_templates(int no_template, const char *template_dir,
 			   const char *init_template_dir)
 {
 	struct strbuf path = STRBUF_INIT;
+	size_t len;
 	struct strbuf template_path = STRBUF_INIT;
 	size_t template_len;
 	struct repository_format template_format = REPOSITORY_FORMAT_INIT;
@@ -108,7 +134,7 @@ static void copy_templates(int no_template, const char *template_dir,
 		return;
 	if (!template_dir && !init_template_dir &&
 	    git_env_bool(GIT_NO_TEMPLATE_DIR_ENVIRONMENT, 0))
-		return;
+		goto no_template;
 	if (!template_dir)
 		template_dir = getenv(TEMPLATE_DIR_ENVIRONMENT);
 	if (!template_dir)
@@ -157,6 +183,19 @@ static void copy_templates(int no_template, const char *template_dir,
 	strbuf_release(&path);
 	strbuf_release(&template_path);
 	clear_repository_format(&template_format);
+	return;
+no_template:
+	if (!git_env_bool("GIT_TEST_BARE_TEMPLATE", 0))
+		return;
+
+	strbuf_addstr(&path, get_git_common_dir());
+	strbuf_complete(&path, '/');
+	len = path.len;
+
+	lazy_mkdir_strbuf_or_die_setlen(&path, len, "hooks");
+	lazy_mkdir_strbuf_or_die_setlen(&path, len, "branches");
+
+	strbuf_release(&path);
 }
 
 /*
diff --git a/t/t0001-init.sh b/t/t0001-init.sh
index 39cf132e9a0..a91de28aadc 100755
--- a/t/t0001-init.sh
+++ b/t/t0001-init.sh
@@ -235,7 +235,7 @@ test_expect_success 'init with --template' '
 
 test_expect_success 'init with --template (blank)' '
 	git init template-plain &&
-	test_path_is_file template-plain/.git/info/exclude &&
+	test_path_is_dir template-plain/.git/info &&
 	git init --template= template-blank &&
 	test_path_is_missing template-blank/.git/info/exclude
 '
@@ -246,19 +246,12 @@ no_templatedir_env () {
 	export NO_SET_GIT_TEMPLATE_DIR
 }
 
-init_no_templatedir_env () {
-	(
-		no_templatedir_env &&
-		git init "$1"
-	)
-}
-
 test_expect_success 'init with init.templatedir set' '
 	mkdir templatedir-source &&
 	echo Content >templatedir-source/file &&
 	test_config_global init.templatedir "${HOME}/templatedir-source" &&
 
-	init_no_templatedir_env templatedir-set &&
+	git init templatedir-set &&
 	test_cmp templatedir-source/file templatedir-set/.git/file
 '
 
@@ -267,7 +260,7 @@ test_expect_success 'init with init.templatedir using ~ expansion' '
 	echo Content >templatedir-source/file &&
 	test_config_global init.templatedir "~/templatedir-source" &&
 
-	init_no_templatedir_env templatedir-expansion &&
+	git init templatedir-expansion &&
 	test_cmp templatedir-source/file templatedir-expansion/.git/file
 '
 
@@ -563,15 +556,19 @@ test_expect_success 'remote init from does not use config from cwd' '
 '
 
 test_expect_success 're-init from a linked worktree' '
-	git init main-worktree &&
 	(
+		git init main-worktree &&
+
 		cd main-worktree &&
 		test_commit first &&
 		git worktree add ../linked-worktree &&
+		>empty &&
+		cp empty .git/info/exclude &&
 		mv .git/info/exclude expected-exclude &&
 		cp .git/config expected-config &&
 		find .git/worktrees -print | sort >expected &&
 		git -C ../linked-worktree init &&
+		cp empty .git/info/exclude &&
 		test_cmp expected-exclude .git/info/exclude &&
 		test_cmp expected-config .git/config &&
 		find .git/worktrees -print | sort >actual &&
diff --git a/t/test-lib.sh b/t/test-lib.sh
index 57efcc5e97a..bd09d691da3 100644
--- a/t/test-lib.sh
+++ b/t/test-lib.sh
@@ -1355,11 +1355,13 @@ else # normal case, use ../bin-wrappers only unless $with_dashes:
 		PATH="$GIT_BUILD_DIR:$GIT_BUILD_DIR/t/helper:$PATH"
 	fi
 fi
-GIT_TEMPLATE_DIR="$GIT_BUILD_DIR"/templates/blt
+GIT_NO_TEMPLATE_DIR=true
+GIT_TEST_BARE_TEMPLATE=true
+export GIT_NO_TEMPLATE_DIR GIT_TEST_BARE_TEMPLATE
 GIT_CONFIG_NOSYSTEM=1
 GIT_ATTR_NOSYSTEM=1
 GIT_CEILING_DIRECTORIES="$TRASH_DIRECTORY/.."
-export PATH GIT_EXEC_PATH GIT_TEMPLATE_DIR GIT_CONFIG_NOSYSTEM GIT_ATTR_NOSYSTEM GIT_CEILING_DIRECTORIES
+export PATH GIT_EXEC_PATH GIT_CONFIG_NOSYSTEM GIT_ATTR_NOSYSTEM GIT_CEILING_DIRECTORIES
 
 if test -z "$GIT_TEST_CMP"
 then
diff --git a/wrap-for-bin.sh b/wrap-for-bin.sh
index 95851b85b6b..26efe0b60ac 100644
--- a/wrap-for-bin.sh
+++ b/wrap-for-bin.sh
@@ -7,10 +7,8 @@
 # @@BUILD_DIR@@ and @@PROG@@.
 
 GIT_EXEC_PATH='@@BUILD_DIR@@'
-if test -n "$NO_SET_GIT_TEMPLATE_DIR"
+if test -z "$GIT_TEMPLATE_DIR" && test -z "$GIT_NO_TEMPLATE_DIR"
 then
-	unset GIT_TEMPLATE_DIR
-else
 	GIT_TEMPLATE_DIR='@@BUILD_DIR@@/templates/blt'
 	export GIT_TEMPLATE_DIR
 fi

-- 
2.34.1.1020.gb1392dd1877
Previous: Ævar Arnfjörð BjarmasonNext: Ævar Arnfjörð Bjarmason
Message 22 of 44 in “tests + init: don't rely on templates & add --no-template + config”
  1. 00/13 tests + init: don't rely on templates & add --no-template + configÆvar Arnfjörð Bjarmason, Dec 12, 2021
  2. 01/13 t0001: fix gaps in "TEMPLATE DIRECTORY" coverageÆvar Arnfjörð Bjarmason, Dec 12, 2021
  3. 02/13 init: split out template population from create_default_files()Ævar Arnfjörð Bjarmason, Dec 12, 2021
  4. 03/13 init: unconditionally create the "info" directoryÆvar Arnfjörð Bjarmason, Dec 12, 2021
  5. Derrick StoleeDec 20, 2021
  6. Ævar Arnfjörð BjarmasonDec 20, 2021
  7. Derrick StoleeDec 20, 2021
  8. Ævar Arnfjörð BjarmasonDec 20, 2021
  9. Junio C HamanoDec 20, 2021
  10. Ævar Arnfjörð BjarmasonDec 21, 2021
  11. Junio C HamanoDec 21, 2021
  12. Ævar Arnfjörð BjarmasonDec 21, 2021
  13. Junio C HamanoDec 21, 2021
  14. Ævar Arnfjörð BjarmasonDec 24, 2021
  15. Junio C HamanoDec 25, 2021
  16. Ævar Arnfjörð BjarmasonJan 12, 2022
  17. Derrick StoleeJan 18, 2022
  18. Ævar Arnfjörð BjarmasonJan 19, 2022
  19. 04/13 t0008: don't rely on default ".git/info/exclude"Ævar Arnfjörð Bjarmason, Dec 12, 2021
  20. 05/13 init & clone: add a --no-template optionÆvar Arnfjörð Bjarmason, Dec 12, 2021
  21. 06/13 init & clone: add init.templateDir=[bool]Ævar Arnfjörð Bjarmason, Dec 12, 2021
  22. 07/13 test-lib: create test data with "git init --no-template" (almost)Ævar Arnfjörð Bjarmason, Dec 12, 2021
  23. 08/13 tests: don't depend on template-created .git/branchesÆvar Arnfjörð Bjarmason, Dec 12, 2021
  24. 11/13 tests: change "cat && chmod +x" to use "write_hook"Ævar Arnfjörð Bjarmason, Dec 12, 2021
  25. 10/13 test-lib-functions: add and use a "write_hook" wrapperÆvar Arnfjörð Bjarmason, Dec 12, 2021
  26. Eric SunshineDec 13, 2021
  27. Ævar Arnfjörð BjarmasonDec 13, 2021
  28. Eric SunshineDec 13, 2021
  29. Ævar Arnfjörð BjarmasonDec 13, 2021
  30. Eric SunshineDec 13, 2021
  31. 09/13 t5540: don't rely on "hook/post-update.sample"Ævar Arnfjörð Bjarmason, Dec 12, 2021
  32. 12/13 tests: migrate miscellaneous "write_script" to "write_hooks"Ævar Arnfjörð Bjarmason, Dec 12, 2021
  33. 13/13 tests: don't depend on template-created .git/hooksÆvar Arnfjörð Bjarmason, Dec 12, 2021
  34. 0/7 tests: don't depend on "git init" using the templateÆvar Arnfjörð Bjarmason, Jun 3, 2022
  35. 1/7 t0008: don't rely on default ".git/info/exclude"Ævar Arnfjörð Bjarmason, Jun 3, 2022
  36. 2/7 tests: don't depend on template-created .git/branchesÆvar Arnfjörð Bjarmason, Jun 3, 2022
  37. 3/7 tests: don't assume a .git/info for .git/info/graftsÆvar Arnfjörð Bjarmason, Jun 3, 2022
  38. 4/7 tests: don't assume a .git/info for .git/info/attributesÆvar Arnfjörð Bjarmason, Jun 3, 2022
  39. 5/7 tests: don't assume a .git/info for .git/info/refsÆvar Arnfjörð Bjarmason, Jun 3, 2022
  40. 6/7 tests: don't assume a .git/info for .git/info/excludeÆvar Arnfjörð Bjarmason, Jun 3, 2022
  41. 7/7 tests: don't assume a .git/info for .git/info/sparse-checkoutÆvar Arnfjörð Bjarmason, Jun 3, 2022
  42. Junio C HamanoJun 3, 2022
  43. Ævar Arnfjörð BjarmasonJun 4, 2022
  44. Junio C HamanoJun 6, 2022

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.