git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 5/5] maintenance: allow custom refspecs during prefetch

From
Derrick Stolee <stolee@gmail.com>
Date
Apr 10, 2021, 00:56 UTC
Message-ID
<36aa6837-722c-9ef0-84cc-77e982db9f6e@gmail.com>
In-Reply-To
<87im4vgsvx.fsf@evledraar.gmail.com>
On 4/9/2021 3:28 PM, Ævar Arnfjörð Bjarmason wrote:
Show 66 quoted lines
> 
> On Fri, Apr 09 2021, Derrick Stolee wrote:
> 
>> On 4/7/2021 6:26 AM, Ævar Arnfjörð Bjarmason wrote:
>>> I think converting the whole thing to something like the WIP/RFC patch
>>> below is much better and more readable.
>>
>> This is an interesting approach. I don't see you using the ERR that you
>> are inputting anywhere, so that seems like an unnecessary bloat to the
>> consumers. But maybe I haven't discovered all of the places where this
>> would be useful, but it seems better to pipe stderr to a file for later
>> comparison when needed.
> 
> Yes, it's probably not a good default here. For the test-lib.sh tests
> there's check_sub_test_lib_test and check_sub_test_lib_test_err, most of
> the tests only test stdout.
> 
>>> +test_expect_process_tree () {
>>> +	depth= &&
>>> +	>actual &&
>>> +	cat >expect &&
>>> +	cat <&3 >expect.err
>>> +	while test $# != 0
>>> +	do
>>> +		case "$1" in
>>> +		--depth)
>>> +			depth="$2"
>>> +			shift
>>> +			;;
>>> +		*)
>>> +			break
>>> +			;;
>>> +		esac
>>> +		shift
>>> +	done &&
>> Do you have an example where this is being checked? Or can depth
>> be left as 1 for now?
> 
> It can probably be hardcoded, but I was hoping someone more familiar
> with trace2 would chime in, but I'm fairly sure there's not a way to do
> it without parsing the existing output with either some clever
> grep/awk-ing of the PERF output, or stateful parsing of the JSON.
> 
> I thought that for git maintenance tests perhaps something wanted to
> assert that we didn't have maintenance invoking maintenance, or that
> something expected to prune refs really invoked the relevant prune
> command via "gc".
> 
>>> +	log="$(pwd)/proc-tree.txt" &&
>>> +	>"$log" &&
>>> +	GIT_TRACE2_PERF="$log" "$@" 2>actual.err &&
>>> +	grep "child_start" proc-tree.txt >proc-tree-start.txt || : &&
>>> +	if test -n "$depth"
>>> +	then
>>> +		grep " d$depth " proc-tree-start.txt >tmp.txt || : &&
>>> +		mv tmp.txt proc-tree-start.txt
>>> +	fi &&
>>> +	sed -e 's/^.*argv:\[//' -e 's/\]$//' <proc-tree-start.txt >actual &&
>>> +	test_cmp expect actual &&
>>> +	test_cmp expect.err actual.err
>>> +} 7>&2 2>&4
>>
>> I think similar ideas could apply to test_region. Giving it a try
>> now.
> 
> Probably, I didn't even notice that one...

I gave this a few hours today, and I'm giving up. I'm the first to admit that I don't have the correct scripting skills to do some of these things.

I've got what I tried below. It certainly looks like it would work. It solves the problem of "what if the test is flaky?" by ensuring that all subcommands (at depth 0) match the inputs exactly.

However, the problem comes when trying to make that work for all of the maintenance tests, specifically the 'incremental-repack' task. That task dynamically computes a --batch-size=X parameter, and that is not stable across runs of the script.

This was avoided in the past by only checking for the first of three subcommands when verifying that the 'incremental-repack' task worked. That is, except for the EXPENSIVE test that checks that the --batch-size maxes out at 2g.

The thing that might make these changing parameters work is to allow the specified lines be a _prefix_ of the actual parameters. Or, let each line be a pattern that is checked against that line. Issues come up with how to handle this line-by-line check that I was unable to overcome.

The good news is that the idea of adding a '--prefetch' option to 'git fetch' makes the change to t7900-maintenance.sh much easier, making this change to test_subcommand less of a priority.

I include my attempt here as a patch. Feel free to take whatever you want of it, or none of it and start over. I do think that it makes the test script look much nicer.

Thanks, -Stolee

-- >8 --
From 449d098f2a13860f44b2e6fb96fb2dd5872b511b Mon Sep 17 00:00:00 2001
From: Derrick Stolee <dstolee@microsoft.com>
Date: Fri, 9 Apr 2021 08:42:26 -0400
Subject: [PATCH 1/2] test-lib: add test_subcommands
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

The test_subcommand helper in test-lib-functions.sh satisfied a need to check for certain subcommands in Git processes. This is especially needed in t7900-maintenance.sh to ensure that the maintenance builtin properly calls certain subcommands based on command-line options, config values, and the state of the repository.

However, test_subcommand has some complexities in its use that can be improved. First, it requires that the caller knows to create a log file with GIT_TRACE2_EVENT then supply that to test_subcommand. Further, it only checks that some exact subcommands exist or do not exist. It does not guarantee that the list of subcommands exactly matches a given list. Because of this drawback, the tests that check a subcommand does _not_ run are particularly flaky to slight changes in behavior.

Introduce a new helper, test_subcommands, that resolves these drawbacks:
1. It runs the supplied command and handles the trace log itself.
2. It takes a list of commands overs stdin and compares this to the
   complete list of subcommands run from the top-level command.

The helper does not test that the full subcommand tree matches, because that would cause tests to be too fragile to changes unrelated to the component being tested. This could easily be extended to allow the full tree with an option, if desired.

To ensure we only check the first level, use GIT_TRACE2_PERF output and scan for " d0 " in the rows that include the "child_start" event. The last column includes a way to scrape the subcommand itself from the trace. Sometimes arguments are quoted, such as when passing a refspec with '*' to the subcommand. This makes it difficult to create a matching string within the single-quoted test definitions, so strip these single quotes from the arguments before matching the input.

Only modify one test in t7900-maintenance.sh. The rest of the callers to test_subcommand will be converted to test_subcommands in a later change.

Helped-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
---
 t/t7900-maintenance.sh  | 16 +++++++---------
 t/test-lib-functions.sh | 24 ++++++++++++++++++++++++
 2 files changed, 31 insertions(+), 9 deletions(-)
diff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh
index 2412d8c5c0..e170ab7862 100755
--- a/t/t7900-maintenance.sh
+++ b/t/t7900-maintenance.sh
@@ -30,15 +30,13 @@ test_expect_success 'help text' '
 '
 
 test_expect_success 'run [--auto|--quiet]' '
-	GIT_TRACE2_EVENT="$(pwd)/run-no-auto.txt" \
-		git maintenance run 2>/dev/null &&
-	GIT_TRACE2_EVENT="$(pwd)/run-auto.txt" \
-		git maintenance run --auto 2>/dev/null &&
-	GIT_TRACE2_EVENT="$(pwd)/run-no-quiet.txt" \
-		git maintenance run --no-quiet 2>/dev/null &&
-	test_subcommand git gc --quiet <run-no-auto.txt &&
-	test_subcommand ! git gc --auto --quiet <run-auto.txt &&
-	test_subcommand git gc --no-quiet <run-no-quiet.txt
+	test_subcommands git maintenance run --quiet <<-EOF &&
+	git gc --quiet
+	EOF
+	test_subcommands git maintenance run --auto --quiet </dev/null &&
+	test_subcommands git maintenance run --no-quiet <<-EOF
+	git gc --no-quiet
+	EOF
 '
 
 test_expect_success 'maintenance.auto config option' '
diff --git a/t/test-lib-functions.sh b/t/test-lib-functions.sh
index 6348e8d733..53ffeb07f8 100644
--- a/t/test-lib-functions.sh
+++ b/t/test-lib-functions.sh
@@ -1658,6 +1658,30 @@ test_subcommand () {
 	fi
 }
 
+# Run a command and ensure it succeeds. Use the
+# GIT_TRACE2_PERF logs to ensure that every subcommand
+# run by this top-level Git process is exactly the
+# set supplied over stdin.
+#
+# Redirects stderr to a file named "err". This can
+# be used by tests, but it also provides consistent
+# use of isatty(2) which can affect subcommand calls.
+test_subcommands () {
+	local log line &&
+
+	cat >expect &&
+
+	log="$(pwd)"/subcommand-trace.txt &&
+
+	GIT_TRACE2_PERF="$log" "$@" 2>err &&
+	grep "child_start" "$log" | grep " d0 " >processes || : &&
+
+	sed -e 's/^.*argv:\[//' -e 's/\]$//' -e "s/'//g" <processes >actual &&
+	test_cmp expect actual &&
+
+	rm -f "$log" expect actual processes
+}
+
 # Check that the given command was invoked as part of the
 # trace2-format trace on stdin.
 #
-- 
2.31.1.vfs.0.0

--- >8 ---

And here is the follow-up that attempts to change the rest of
the tests in t7900-maintenance.sh. However, this leads to flaky
tests because of the --batch-size changes.

--- >8 ---

From 61f142ba7fcde8ae2a84f18005f384a4544b7741 Mon Sep 17 00:00:00 2001
From: Derrick Stolee <dstolee@microsoft.com>
Date: Fri, 9 Apr 2021 11:13:42 -0400
Subject: [PATCH 2/2] t7900: convert to test_subcommands

Replace the remaining uses of test_subcommand with test_subcommands,
allowing us to delete the old helper.

Most of these replacements are straightforward. However, some are a bit
more subtle, specifically because we now are checking the full ordered
set of subcommands. Some places we were only testing one of multiple
subcommands that would be run by a task. These are expanded to include
the full set.

When working with the prefetch task, the refspecs that are passed to the
subcommand are normally quoted with single quotes in the GIT_TRACE2_PERF
output, but those characters are removed in test_subcommands, allowing
our tests in t7900-maintenance.sh to avoid escaping such quotes.

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
---
 t/t7900-maintenance.sh  | 285 +++++++++++++++++++---------------------
 t/test-lib-functions.sh |  33 -----
 2 files changed, 133 insertions(+), 185 deletions(-)

diff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh
index e170ab7862..8861435dcf 100755
--- a/t/t7900-maintenance.sh
+++ b/t/t7900-maintenance.sh
@@ -40,78 +40,65 @@ test_expect_success 'run [--auto|--quiet]' '
 '
 
 test_expect_success 'maintenance.auto config option' '
-	GIT_TRACE2_EVENT="$(pwd)/default" git commit --quiet --allow-empty -m 1 &&
-	test_subcommand git maintenance run --auto --quiet <default &&
-	GIT_TRACE2_EVENT="$(pwd)/true" \
-		git -c maintenance.auto=true \
-		commit --quiet --allow-empty -m 2 &&
-	test_subcommand git maintenance run --auto --quiet  <true &&
-	GIT_TRACE2_EVENT="$(pwd)/false" \
-		git -c maintenance.auto=false \
-		commit --quiet --allow-empty -m 3 &&
-	test_subcommand ! git maintenance run --auto --quiet  <false
+	test_subcommands git commit --allow-empty -m 1 <<-EOF &&
+	git maintenance run --auto --no-quiet
+	EOF
+	test_subcommands git -c maintenance.auto=true \
+		commit --allow-empty -m 2 <<-EOF &&
+	git maintenance run --auto --no-quiet
+	EOF
+	test_subcommands git -c maintenance.auto=false \
+		commit --allow-empty -m 3 </dev/null
 '
 
 test_expect_success 'maintenance.<task>.enabled' '
 	git config maintenance.gc.enabled false &&
 	git config maintenance.commit-graph.enabled true &&
-	GIT_TRACE2_EVENT="$(pwd)/run-config.txt" git maintenance run 2>err &&
-	test_subcommand ! git gc --quiet <run-config.txt &&
-	test_subcommand git commit-graph write --split --reachable --no-progress <run-config.txt
+
+	# This also verifies that "git gc" is not run.
+	test_subcommands git maintenance run --quiet <<-EOF
+	git commit-graph write --split --reachable --no-progress
+	EOF
 '
 
 test_expect_success 'run --task=<task>' '
-	GIT_TRACE2_EVENT="$(pwd)/run-commit-graph.txt" \
-		git maintenance run --task=commit-graph 2>/dev/null &&
-	GIT_TRACE2_EVENT="$(pwd)/run-gc.txt" \
-		git maintenance run --task=gc 2>/dev/null &&
-	GIT_TRACE2_EVENT="$(pwd)/run-commit-graph.txt" \
-		git maintenance run --task=commit-graph 2>/dev/null &&
-	GIT_TRACE2_EVENT="$(pwd)/run-both.txt" \
-		git maintenance run --task=commit-graph --task=gc 2>/dev/null &&
-	test_subcommand ! git gc --quiet <run-commit-graph.txt &&
-	test_subcommand git gc --quiet <run-gc.txt &&
-	test_subcommand git gc --quiet <run-both.txt &&
-	test_subcommand git commit-graph write --split --reachable --no-progress <run-commit-graph.txt &&
-	test_subcommand ! git commit-graph write --split --reachable --no-progress <run-gc.txt &&
-	test_subcommand git commit-graph write --split --reachable --no-progress <run-both.txt
+	test_subcommands git maintenance run --task=commit-graph <<-EOF &&
+	git commit-graph write --split --reachable --no-progress
+	EOF
+	test_subcommands git maintenance run --task=gc <<-EOF &&
+	git gc --quiet
+	EOF
+	test_subcommands git maintenance run --task=commit-graph --task=gc <<-EOF
+	git gc --quiet
+	git commit-graph write --split --reachable --no-progress
+	EOF
 '
 
 test_expect_success 'core.commitGraph=false prevents write process' '
-	GIT_TRACE2_EVENT="$(pwd)/no-commit-graph.txt" \
-		git -c core.commitGraph=false maintenance run \
-		--task=commit-graph 2>/dev/null &&
-	test_subcommand ! git commit-graph write --split --reachable --no-progress \
-		<no-commit-graph.txt
+	test_subcommands git -c core.commitGraph=false maintenance \
+		run --task=commit-graph <<-EOF
+	EOF
 '
 
 test_expect_success 'commit-graph auto condition' '
 	COMMAND="maintenance run --task=commit-graph --auto --quiet" &&
+	cat >did-run.txt <<-EOF &&
+	git commit-graph write --split --reachable --no-progress
+	EOF
 
-	GIT_TRACE2_EVENT="$(pwd)/cg-no.txt" \
-		git -c maintenance.commit-graph.auto=1 $COMMAND &&
-	GIT_TRACE2_EVENT="$(pwd)/cg-negative-means-yes.txt" \
-		git -c maintenance.commit-graph.auto="-1" $COMMAND &&
+	test_subcommands git -c maintenance.commit-graph.auto=1 $COMMAND </dev/null &&
+	test_subcommands git -c maintenance.commit-graph.auto="-1" $COMMAND <did-run.txt &&
 
 	test_commit first &&
 
-	GIT_TRACE2_EVENT="$(pwd)/cg-zero-means-no.txt" \
-		git -c maintenance.commit-graph.auto=0 $COMMAND &&
-	GIT_TRACE2_EVENT="$(pwd)/cg-one-satisfied.txt" \
-		git -c maintenance.commit-graph.auto=1 $COMMAND &&
+	test_subcommands git -c maintenance.commit-graph.auto=0 $COMMAND </dev/null &&
+	test_subcommands git -c maintenance.commit-graph.auto="1" $COMMAND <did-run.txt &&
 
 	git commit --allow-empty -m "second" &&
 	git commit --allow-empty -m "third" &&
 
-	GIT_TRACE2_EVENT="$(pwd)/cg-two-satisfied.txt" \
-		git -c maintenance.commit-graph.auto=2 $COMMAND &&
-
-	COMMIT_GRAPH_WRITE="git commit-graph write --split --reachable --no-progress" &&
-	test_subcommand ! $COMMIT_GRAPH_WRITE <cg-no.txt &&
-	test_subcommand $COMMIT_GRAPH_WRITE <cg-negative-means-yes.txt &&
-	test_subcommand ! $COMMIT_GRAPH_WRITE <cg-zero-means-no.txt &&
-	test_subcommand $COMMIT_GRAPH_WRITE <cg-one-satisfied.txt &&
-	test_subcommand $COMMIT_GRAPH_WRITE <cg-two-satisfied.txt
+	test_subcommands git -c maintenance.commit-graph.auto=3 $COMMAND </dev/null &&
+	test_subcommands git -c maintenance.commit-graph.auto=2 $COMMAND <did-run.txt
 '
 
 test_expect_success 'run --task=bogus' '
@@ -138,10 +125,11 @@ test_expect_success 'prefetch multiple remotes' '
 	git -C clone2 switch -c two &&
 	test_commit -C clone1 one &&
 	test_commit -C clone2 two &&
-	GIT_TRACE2_EVENT="$(pwd)/run-prefetch.txt" git maintenance run --task=prefetch 2>/dev/null &&
 	fetchargs="--prune --no-tags --no-write-fetch-head --recurse-submodules=no --refmap= --quiet" &&
-	test_subcommand git fetch remote1 $fetchargs +refs/heads/\\*:refs/prefetch/remote1/\\* <run-prefetch.txt &&
-	test_subcommand git fetch remote2 $fetchargs +refs/heads/\\*:refs/prefetch/remote2/\\* <run-prefetch.txt &&
+	test_subcommands git maintenance run --task=prefetch <<-EOF &&
+	git fetch remote1 $fetchargs +refs/heads/*:refs/prefetch/remote1/*
+	git fetch remote2 $fetchargs +refs/heads/*:refs/prefetch/remote2/*
+	EOF
 	test_path_is_missing .git/refs/remotes &&
 	git log prefetch/remote1/one &&
 	git log prefetch/remote2/two &&
@@ -215,24 +203,23 @@ test_expect_success 'loose-objects task' '
 
 test_expect_success 'maintenance.loose-objects.auto' '
 	git repack -adk &&
-	GIT_TRACE2_EVENT="$(pwd)/trace-lo1.txt" \
-		git -c maintenance.loose-objects.auto=1 maintenance \
-		run --auto --task=loose-objects 2>/dev/null &&
-	test_subcommand ! git prune-packed --quiet <trace-lo1.txt &&
+	test_subcommands git -c maintenance.loose-objects.auto=1 maintenance \
+		run --auto --task=loose-objects </dev/null &&
 	printf data-A | git hash-object -t blob --stdin -w &&
-	GIT_TRACE2_EVENT="$(pwd)/trace-loA" \
-		git -c maintenance.loose-objects.auto=2 \
-		maintenance run --auto --task=loose-objects 2>/dev/null &&
-	test_subcommand ! git prune-packed --quiet <trace-loA &&
+	test_subcommands git -c maintenance.loose-objects.auto=2 \
+		maintenance run --auto --task=loose-objects </dev/null &&
 	printf data-B | git hash-object -t blob --stdin -w &&
-	GIT_TRACE2_EVENT="$(pwd)/trace-loB" \
-		git -c maintenance.loose-objects.auto=2 \
-		maintenance run --auto --task=loose-objects 2>/dev/null &&
-	test_subcommand git prune-packed --quiet <trace-loB &&
-	GIT_TRACE2_EVENT="$(pwd)/trace-loC" \
-		git -c maintenance.loose-objects.auto=2 \
-		maintenance run --auto --task=loose-objects 2>/dev/null &&
-	test_subcommand git prune-packed --quiet <trace-loC
+
+	test_subcommands git -c maintenance.loose-objects.auto=2 \
+		maintenance run --auto --task=loose-objects <<-EOF &&
+	git prune-packed --quiet
+	git pack-objects --quiet .git/objects/pack/loose
+	EOF
+
+	test_subcommands git -c maintenance.loose-objects.auto=2 \
+		maintenance run --auto --task=loose-objects <<-EOF
+	git prune-packed --quiet
+	EOF
 '
 
 test_expect_success 'incremental-repack task' '
@@ -307,38 +294,40 @@ test_expect_success EXPENSIVE 'incremental-repack 2g limit' '
 	git maintenance run --task=loose-objects &&
 
 	# Now run the incremental-repack task and check the batch-size
-	GIT_TRACE2_EVENT="$(pwd)/run-2g.txt" git maintenance run \
-		--task=incremental-repack 2>/dev/null &&
-	test_subcommand git multi-pack-index repack \
-		 --no-progress --batch-size=2147483647 <run-2g.txt
+	test_subcommands git maintenance run \
+		--task=incremental-repack <<-EOF
+	git multi-pack-index repack --no-progress --batch-size=2147483647
+	EOF
 '
 
 test_expect_success 'maintenance.incremental-repack.auto' '
 	git repack -adk &&
 	git config core.multiPackIndex true &&
 	git multi-pack-index write &&
-	GIT_TRACE2_EVENT="$(pwd)/midx-init.txt" git \
-		-c maintenance.incremental-repack.auto=1 \
-		maintenance run --auto --task=incremental-repack 2>/dev/null &&
-	test_subcommand ! git multi-pack-index write --no-progress <midx-init.txt &&
+	test_subcommands git -c maintenance.incremental-repack.auto=1 \
+		maintenance run --auto --task=incremental-repack </dev/null &&
+
 	test_commit A &&
 	git pack-objects --revs .git/objects/pack/pack <<-\EOF &&
 	HEAD
 	^HEAD~1
 	EOF
-	GIT_TRACE2_EVENT=$(pwd)/trace-A git \
-		-c maintenance.incremental-repack.auto=2 \
-		maintenance run --auto --task=incremental-repack 2>/dev/null &&
-	test_subcommand ! git multi-pack-index write --no-progress <trace-A &&
+
+	test_subcommands git -c maintenance.incremental-repack.auto=2 \
+		maintenance run --auto --task=incremental-repack </dev/null &&
+
 	test_commit B &&
 	git pack-objects --revs .git/objects/pack/pack <<-\EOF &&
 	HEAD
 	^HEAD~1
 	EOF
-	GIT_TRACE2_EVENT=$(pwd)/trace-B git \
-		-c maintenance.incremental-repack.auto=2 \
-		maintenance run --auto --task=incremental-repack 2>/dev/null &&
-	test_subcommand git multi-pack-index write --no-progress <trace-B
+
+	test_subcommands git -c maintenance.incremental-repack.auto=2 \
+		maintenance run --auto --task=incremental-repack <<-EOF
+	git multi-pack-index write --no-progress
+	git multi-pack-index expire --no-progress
+	git multi-pack-index repack --no-progress --batch-size=469
+	EOF
 '
 
 test_expect_success 'pack-refs task' '
@@ -346,11 +335,12 @@ test_expect_success 'pack-refs task' '
 	do
 		git branch -f to-pack/$n HEAD || return 1
 	done &&
-	GIT_TRACE2_EVENT="$(pwd)/pack-refs.txt" \
-		git maintenance run --task=pack-refs &&
+	test_subcommands git maintenance run --task=pack-refs <<-EOF &&
+	git pack-refs --all --prune
+	EOF
+
 	ls .git/refs/heads/ >after &&
-	test_must_be_empty after &&
-	test_subcommand git pack-refs --all --prune <pack-refs.txt
+	test_must_be_empty after
 '
 
 test_expect_success '--auto and --schedule incompatible' '
@@ -371,26 +361,26 @@ test_expect_success '--schedule inheritance weekly -> daily -> hourly' '
 	git config maintenance.incremental-repack.enabled true &&
 	git config maintenance.incremental-repack.schedule weekly &&
 
-	GIT_TRACE2_EVENT="$(pwd)/hourly.txt" \
-		git maintenance run --schedule=hourly 2>/dev/null &&
-	test_subcommand git prune-packed --quiet <hourly.txt &&
-	test_subcommand ! git commit-graph write --split --reachable \
-		--no-progress <hourly.txt &&
-	test_subcommand ! git multi-pack-index write --no-progress <hourly.txt &&
+	cat >hourly <<-EOF &&
+	git prune-packed --quiet
+	EOF
 
-	GIT_TRACE2_EVENT="$(pwd)/daily.txt" \
-		git maintenance run --schedule=daily 2>/dev/null &&
-	test_subcommand git prune-packed --quiet <daily.txt &&
-	test_subcommand git commit-graph write --split --reachable \
-		--no-progress <daily.txt &&
-	test_subcommand ! git multi-pack-index write --no-progress <daily.txt &&
+	cat >daily <<-EOF &&
+	git prune-packed --quiet
+	git commit-graph write --split --reachable --no-progress
+	EOF
 
-	GIT_TRACE2_EVENT="$(pwd)/weekly.txt" \
-		git maintenance run --schedule=weekly 2>/dev/null &&
-	test_subcommand git prune-packed --quiet <weekly.txt &&
-	test_subcommand git commit-graph write --split --reachable \
-		--no-progress <weekly.txt &&
-	test_subcommand git multi-pack-index write --no-progress <weekly.txt
+	cat >weekly <<-EOF &&
+	git prune-packed --quiet
+	git multi-pack-index write --no-progress
+	git multi-pack-index expire --no-progress
+	git multi-pack-index repack --no-progress --batch-size=655
+	git commit-graph write --split --reachable --no-progress
+	EOF
+
+	test_subcommands git maintenance run --schedule=hourly <hourly &&
+	test_subcommands git maintenance run --schedule=daily <daily &&
+	test_subcommands git maintenance run --schedule=weekly <weekly
 '
 
 test_expect_success 'maintenance.strategy inheritance' '
@@ -402,58 +392,49 @@ test_expect_success 'maintenance.strategy inheritance' '
 	test_when_finished git config --unset maintenance.strategy &&
 	git config maintenance.strategy incremental &&
 
-	GIT_TRACE2_EVENT="$(pwd)/incremental-hourly.txt" \
-		git maintenance run --schedule=hourly --quiet &&
-	GIT_TRACE2_EVENT="$(pwd)/incremental-daily.txt" \
-		git maintenance run --schedule=daily --quiet &&
-	GIT_TRACE2_EVENT="$(pwd)/incremental-weekly.txt" \
-		git maintenance run --schedule=weekly --quiet &&
-
-	test_subcommand git commit-graph write --split --reachable \
-		--no-progress <incremental-hourly.txt &&
-	test_subcommand ! git prune-packed --quiet <incremental-hourly.txt &&
-	test_subcommand ! git multi-pack-index write --no-progress \
-		<incremental-hourly.txt &&
-	test_subcommand ! git pack-refs --all --prune \
-		<incremental-hourly.txt &&
-
-	test_subcommand git commit-graph write --split --reachable \
-		--no-progress <incremental-daily.txt &&
-	test_subcommand git prune-packed --quiet <incremental-daily.txt &&
-	test_subcommand git multi-pack-index write --no-progress \
-		<incremental-daily.txt &&
-	test_subcommand ! git pack-refs --all --prune \
-		<incremental-daily.txt &&
-
-	test_subcommand git commit-graph write --split --reachable \
-		--no-progress <incremental-weekly.txt &&
-	test_subcommand git prune-packed --quiet <incremental-weekly.txt &&
-	test_subcommand git multi-pack-index write --no-progress \
-		<incremental-weekly.txt &&
-	test_subcommand git pack-refs --all --prune \
-		<incremental-weekly.txt &&
+	# Modify this default for simplicity
+	git config maintenance.prefetch.enabled false &&
+
+	test_subcommands git maintenance run --schedule=hourly <<-EOF &&
+	git commit-graph write --split --reachable --no-progress
+	EOF
+
+	test_subcommands git maintenance run --schedule=daily <<-EOF &&
+	git prune-packed --quiet
+	git multi-pack-index write --no-progress
+	git multi-pack-index expire --no-progress
+	git multi-pack-index repack --no-progress --batch-size=655
+	git commit-graph write --split --reachable --no-progress
+	EOF
+
+	test_subcommands git maintenance run --schedule=weekly <<-EOF &&
+	git prune-packed --quiet
+	git multi-pack-index write --no-progress
+	git multi-pack-index expire --no-progress
+	git multi-pack-index repack --no-progress --batch-size=655
+	git commit-graph write --split --reachable --no-progress
+	git pack-refs --all --prune
+	EOF
 
 	# Modify defaults
 	git config maintenance.commit-graph.schedule daily &&
 	git config maintenance.loose-objects.schedule hourly &&
 	git config maintenance.incremental-repack.enabled false &&
 
-	GIT_TRACE2_EVENT="$(pwd)/modified-hourly.txt" \
-		git maintenance run --schedule=hourly --quiet &&
-	GIT_TRACE2_EVENT="$(pwd)/modified-daily.txt" \
-		git maintenance run --schedule=daily --quiet &&
-
-	test_subcommand ! git commit-graph write --split --reachable \
-		--no-progress <modified-hourly.txt &&
-	test_subcommand git prune-packed --quiet <modified-hourly.txt &&
-	test_subcommand ! git multi-pack-index write --no-progress \
-		<modified-hourly.txt &&
-
-	test_subcommand git commit-graph write --split --reachable \
-		--no-progress <modified-daily.txt &&
-	test_subcommand git prune-packed --quiet <modified-daily.txt &&
-	test_subcommand ! git multi-pack-index write --no-progress \
-		<modified-daily.txt
+	test_subcommands git maintenance run --schedule=hourly <<-EOF &&
+	git prune-packed --quiet
+	EOF
+
+	test_subcommands git maintenance run --schedule=daily <<-EOF &&
+	git prune-packed --quiet
+	git commit-graph write --split --reachable --no-progress
+	EOF
+
+	test_subcommands git maintenance run --schedule=weekly <<-EOF
+	git prune-packed --quiet
+	git commit-graph write --split --reachable --no-progress
+	git pack-refs --all --prune
+	EOF
 '
 
 test_expect_success 'register and unregister' '
diff --git a/t/test-lib-functions.sh b/t/test-lib-functions.sh
index 53ffeb07f8..444dd3c6c7 100644
--- a/t/test-lib-functions.sh
+++ b/t/test-lib-functions.sh
@@ -1625,39 +1625,6 @@ test_path_is_hidden () {
 	return 1
 }
 
-# Check that the given command was invoked as part of the
-# trace2-format trace on stdin.
-#
-#	test_subcommand [!] <command> <args>... < <trace>
-#
-# For example, to look for an invocation of "git upload-pack
-# /path/to/repo"
-#
-#	GIT_TRACE2_EVENT=event.log git fetch ... &&
-#	test_subcommand git upload-pack "$PATH" <event.log
-#
-# If the first parameter passed is !, this instead checks that
-# the given command was not called.
-#
-test_subcommand () {
-	local negate=
-	if test "$1" = "!"
-	then
-		negate=t
-		shift
-	fi
-
-	local expr=$(printf '"%s",' "$@")
-	expr="${expr%,}"
-
-	if test -n "$negate"
-	then
-		! grep "\[$expr\]"
-	else
-		grep "\[$expr\]"
-	fi
-}
-
 # Run a command and ensure it succeeds. Use the
 # GIT_TRACE2_PERF logs to ensure that every subcommand
 # run by this top-level Git process is exactly the
-- 
2.31.1.vfs.0.0
Previous: Ævar Arnfjörð BjarmasonNext: Ævar Arnfjörð Bjarmason
Message 31 of 72 in “Maintenance: adapt custom refspecs”
  1. 0/5 Maintenance: adapt custom refspecsDerrick Stolee via GitGitGadget, Apr 5, 2021
  2. 2/5 test-lib: use exact match for test_subcommandDerrick Stolee via GitGitGadget, Apr 5, 2021
  3. Eric SunshineApr 5, 2021
  4. Junio C HamanoApr 5, 2021
  5. 1/5 maintenance: simplify prefetch logicDerrick Stolee via GitGitGadget, Apr 5, 2021
  6. Tom SaegerApr 5, 2021
  7. 3/5 refspec: output a refspec itemDerrick Stolee via GitGitGadget, Apr 5, 2021
  8. Tom SaegerApr 5, 2021
  9. Eric SunshineApr 5, 2021
  10. Junio C HamanoApr 5, 2021
  11. Derrick StoleeApr 6, 2021
  12. Eric SunshineApr 6, 2021
  13. Derrick StoleeApr 6, 2021
  14. Ævar Arnfjörð BjarmasonApr 7, 2021
  15. Derrick StoleeApr 7, 2021
  16. Ævar Arnfjörð BjarmasonApr 7, 2021
  17. Junio C HamanoApr 7, 2021
  18. Ævar Arnfjörð BjarmasonApr 7, 2021
  19. Junio C HamanoApr 8, 2021
  20. 4/5 test-tool: test refspec input/outputDerrick Stolee via GitGitGadget, Apr 5, 2021
  21. Eric SunshineApr 5, 2021
  22. Derrick StoleeApr 6, 2021
  23. Ævar Arnfjörð BjarmasonApr 7, 2021
  24. 5/5 maintenance: allow custom refspecs during prefetchDerrick Stolee via GitGitGadget, Apr 5, 2021
  25. Tom SaegerApr 5, 2021
  26. Derrick StoleeApr 6, 2021
  27. Ævar Arnfjörð BjarmasonApr 7, 2021
  28. Ævar Arnfjörð BjarmasonApr 7, 2021
  29. Derrick StoleeApr 9, 2021
  30. Ævar Arnfjörð BjarmasonApr 9, 2021
  31. Derrick StoleeApr 10, 2021
  32. Ævar Arnfjörð BjarmasonApr 10, 2021
  33. Ævar Arnfjörð BjarmasonApr 7, 2021
  34. 0/5 Maintenance: adapt custom refspecsDerrick Stolee via GitGitGadget, Apr 6, 2021
  35. 1/5 maintenance: simplify prefetch logicDerrick Stolee via GitGitGadget, Apr 6, 2021
  36. Emily ShafferApr 7, 2021
  37. Derrick StoleeApr 9, 2021
  38. 2/5 test-lib: use exact match for test_subcommandDerrick Stolee via GitGitGadget, Apr 6, 2021
  39. 4/5 test-tool: test refspec input/outputDerrick Stolee via GitGitGadget, Apr 6, 2021
  40. Josh SteadmonApr 7, 2021
  41. Emily ShafferApr 7, 2021
  42. 3/5 refspec: output a refspec itemDerrick Stolee via GitGitGadget, Apr 6, 2021
  43. 5/5 maintenance: allow custom refspecs during prefetchDerrick Stolee via GitGitGadget, Apr 6, 2021
  44. Tom SaegerApr 6, 2021
  45. Derrick StoleeApr 6, 2021
  46. Josh SteadmonApr 7, 2021
  47. Emily ShafferApr 7, 2021
  48. Jonathan TanApr 8, 2021
  49. 0/3 Maintenance: adapt custom refspecsDerrick Stolee via GitGitGadget, Apr 10, 2021
  50. 2/3 fetch: add --prefetch optionDerrick Stolee via GitGitGadget, Apr 10, 2021
  51. Ramsay JonesApr 11, 2021
  52. Derrick StoleeApr 12, 2021
  53. 3/3 maintenance: use 'git fetch --prefetch'Derrick Stolee via GitGitGadget, Apr 10, 2021
  54. 1/3 maintenance: simplify prefetch logicDerrick Stolee via GitGitGadget, Apr 10, 2021
  55. Tom SaegerApr 12, 2021
  56. Derrick StoleeApr 12, 2021
  57. Junio C HamanoApr 11, 2021
  58. Tom SaegerApr 12, 2021
  59. Tom SaegerApr 12, 2021
  60. Tom SaegerApr 12, 2021
  61. Derrick StoleeApr 12, 2021
  62. 0/4 Maintenance: adapt custom refspecsDerrick Stolee via GitGitGadget, Apr 16, 2021
  63. 1/4 maintenance: simplify prefetch logicDerrick Stolee via GitGitGadget, Apr 16, 2021
  64. Tom SaegerApr 16, 2021
  65. 2/4 fetch: add --prefetch optionDerrick Stolee via GitGitGadget, Apr 16, 2021
  66. Tom SaegerApr 16, 2021
  67. Tom SaegerApr 16, 2021
  68. 3/4 maintenance: use 'git fetch --prefetch'Derrick Stolee via GitGitGadget, Apr 16, 2021
  69. 4/4 maintenance: respect remote.*.skipFetchAllDerrick Stolee via GitGitGadget, Apr 16, 2021
  70. Ævar Arnfjörð BjarmasonApr 16, 2021
  71. Tom SaegerApr 16, 2021
  72. Tom SaegerApr 16, 2021

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.