threads / patch / 62112

patcht9001: use a more distinct fake BugID

Subject: [PATCH] t9001: use a more distinct fake BugID

## tl;dr

2 messages between Sep 15, 2024 and Sep 16, 2024. Diffs are folded; open one to read it.

replies: 1people: 2as markdown or json

Jeff King· Sep 15, 2024, 11:31 UTC · lore

In the test "cc list is sanitized", we feed a commit with a variety of trailers to send-email, and then check its output to see how it handled them. For most of them, we are grepping for a specific mention of the header, but there's a "BugID" header which we expect to be ignored. We confirm this by grepping for "12345", the fake BugID, and making sure it is not present.

But we can be fooled by false positives! I just tracked down a flaky test failure here that was caused by matching this unrelated line in the output:

  <20240914090449.612345-1-author@example.com>
which will change from run to run based on the time, pid, etc.

Ideally we'd tighten the regex to make this more specifically, but since the point is that it _shouldn't_ be mentioned, it's hard to say what the right match would be (e.g., would there be a leading space?).

Instead, let's just choose a match that is much less likely to appear. The actual content of the header isn't important, since it's supposed to be ignored.

Signed-off-by: Jeff King <peff@peff.net>
---
I guess this is fairly unlikely, as re-running the test with --stress
didn't reproduce after a few hundred attempts. Back of the envelope, I
guess any 5-digit sequence has a 1-in-10^5 chance of matching our
target. There are 2 in chances in a 6-digit pid. Some in the date, but
as there's no December 34th or hour 34, you're limited to a few specific
times like 01:23:45 (or at 11am), 12:34:5x, or perhaps 23:45 on the 1st,
11th, or 21st of the month.

So I think I just got really unlucky, but after spending several minutes debugging, I wouldn't want to wish it on anybody else. ;)

 t/t9001-send-email.sh | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
Show changes to t/t9001-send-email.sh +2 −2
diff --git a/t/t9001-send-email.sh b/t/t9001-send-email.sh
index df5336bb7e..e2430f7bfa 100755
--- a/t/t9001-send-email.sh
+++ b/t/t9001-send-email.sh
@@ -1324,7 +1324,7 @@ test_expect_success $PREREQ 'cc list is sanitized' '
 	Reviewed-by: Füñný Nâmé <odd_?=mail@example.com>
 	Reported-by: bugger on Jira
 	Reported-by: Douglas Reporter <doug@example.com> [from Jira profile]
-	BugID: 12345
+	BugID: 12345should-not-appear
 	Co-developed-by: "C. O. Developer" <codev@example.com>
 	Signed-off-by: A. U. Thor <thor.au@example.com>
 	EOF
@@ -1337,7 +1337,7 @@ test_expect_success $PREREQ 'cc list is sanitized' '
 " <odd_?=mail@example.com>" actual-show-all-headers &&
 	test_grep "^(body) Ignoring Reported-by .* bugger on Jira" actual-show-all-headers &&
 	test_grep "^(body) Adding cc: Douglas Reporter <doug@example.com>" actual-show-all-headers &&
-	test_grep ! "12345" actual-show-all-headers &&
+	test_grep ! "12345should-not-appear" actual-show-all-headers &&
 	test_grep "^(body) Adding cc: \"C. O. Developer\" <codev@example.com>" actual-show-all-headers &&
 	test_grep "^(body) Adding cc: \"A. U. Thor\" <thor.au@example.com>" actual-show-all-headers
 '
-- 
2.46.1.893.gc4b01a7614
Patrick Steinhardt· Sep 16, 2024, 07:27 UTC · re: Jeff King · lore

Re: [PATCH] t9001: use a more distinct fake BugID

On Sun, Sep 15, 2024 at 07:31:15AM -0400, Jeff King wrote:
Show 35 quoted lines
> In the test "cc list is sanitized", we feed a commit with a variety of
> trailers to send-email, and then check its output to see how it handled
> them. For most of them, we are grepping for a specific mention of the
> header, but there's a "BugID" header which we expect to be ignored. We
> confirm this by grepping for "12345", the fake BugID, and making sure it
> is not present.
> 
> But we can be fooled by false positives! I just tracked down a flaky
> test failure here that was caused by matching this unrelated line in the
> output:
> 
>   <20240914090449.612345-1-author@example.com>
> 
> which will change from run to run based on the time, pid, etc.
> 
> Ideally we'd tighten the regex to make this more specifically, but since
> the point is that it _shouldn't_ be mentioned, it's hard to say what the
> right match would be (e.g., would there be a leading space?).
> 
> Instead, let's just choose a match that is much less likely to appear.
> The actual content of the header isn't important, since it's supposed to
> be ignored.
> 
> Signed-off-by: Jeff King <peff@peff.net>
> ---
> I guess this is fairly unlikely, as re-running the test with --stress
> didn't reproduce after a few hundred attempts. Back of the envelope, I
> guess any 5-digit sequence has a 1-in-10^5 chance of matching our
> target. There are 2 in chances in a 6-digit pid. Some in the date, but
> as there's no December 34th or hour 34, you're limited to a few specific
> times like 01:23:45 (or at 11am), 12:34:5x, or perhaps 23:45 on the 1st,
> 11th, or 21st of the month.
> 
> So I think I just got really unlucky, but after spending several minutes
> debugging, I wouldn't want to wish it on anybody else. ;)
Heh, unlucky indeed. In any case, the change makes sense to me, thanks!
Patrick

← back to recent threads