git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v2 4/5] repack: use tempfiles for signal cleanup

From
Jeff King <peff@peff.net>
Date
Oct 23, 2022, 17:00 UTC
Message-ID
<Y1VzPY4zQyZbVAsm@coredump.intra.peff.net>
In-Reply-To
<xmqqtu3vflau.fsf@gitster.g>
On Sat, Oct 22, 2022 at 05:14:33PM -0700, Junio C Hamano wrote:
Show 22 quoted lines
> > Some options:
> >
> >   - find a different way to convince repack to die. The most likely is
> >     probably corrupting the cruft objects in some way such that
> >     pack-objects dies, but not until it starts doing real work.
> >
> >   - mark the test_must_fail with ok=sigpipe. In most case this is a
> >     band-aid, but here we still test what we want. The .tmp cleanup
> >     should kick in from a die() or from a signal death, so either is
> >     sufficient for our purposes.
> >
> >   - teach git-repack to ignore sigpipe. We don't actually check the
> >     writes to pack-objects (since they're done by fprintf), but we'd
> >     notice its failing exit code. And arguably this is improving the
> >     user experience. Saying "pack-objects died with an error" is more
> >     useful than a sigpipe.
> >
> > Thoughts?
> 
> I agree that for this particular one the second one is a reasonable
> thing to do in the context of the test.  The third one may actually
> be a better fix, exactly for the reason you state here.

Here's a patch on top of of jk/repack-tempfile-cleanup that adjusts the test (and should make the immediate racy CI pain go away). It gives some explanation why the third option isn't as interesting as you'd think. If somebody later wants to add such a "pack-objects died" error, we can adjust sigpipe handling there.

-- >8 --
Subject: [PATCH] t7700: annotate cruft-pack failure with ok=sigpipe

One of our tests intentionally causes the cruft-pack generation phase of repack to fail, in order to stimulate an exit from repack at the desired moment. It does so by feeding a bogus option argument to pack-objects. This is a simple and reliable way to get pack-objects to fail, but it has one downside: pack-objects will die before reading its stdin, which means the caller repack may racily get SIGPIPE writing to it.

For the purposes of this test, that's OK. We are checking whether repack cleans up already-created .tmp files, and it will do so whether it exits or dies by signal (because the tempfile API hooks both).

But we have to tell test_must_fail that either outcome is OK, or it complains about the signal. Arguably this is a workaround (compared to fixing repack), as repack dying to SIGPIPE means that it loses the opportunity to give a more detailed message. But we don't actually write such a message anyway; we rely on pack-objects to have written something useful to stderr, and it does. In either case (signal or exit), that is the main thing the user will see.

Signed-off-by: Jeff King <peff@peff.net>
---
 t/t7700-repack.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/t/t7700-repack.sh b/t/t7700-repack.sh
index edcda849b9..9164acbe02 100755
--- a/t/t7700-repack.sh
+++ b/t/t7700-repack.sh
@@ -433,7 +433,7 @@ test_expect_success TTY '--quiet disables progress' '
 '
 
 test_expect_success 'clean up .tmp-* packs on error' '
-	test_must_fail git \
+	test_must_fail ok=sigpipe git \
 		-c repack.cruftwindow=bogus \
 		repack -ad --cruft &&
 	find $objdir/pack -name '.tmp-*' >tmpfiles &&
-- 
2.38.1.500.ge50734af7a
Previous: Junio C HamanoNext: Junio C Hamano
Message 30 of 34 in “repack tempfile-cleanup signal deadlock”
  1. 0/4 repack tempfile-cleanup signal deadlockJeff King, Oct 21, 2022
  2. 1/4 repack: convert "names" util bitfield to arrayJeff King, Oct 21, 2022
  3. Junio C HamanoOct 21, 2022
  4. Taylor BlauOct 21, 2022
  5. Jeff KingOct 21, 2022
  6. Junio C HamanoOct 21, 2022
  7. Jeff KingOct 21, 2022
  8. Junio C HamanoOct 21, 2022
  9. Jeff KingOct 22, 2022
  10. 2/4 repack: populate extension bits incrementallyJeff King, Oct 21, 2022
  11. Taylor BlauOct 21, 2022
  12. Jeff KingOct 21, 2022
  13. Taylor BlauOct 21, 2022
  14. Jeff KingOct 21, 2022
  15. 3/4 repack: use tempfiles for signal cleanupJeff King, Oct 21, 2022
  16. Junio C HamanoOct 21, 2022
  17. Jeff KingOct 21, 2022
  18. Taylor BlauOct 21, 2022
  19. Jeff KingOct 22, 2022
  20. Jeff KingOct 22, 2022
  21. 4/4 repack: drop remove_temporary_files()Jeff King, Oct 21, 2022
  22. Taylor BlauOct 21, 2022
  23. 0/5 repack tempfile-cleanup signal deadlockJeff King, Oct 22, 2022
  24. 1/5 repack: convert "names" util bitfield to arrayJeff King, Oct 22, 2022
  25. 2/5 repack: populate extension bits incrementallyJeff King, Oct 22, 2022
  26. 3/5 repack: expand error message for missing pack filesJeff King, Oct 22, 2022
  27. 4/5 repack: use tempfiles for signal cleanupJeff King, Oct 22, 2022
  28. Jeff KingOct 22, 2022
  29. Junio C HamanoOct 23, 2022
  30. Jeff KingOct 23, 2022
  31. Junio C HamanoOct 23, 2022
  32. Jeff KingOct 23, 2022
  33. Junio C HamanoOct 23, 2022
  34. 5/5 repack: drop remove_temporary_files()Jeff King, Oct 22, 2022

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.