patch, 3 partsLSan quality of life improvements

I was fixing some leaks the other day and came up with a few changes that made the process a bit less painful, especially when the leaks are hidden in sub-processes (which is most of them when you are digging into http push/fetch, as I was).

I hope we're not too far off from a world where leaks are something that pop up in your newly written code, and you're not slogging through existing test scripts. But until then, I hope these might help others.

If you want to see the before/after, try:

  cd t
  ./t5550-http-fetch-dumb.sh -i

before and after this series. Before you get no leaks reported to stdout with "-i", and way too many without it. After, you get the leaks for the first test that generates them.

  [1/3]: test-lib: stop showing old leak logs
  [2/3]: test-lib: show leak-sanitizer logs on --immediate failure
  [3/3]: test-lib: check for leak logs after every test

 t/test-lib-functions.sh |  3 ++-
 t/test-lib.sh           | 41 ++++++-----------------------------------
 2 files changed, 8 insertions(+), 36 deletions(-)

-Peff

We ask LSan to record the logs of all leaks in test-results/, which is useful for finding leaks that didn't trigger a test failure.

We don't clean out the leak/ directory for each test before running it, though. Instead, we count the number of files it has, and complain only if we ended up with more when the script finishes. So we shouldn't trigger any output if you've made a script leak free. But if you simply _reduced_ the number of leaks, then there is an annoying outcome: we do not record which logs were from this run and which were from previous ones. So when we dump them to stdout, you get a mess of possibly-outdated leaks. This is very confusing when you are in an edit-compile-test cycle trying to fix leaks.

The instructions do note that you should "rm -rf test-results/" if you want to avoid this. But I'm having trouble seeing how this cumulative count could ever be useful. It is not even counting the number of leaks, but rather the number of processes with at least one leak!

So let's just blow away the per-test leak/ directory before running. We already overwrite the ".out" file in test-results/ in the same way, so this is following that pattern.

Running "make test" isn't affected by this, since it blows away all of test-results/ already. This only comes up when you are iterating on a single script that you're running manually.

Signed-off-by: Jeff King <peff@peff.net>
---
 t/test-lib.sh | 35 ++---------------------------------
 1 file changed, 2 insertions(+), 33 deletions(-)

On Tue, Sep 24, 2024 at 05:35:40PM -0400, Jeff King wrote:

I'm very much in favor of this change. I frequently re-ran tests with leak checking enabled only to realize that, oops, I forgot to "rm -rf" the leak directory first. So eventually I ended up with the following command:

    $ rm -rf /tmp/git-tests/ && make -C .. -j20 SANITIZE=leak && ./t5310-pack-bitmaps.sh -ix

Every time I didn't use it I soon came to regret it.

Patrick

When we've compiled with SANITIZE=leak, at the end of the test script we'll dump any collected logs to stdout. These logs have two uses:

  1. Leaks don't always cause a test snippet to fail (e.g., if they
     happen in a sub-process that we expect to return non-zero).
     Checking the logs catches these cases that we'd otherwise miss
     entirely.

  2. LSan will dump the leak info to stderr, but that is sometimes
     hidden (e.g., because it's redirected by the test, or because it's
     in a sub-process whose stderr goes elsewhere). Dumping the logs is
     the easiest way for the developer to see them.

One downside is that the set of logs for an entire script may be very long, especially when you're trying to fix existing test scripts. You can run with --immediate to stop at the first failing test, which means we'll have accrued fewer logs. But we don't show the logs in that case!

Let's start doing so. This can only help case (2), of course (since it depends on test failure). And it's somewhat weakened by the fact that any cases of (1) will pollute the logs. But we can improve things further in the next patch.

Signed-off-by: Jeff King <peff@peff.net>
---
 t/test-lib.sh | 1 +
 1 file changed, 1 insertion(+)

On Tue, Sep 24, 2024 at 05:36:36PM -0400, Jeff King wrote:

Yes, please!

Patrick

If you are trying to find and fix leaks in a large test script, it can be overwhelming to see the leak logs for every test at once. The previous commit let you use "--immediate" to see the logs after the first failing test, but this isn't always the first leak. As discussed there, we may see leaks from previous tests that didn't happen to fail.

To catch those, let's check for any logs that appeared after each test snippet is run, meaning that in a SANITIZE=leak build, any leak is an immediate failure of the test snippet.

This check is mostly free in non-leak builds (just a "test -z"), and only a few extra processes in a leak build, so I don't think the overhead should matter (if it does, we could probably optimize for the common "no logs" case without even spending a process).

Signed-off-by: Jeff King <peff@peff.net>
---
 t/test-lib-functions.sh |  3 ++-
 t/test-lib.sh           | 11 ++++++-----
 2 files changed, 8 insertions(+), 6 deletions(-)

On Tue, Sep 24, 2024 at 05:38:36PM -0400, Jeff King wrote:

So previously, `--immediate` didn't detect tests that should have failed because they were leaks, and now it does? Sounds like a sensible change to me, too.

Patrick

On Thu, Sep 26, 2024 at 04:19:24PM +0200, Patrick Steinhardt wrote:

Yes, though just to be pedantic, they are marked as failures even without --immediate. It is just that doing so is a lot more useful with --immediate, since otherwise we'd find the leaks at the end (but even that it may still be useful to point to a particular test).

So it really is "if you are in a SANITIZE=leak build, generating a leak log fails the test even if it would otherwise pass".

-Peff

On Tue, Sep 24, 2024 at 05:34:04PM -0400, Jeff King wrote:

Thanks, all of these look like strict improvements to me! Now I wish that I had all of these changes earlier, as that would have made my leak-fixing sessions way more enjoyable.

Patrick