Re: Covierty Integration / Improvement

From: Johannes Schindelin <johannes.schindelin@gmx.de>
Date: Apr 6, 2022, 15:08 UTC
Message-ID: <nycvar.QRO.7.76.6.2204061620110.379@tvgsbejvaqbjf.bet>
In-Reply-To: <nycvar.QRO.7.76.6.2204052352030.379@tvgsbejvaqbjf.bet>

Team,

On Wed, 6 Apr 2022, Johannes Schindelin wrote:

Show 32 quoted lines

> On Fri, 1 Apr 2022, Markus Vervier wrote:
>
> > - Could you tell us more about the amount and types of false positives
> > and problems you've faced trying to eliminate them? This will help us
> > to understand the expectations / requirements for a successful
> > integration of Coverity.
>
> From the top of my head, I would estimate about 60-70% of the results to
> be false positives.
>
> As Junio pointed out, we do not consider memory to be leaked in one-shot
> processes where memory is allocated, once, in the equivalent of a
> `main()` function. Sure, we could add a slew of `free()` calls right
> before exiting the process, but that's kind of pointless.
>
> Another major source of false positives is our string data structure,
> which offers a small-ish static, read-only buffer to get started, but
> replaces that with something `malloc()`ed/`realloc()`ed as soon as the
> string is about to be manipulated. Yet Coverity insists that we're
> writing into a read-only buffer, and get out of bounds, which is simply
> not true.
>
> Similar issues are reported with our `strvec` data structure that has
> the same allocation pattern.
>
> Since the false positives outnumber the valid issues reported by
> Coverity, we have not been very eager to sift through new reports.
>
> The list of categories of false positives listed above is not
> exhaustive, of course, but combined with how cumbersome it is to get
> access to the reports (they cannot be viewed anonymously), you get an
> idea why we do not pay all that much attention to Coverity.

I have fixed Git for Windows' Coverity build and started to sift through the 154 new defects reported as of v2.36.0-rc0.

Sadly, there is now a new class of overwhelming false positives: Coverity claims that "strbuf_addstr does not [NUL-]terminate", which is of course false. Specifically, Coverity explains that:

/strbuf.c
296 void strbuf_add(struct strbuf *sb, const void *data, size_t len)
297 {
298        strbuf_grow(sb, len);
   1. string_copy: Calling memcpy copies a source string data to sb->buf.
   2. string_null_source: The argument sb->buf will not be
      null[sic!]-terminated, because either the source string is not
      null-terminated, or the length of source string data is greater than
      or equal to the size argument len.
299        memcpy(sb->buf + sb->len, data, len);
300        strbuf_setlen(sb, sb->len + len);
301 }

In other words, it misses the fact that `strbuf_setlen()` _does_ set `sb->buf[sb->len] = '\0'` (I assume that Coverity gets confused by the `slopbuf` once again).

I stopped after the first 30-40 instances of "String not null terminated" reports because my time is a bit too expensive to spend on reports like that. Among the reported issues I looked at, there were two false positives where Coverity misinterpreted how much space was allocated (and thought we'd overrun, which we don't), the rest were those NUL-termination false positives.

Ciao, Johannes