clang static analyzer

There have been several attempts at running the clang static analyzer on the git source code, some even resulted in patches. I tried it, too, and among the many false positives I think clang found a few real issues. The results can be seen at [1].

Clang again found many dead assignments/increments, but in the earlier discussions you concluded that you want to keep those around. So I focussed on another class of potential bugs: Argument with 'nonnull' attribute passed null. There were a total of seven such issues. I then tried to look through the code and see if they are valid or false positives:

xdiff-interface.c:xdiff_set_find_func() - When 'value' is a string with no newline character in it, the loop at line 291 sets 'value' to NULL on its first iteration and then passes 'value' to strchr() in the second iteration.

utf8.c:utf8_strwidth() - 'string' may be set to NULL in utf8_width() which makes this one a false positive.

pretty.c:get_header() - if 'line' doesn't contain a newline character, line is set to NULL on first iteration and then passed to strchr() in the second itration.

attr.c:prepare_attr_stack() - bootstrap_attr_stack() sets attr_stack so this one is a false positive as well.

test-parse-options.c:length_callback() - if arg == NULL and unset == 0 then the function passes NULL to strlen().

builtin-pack-objects.c:check_pbase_path() - false positive, if done_pbase_paths == NULL then also done_pbase_paths_alloc == 0 and so step 4 can't take the false branch.

builtin-ls-files.c:verify_pathspec() - false positive, pathspec is not NULL when the function is called.

- Some of the issues might be purely hypothetical, for example I don't know if it's possible that get_header() can be passed a string with no newlines, maybe this is prevented earlier in the code path.
- Some of the false positives (such as the last one) could be avoided by giving clang a hint that a certain variable can't be NULL (by using assert() or if (!foo) return).

tom

[1] http://78.46.209.101/stuff/clang-static-analyzer/git/v1.6.6-rc1-32-g97f3d79/

On Sun, Dec 06, 2009 at 07:11:24AM +0100, Tomas Carnecky wrote:

Thanks, I think you are moving in the right direction to manually investigate the output of clang, since it obviously does generate some false positives.

I think the next step for each site you found would be:

  1. If it really is a problem, then it should be easy to show a simple
     case that can trigger the issue. Submit a patch fixing that site,
     either describing the test case in the commit message, or adding a
     case to the test suite.

  2. If it is a false positive, see what it would take to silence clang
     and submit a patch.  I don't think we are opposed to annotations
     that help analysis tools as long as those annotations aren't too
     intrusive or make the code less readable.

-Peff

On Sun, 6 Dec 2009, Jeff King wrote:

I'm a bit skeptical here. Going down that route might mean that we'll eventually have to add all sort of crap to accommodate everyone's preferred static analysis tool of the day. Would be far nicer to try to make those tools more intelligent instead, or at least make them understand an out-of-line annotation format that does not clutter the code itself.

Nicolas

On Sun, Dec 06, 2009 at 10:39:56AM -0500, Nicolas Pitre wrote:

To be clear, I am a bit skeptical, too. I would really prefer an out-of-line annotation if one is available. But I am trying to encourage the OP to actually make a patch for one instance so we can see just what it would look like. Then we actually have a data point to discuss.

-Peff

The 06/12/09, Jeff King wrote:

Me too. I have no idea about how clang works but if there are enough work done to support such a tool in the code itself, it would be sad to not share and promote this work. If Junio cares enough himself, he could set up a public dedicated branch. Now if he doesn't, it's not necessary at all. Tomas or anyone else with enough time and motivation can fork the repository for this purpose.

The latter option would be good and appreciated by everybody here, I think.

Tomas Carnecky <tom@dbservice.com> writes:

Thanks.

In practice, we will always have a newline, as we are reading from a valid commit object in this codepath.

 pretty.c |    5 +++--
 1 files changed, 3 insertions(+), 2 deletions(-)

Tomas Carnecky <tom@dbservice.com> writes:

Thanks, but I am confused with your analysis.

If value doesn't have '\n', then regs->nr is 1 when you go into the loop at ll. 291-, because we counted the number of LF in the first loop in the function.

The first iteration of the second loop sets ep to NULL, expression is set to value, then we run regcomp on the expression. Then at the end of the loop we do set value to a bogus "(char*)1". But incrementing i makes it go over regs->nr and satisfy the termination condition of the loop; we happily exit the loop before we use the now bogus "value".