Re: [PATCH v3 2/5] strvec: add a strvec_pushvec()

From: Jeff King <peff@peff.net>
Date: Sep 3, 2021, 11:06 UTC
Message-ID: <YTIBnT8Ue1HZXs82@coredump.intra.peff.net>
In-Reply-To: <87v93i8svd.fsf@evledraar.gmail.com>

On Fri, Sep 03, 2021 at 01:19:33AM +0200, Ævar Arnfjörð Bjarmason wrote:

Show 8 quoted lines

> > One thing that did surprise me: the use of "int" here for iterating,
> > rather than size_t. But it seems that strvec is already storing ints,
> > which is an accident!
> 
> Is it really? If you temporarily try to say convert that to "size_t *nr"
> to have the compiler catch all the cases where we use "nr", and then
> s/size_t/int/g those all, you'll find that e.g. setup_revisions() and
> the like expect to take either "int argc" or the strvec equivalent.

It most definitely is an accident, in the sense that the commit which changed it did not mean to. :)

Show 6 quoted lines

> We can sensibly convert some of those to size_t, but not all, and the
> int v.s. size_t inconsistency as a result feels weird.
> 
> Since the main point of this API is to be a wrapper for what a C main()
> would take, shouldn't its prototype mirror its prototype? I.e. we should
> stick to "int" here?

That isn't the main point of this API. The reason the name changed away from argv_array was so that it would be more obviously a generally useful array of strings.

But even if that were the use, the point isn't that we expect to see 2B or even 4B strings in any reasonable use case. The point is that we don't want to accidentally overflow the integer counter, leading to reads and writes of random bits of memory. And all it takes to do that is some code like:

  while (strbuf_readline(stdin, &buf))
	strvec_push(&foo, buf.buf);

On a system with 32-bit size_t, you are not likely to actually allocate 2B strings before you'd fail. But on most 64-bit systems, you have plenty of address space (and these days, often RAM) to do so, but you still only need 2B strings, because "int" is 32 bits.

So code like setup_revisions() which uses an int is fine. It is reading from a source with that much smaller "int" limit in the first place. Whether strvec can handle bigger arrays or not does not matter either way. And when it later uses ints to operate on such a strvec, it's OK in practice; even if the size_t gets truncated on the fly to an int, we know we did not put more than an int's worth of items into the array in the first place.

But code which has potentially larger inputs (either because it reads iteratively into the strvec as above, or because it is itself using a larger data type) is now also OK, because it's using size_t.

What's not OK is code which operates on a potentially-unbounded strvec using an int. E.g., following the code I showed above with something like:

  int nr = foo.nr;
  foo.v[nr] = xstrdup("replacement string");

which may write to memory outside of foo.v.

Obviously that's nonsense nobody would ever write directly. It is possible for some code to do this inadvertently (say, passing a ptr/int pair through a function interface), but I think it's fairly unlikely in practice. So while I do think more code ought to be using size_t in general, I don't think it's necessary to audit this carefully. The important thing to me is protecting strvec itself.

-Peff