git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v2 6/9] list-objects-filter-options: make filter_spec a strbuf

From
Matthew DeVore <matvore@comcast.net>
Date
Jun 11, 2019, 00:34 UTC
Message-ID
<20190611003456.GB10396@comcast.net>
In-Reply-To
<xmqqimtdmc59.fsf@gitster-ct.c.googlers.com>
On Mon, Jun 10, 2019 at 01:13:54PM -0700, Junio C Hamano wrote:
Show 10 quoted lines
> Matthew DeVore <matvore@google.com> writes:
> 
> > -	filter_options->filter_spec = strdup(core_partial_clone_filter_default);
> > +	if (!filter_options->filter_spec.buf)
> > +		strbuf_init(&filter_options->filter_spec, 0);
> 
> This part made me go "Huh?" a bit.
> 
> Do we document that .buf==NULL means an uninitialized strbuf that is
> safe to run strbuf_init() on?  I do not mind that as a general
Kind of. The first bullet point in strbuf.h says:
 *  - The `buf` member is never NULL, so it can be used in any usual C
 *    string operations safely. strbuf's _have_ to be initialized either by
 *    `strbuf_init()` or by `= STRBUF_INIT` before the invariants, though.

So I extrapolated that if buf is NULL, it must be because it was just xcalloc'd and not initialized. One possible improvement to the API would be to refactor it such that there is no STRBUF_INIT, but a zero-initialized strbuf is valid. If you expect to get a non-NULL buf, even for a zero-initialized strbuf, you should call a function like strbuf_nonnull_buf(&buf), and that will return the slop buf if buf is null, or the actual buf if it is non-null.

I don't understand why the API designer was so strict about requiring the buffer to be set to non-null, since it's quite a burden for API users. If I eagerly set all filter_options's strbuf's to STRBUF_INIT, it involves changing a couple of global variables which currently do not need an initializer, and it would make the code a bit messy. The structs which have a strbuf somewhere in their nested fields would need to know that, and set up an initialization macro to avoid the null buf.

I kind of suspect the right short-term fix is to avoid strbuf's and use a string_list, which I join later to a full string when needed.

> convention, and it may even be a useful one (i.e. it allows you to
> calloc() a structure with an embedded strbuf in it and the "if
> .buf==NULL, call strbuf_init() lazily" can become an established
> pattern), but at the same time it feels a bit brittle.  

Is it brittle because a strbuf may be initialized to non-zero memory, and so the "if (buf.buf == NULL)" may evaluate to false, and then go on treating garbage like a valid buffer? I would think that's almost impossible because of the use of xcalloc.

The only reason I realized the strbuf_init was necessary was not because I read the documentation, but because I mistakenly called strbuf_reset, which calls strbuf_setlen, which doesn't handle a null buf. Many other functions seem to handle it well semi-accidentially. After I ran into the crash, I finally read the documentation I cited above.

Show 7 quoted lines
> 
> Such a convention forces everybody who might want to use such an
> embedded strbuf to first check .buf==NULL and lazily initialize
> it---and at some point when the embedded strbuf to be used by enough
> codepaths, it would make the code more robust by giving up on the
> lazy initialization (iow, when *filter_options is initialized, run
> strbuf_init() on its .filter_spec field).
Previous: Junio C HamanoNext: Junio C Hamano
Message 12 of 74 in “Filter combination”
  1. 0/9 Filter combinationMatthew DeVore, Jun 1, 2019
  2. 1/9 list-objects-filter: make API easier to useMatthew DeVore, Jun 1, 2019
  3. 2/9 list-objects-filter: put omits set in filter structMatthew DeVore, Jun 1, 2019
  4. 3/9 list-objects-filter-options: always supply *errbufMatthew DeVore, Jun 1, 2019
  5. 4/9 list-objects-filter: implement composite filtersMatthew DeVore, Jun 1, 2019
  6. Jeff HostetlerJun 3, 2019
  7. Matthew DeVoreJun 6, 2019
  8. Jeff HostetlerJun 7, 2019
  9. 5/9 list-objects-filter-options: move error check upMatthew DeVore, Jun 1, 2019
  10. 6/9 list-objects-filter-options: make filter_spec a strbufMatthew DeVore, Jun 1, 2019
  11. Junio C HamanoJun 10, 2019
  12. Matthew DeVoreJun 11, 2019
  13. Junio C HamanoJun 11, 2019
  14. Matthew DeVoreJun 11, 2019
  15. Matthew DeVoreJun 11, 2019
  16. Junio C HamanoJun 11, 2019
  17. Matthew DeVoreJun 12, 2019
  18. Matthew DeVoreJun 12, 2019
  19. 7/9 list-objects-filter-options: allow mult. --filterMatthew DeVore, Jun 1, 2019
  20. 8/9 list-objects-filter-options: clean up use of ALLOC_GROWMatthew DeVore, Jun 1, 2019
  21. Jacob KellerJun 3, 2019
  22. Matthew DeVoreJun 3, 2019
  23. Jacob KellerJun 4, 2019
  24. 9/9 list-objects-filter-options: make parser voidMatthew DeVore, Jun 1, 2019
  25. Jeff HostetlerJun 3, 2019
  26. 00/10 Filter combinationMatthew DeVore, Jun 13, 2019
  27. 01/10 list-objects-filter: make API easier to useMatthew DeVore, Jun 13, 2019
  28. 02/10 list-objects-filter: put omits set in filter structMatthew DeVore, Jun 13, 2019
  29. 03/10 list-objects-filter-options: always supply *errbufMatthew DeVore, Jun 13, 2019
  30. 04/10 list-objects-filter: implement composite filtersMatthew DeVore, Jun 13, 2019
  31. 05/10 list-objects-filter-options: move error check upMatthew DeVore, Jun 13, 2019
  32. 06/10 list-objects-filter-options: make filter_spec a string_listMatthew DeVore, Jun 13, 2019
  33. 07/10 strbuf: give URL-encoding API a char predicate fnMatthew DeVore, Jun 13, 2019
  34. 08/10 list-objects-filter-options: allow mult. --filterMatthew DeVore, Jun 13, 2019
  35. 09/10 list-objects-filter-options: clean up use of ALLOC_GROWMatthew DeVore, Jun 13, 2019
  36. 10/10 list-objects-filter-options: make parser voidMatthew DeVore, Jun 13, 2019
  37. Junio C HamanoJun 14, 2019
  38. 00/10 Filter combinationMatthew DeVore, Jun 15, 2019
  39. 01/10 list-objects-filter: make API easier to useMatthew DeVore, Jun 15, 2019
  40. Jonathan TanJun 21, 2019
  41. Matthew DeVoreJun 27, 2019
  42. 02/10 list-objects-filter: put omits set in filter structMatthew DeVore, Jun 15, 2019
  43. 03/10 list-objects-filter-options: always supply *errbufMatthew DeVore, Jun 15, 2019
  44. 04/10 list-objects-filter: implement composite filtersMatthew DeVore, Jun 15, 2019
  45. Johannes SchindelinJun 18, 2019
  46. Matthew DeVoreJun 18, 2019
  47. Johannes SchindelinJun 21, 2019
  48. Jonathan TanJun 22, 2019
  49. Matthew DeVoreJun 27, 2019
  50. 05/10 list-objects-filter-options: move error check upMatthew DeVore, Jun 15, 2019
  51. 06/10 list-objects-filter-options: make filter_spec a string_listMatthew DeVore, Jun 15, 2019
  52. Jonathan TanJun 22, 2019
  53. Matthew DeVoreJun 27, 2019
  54. 07/10 strbuf: give URL-encoding API a char predicate fnMatthew DeVore, Jun 15, 2019
  55. 08/10 list-objects-filter-options: allow mult. --filterMatthew DeVore, Jun 15, 2019
  56. 09/10 list-objects-filter-options: clean up use of ALLOC_GROWMatthew DeVore, Jun 15, 2019
  57. 10/10 list-objects-filter-options: make parser voidMatthew DeVore, Jun 15, 2019
  58. Jonathan TanJun 22, 2019
  59. Matthew DeVoreJun 27, 2019
  60. Matthew DeVoreJun 27, 2019
  61. Junio C HamanoJun 18, 2019
  62. 00/10 Filter combinationMatthew DeVore, Jun 27, 2019
  63. 01/10 list-objects-filter: encapsulate filter componentsMatthew DeVore, Jun 27, 2019
  64. 03/10 list-objects-filter-options: always supply *errbufMatthew DeVore, Jun 27, 2019
  65. 02/10 list-objects-filter: put omits set in filter structMatthew DeVore, Jun 27, 2019
  66. 04/10 list-objects-filter: implement composite filtersMatthew DeVore, Jun 27, 2019
  67. 05/10 list-objects-filter-options: move error check upMatthew DeVore, Jun 27, 2019
  68. 06/10 list-objects-filter-options: make filter_spec a string_listMatthew DeVore, Jun 27, 2019
  69. 07/10 strbuf: give URL-encoding API a char predicate fnMatthew DeVore, Jun 27, 2019
  70. 08/10 list-objects-filter-options: allow mult. --filterMatthew DeVore, Jun 27, 2019
  71. 09/10 list-objects-filter-options: clean up use of ALLOC_GROWMatthew DeVore, Jun 27, 2019
  72. 10/10 list-objects-filter-options: make parser voidMatthew DeVore, Jun 27, 2019
  73. Junio C HamanoJun 28, 2019
  74. Jonathan TanJun 28, 2019

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.