git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v7 2/3] unit tests: add TAP unit test framework

From
Josh Steadmon <steadmon@google.com>
Date
Oct 9, 2023, 17:37 UTC
Message-ID
<ZSQ6ZwoZDYtS5hmz@google.com>
In-Reply-To
<xmqq350hw6n7.fsf@gitster.g>
On 2023.08.17 17:12, Junio C Hamano wrote:
Show 30 quoted lines
> 
> What I am going to utter from here on are not complaints but purely
> meant as questions.  
> 
> Would the resulting output and maintainability of the tests change
> (improve, or worsen) if we introduce
> 
> 	static void assert_empty_strbuf(struct strbuf *buf)
> 	{
> 		check_uint(buf->len, ==, 0);
>                 check_uint(buf->alloc, ==, 0);
> 		check(buf.buf == strbuf_slopbuf);
> 		check_char(buf.buf[0], ==, '\0');
> 	}
> 
> and call it from the setup() function to ensure that
> strbuf_release(&buf) it calls after running customer test f() brings
> the buffer in a reasonably initialized state?  The t_static_init()
> test should be able to say
> 
> 	static void t_static_init(void)
> 	{
> 		struct strbuf buf = STRBUF_INIT;
> 		assert_empty_strbuf(&buf);
> 	}
> 
> if we did so, but is that a good thing or a bad thing (e.g. it may
> make it harder to figure out where the real error came from, because
> of the "line number" thing will not easily capture the caller of the
> caller, perhaps)?  

I am unsure whether or not this is an improvement. While it would certainly help readability and reduce duplication if this were production code, in test code it can often be more valuable to be verbose and explicit, so that individual broken test cases can be quickly understood without having to do a lot of cross referencing.

I'll hold off on adding any more utility functions in t-strbuf for V8, but if you or other folks feel strongly about it we can address it in V9.

Show 15 quoted lines
> > +	check_char(buf.buf[0], ==, '\0');
> > +}
> 
> > +static void t_dynamic_init(void)
> > +{
> > +	struct strbuf buf;
> > +
> > +	strbuf_init(&buf, 1024);
> > +	check_uint(buf.len, ==, 0);
> > +	check_uint(buf.alloc, >=, 1024);
> > +	check_char(buf.buf[0], ==, '\0');
> 
> Is it sensible to check buf.buf is not slopbuf at this point, or
> does it make the test TOO intimate with the current implementation
> detail?

Yes, I think this is too much of an internal detail. None of the users of strbuf ever reference it directly. Presumably for library-ish code, we should stick to testing just the user-observable parts, not the implementation.

Show 31 quoted lines
> > +	check_char(buf->buf[0], ==, ch);
> > +	check_char(buf->buf[1], ==, '\0');
> > +}
> 
> In any case, this t_addch() REQUIRES that incoming buf is empty,
> doesn't it?  I do not think it is sensible.  I would have expected
> that it would be more like
> 
> 	t_addch(struct strbuf *buf, void *data)
> 	{
> 		char ch = *(char *)data;
> 		size_t orig_alloc = buf->alloc;
> 		size_t orig_len = buf->len;
> 
> 		if (!assert_sane_strbuf(buf))
> 			return;
>                 strbuf_addch(buf, ch);
> 		if (!assert_sane_strbuf(buf))
> 			return;
> 		check_uint(buf->len, ==, orig_len + 1);
> 		check_uint(buf->alloc, >=, orig_alloc);
>                 check_char(buf->buf[buf->len - 1], ==, ch);
>                 check_char(buf->buf[buf->len], ==, '\0');
> 	}
> 
> to ensure that we can add a ch to a strbuf with any existing
> contents and get a one-byte longer contents than before, with the
> last byte of the buffer becoming 'ch' and still NUL terminated.
> 
> And we protect ourselves with a helper that checks if the given
> strbuf looks *sane*.

Yeah, in general I think this is a good improvement, but again I'm not sure if it's worth adding additional helpers. I'll try to rework this a bit in V8.

Show 21 quoted lines
> 	static int assert_sane_strbuf(struct strbuf *buf)
> 	{
>         	/* can use slopbuf only when the length is 0 */
> 		if (buf->buf == strbuf_slopbuf)
>                 	return (buf->len == 0);
> 		/* everybody else must have non-NULL buffer */
> 		if (buf->buf == NULL)
> 			return 0;
>                 /* 
> 		 * alloc must be at least 1 byte larger than len
> 		 * for the terminating NUL at the end.
> 		 */
> 		return ((buf->len + 1 <= buf->alloc) &&
> 		    	(buf->buf[buf->len] == '\0'));
> 	}
> 
> You can obviously use your check_foo() for the individual checks
> done in this function to get a more detailed diagnosis, but because
> I have confused myself enough by thinking about their polarity, I
> wrote this in barebones comparison instead.
> 
Previous: Josh SteadmonNext: Junio C Hamano
Message 24 of 67 in “unit tests: Add a project plan document”
  1. unit tests: Add a project plan documentJosh Steadmon, Jun 30, 2023
  2. Junio C HamanoJul 1, 2023
  3. Junio C HamanoJul 1, 2023
  4. unit tests: Add a project plan documentJosh Steadmon, Aug 7, 2023
  5. Phillip WoodAug 14, 2023
  6. Josh SteadmonAug 15, 2023
  7. Phillip WoodAug 17, 2023
  8. 0/3 Add unit test framework and project plan MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bitJosh Steadmon, Aug 16, 2023
  9. 3/3 ci: run unit tests in CIJosh Steadmon, Aug 16, 2023
  10. 1/3 unit tests: Add a project plan documentJosh Steadmon, Aug 16, 2023
  11. 2/3 unit tests: add TAP unit test frameworkJosh Steadmon, Aug 16, 2023
  12. Junio C HamanoAug 17, 2023
  13. Junio C HamanoAug 17, 2023
  14. Josh SteadmonAug 17, 2023
  15. 0/3 Add unit test framework and project planJosh Steadmon, Aug 17, 2023
  16. 3/3 ci: run unit tests in CIJosh Steadmon, Aug 17, 2023
  17. 1/3 unit tests: Add a project plan documentJosh Steadmon, Aug 17, 2023
  18. 2/3 unit tests: add TAP unit test frameworkJosh Steadmon, Aug 17, 2023
  19. Junio C HamanoAug 18, 2023
  20. Junio C HamanoSep 22, 2023
  21. phillip.wood123@gmail.comSep 24, 2023
  22. Junio C HamanoSep 25, 2023
  23. Josh SteadmonOct 6, 2023
  24. Josh SteadmonOct 9, 2023
  25. Junio C HamanoAug 17, 2023
  26. Josh SteadmonAug 24, 2023
  27. Junio C HamanoSep 13, 2023
  28. 0/3 Add unit test framework and project planJosh Steadmon, Oct 9, 2023
  29. 1/3 unit tests: Add a project plan documentJosh Steadmon, Oct 9, 2023
  30. Oswald BuddenhagenOct 10, 2023
  31. Josh SteadmonOct 11, 2023
  32. Oswald BuddenhagenOct 11, 2023
  33. Josh SteadmonNov 1, 2023
  34. Christian CouderOct 27, 2023
  35. Josh SteadmonNov 1, 2023
  36. Junio C HamanoNov 1, 2023
  37. 2/3 unit tests: add TAP unit test frameworkJosh Steadmon, Oct 9, 2023
  38. Junio C HamanoOct 11, 2023
  39. 5/3 fixup! unit tests: add TAP unit test frameworkPhillip Wood, Oct 16, 2023
  40. Junio C HamanoOct 16, 2023
  41. Josh SteadmonNov 1, 2023
  42. Junio C HamanoNov 1, 2023
  43. Josh SteadmonNov 1, 2023
  44. Junio C HamanoNov 1, 2023
  45. Christian CouderOct 27, 2023
  46. Josh SteadmonNov 1, 2023
  47. 3/3 ci: run unit tests in CIJosh Steadmon, Oct 9, 2023
  48. Junio C HamanoOct 9, 2023
  49. 0/3 CMake unit test fixupsPhillip Wood, Oct 19, 2023
  50. 1/3 fixup! cmake: also build unit testsPhillip Wood, Oct 19, 2023
  51. 2/3 fixup! artifacts-tar: when including `.dll` files, don't forget the unit-testsPhillip Wood, Oct 19, 2023
  52. 3/3 fixup! cmake: handle also unit testsPhillip Wood, Oct 19, 2023
  53. Junio C HamanoOct 19, 2023
  54. phillip.wood123@gmail.comOct 16, 2023
  55. Josh SteadmonNov 1, 2023
  56. Christian CouderOct 27, 2023
  57. 0/3 Add unit test framework and project planJosh Steadmon, Nov 1, 2023
  58. 1/3 unit tests: Add a project plan documentJosh Steadmon, Nov 1, 2023
  59. 3/3 ci: run unit tests in CIJosh Steadmon, Nov 1, 2023
  60. 2/3 unit tests: add TAP unit test frameworkJosh Steadmon, Nov 1, 2023
  61. Christian CouderNov 3, 2023
  62. Josh SteadmonNov 9, 2023
  63. 0/3 Add unit test framework and project planJosh Steadmon, Nov 9, 2023
  64. 1/3 unit tests: Add a project plan documentJosh Steadmon, Nov 9, 2023
  65. Junio C HamanoNov 9, 2023
  66. 2/3 unit tests: add TAP unit test frameworkJosh Steadmon, Nov 9, 2023
  67. 3/3 ci: run unit tests in CIJosh Steadmon, Nov 9, 2023

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.