git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v2 0/4] utf8: bounds-related fixes and a doc fix

From
Dirk Gouders <dirk@gouders.net>
Date
Nov 18, 2026, 21:08 UTC
Message-ID
<20261118-master-v2-0-3f2aaec15d31@gouders.net>
In-Reply-To
<20261116-master-v1-0-dc2a50ce0aa0@gouders.net>

A patch here on the list made me look at display_mode_esc_sequence_len() in utf8.c and I found two details in the file that are perhaps worth a fix:

1) utf8_strnwidth() disregards the given len argument when fed
   with proper string and len.
2) The comment for utf8_strnwidth() is incorrect: it states the
   function returns strlen() for cases when it actually returns its
   len argument.
Signed-off-by: Dirk Gouders <dirk@gouders.net>
---
Changes in v2:
- Add a patch by Junio that introduces a bounded function display_mode_esc_sequence_len_counted()
  I added some tests for this new function.
- 2/2 became 2/4
- 3/4 is the modified 1/2 and uses the new
  display_mode_esc_sequence_len_counted() so that the while loop stays
  within the given bounds.
  The commit message was adjusted.
  One of the tests was wrong: "\033" has width 0, so the sum must be 3 instead of 4.
- 4/4 is an addition to fix a similar problem with utf8_width().
  That function is used without its optional bounds argument and
  can violate boundaries as well.  Test for such cases are added.
- Link to v1: https://patch.msgid.link/20261116-master-v1-0-dc2a50ce0aa0@gouders.net
---
Dirk Gouders (3):
      utf8: fix comment for utf8_strnwidth()
      utf8: respect string boundaries after skipping ANSI escape sequences
      utf8: respect boundaries in utf8_strnwidth()
Junio C Hamano (1):
      utf8: add display_mode_esc_sequence_len_counted()
 Makefile                    |  1 +
 t/meson.build               |  1 +
 t/unit-tests/u-utf8-ansi.c  | 14 +++++++++++++
 t/unit-tests/u-utf8-width.c | 20 ++++++++++++++++++
 utf8.c                      | 50 +++++++++++++++++++++++++++++++++------------
 utf8.h                      |  1 +
 6 files changed, 74 insertions(+), 13 deletions(-)
Range-diff versus v1:
-:  ---------- > 1:  3b2ab53858 utf8: add display_mode_esc_sequence_len_counted()
2:  aab04775f6 = 2:  efeff5cfd7 utf8: fix comment for utf8_strnwidth()
1:  21baf82731 ! 3:  152210dd15 utf8: respect bounds in utf8_strnwidth()
    @@ Metadata
     Author: Dirk Gouders <dirk@gouders.net>
     
      ## Commit message ##
    -    utf8: respect bounds in utf8_strnwidth()
    +    utf8: respect string boundaries after skipping ANSI escape sequences
     
    -    utf8_strnwidth() takes a len argument, but because
    -    display_mode_esc_sequence_len() works without an explicit boundary,
    -    there are cases when utf8_strnwidth() operates beyond the given len
    -    boundary.
    +    utf8_strnwidth() can skip ANSI escape sequences and it is possible
    +    that such sequences end the string within the given boundary.
     
    -    Fix this by adding a boundary-check after looping with
    -    display_mode_esc_sequence_len() to ensure we operate strictly within
    -    the given len.
    +    Check for such situations before proceeding and calling utf8_width().
     
    -    Further, add three corner case tests of which two will fail without
    -    the fix:
    -    ------------------------------------------------------------------------
    -    not ok 257 - utf8_width::strnwidth_ansi
    -        ---
    -        reason: |
    -          Expected comparison to hold: 0 == utf8_strnwidth("\033[31mhello", 4, 1)
    -          0 != 1
    -        at:
    -          file: 't/unit-tests/u-utf8-width.c'
    -          line: 11
    -          function: 'test_utf8_width__strnwidth_ansi'
    -        ...
    -    ------------------------------------------------------------------------
    -
    -    With the fix:
    -    ------------------------------------------------------------------------
    -    ok 257 - utf8_width::strnwidth_ansi
    -    ------------------------------------------------------------------------
    +    Add tests that test such corner cases.
     
         Signed-off-by: Dirk Gouders <dirk@gouders.net>
     
    @@ t/unit-tests/u-utf8-width.c
     + */
     +void test_utf8_width__strnwidth_ansi(void)
     +{
    -+	cl_assert_equal_i(0, utf8_strnwidth("\033[31mhello", 4, 1));
    ++	cl_assert_equal_i(3, utf8_strnwidth("\033[31mhello", 4, 1));
     +	cl_assert_equal_i(0, utf8_strnwidth("\033[31mhello", 5, 1));
     +	cl_assert_equal_i(1, utf8_strnwidth("\033[31mhello", 6, 1));
     +}
    @@ utf8.c: int utf8_strnwidth(const char *string, size_t len, int skip_ansi)
      		size_t skip;
      
     -		while (skip_ansi &&
    --		       (skip = display_mode_esc_sequence_len(string)) != 0)
    +-		       (skip = display_mode_esc_sequence_len_counted(string,
    +-							orig + len - string)) != 0)
     -			string += skip;
    -+		if (skip_ansi) {
    -+			while ((skip = display_mode_esc_sequence_len(string)) != 0)
    ++		if (skip_ansi ) {
    ++			while ((skip = display_mode_esc_sequence_len_counted(string,
    ++									     orig + len - string)) != 0)
     +				string += skip;
     +
     +			if (string >= orig + len)
-:  ---------- > 4:  044d188fa8 utf8: respect boundaries in utf8_strnwidth()

--- base-commit: 7fb6055e678f800320d8ab8cd8f201d9493d1ba0 change-id: 20261116-master-69b171612823

Previous: Dirk GoudersNext: Dirk Gouders
Message 9 of 26 in “utf8: two minor fixes”
  1. 0/2 utf8: two minor fixesDirk Gouders, Nov 16, 2026
  2. 1/2 utf8: respect bounds in utf8_strnwidth()Dirk Gouders, Nov 16, 2026
  3. Junio C HamanoNov 17, 2026
  4. Dirk GoudersNov 17, 2026
  5. Junio C HamanoNov 17, 2026
  6. Dirk GoudersNov 17, 2026
  7. Junio C HamanoNov 17, 2026
  8. 2/2 utf8: fix comment for utf8_strnwidth()Dirk Gouders, Nov 16, 2026
  9. 0/4 utf8: bounds-related fixes and a doc fixDirk Gouders, Nov 18, 2026
  10. 1/4 utf8: add display_mode_esc_sequence_len_counted()Dirk Gouders, Nov 18, 2026
  11. Andy KoppeNov 19, 2026
  12. Dirk GoudersNov 19, 2026
  13. Andy KoppeNov 19, 2026
  14. Andy KoppeNov 19, 2026
  15. Dirk GoudersNov 19, 2026
  16. Dirk GoudersNov 19, 2026
  17. Dirk GoudersNov 19, 2026
  18. Junio C HamanoNov 19, 2026
  19. Dirk GoudersNov 19, 2026
  20. 2/4 utf8: fix comment for utf8_strnwidth()Dirk Gouders, Nov 18, 2026
  21. Andy KoppeNov 18, 2026
  22. Dirk GoudersNov 19, 2026
  23. 3/4 utf8: respect string boundaries after skipping ANSI escape sequencesDirk Gouders, Nov 18, 2026
  24. Junio C HamanoNov 18, 2026
  25. 4/4 utf8: respect boundaries in utf8_strnwidth()Dirk Gouders, Nov 18, 2026
  26. Junio C HamanoNov 18, 2026

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.