git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v2] regexec: work around macOS TRE leak on invalid UTF-8

From
Patrick Steinhardt <ps@pks.im>
Date
Aug 5, 2026, 09:00 UTC
Message-ID
<anL7qL2-4h8ZlLcg@pks.im>
In-Reply-To
<20260728052538.12429-1-chungmin@chungminlee.com>
On Mon, Jul 27, 2026 at 10:25:38PM -0700, Chungmin Lee wrote:
Show 18 quoted lines
> On macOS, the system regex engine leaks an internal buffer when
> regexec() encounters an invalid multibyte sequence in a UTF-8 locale.
> The line-by-line path can call regexec_buf() for each pattern on every
> line, so "git grep" can leak repeatedly on a file containing invalid
> UTF-8.  The total leak grows with the number of calls, and the per-call
> allocation grows with the pattern's automaton.  In one case, grepping a
> repository containing PDFs exhausted memory and caused the machine to
> restart.
> 
> ce025ae4f61e (grep: disable lookahead on error, 2024-10-20) made "git
> grep" fall back to line-by-line matching when regexec() reports an error
> on invalid UTF-8.  That fallback cannot prevent this leak: the allocation
> has already leaked when regexec() returns REG_ILLSEQ.
> 
> Avoid the leaking path by providing a Darwin-specific regexec_buf().
> Walk the input with mbrtowc(), split it at bytes that cannot form a
> complete multibyte character, and search each valid segment separately.
> This preserves matches in valid text on either side of an invalid byte.

Hm. I feel like we're adding quite a lot of logic only to fix an upstream bug that we expect will be eventually fixed. At the same time we already have a compatibility "regexec" implementation that I'd expect doesn't have the bug. So would an alternative be to detect whether the given platform is susceptible to the bug and, if so, define NO_REGEX and then use our own regex implementation? Or are there good reasons to not do that?

Patrick
Previous: Junio C Hamano
Message 5 of 5 in “regexec: work around macOS TRE memory leak on invalid UTF-8”
  1. regexec: work around macOS TRE memory leak on invalid UTF-8Chungmin Lee, Jul 22, 2026
  2. Junio C HamanoJul 24, 2026
  3. regexec: work around macOS TRE leak on invalid UTF-8Chungmin Lee, Jul 28, 2026
  4. Junio C HamanoJul 29, 2026
  5. Patrick SteinhardtAug 5, 2026

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.