git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v4 2/3] regex: add regexec_buf() that can work on a non NUL-terminated string

From
Johannes Schindelin <johannes.schindelin@gmx.de>
Date
Sep 21, 2016, 18:24 UTC
Message-ID
<270cea11c4d8bfb332a6c014a11673b7f4666ee4.1474482164.git.johannes.schindelin@gmx.de>
In-Reply-To
<cover.1474482164.git.johannes.schindelin@gmx.de>

We just introduced a test that demonstrates that our sloppy use of regexec() on a mmap()ed area can result in incorrect results or even hard crashes.

So what we need to fix this is a function that calls regexec() on a length-delimited, rather than a NUL-terminated, string.

Happily, there is an extension to regexec() introduced by the NetBSD project and present in all major regex implementation including Linux', MacOSX' and the one Git includes in compat/regex/: by using the (non-POSIX) REG_STARTEND flag, it is possible to tell the regexec() function that it should only look at the offsets between pmatch[0].rm_so and pmatch[0].rm_eo.

That is exactly what we need.

Since support for REG_STARTEND is so widespread by now, let's just introduce a helper function that uses it, and fall back to allocating and constructing a NUL-terminated when REG_STARTEND is not available.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 Makefile          |  3 ++-
 git-compat-util.h | 13 +++++++++++++
 2 files changed, 15 insertions(+), 1 deletion(-)
diff --git a/Makefile b/Makefile
index df4f86b..c6f7f66 100644
--- a/Makefile
+++ b/Makefile
@@ -301,7 +301,8 @@ all::
 # crashes due to allocation and free working on different 'heaps'.
 # It's defined automatically if USE_NED_ALLOCATOR is set.
 #
-# Define NO_REGEX if you have no or inferior regex support in your C library.
+# Define NO_REGEX if your C library lacks regex support with REG_STARTEND
+# feature.
 #
 # Define HAVE_DEV_TTY if your system can open /dev/tty to interact with the
 # user.
diff --git a/git-compat-util.h b/git-compat-util.h
index 37cce07..8aab0c3 100644
--- a/git-compat-util.h
+++ b/git-compat-util.h
@@ -977,6 +977,19 @@ void git_qsort(void *base, size_t nmemb, size_t size,
 #define qsort git_qsort
 #endif
 
+#ifndef REG_STARTEND
+#error "Git requires REG_STARTEND support. Compile with NO_REGEX=NeedsStartEnd"
+#endif
+
+static inline int regexec_buf(const regex_t *preg, const char *buf, size_t size,
+			      size_t nmatch, regmatch_t pmatch[], int eflags)
+{
+	assert(nmatch > 0 && pmatch);
+	pmatch[0].rm_so = 0;
+	pmatch[0].rm_eo = size;
+	return regexec(preg, buf, nmatch, pmatch, eflags | REG_STARTEND);
+}
+
 #ifndef DIR_HAS_BSD_GROUP_SEMANTICS
 # define FORCE_DIR_SET_GID S_ISGID
 #else
-- 
2.10.0.windows.1.10.g803177d
Previous: Johannes SchindelinNext: Junio C Hamano
Message 58 of 66 in “Fix a segfault caused by regexec() being called on mmap()ed data”
  1. 0/3 Fix a segfault caused by regexec() being called on mmap()ed dataJohannes Schindelin, Sep 5, 2016
  2. 2/3 diff_populate_filespec: NUL-terminate buffersJohannes Schindelin, Sep 5, 2016
  3. Jeff KingSep 6, 2016
  4. Johannes SchindelinSep 6, 2016
  5. Jeff KingSep 6, 2016
  6. Junio C HamanoSep 7, 2016
  7. Johannes SchindelinSep 8, 2016
  8. Johannes SchindelinSep 8, 2016
  9. Jeff KingSep 8, 2016
  10. Junio C HamanoSep 8, 2016
  11. Johannes SchindelinSep 8, 2016
  12. Jeff KingSep 8, 2016
  13. 3/3 diff_grep: add assertions verifying that the buffers are NUL-terminatedJohannes Schindelin, Sep 5, 2016
  14. Jeff KingSep 6, 2016
  15. Johannes SchindelinSep 6, 2016
  16. 1/3 Demonstrate a problem: our pickaxe code assumes NUL-terminated buffersJohannes Schindelin, Sep 5, 2016
  17. Jeff KingSep 6, 2016
  18. Johannes SchindelinSep 8, 2016
  19. Junio C HamanoSep 5, 2016
  20. Jeff KingSep 6, 2016
  21. Johannes SchindelinSep 6, 2016
  22. Jeff KingSep 6, 2016
  23. Johannes SchindelinSep 8, 2016
  24. Jeff KingSep 8, 2016
  25. Johannes SchindelinSep 9, 2016
  26. Junio C HamanoSep 9, 2016
  27. Johannes SchindelinSep 6, 2016
  28. Jeff KingSep 6, 2016
  29. Johannes SchindelinSep 6, 2016
  30. 0/3 Fix a segfault caused by regexec() being called on mmap()ed dataJohannes Schindelin, Sep 8, 2016
  31. 1/3 Demonstrate a problem: our pickaxe code assumes NUL-terminated buffersJohannes Schindelin, Sep 8, 2016
  32. 2/3 Introduce a function to run regexec() on non-NUL-terminated buffersJohannes Schindelin, Sep 8, 2016
  33. Jeff KingSep 8, 2016
  34. Johannes SchindelinSep 9, 2016
  35. Jeff KingSep 9, 2016
  36. 3/3 Use the newly-introduced regexec_buf() functionJohannes Schindelin, Sep 8, 2016
  37. Johannes SchindelinSep 8, 2016
  38. Jeff KingSep 8, 2016
  39. Jeff KingSep 8, 2016
  40. Jeff KingSep 8, 2016
  41. Ramsay JonesSep 8, 2016
  42. Jeff KingSep 8, 2016
  43. Junio C HamanoSep 8, 2016
  44. Johannes SchindelinSep 8, 2016
  45. Jeff KingSep 8, 2016
  46. 0/3 Fix a segfault caused by regexec() being called on mmap()ed dataJohannes Schindelin, Sep 8, 2016
  47. 1/3 Demonstrate a problem: our pickaxe code assumes NUL-terminated buffersJohannes Schindelin, Sep 8, 2016
  48. 2/3 Introduce a function to run regexec() on non-NUL-terminated buffersJohannes Schindelin, Sep 8, 2016
  49. Junio C HamanoSep 8, 2016
  50. 3/3 Use the newly-introduced regexec_buf() functionJohannes Schindelin, Sep 8, 2016
  51. Junio C HamanoSep 8, 2016
  52. Johannes SchindelinSep 9, 2016
  53. Jeff KingSep 9, 2016
  54. Johannes SchindelinSep 9, 2016
  55. Junio C HamanoSep 9, 2016
  56. 0/3 Fix a segfault caused by regexec() being called on mmap()ed dataJohannes Schindelin, Sep 21, 2016
  57. 1/3 regex: -G<pattern> feeds a non NUL-terminated string to regexec() and failsJohannes Schindelin, Sep 21, 2016
  58. 2/3 regex: add regexec_buf() that can work on a non NUL-terminated stringJohannes Schindelin, Sep 21, 2016
  59. Junio C HamanoSep 21, 2016
  60. Johannes SchindelinSep 22, 2016
  61. 3/3 regex: use regexec_buf()Johannes Schindelin, Sep 21, 2016
  62. Junio C HamanoSep 21, 2016
  63. Junio C HamanoSep 21, 2016
  64. Jeff KingSep 21, 2016
  65. Johannes SchindelinSep 25, 2016
  66. Jeff KingSep 21, 2016

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.