git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 1/3] grep: make PCRE1 aware of custom allocator

From
Carlo Marcelo Arenas Belón via GitGitGadget <gitgitgadget@gmail.com>
Date
Oct 16, 2019, 12:06 UTC
Message-ID
<4feb8cc83a6d0b73bd7e47a513b91ce193c6ed6f.1571227613.git.gitgitgadget@gmail.com>
In-Reply-To
<pull.402.git.1571227613.gitgitgadget@gmail.com>
From: =?UTF-8?q?Carlo=20Marcelo=20Arenas=20Bel=C3=B3n?= <carenas@gmail.com>

63e7e9d8b6 ("git-grep: Learn PCRE", 2011-05-09) didn't include a way to override the system alocator, and so it is incompatible with USE_NED_ALLOCATOR as reported by Dscho[1] (in similar code from PCRE2)

Note that nedmalloc, as well as other custom allocators like jemalloc and mi-malloc, can be configured at runtime (via `LD_PRELOAD`), therefore we cannot know at compile time whether a custom allocator is used or not.

Make the minimum change possible to ensure this combination is supported by extending `grep_init()` to set the PCRE1 specific functions to Git's idea of `malloc()` and `free()` and therefore making sure all allocations are done inside PCRE1 with the same allocator than the rest of Git.

This change has negligible performance impact: PCRE needs to allocate memory once per program run for the character table and for each pattern compilation. These are both rare events compared to matching patterns against lines. Actual measurements[2] show that the impact is lost in the noise.

[1] https://public-inbox.org/git/pull.306.git.gitgitgadget@gmail.com [2] https://public-inbox.org/git/7f42007f-911b-c570-17f6-1c6af0429586@web.de

Signed-off-by: Carlo Marcelo Arenas Belón <carenas@gmail.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 grep.c | 8 ++++++++
 1 file changed, 8 insertions(+)
diff --git a/grep.c b/grep.c
index cd952ef5d3..db6e0d895f 100644
--- a/grep.c
+++ b/grep.c
@@ -150,12 +150,20 @@ int grep_config(const char *var, const char *value, void *cb)
  * Initialize one instance of grep_opt and copy the
  * default values from the template we read the configuration
  * information in an earlier call to git_config(grep_config).
+ *
+ * If using PCRE, make sure that the library is configured
+ * to use the same allocator as Git (e.g. nedmalloc on Windows).
  */
 void grep_init(struct grep_opt *opt, struct repository *repo, const char *prefix)
 {
 	struct grep_opt *def = &grep_defaults;
 	int i;
 
+#ifdef USE_LIBPCRE1
+	pcre_malloc = malloc;
+	pcre_free = free;
+#endif
+
 	memset(opt, 0, sizeof(*opt));
 	opt->repo = repo;
 	opt->prefix = prefix;
-- 
gitgitgadget
Previous: Carlo Marcelo Arenas Belón via GitGitGadgetNext: Johannes Schindelin via GitGitGadget
Message 3 of 42 in “Revive 'pcre2-chartables-leakfix'”
  1. 0/3 Revive 'pcre2-chartables-leakfix'Johannes Schindelin via GitGitGadget, Oct 16, 2019
  2. 2/3 grep: make PCRE2 aware of custom allocatorCarlo Marcelo Arenas Belón via GitGitGadget, Oct 16, 2019
  3. 1/3 grep: make PCRE1 aware of custom allocatorCarlo Marcelo Arenas Belón via GitGitGadget, Oct 16, 2019
  4. 3/3 grep: avoid leak of chartables in PCRE2Johannes Schindelin via GitGitGadget, Oct 16, 2019
  5. 0/3 Revive 'pcre2-chartables-leakfix'Johannes Schindelin via GitGitGadget, Oct 16, 2019
  6. 1/3 grep: make PCRE1 aware of custom allocatorCarlo Marcelo Arenas Belón via GitGitGadget, Oct 16, 2019
  7. 2/3 grep: make PCRE2 aware of custom allocatorCarlo Marcelo Arenas Belón via GitGitGadget, Oct 16, 2019
  8. Junio C HamanoOct 18, 2019
  9. 00/10 grep/pcre2: memory allocation fixesÆvar Arnfjörð Bjarmason, Feb 4, 2021
  10. Junio C HamanoFeb 10, 2021
  11. 00/10 grep/pcre2: memory allocation fixesÆvar Arnfjörð Bjarmason, Feb 18, 2021
  12. Junio C HamanoMar 4, 2021
  13. 03/10 grep/pcre2: correct reference to grep_init() in commentÆvar Arnfjörð Bjarmason, Feb 18, 2021
  14. 04/10 grep/pcre2: prepare to add debugging to pcre2_malloc()Ævar Arnfjörð Bjarmason, Feb 18, 2021
  15. 02/10 grep/pcre2: drop needless assignment to NULLÆvar Arnfjörð Bjarmason, Feb 18, 2021
  16. 01/10 grep/pcre2: drop needless assignment + assert() on opt->pcre2Ævar Arnfjörð Bjarmason, Feb 18, 2021
  17. 05/10 grep/pcre2: add GREP_PCRE2_DEBUG_MALLOC debug modeÆvar Arnfjörð Bjarmason, Feb 18, 2021
  18. 06/10 grep/pcre2: use compile-time PCREv2 version testÆvar Arnfjörð Bjarmason, Feb 18, 2021
  19. Junio C HamanoMar 4, 2021
  20. 07/10 grep/pcre2: use pcre2_maketables_free() functionÆvar Arnfjörð Bjarmason, Feb 18, 2021
  21. 08/10 grep/pcre2: actually make pcre2 use custom allocatorÆvar Arnfjörð Bjarmason, Feb 18, 2021
  22. Junio C HamanoMar 4, 2021
  23. 10/10 grep/pcre2: move definitions of pcre2_{malloc,free}Ævar Arnfjörð Bjarmason, Feb 18, 2021
  24. Junio C HamanoMar 4, 2021
  25. 09/10 grep/pcre2: move back to thread-only PCREv2 structuresÆvar Arnfjörð Bjarmason, Feb 18, 2021
  26. Junio C HamanoMar 4, 2021
  27. 01/10 grep/pcre2: drop needless assignment + assert() on opt->pcre2Ævar Arnfjörð Bjarmason, Feb 4, 2021
  28. 02/10 grep/pcre2: drop needless assignment to NULLÆvar Arnfjörð Bjarmason, Feb 4, 2021
  29. 03/10 grep/pcre2: correct reference to grep_init() in commentÆvar Arnfjörð Bjarmason, Feb 4, 2021
  30. 04/10 grep/pcre2: prepare to add debugging to pcre2_malloc()Ævar Arnfjörð Bjarmason, Feb 4, 2021
  31. 05/10 grep/pcre2: add GREP_PCRE2_DEBUG_MALLOC debug modeÆvar Arnfjörð Bjarmason, Feb 4, 2021
  32. Johannes SchindelinFeb 10, 2021
  33. 06/10 grep/pcre2: use compile-time PCREv2 version testÆvar Arnfjörð Bjarmason, Feb 4, 2021
  34. 08/10 grep/pcre2: actually make pcre2 use custom allocatorÆvar Arnfjörð Bjarmason, Feb 4, 2021
  35. Johannes SchindelinFeb 10, 2021
  36. 07/10 grep/pcre2: use pcre2_maketables_free() functionÆvar Arnfjörð Bjarmason, Feb 4, 2021
  37. Johannes SchindelinFeb 10, 2021
  38. Junio C HamanoMar 4, 2021
  39. 09/10 grep/pcre2: move back to thread-only PCREv2 structuresÆvar Arnfjörð Bjarmason, Feb 4, 2021
  40. 10/10 grep/pcre2: move definitions of pcre2_{malloc,free}Ævar Arnfjörð Bjarmason, Feb 4, 2021
  41. Johannes SchindelinFeb 10, 2021
  42. 3/3 grep: avoid leak of chartables in PCRE2Carlo Marcelo Arenas Belón via GitGitGadget, Oct 16, 2019

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.