git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v3 5/8] grep: allocate subrepos on heap

From
JTJonathan Tan <jonathantanmy@google.com>
Date
Aug 16, 2021, 21:09 UTC
Message-ID
<7d1eeac4b50abd46c99f61eaddcc72f8bb1de043.1629148153.git.jonathantanmy@google.com>
In-Reply-To
<cover.1629148153.git.jonathantanmy@google.com>

Currently, struct repository objects corresponding to submodules are allocated on the stack in grep_submodule(). This currently works because they will not be used once grep_submodule() exits, but a subsequent patch will require these structs to be accessible for longer (perhaps even in another thread). Allocate them on the heap and clear them only at the very end.

Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
---
 builtin/grep.c | 39 ++++++++++++++++++++++++++++++---------
 1 file changed, 30 insertions(+), 9 deletions(-)
diff --git a/builtin/grep.c b/builtin/grep.c
index 9e61c7c993..fa7fd08150 100644
--- a/builtin/grep.c
+++ b/builtin/grep.c
@@ -65,6 +65,9 @@ static int todo_done;
 /* Has all work items been added? */
 static int all_work_added;
 
+static struct repository **repos_to_free;
+static size_t repos_to_free_nr, repos_to_free_alloc;
+
 /* This lock protects all the variables above. */
 static pthread_mutex_t grep_mutex;
 
@@ -168,6 +171,19 @@ static void work_done(struct work_item *w)
 	grep_unlock();
 }
 
+static void free_repos(void)
+{
+	int i;
+
+	for (i = 0; i < repos_to_free_nr; i++) {
+		repo_clear(repos_to_free[i]);
+		free(repos_to_free[i]);
+	}
+	FREE_AND_NULL(repos_to_free);
+	repos_to_free_nr = 0;
+	repos_to_free_alloc = 0;
+}
+
 static void *run(void *arg)
 {
 	int hit = 0;
@@ -415,19 +431,24 @@ static int grep_submodule(struct grep_opt *opt,
 			  const struct object_id *oid,
 			  const char *filename, const char *path, int cached)
 {
-	struct repository subrepo;
+	struct repository *subrepo;
 	struct repository *superproject = opt->repo;
 	const struct submodule *sub;
 	struct grep_opt subopt;
-	int hit;
+	int hit = 0;
 
 	sub = submodule_from_path(superproject, null_oid(), path);
 
 	if (!is_submodule_active(superproject, path))
 		return 0;
 
-	if (repo_submodule_init(&subrepo, superproject, sub))
+	subrepo = xmalloc(sizeof(*subrepo));
+	if (repo_submodule_init(subrepo, superproject, sub)) {
+		free(subrepo);
 		return 0;
+	}
+	ALLOC_GROW(repos_to_free, repos_to_free_nr + 1, repos_to_free_alloc);
+	repos_to_free[repos_to_free_nr++] = subrepo;
 
 	/*
 	 * NEEDSWORK: repo_read_gitmodules() might call
@@ -438,7 +459,7 @@ static int grep_submodule(struct grep_opt *opt,
 	 * subrepo's odbs to the in-memory alternates list.
 	 */
 	obj_read_lock();
-	repo_read_gitmodules(&subrepo, 0);
+	repo_read_gitmodules(subrepo, 0);
 
 	/*
 	 * NEEDSWORK: This adds the submodule's object directory to the list of
@@ -450,11 +471,11 @@ static int grep_submodule(struct grep_opt *opt,
 	 * store is no longer global and instead is a member of the repository
 	 * object.
 	 */
-	add_submodule_odb_by_path(subrepo.objects->odb->path);
+	add_submodule_odb_by_path(subrepo->objects->odb->path);
 	obj_read_unlock();
 
 	memcpy(&subopt, opt, sizeof(subopt));
-	subopt.repo = &subrepo;
+	subopt.repo = subrepo;
 
 	if (oid) {
 		enum object_type object_type;
@@ -464,9 +485,9 @@ static int grep_submodule(struct grep_opt *opt,
 		struct strbuf base = STRBUF_INIT;
 
 		obj_read_lock();
-		object_type = oid_object_info(&subrepo, oid, NULL);
+		object_type = oid_object_info(subrepo, oid, NULL);
 		obj_read_unlock();
-		data = read_object_with_reference(&subrepo,
+		data = read_object_with_reference(subrepo,
 						  oid, tree_type,
 						  &size, NULL);
 		if (!data)
@@ -484,7 +505,6 @@ static int grep_submodule(struct grep_opt *opt,
 		hit = grep_cache(&subopt, pathspec, cached);
 	}
 
-	repo_clear(&subrepo);
 	return hit;
 }
 
@@ -1182,5 +1202,6 @@ int cmd_grep(int argc, const char **argv, const char *prefix)
 		run_pager(&opt, prefix);
 	clear_pathspec(&pathspec);
 	free_grep_patterns(&opt);
+	free_repos();
 	return !hit;
 }
-- 
2.33.0.rc1.237.g0d66db33f3-goog
Previous: Jonathan TanNext: Jonathan Tan
Message 61 of 71 in “In grep, no adding submodule ODB as alternates”
  1. 0/7 In grep, no adding submodule ODB as alternatesJonathan Tan, Aug 10, 2021
  2. 4/7 grep: read submodule entry with explicit repoJonathan Tan, Aug 10, 2021
  3. Emily ShafferAug 11, 2021
  4. Jonathan TanAug 13, 2021
  5. 3/7 grep: typesafe versions of grep_source_initJonathan Tan, Aug 10, 2021
  6. Junio C HamanoAug 10, 2021
  7. Emily ShafferAug 11, 2021
  8. Ramsay JonesAug 11, 2021
  9. Jonathan TanAug 13, 2021
  10. Matheus Tavares BernardinoAug 11, 2021
  11. Junio C HamanoAug 12, 2021
  12. Jonathan TanAug 13, 2021
  13. 2/7 grep: use submodule-ODB-as-alternate lazy-additionJonathan Tan, Aug 10, 2021
  14. Emily ShafferAug 11, 2021
  15. Jonathan TanAug 13, 2021
  16. 1/7 submodule: lazily add submodule ODBs as alternatesJonathan Tan, Aug 10, 2021
  17. Junio C HamanoAug 10, 2021
  18. Jonathan TanAug 13, 2021
  19. Emily ShafferAug 11, 2021
  20. Jonathan TanAug 13, 2021
  21. 5/7 grep: allocate subrepos on heapJonathan Tan, Aug 10, 2021
  22. Emily ShafferAug 11, 2021
  23. Jonathan TanAug 13, 2021
  24. 6/7 grep: add repository to OID grep sourcesJonathan Tan, Aug 10, 2021
  25. Emily ShafferAug 11, 2021
  26. Jonathan TanAug 13, 2021
  27. Matheus Tavares BernardinoAug 11, 2021
  28. Jonathan TanAug 13, 2021
  29. 7/7 t7814: show lack of alternate ODB-addingJonathan Tan, Aug 10, 2021
  30. Emily ShafferAug 11, 2021
  31. Matheus Tavares BernardinoAug 11, 2021
  32. Jonathan TanAug 13, 2021
  33. Emily ShafferAug 11, 2021
  34. Josh SteadmonAug 11, 2021
  35. 0/8 In grep, no adding submodule ODB as alternatesJonathan Tan, Aug 13, 2021
  36. 2/8 grep: use submodule-ODB-as-alternate lazy-additionJonathan Tan, Aug 13, 2021
  37. 1/8 submodule: lazily add submodule ODBs as alternatesJonathan Tan, Aug 13, 2021
  38. 3/8 grep: typesafe versions of grep_source_initJonathan Tan, Aug 13, 2021
  39. Matheus Tavares BernardinoAug 16, 2021
  40. 4/8 grep: read submodule entry with explicit repoJonathan Tan, Aug 13, 2021
  41. 5/8 grep: allocate subrepos on heapJonathan Tan, Aug 13, 2021
  42. Junio C HamanoAug 13, 2021
  43. Jonathan TanAug 16, 2021
  44. 6/8 grep: add repository to OID grep sourcesJonathan Tan, Aug 13, 2021
  45. Matheus Tavares BernardinoAug 16, 2021
  46. Jonathan TanAug 16, 2021
  47. 7/8 submodule-config: pass repo upon blob config readJonathan Tan, Aug 13, 2021
  48. Matheus Tavares BernardinoAug 16, 2021
  49. Matheus Tavares BernardinoAug 16, 2021
  50. Jonathan TanAug 16, 2021
  51. Matheus Tavares BernardinoAug 16, 2021
  52. Jonathan TanAug 16, 2021
  53. Jonathan TanAug 16, 2021
  54. 8/8 t7814: show lack of alternate ODB-addingJonathan Tan, Aug 13, 2021
  55. Matheus Tavares BernardinoAug 16, 2021
  56. 0/8 In grep, no adding submodule ODB as alternatesJonathan Tan, Aug 16, 2021
  57. 1/8 submodule: lazily add submodule ODBs as alternatesJonathan Tan, Aug 16, 2021
  58. 2/8 grep: use submodule-ODB-as-alternate lazy-additionJonathan Tan, Aug 16, 2021
  59. 3/8 grep: typesafe versions of grep_source_initJonathan Tan, Aug 16, 2021
  60. 4/8 grep: read submodule entry with explicit repoJonathan Tan, Aug 16, 2021
  61. 5/8 grep: allocate subrepos on heapJonathan Tan, Aug 16, 2021
  62. 6/8 grep: add repository to OID grep sourcesJonathan Tan, Aug 16, 2021
  63. Ævar Arnfjörð BjarmasonSep 27, 2021
  64. Matheus TavaresSep 27, 2021
  65. Ævar Arnfjörð BjarmasonSep 27, 2021
  66. 7/8 submodule-config: pass repo upon blob config readJonathan Tan, Aug 16, 2021
  67. 8/8 t7814: show lack of alternate ODB-addingJonathan Tan, Aug 16, 2021
  68. Matheus Tavares BernardinoAug 17, 2021
  69. Junio C HamanoSep 8, 2021
  70. Matheus Tavares BernardinoSep 8, 2021
  71. Junio C HamanoSep 8, 2021

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.