git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v2 2/8] builtin/checkout.c: complete parallel checkout support

From
Matheus Tavares <matheus.bernardino@usp.br>
Date
Apr 30, 2021, 21:40 UTC
Message-ID
<e2d82c4337dc002d7296454c59d4006c3bb43c27.1619818517.git.matheus.bernardino@usp.br>
In-Reply-To
<cover.1619818517.git.matheus.bernardino@usp.br>

Pathspec-limited checkouts (like `git checkout *.txt`) are performed by a code path that doesn't yet support parallel checkout because it calls checkout_entry() directly, instead of unpack_trees(). Let's add parallel checkout support for this code path too.

Note: the transient cache entries allocated in checkout_merged() are now
allocated in a mem_pool which is only discarded after parallel checkout
finishes. This is done because the entries need to be valid when
run_parallel_checkout() is called.
Signed-off-by: Matheus Tavares <matheus.bernardino@usp.br>
---
 builtin/checkout.c | 20 ++++++++++++++++----
 1 file changed, 16 insertions(+), 4 deletions(-)
diff --git a/builtin/checkout.c b/builtin/checkout.c
index db667d0267..b71dc08430 100644
--- a/builtin/checkout.c
+++ b/builtin/checkout.c
@@ -27,6 +27,7 @@
 #include "wt-status.h"
 #include "xdiff-interface.h"
 #include "entry.h"
+#include "parallel-checkout.h"
 
 static const char * const checkout_usage[] = {
 	N_("git checkout [<options>] <branch>"),
@@ -230,7 +231,8 @@ static int checkout_stage(int stage, const struct cache_entry *ce, int pos,
 		return error(_("path '%s' does not have their version"), ce->name);
 }
 
-static int checkout_merged(int pos, const struct checkout *state, int *nr_checkouts)
+static int checkout_merged(int pos, const struct checkout *state,
+			   int *nr_checkouts, struct mem_pool *ce_mem_pool)
 {
 	struct cache_entry *ce = active_cache[pos];
 	const char *path = ce->name;
@@ -291,11 +293,10 @@ static int checkout_merged(int pos, const struct checkout *state, int *nr_checko
 	if (write_object_file(result_buf.ptr, result_buf.size, blob_type, &oid))
 		die(_("Unable to add merge result for '%s'"), path);
 	free(result_buf.ptr);
-	ce = make_transient_cache_entry(mode, &oid, path, 2, NULL);
+	ce = make_transient_cache_entry(mode, &oid, path, 2, ce_mem_pool);
 	if (!ce)
 		die(_("make_cache_entry failed for path '%s'"), path);
 	status = checkout_entry(ce, state, NULL, nr_checkouts);
-	discard_cache_entry(ce);
 	return status;
 }
 
@@ -359,16 +360,22 @@ static int checkout_worktree(const struct checkout_opts *opts,
 	int nr_checkouts = 0, nr_unmerged = 0;
 	int errs = 0;
 	int pos;
+	int pc_workers, pc_threshold;
+	struct mem_pool ce_mem_pool;
 
 	state.force = 1;
 	state.refresh_cache = 1;
 	state.istate = &the_index;
 
+	mem_pool_init(&ce_mem_pool, 0);
+	get_parallel_checkout_configs(&pc_workers, &pc_threshold);
 	init_checkout_metadata(&state.meta, info->refname,
 			       info->commit ? &info->commit->object.oid : &info->oid,
 			       NULL);
 
 	enable_delayed_checkout(&state);
+	if (pc_workers > 1)
+		init_parallel_checkout();
 	for (pos = 0; pos < active_nr; pos++) {
 		struct cache_entry *ce = active_cache[pos];
 		if (ce->ce_flags & CE_MATCHED) {
@@ -384,10 +391,15 @@ static int checkout_worktree(const struct checkout_opts *opts,
 						       &nr_checkouts, opts->overlay_mode);
 			else if (opts->merge)
 				errs |= checkout_merged(pos, &state,
-							&nr_unmerged);
+							&nr_unmerged,
+							&ce_mem_pool);
 			pos = skip_same_name(ce, pos) - 1;
 		}
 	}
+	if (pc_workers > 1)
+		errs |= run_parallel_checkout(&state, pc_workers, pc_threshold,
+					      NULL, NULL);
+	mem_pool_discard(&ce_mem_pool, should_validate_cache_entries());
 	remove_marked_cache_entries(&the_index, 1);
 	remove_scheduled_dirs();
 	errs |= finish_delayed_checkout(&state, &nr_checkouts);
-- 
2.30.1
Previous: Matheus Tavares BernardinoNext: Christian Couder
Message 21 of 50 in “Parallel Checkout (part 3)”
  1. 0/7 Parallel Checkout (part 3)Matheus Tavares, Apr 22, 2021
  2. 1/7 make_transient_cache_entry(): optionally alloc from mem_poolMatheus Tavares, Apr 22, 2021
  3. 2/7 builtin/checkout.c: complete parallel checkout supportMatheus Tavares, Apr 22, 2021
  4. Derrick StoleeApr 23, 2021
  5. Matheus Tavares BernardinoApr 26, 2021
  6. 3/7 checkout-index: add parallel checkout supportMatheus Tavares, Apr 22, 2021
  7. Derrick StoleeApr 23, 2021
  8. Matheus Tavares BernardinoApr 26, 2021
  9. 4/7 parallel-checkout: add tests for basic operationsMatheus Tavares, Apr 22, 2021
  10. Derrick StoleeApr 23, 2021
  11. Matheus Tavares BernardinoApr 27, 2021
  12. 5/7 parallel-checkout: add tests related to path collisionsMatheus Tavares, Apr 22, 2021
  13. 6/7 parallel-checkout: add tests related to .gitattributesMatheus Tavares, Apr 22, 2021
  14. Derrick StoleeApr 23, 2021
  15. 7/7 ci: run test round with parallel-checkout enabledMatheus Tavares, Apr 22, 2021
  16. Derrick StoleeApr 23, 2021
  17. 0/8 Parallel Checkout (part 3)Matheus Tavares, Apr 30, 2021
  18. 1/8 make_transient_cache_entry(): optionally alloc from mem_poolMatheus Tavares, Apr 30, 2021
  19. Christian CouderMay 1, 2021
  20. Matheus Tavares BernardinoMay 3, 2021
  21. 2/8 builtin/checkout.c: complete parallel checkout supportMatheus Tavares, Apr 30, 2021
  22. Christian CouderMay 1, 2021
  23. Matheus Tavares BernardinoMay 3, 2021
  24. 3/8 checkout-index: add parallel checkout supportMatheus Tavares, Apr 30, 2021
  25. Christian CouderMay 1, 2021
  26. Matheus Tavares BernardinoMay 3, 2021
  27. 4/8 parallel-checkout: add tests for basic operationsMatheus Tavares, Apr 30, 2021
  28. 5/8 parallel-checkout: add tests related to path collisionsMatheus Tavares, Apr 30, 2021
  29. Torsten BögershausenMay 2, 2021
  30. Matheus Tavares BernardinoMay 3, 2021
  31. 6/8 t0028: extract encoding helpers to lib-encoding.shMatheus Tavares, Apr 30, 2021
  32. 7/8 parallel-checkout: add tests related to .gitattributesMatheus Tavares, Apr 30, 2021
  33. 8/8 ci: run test round with parallel-checkout enabledMatheus Tavares, Apr 30, 2021
  34. Torsten BögershausenMay 2, 2021
  35. Matheus Tavares BernardinoMay 3, 2021
  36. 0/8 Parallel Checkout (part 3)Matheus Tavares, May 4, 2021
  37. 1/8 make_transient_cache_entry(): optionally alloc from mem_poolMatheus Tavares, May 4, 2021
  38. 2/8 builtin/checkout.c: complete parallel checkout supportMatheus Tavares, May 4, 2021
  39. Derrick StoleeMay 5, 2021
  40. 3/8 checkout-index: add parallel checkout supportMatheus Tavares, May 4, 2021
  41. 4/8 parallel-checkout: add tests for basic operationsMatheus Tavares, May 4, 2021
  42. AIX failures on parallel checkout (new in v2.32.0-rc*)Ævar Arnfjörð Bjarmason, May 26, 2021
  43. Matheus Tavares BernardinoMay 26, 2021
  44. Junio C HamanoMay 26, 2021
  45. 5/8 parallel-checkout: add tests related to path collisionsMatheus Tavares, May 4, 2021
  46. 6/8 t0028: extract encoding helpers to lib-encoding.shMatheus Tavares, May 4, 2021
  47. 7/8 parallel-checkout: add tests related to .gitattributesMatheus Tavares, May 4, 2021
  48. 8/8 ci: run test round with parallel-checkout enabledMatheus Tavares, May 4, 2021
  49. Derrick StoleeMay 5, 2021
  50. Junio C HamanoMay 6, 2021

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.