git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[RFC PATCH 16/21] checkout-index: add parallel checkout support

From
Matheus Tavares <matheus.bernardino@usp.br>
Date
Aug 10, 2020, 21:33 UTC
Message-ID
<feed9d435589a308759a3e6b623db13e469c242a.1597093021.git.matheus.bernardino@usp.br>
In-Reply-To
<cover.1597093021.git.matheus.bernardino@usp.br>
Signed-off-by: Matheus Tavares <matheus.bernardino@usp.br>
---
 builtin/checkout-index.c | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)
diff --git a/builtin/checkout-index.c b/builtin/checkout-index.c
index 0f1ff73129..33fb933c30 100644
--- a/builtin/checkout-index.c
+++ b/builtin/checkout-index.c
@@ -12,6 +12,7 @@
 #include "cache-tree.h"
 #include "parse-options.h"
 #include "entry.h"
+#include "parallel-checkout.h"
 
 #define CHECKOUT_ALL 4
 static int nul_term_line;
@@ -160,6 +161,7 @@ int cmd_checkout_index(int argc, const char **argv, const char *prefix)
 	int prefix_length;
 	int force = 0, quiet = 0, not_new = 0;
 	int index_opt = 0;
+	int pc_workers, pc_threshold;
 	struct option builtin_checkout_index_options[] = {
 		OPT_BOOL('a', "all", &all,
 			N_("check out all files in the index")),
@@ -214,6 +216,14 @@ int cmd_checkout_index(int argc, const char **argv, const char *prefix)
 		hold_locked_index(&lock_file, LOCK_DIE_ON_ERROR);
 	}
 
+	if (!to_tempfile)
+		get_parallel_checkout_configs(&pc_workers, &pc_threshold);
+	else
+		pc_workers = 1;
+
+	if (pc_workers > 1)
+		init_parallel_checkout();
+
 	/* Check out named files first */
 	for (i = 0; i < argc; i++) {
 		const char *arg = argv[i];
@@ -256,6 +266,12 @@ int cmd_checkout_index(int argc, const char **argv, const char *prefix)
 	if (all)
 		checkout_all(prefix, prefix_length);
 
+	if (pc_workers > 1) {
+		/* Errors were already reported */
+		run_parallel_checkout(&state, pc_workers, pc_threshold,
+				      NULL, NULL);
+	}
+
 	if (is_lock_file_locked(&lock_file) &&
 	    write_locked_index(&the_index, &lock_file, COMMIT_LOCK))
 		die("Unable to write new index file");
-- 
2.27.0
Previous: Matheus TavaresNext: Matheus Tavares
Message 20 of 61 in “[RFC] Parallel checkout”
  1. 00/21 [RFC] Parallel checkoutMatheus Tavares, Aug 10, 2020
  2. 01/21 convert: make convert_attrs() and convert structs publicMatheus Tavares, Aug 10, 2020
  3. 02/21 convert: add [async_]convert_to_working_tree_ca() variantsMatheus Tavares, Aug 10, 2020
  4. 03/21 convert: add get_stream_filter_ca() variantMatheus Tavares, Aug 10, 2020
  5. 04/21 convert: add conv_attrs classificationMatheus Tavares, Aug 10, 2020
  6. 05/21 entry: extract a header file for entry.c functionsMatheus Tavares, Aug 10, 2020
  7. 06/21 entry: make fstat_output() and read_blob_entry() publicMatheus Tavares, Aug 10, 2020
  8. 07/21 entry: extract cache_entry update from write_entry()Matheus Tavares, Aug 10, 2020
  9. 08/21 entry: move conv_attrs lookup up to checkout_entry()Matheus Tavares, Aug 10, 2020
  10. 09/21 entry: add checkout_entry_ca() which takes preloaded conv_attrsMatheus Tavares, Aug 10, 2020
  11. 10/21 unpack-trees: add basic support for parallel checkoutMatheus Tavares, Aug 10, 2020
  12. 11/21 parallel-checkout: make it truly parallelMatheus Tavares, Aug 10, 2020
  13. Jeff HostetlerAug 19, 2020
  14. Matheus Tavares BernardinoAug 20, 2020
  15. Jeff HostetlerAug 20, 2020
  16. 12/21 parallel-checkout: add configuration optionsMatheus Tavares, Aug 10, 2020
  17. 13/21 parallel-checkout: support progress displayingMatheus Tavares, Aug 10, 2020
  18. 14/21 make_transient_cache_entry(): optionally alloc from mem_poolMatheus Tavares, Aug 10, 2020
  19. 15/21 builtin/checkout.c: complete parallel checkout supportMatheus Tavares, Aug 10, 2020
  20. 16/21 checkout-index: add parallel checkout supportMatheus Tavares, Aug 10, 2020
  21. 17/21 parallel-checkout: avoid stat() calls in workersMatheus Tavares, Aug 10, 2020
  22. 18/21 entry: use is_dir_sep() when checking leading dirsMatheus Tavares, Aug 10, 2020
  23. 19/21 symlinks: make has_dirs_only_path() track FL_NOENTMatheus Tavares, Aug 10, 2020
  24. 20/21 parallel-checkout: create leading dirs in workersMatheus Tavares, Aug 10, 2020
  25. 21/21 parallel-checkout: skip checking the working tree on cloneMatheus Tavares, Aug 10, 2020
  26. Jeff HostetlerAug 12, 2020
  27. 00/19 Parallel Checkout (part I)Matheus Tavares, Sep 22, 2020
  28. 01/19 convert: make convert_attrs() and convert structs publicMatheus Tavares, Sep 22, 2020
  29. 03/19 convert: add get_stream_filter_ca() variantMatheus Tavares, Sep 22, 2020
  30. 02/19 convert: add [async_]convert_to_working_tree_ca() variantsMatheus Tavares, Sep 22, 2020
  31. 04/19 convert: add conv_attrs classificationMatheus Tavares, Sep 22, 2020
  32. 05/19 entry: extract a header file for entry.c functionsMatheus Tavares, Sep 22, 2020
  33. 06/19 entry: make fstat_output() and read_blob_entry() publicMatheus Tavares, Sep 22, 2020
  34. 07/19 entry: extract cache_entry update from write_entry()Matheus Tavares, Sep 22, 2020
  35. 08/19 entry: move conv_attrs lookup up to checkout_entry()Matheus Tavares, Sep 22, 2020
  36. Jeff HostetlerOct 1, 2020
  37. Jeff HostetlerOct 1, 2020
  38. 09/19 entry: add checkout_entry_ca() which takes preloaded conv_attrsMatheus Tavares, Sep 22, 2020
  39. 10/19 unpack-trees: add basic support for parallel checkoutMatheus Tavares, Sep 22, 2020
  40. parallel-checkout: drop unused checkout state parameterJeff King, Oct 5, 2020
  41. Matheus Tavares BernardinoOct 5, 2020
  42. Jeff KingOct 5, 2020
  43. 11/19 parallel-checkout: make it truly parallelMatheus Tavares, Sep 22, 2020
  44. Martin ÅgrenSep 29, 2020
  45. Matheus Tavares BernardinoSep 30, 2020
  46. 12/19 parallel-checkout: support progress displayingMatheus Tavares, Sep 22, 2020
  47. 13/19 make_transient_cache_entry(): optionally alloc from mem_poolMatheus Tavares, Sep 22, 2020
  48. 14/19 builtin/checkout.c: complete parallel checkout supportMatheus Tavares, Sep 22, 2020
  49. 15/19 checkout-index: add parallel checkout supportMatheus Tavares, Sep 22, 2020
  50. 16/19 parallel-checkout: add tests for basic operationsMatheus Tavares, Sep 22, 2020
  51. Jonathan NiederOct 20, 2020
  52. Taylor BlauOct 20, 2020
  53. Matheus Tavares BernardinoOct 20, 2020
  54. Junio C HamanoOct 20, 2020
  55. Matheus Tavares BernardinoOct 20, 2020
  56. Jonathan NiederOct 20, 2020
  57. Junio C HamanoOct 20, 2020
  58. 17/19 parallel-checkout: add tests related to clone collisionsMatheus Tavares, Sep 22, 2020
  59. 18/19 parallel-checkout: add tests related to .gitattributesMatheus Tavares, Sep 22, 2020
  60. 19/19 ci: run test round with parallel-checkout enabledMatheus Tavares, Sep 22, 2020
  61. Jeff HostetlerOct 1, 2020

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.