git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v7 7/7] diff-lib: parallelize run_diff_files for submodules

From
Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Date
Feb 7, 2023, 23:06 UTC
Message-ID
<230208.86ilgd0yej.gmgdl@evledraar.gmail.com>
In-Reply-To
<20230207181706.363453-8-calvinwan@google.com>
On Tue, Feb 07 2023, Calvin Wan wrote:
Show 13 quoted lines
> [...]
> +	sps->result = 1;
> +	strbuf_addf(err,
> +	    _(status_porcelain_start_error),
> +	    task->path);
> +	return 0;
> [...]
> +	if (retvalue) {
> +		sps->result = 1;
> +		strbuf_addf(err,
> +		    _(status_porcelain_fail_error),
> +		    task->path);
> [...]
This is nitpicky, but what's with the short lines and over-wrapping?

If you change these two to (just using my macro version on top, but it's the same with yours):

	strbuf_addf(err, _(STATUS_PORCELAIN_START_ERROR), task->path);
And:
	strbuf_addf(err, _(STATUS_PORCELAIN_FAIL_ERROR), task->path);

Both of these are under our usual line limit at their respective indentation (the latter at 77, rule of thumb is to wrap at 79-80).

> +	if (submodules.nr > 0) {
Don't compare unsigned to >0, just use "submodules.nr".
> +		int parallel_jobs;

nit: add extra \n, or maybe just call this "int v", as it's clear from the scope what it's about...

Show 6 quoted lines
> +		if (git_config_get_int("submodule.diffjobs", &parallel_jobs))
> +			parallel_jobs = 1;
> +		else if (!parallel_jobs)
> +			parallel_jobs = online_cpus();
> +		else if (parallel_jobs < 0)
> +			die(_("submodule.diffjobs cannot be negative"));

Can't you use the "ulong" instead of "int" and have it handle this "is negative?" error check for you?

> +
> +		if (get_submodules_status(&submodules, parallel_jobs))
> +			die(_("submodule status failed"));
> +		for (size_t i = 0; i < submodules.nr; i++) {
Another case that can use for_each_string_list_item().
Show 9 quoted lines
> +struct submodule_parallel_status {
> +	size_t index_count;
> +	int result;
> +
> +	struct string_list *submodule_names;
> +
> +	/* Pending statuses by OIDs */
> +	struct status_task **oid_status_tasks;
> +	int oid_status_tasks_nr, oid_status_tasks_alloc;
For new structs, let's use size_t, not "int" for alloc/nr.

Also, as this is 7/7 and we're not adding another such pattern for the forseeable future, can we just call these "size_t nr", "size_t alloc" and "tasks"?

And having said all that, it turns out this is just dead code that can be removed? Blindly copied from submodule_parallel_fetch?

Previous: Calvin WanNext: Calvin Wan
Message 69 of 86 in “submodule: parallelize diff”
  1. 0/6 submodule: parallelize diffCalvin Wan, Jan 4, 2023
  2. Calvin WanJan 5, 2023
  3. 0/6 submodule: parallelize diffCalvin Wan, Jan 17, 2023
  4. 1/7 run-command: add duplicate_output_fn to run_processes_parallel_optsCalvin Wan, Feb 7, 2023
  5. Ævar Arnfjörð BjarmasonFeb 7, 2023
  6. Calvin WanFeb 8, 2023
  7. Phillip WoodFeb 8, 2023
  8. Calvin WanFeb 8, 2023
  9. Phillip WoodFeb 9, 2023
  10. 0/7 submodule: parallelize diffCalvin Wan, Feb 7, 2023
  11. Ævar Arnfjörð BjarmasonFeb 8, 2023
  12. 0/6 submodule: parallelize diffCalvin Wan, Feb 9, 2023
  13. Ævar Arnfjörð BjarmasonFeb 9, 2023
  14. Junio C HamanoFeb 9, 2023
  15. Calvin WanFeb 9, 2023
  16. Junio C HamanoFeb 9, 2023
  17. Ævar Arnfjörð BjarmasonFeb 10, 2023
  18. Junio C HamanoFeb 10, 2023
  19. Phillip WoodFeb 9, 2023
  20. 0/6 submodule: parallelize diffCalvin Wan, Mar 2, 2023
  21. 1/6 run-command: add on_stderr_output_fn to run_processes_parallel_optsCalvin Wan, Mar 2, 2023
  22. 2/6 submodule: rename strbuf variableCalvin Wan, Mar 2, 2023
  23. Junio C HamanoMar 3, 2023
  24. Calvin WanMar 6, 2023
  25. Junio C HamanoMar 6, 2023
  26. Calvin WanMar 6, 2023
  27. 3/6 submodule: move status parsing into functionCalvin Wan, Mar 2, 2023
  28. Glen ChooMar 17, 2023
  29. 5/6 diff-lib: refactor out diff_change logicCalvin Wan, Mar 2, 2023
  30. 4/6 submodule: refactor is_submodule_modified()Calvin Wan, Mar 2, 2023
  31. 6/6 diff-lib: parallelize run_diff_files for submodulesCalvin Wan, Mar 2, 2023
  32. Ævar Arnfjörð BjarmasonMar 7, 2023
  33. Ævar Arnfjörð BjarmasonMar 7, 2023
  34. Junio C HamanoMar 7, 2023
  35. Glen ChooMar 17, 2023
  36. Glen ChooMar 17, 2023
  37. 1/6 run-command: add duplicate_output_fn to run_processes_parallel_optsCalvin Wan, Feb 9, 2023
  38. Glen ChooFeb 13, 2023
  39. Junio C HamanoFeb 13, 2023
  40. Calvin WanFeb 13, 2023
  41. 2/6 submodule: strbuf variable renameCalvin Wan, Feb 9, 2023
  42. Glen ChooFeb 13, 2023
  43. 3/6 submodule: move status parsing into functionCalvin Wan, Feb 9, 2023
  44. 4/6 submodule: refactor is_submodule_modified()Calvin Wan, Feb 9, 2023
  45. Glen ChooFeb 13, 2023
  46. 5/6 diff-lib: refactor out diff_change logicCalvin Wan, Feb 9, 2023
  47. Ævar Arnfjörð BjarmasonFeb 9, 2023
  48. Glen ChooFeb 13, 2023
  49. Calvin WanFeb 13, 2023
  50. Glen ChooFeb 14, 2023
  51. 6/6 diff-lib: parallelize run_diff_files for submodulesCalvin Wan, Feb 9, 2023
  52. Glen ChooFeb 13, 2023
  53. 2/7 submodule: strbuf variable renameCalvin Wan, Feb 7, 2023
  54. Ævar Arnfjörð BjarmasonFeb 7, 2023
  55. Calvin WanFeb 8, 2023
  56. 3/7 submodule: move status parsing into functionCalvin Wan, Feb 7, 2023
  57. 4/7 submodule: refactor is_submodule_modified()Calvin Wan, Feb 7, 2023
  58. Ævar Arnfjörð BjarmasonFeb 7, 2023
  59. 5/7 diff-lib: refactor out diff_change logicCalvin Wan, Feb 7, 2023
  60. Phillip WoodFeb 8, 2023
  61. Calvin WanFeb 8, 2023
  62. Phillip WoodFeb 9, 2023
  63. 6/7 diff-lib: refactor match_stat_with_submoduleCalvin Wan, Feb 7, 2023
  64. Ævar Arnfjörð BjarmasonFeb 8, 2023
  65. Phillip WoodFeb 8, 2023
  66. Calvin WanFeb 8, 2023
  67. Phillip WoodFeb 8, 2023
  68. 7/7 diff-lib: parallelize run_diff_files for submodulesCalvin Wan, Feb 7, 2023
  69. Ævar Arnfjörð BjarmasonFeb 7, 2023
  70. 1/6 run-command: add duplicate_output_fn to run_processes_parallel_optsCalvin Wan, Jan 17, 2023
  71. 2/6 submodule: strbuf variable renameCalvin Wan, Jan 17, 2023
  72. 3/6 submodule: move status parsing into functionCalvin Wan, Jan 17, 2023
  73. 4/6 diff-lib: refactor match_stat_with_submoduleCalvin Wan, Jan 17, 2023
  74. 5/6 diff-lib: parallelize run_diff_files for submodulesCalvin Wan, Jan 17, 2023
  75. Glen ChooJan 26, 2023
  76. Glen ChooJan 26, 2023
  77. Calvin WanJan 26, 2023
  78. 6/6 submodule: call parallel code from serial statusCalvin Wan, Jan 17, 2023
  79. Glen ChooJan 26, 2023
  80. Glen ChooJan 26, 2023
  81. 1/6 run-command: add duplicate_output_fn to run_processes_parallel_optsCalvin Wan, Jan 4, 2023
  82. 2/6 submodule: strbuf variable renameCalvin Wan, Jan 4, 2023
  83. 3/6 submodule: move status parsing into functionCalvin Wan, Jan 4, 2023
  84. 4/6 diff-lib: refactor match_stat_with_submoduleCalvin Wan, Jan 4, 2023
  85. 5/6 diff-lib: parallelize run_diff_files for submodulesCalvin Wan, Jan 4, 2023
  86. 6/6 submodule: call parallel code from serial statusCalvin Wan, Jan 4, 2023

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.