git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v3 1/8] [RFC] dir: convert trace calls to trace2 equivalents

From
Junio C Hamano <gitster@pobox.com>
Date
May 10, 2021, 04:49 UTC
Message-ID
<xmqqk0o7ry5c.fsf@gitster.g>
In-Reply-To
<9f1c0d78d739cc1da8517eea92e81a7bd162578f.1620503945.git.gitgitgadget@gmail.com>
"Elijah Newren via GitGitGadget" <gitgitgadget@gmail.com> writes:
Show 17 quoted lines
> +static void trace2_read_directory_statistics(struct dir_struct *dir,
> +					     struct repository *repo)
> +{
> +	if (!dir->untracked)
> +		return;
> +	trace2_data_intmax("read_directory", repo,
> +			   "node-creation", dir->untracked->dir_created);
> +	trace2_data_intmax("read_directory", repo,
> +			   "gitignore-invalidation",
> +			   dir->untracked->gitignore_invalidated);
> +	trace2_data_intmax("read_directory", repo,
> +			   "directory-invalidation",
> +			   dir->untracked->dir_invalidated);
> +	trace2_data_intmax("read_directory", repo,
> +			   "opendir", dir->untracked->dir_opened);
> +}
> +

This obviously looks like an equivalent to what happens in the original inside the "if (dir->untracked)" block.

And we have a performance_{enter,leave} pair replaced with a region_[enter,leave} pair.

Show 5 quoted lines
> -	trace_performance_enter();
> +	trace2_region_enter("dir", "read_directory", istate->repo);
>   ...
> -	trace_performance_leave("read directory %.*s", len, path);
> +	trace2_region_leave("dir", "read_directory", istate->repo);
Show 19 quoted lines
>  	if (dir->untracked) {
>  		static int force_untracked_cache = -1;
> -		static struct trace_key trace_untracked_stats = TRACE_KEY_INIT(UNTRACKED_STATS);
>  
>  		if (force_untracked_cache < 0)
>  			force_untracked_cache =
>  				git_env_bool("GIT_FORCE_UNTRACKED_CACHE", 0);
> -		trace_printf_key(&trace_untracked_stats,
> -				 "node creation: %u\n"
> -				 "gitignore invalidation: %u\n"
> -				 "directory invalidation: %u\n"
> -				 "opendir: %u\n",
> -				 dir->untracked->dir_created,
> -				 dir->untracked->gitignore_invalidated,
> -				 dir->untracked->dir_invalidated,
> -				 dir->untracked->dir_opened);
>  		if (force_untracked_cache &&
>  			dir->untracked == istate->untracked &&
>  		    (dir->untracked->dir_opened ||

Removal of the trace_printf() in the middle made the body of this if() statement much less distracting, which is good.

Show 7 quoted lines
> @@ -2811,6 +2818,9 @@ int read_directory(struct dir_struct *dir, struct index_state *istate,
>  			FREE_AND_NULL(dir->untracked);
>  		}
>  	}
> +
> +	if (trace2_is_enabled())
> +		trace2_read_directory_statistics(dir, istate->repo);

This slightly changes the semantics in that the original did an equivalent emitting from inside the "if (dir->untracked)" block, but this call is hoisted outside, and the new helper knows how to be silent when untracked thing is not in effect, so the net effect at this step is the same. And if we ever add tracing statics that is relevant when !dir->untracked is true, the new code organization is easier to work with.

The only curious thing is the guard "if (trace2_is_enabled())"; correctness-wise, are there bad things going to happen if it is not here, or is this a performance hack, or is it more for its documentation value (meaning, it would be a bug if we later added things that are irrelevant when trace is not enabled to the helper)?

Show 5 quoted lines
> @@ -57,6 +57,19 @@ iuc () {
>  	return $ret
>  }
>  
> +get_relevant_traces() {
Style.  SP on both sides of "()".
Show 9 quoted lines
> +	# From the GIT_TRACE2_PERF data of the form
> +	#    $TIME $FILE:$LINE | d0 | main | data | r1 | ? | ? | read_directo | $RELEVANT_STAT
> +	# extract the $RELEVANT_STAT fields.  We don't care about region_enter
> +	# or region_leave, or stats for things outside read_directory.
> +	INPUT_FILE=$1
> +	OUTPUT_FILE=$2
> +	grep data.*read_directo $INPUT_FILE \
> +	    | cut -d "|" -f 9 \
> +	    >$OUTPUT_FILE

Style. Wrapping the line after pipe '|' will allow you to omit the backslash. Also quote the redirection target, i.e. >"$OUTPUT_FILE", to help certain vintage of bash.

Those who are more familiar with the trace2 infrastructure may want to further comment, but it looked obvious and straightforward to me.

Thanks.
Previous: Elijah Newren via GitGitGadgetNext: Elijah Newren
Message 37 of 90 in “Directory traversal fixes”
  1. 0/5 Directory traversal fixesElijah Newren via GitGitGadget, May 7, 2021
  2. 1/5 t7300: add testcase showing unnecessary traversal into ignored directoryElijah Newren via GitGitGadget, May 7, 2021
  3. Eric SunshineMay 7, 2021
  4. Elijah NewrenMay 7, 2021
  5. Eric SunshineMay 7, 2021
  6. Elijah NewrenMay 7, 2021
  7. Eric SunshineMay 7, 2021
  8. Jeff KingMay 7, 2021
  9. Eric SunshineMay 7, 2021
  10. Elijah NewrenMay 8, 2021
  11. Eric SunshineMay 8, 2021
  12. Elijah NewrenMay 8, 2021
  13. Philip OakleyMay 8, 2021
  14. Elijah NewrenMay 8, 2021
  15. 2/5 t3001, t7300: add testcase showcasing missed directory traversalElijah Newren via GitGitGadget, May 7, 2021
  16. 3/5 dir: avoid unnecessary traversal into ignored directoryElijah Newren via GitGitGadget, May 7, 2021
  17. 4/5 dir: traverse into untracked directories if they may have ignored subfilesElijah Newren via GitGitGadget, May 7, 2021
  18. 5/5 [RFC] ls-files: error out on -i unless -o or -c are specifiedElijah Newren via GitGitGadget, May 7, 2021
  19. 6/5 dir: update stale description of treat_directory()Derrick Stolee, May 7, 2021
  20. Elijah NewrenMay 7, 2021
  21. Derrick StoleeMay 7, 2021
  22. 0/8 Directory traversal fixesElijah Newren via GitGitGadget, May 8, 2021
  23. 1/8 t7300: add testcase showing unnecessary traversal into ignored directoryElijah Newren via GitGitGadget, May 8, 2021
  24. Junio C HamanoMay 8, 2021
  25. Elijah NewrenMay 8, 2021
  26. Junio C HamanoMay 8, 2021
  27. Elijah NewrenMay 8, 2021
  28. 3/8 dir: avoid unnecessary traversal into ignored directoryElijah Newren via GitGitGadget, May 8, 2021
  29. 2/8 t3001, t7300: add testcase showcasing missed directory traversalElijah Newren via GitGitGadget, May 8, 2021
  30. 4/8 dir: traverse into untracked directories if they may have ignored subfilesElijah Newren via GitGitGadget, May 8, 2021
  31. 5/8 [RFC] ls-files: error out on -i unless -o or -c are specifiedElijah Newren via GitGitGadget, May 8, 2021
  32. 6/8 dir: update stale description of treat_directory()Derrick Stolee via GitGitGadget, May 8, 2021
  33. 7/8 [RFC] dir: convert trace calls to trace2 equivalentsElijah Newren via GitGitGadget, May 8, 2021
  34. 8/8 [RFC] dir: reported number of visited directories and paths with trace2Elijah Newren via GitGitGadget, May 8, 2021
  35. 0/8 Directory traversal fixesElijah Newren via GitGitGadget, May 8, 2021
  36. 1/8 [RFC] dir: convert trace calls to trace2 equivalentsElijah Newren via GitGitGadget, May 8, 2021
  37. Junio C HamanoMay 10, 2021
  38. Elijah NewrenMay 11, 2021
  39. Jeff HostetlerMay 11, 2021
  40. Elijah NewrenMay 11, 2021
  41. 2/8 [RFC] dir: report number of visited directories and paths with trace2Elijah Newren via GitGitGadget, May 8, 2021
  42. Junio C HamanoMay 10, 2021
  43. 3/8 [RFC] ls-files: error out on -i unless -o or -c are specifiedElijah Newren via GitGitGadget, May 8, 2021
  44. Junio C HamanoMay 10, 2021
  45. Elijah NewrenMay 11, 2021
  46. Junio C HamanoMay 11, 2021
  47. 4/8 t7300: add testcase showing unnecessary traversal into ignored directoryElijah Newren via GitGitGadget, May 8, 2021
  48. Junio C HamanoMay 10, 2021
  49. Elijah NewrenMay 11, 2021
  50. Junio C HamanoMay 11, 2021
  51. Elijah NewrenMay 12, 2021
  52. Junio C HamanoMay 12, 2021
  53. 5/8 t3001, t7300: add testcase showcasing missed directory traversalElijah Newren via GitGitGadget, May 8, 2021
  54. 6/8 dir: avoid unnecessary traversal into ignored directoryElijah Newren via GitGitGadget, May 8, 2021
  55. Junio C HamanoMay 10, 2021
  56. Elijah NewrenMay 11, 2021
  57. 7/8 dir: traverse into untracked directories if they may have ignored subfilesElijah Newren via GitGitGadget, May 8, 2021
  58. 8/8 dir: update stale description of treat_directory()Derrick Stolee via GitGitGadget, May 8, 2021
  59. 0/8 Directory traversal fixesElijah Newren via GitGitGadget, May 11, 2021
  60. 1/8 dir: convert trace calls to trace2 equivalentsElijah Newren via GitGitGadget, May 11, 2021
  61. Jeff HostetlerMay 11, 2021
  62. Elijah NewrenMay 11, 2021
  63. Jeff HostetlerMay 11, 2021
  64. Elijah NewrenMay 12, 2021
  65. Jeff HostetlerMay 12, 2021
  66. Elijah NewrenMay 12, 2021
  67. 3/8 ls-files: error out on -i unless -o or -c are specifiedElijah Newren via GitGitGadget, May 11, 2021
  68. 2/8 dir: report number of visited directories and paths with trace2Elijah Newren via GitGitGadget, May 11, 2021
  69. 4/8 t7300: add testcase showing unnecessary traversal into ignored directoryElijah Newren via GitGitGadget, May 11, 2021
  70. 5/8 t3001, t7300: add testcase showcasing missed directory traversalElijah Newren via GitGitGadget, May 11, 2021
  71. 7/8 dir: traverse into untracked directories if they may have ignored subfilesElijah Newren via GitGitGadget, May 11, 2021
  72. 6/8 dir: avoid unnecessary traversal into ignored directoryElijah Newren via GitGitGadget, May 11, 2021
  73. 8/8 dir: update stale description of treat_directory()Derrick Stolee via GitGitGadget, May 11, 2021
  74. 0/9 Directory traversal fixesElijah Newren via GitGitGadget, May 12, 2021
  75. 1/9 dir: convert trace calls to trace2 equivalentsElijah Newren via GitGitGadget, May 12, 2021
  76. 2/9 dir: report number of visited directories and paths with trace2Elijah Newren via GitGitGadget, May 12, 2021
  77. 4/9 t7300: add testcase showing unnecessary traversal into ignored directoryElijah Newren via GitGitGadget, May 12, 2021
  78. 6/9 dir: avoid unnecessary traversal into ignored directoryElijah Newren via GitGitGadget, May 12, 2021
  79. 8/9 dir: update stale description of treat_directory()Derrick Stolee via GitGitGadget, May 12, 2021
  80. Derrick StoleeMay 17, 2021
  81. Junio C HamanoMay 17, 2021
  82. Elijah NewrenMay 18, 2021
  83. Junio C HamanoMay 19, 2021
  84. 7/9 dir: traverse into untracked directories if they may have ignored subfilesElijah Newren via GitGitGadget, May 12, 2021
  85. 3/9 ls-files: error out on -i unless -o or -c are specifiedElijah Newren via GitGitGadget, May 12, 2021
  86. 5/9 t3001, t7300: add testcase showcasing missed directory traversalElijah Newren via GitGitGadget, May 12, 2021
  87. 9/9 dir: introduce readdir_skip_dot_and_dotdot() helperElijah Newren via GitGitGadget, May 12, 2021
  88. Derrick StoleeMay 17, 2021
  89. Elijah NewrenMay 18, 2021
  90. Derrick StoleeMay 17, 2021

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.