git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v3 1/1] fetch: add trace2 instrumentation

From
Derrick Stolee <stolee@gmail.com>
Date
Nov 18, 2019, 15:46 UTC
Message-ID
<944c956e-dd9a-1a12-5cb1-0c263ee7d5bd@gmail.com>
In-Reply-To
<xmqq5zjws12y.fsf@gitster-ct.c.googlers.com>
On 11/7/2019 12:32 AM, Junio C Hamano wrote:
Show 113 quoted lines
> "Erik Chen via GitGitGadget" <gitgitgadget@gmail.com> writes:
> 
>> From: Erik Chen <erikchen@chromium.org>
>>
>> Add trace2 regions to fetch-pack.c to better track time spent in the various
>> phases of a fetch:
>>
>>     * matching common remote and local refs
>>     * marking local refs as complete (part of the matching process)
>>
>> Both of these stages can be slow for repositories with many refs.
>>
>> Signed-off-by: Erik Chen <erikchen@chromium.org>
>> ---
>>  fetch-pack.c | 8 ++++++++
>>  1 file changed, 8 insertions(+)
> 
> OK.
> 
>> diff --git a/fetch-pack.c b/fetch-pack.c
>> index 0130b44112..5e3eee0477 100644
>> --- a/fetch-pack.c
>> +++ b/fetch-pack.c
>> @@ -669,6 +669,8 @@ static void mark_complete_and_common_ref(struct fetch_negotiator *negotiator,
>>  
>>  	save_commit_buffer = 0;
>>  
>> +	trace2_region_enter("fetch-pack", "mark_complete_and_common_ref", NULL);
>> +
>>  	for (ref = *refs; ref; ref = ref->next) {
>>  		struct object *o;
>>  
>> @@ -690,6 +692,10 @@ static void mark_complete_and_common_ref(struct fetch_negotiator *negotiator,
>>  		}
>>  	}
>>  
>> +	/* This block marks all local refs as COMPLETE, and then recursively marks all
>> +	 * parents of those refs as COMPLETE.
>> +	 */
> 
>         /*
>          * We write our multi-line comments like this, with the
>          * slash-asterisk at the beginning and the asterisk-slash
>          * at the end on its own line.  Learn such local conventions
>          * from the existing surrounding code and imitate, which
>          * would reduce stylistic errors.
>          */
> 
> Will fix-up while queuing (no need to reroll only to fix this).
> 
>> +	trace2_region_enter("fetch-pack", "mark_complete_local_refs", NULL);
>>  	if (!args->deepen) {
>>  		for_each_ref(mark_complete_oid, NULL);
>>  		for_each_cached_alternate(NULL, mark_alternate_complete);
>> @@ -697,6 +703,7 @@ static void mark_complete_and_common_ref(struct fetch_negotiator *negotiator,
>>  		if (cutoff)
>>  			mark_recent_complete_commits(args, cutoff);
>>  	}
>> +	trace2_region_leave("fetch-pack", "mark_complete_local_refs", NULL);
>>  
>>  	/*
>>  	 * Mark all complete remote refs as common refs.
>> @@ -716,6 +723,7 @@ static void mark_complete_and_common_ref(struct fetch_negotiator *negotiator,
>>  	}
>>  
>>  	save_commit_buffer = old_save_commit_buffer;
>> +	trace2_region_leave("fetch-pack", "mark_complete_and_common_ref", NULL);
> 
> 
> So this introduces a single region around the entire function body
> of mark_complete_and_common_ref(), within which only one subpart is
> also enclosed in a nested region.  Is that because the parts inside
> the outer region before and after the inner region are known to
> consume negligible time?  IOW I would understand
> 
>         F () {
>                 <region 1 begin>
>                     <region 1.1 begin>
>                         code
>                     <region 1.1 end>
>                     <region 1.2 begin>
>                         code
>                     <region 1.2 end>
>                     <region 1.3 begin>
>                         code
>                     <region 1.3 end>
>                 <region 1 end>
>         }
> 
> or
> 
>         F () {
>                         trivial code
>                 <region 1 begin>
>                         heavy code
>                 <region 1 end>
>                         trivial code
>         }
> 
> but this appears to do
> 
> 
>         F () {
>                 <region 1 begin>
>                         code
>                     <region 1.1 begin>
>                         code
>                     <region 1.1 end>
>                         code
>                 <region 1 end>
>         }
> 
> which is somewhat puzzling.

I notice that a v4 was sent that adds more sub-regions without actually responding to this request. (It is worth also pointing out that you ignored Junio's request you use the cover letter to explain your reasoning for changes between versions.)

There is a real downside to nesting regions like this. Specifically, we frequently limit the depth that we report nested regions to avoid overwhelming the logs.

In general, these sub-regions should be avoided when possible and instead create regions around important sections, such as the second option Junio lists above.

Thanks, -Stolee

Previous: Junio C HamanoNext: Junio C Hamano
Message 10 of 22 in “fetch: add trace2 instrumentation”
  1. 0/1 fetch: add trace2 instrumentationerik chen via GitGitGadget, Nov 5, 2019
  2. 1/1 fetch: add trace2 instrumentationErik Chen via GitGitGadget, Nov 5, 2019
  3. Johannes SchindelinNov 6, 2019
  4. 0/2 fetch: add trace2 instrumentationerik chen via GitGitGadget, Nov 6, 2019
  5. 1/2 fetch: add trace2 instrumentationErik Chen via GitGitGadget, Nov 6, 2019
  6. 2/2 add whitespaceErik Chen via GitGitGadget, Nov 6, 2019
  7. 0/1 fetch: add trace2 instrumentationerik chen via GitGitGadget, Nov 6, 2019
  8. 1/1 fetch: add trace2 instrumentationErik Chen via GitGitGadget, Nov 6, 2019
  9. Junio C HamanoNov 7, 2019
  10. Derrick StoleeNov 18, 2019
  11. Junio C HamanoNov 19, 2019
  12. Erik ChenNov 19, 2019
  13. Erik ChenNov 19, 2019
  14. Erik ChenNov 19, 2019
  15. Erik ChenNov 19, 2019
  16. Junio C HamanoNov 7, 2019
  17. Erik ChenNov 19, 2019
  18. 0/1 fetch: add trace2 instrumentationerik chen via GitGitGadget, Nov 18, 2019
  19. 1/1 fetch: add trace2 instrumentationErik Chen via GitGitGadget, Nov 18, 2019
  20. 0/1 fetch: add trace2 instrumentationerik chen via GitGitGadget, Nov 19, 2019
  21. 1/1 fetch: add trace2 instrumentationErik Chen via GitGitGadget, Nov 19, 2019
  22. Junio C HamanoNov 20, 2019

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.