git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH] pack-bitmap: remove trace2 region from hot path

From
Derrick Stolee via GitGitGadget <gitgitgadget@gmail.com>
Date
Sep 23, 2022, 13:00 UTC
Message-ID
<pull.1365.git.1663938034607.gitgitgadget@gmail.com>
From: Derrick Stolee <derrickstolee@github.com>

The trace2 region around the call to lazy_bitmap_for_commit() in bitmap_for_commit() was added in 28cd730680d (pack-bitmap: prepare to read lookup table extension, 2022-08-14). While adding trace2 regions is typically helpful for tracking performance, this method is called possibly thousands of times as a commit walk explores commit history looking for a matching bitmap. When trace2 output is enabled, this region is emitted many times and performance is throttled by that output.

For now, remove these regions entirely.

This is a critical path, and it would be valuable to measure that the time spent in bitmap_for_commit() does not increase when using the commit lookup table. The best way to do that would be to use a mechanism that sums the time spent in a region and reports a single value at the end of the process. This technique was introduced but not merged by [1] so maybe this example presents some justification to revisit that approach.

[1] https://lore.kernel.org/git/pull.1099.v2.git.1640720202.gitgitgadget@gmail.com/
Signed-off-by: Derrick Stolee <derrickstolee@github.com>
---
    [For 2.38.0] pack-bitmap: remove trace2 region from hot path
    
    I noticed this while trying to backport the Abhradeep's lookup table
    work into GitHub's fork. Something went wrong in that process, causing
    this region to significantly slow down. It turns out that slow down does
    not reproduce on current 'master', which is good. I must have missed
    something while I was backporting.
    
    Regardless, the use of trace2_region_enter() here should be removed or
    replaced. For the sake of 2.38.0, this simple removal is safe enough.
    However, to really dig into what was happening I had to construct a
    rebase [2] of Jeff's trace2 stopwatch work, then apply changes on top
    [3] that could replace this region with trace2_timer_*() methods.
    
    [2]
    https://github.com/git/git/compare/master...derrickstolee:trace2-stopwatch
    [3]
    https://github.com/derrickstolee/git/compare/trace2-stopwatch...bitmap-trace2
    
    As a separate discussion, it might be worth revisiting that stopwatch
    work so we have it available as a tool when doing these kinds of
    investigations.
    
    Thanks, -Stolee
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1365%2Fderrickstolee%2Fbitmap-remove-trace2-v1
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1365/derrickstolee/bitmap-remove-trace2-v1
Pull-Request: https://github.com/gitgitgadget/git/pull/1365
 pack-bitmap.c | 2 --
 1 file changed, 2 deletions(-)
diff --git a/pack-bitmap.c b/pack-bitmap.c
index 9a208abc1fd..13457dd77e5 100644
--- a/pack-bitmap.c
+++ b/pack-bitmap.c
@@ -830,10 +830,8 @@ struct ewah_bitmap *bitmap_for_commit(struct bitmap_index *bitmap_git,
 		if (!bitmap_git->table_lookup)
 			return NULL;
 
-		trace2_region_enter("pack-bitmap", "reading_lookup_table", the_repository);
 		/* NEEDSWORK: cache misses aren't recorded */
 		bitmap = lazy_bitmap_for_commit(bitmap_git, commit);
-		trace2_region_leave("pack-bitmap", "reading_lookup_table", the_repository);
 		if (!bitmap)
 			return NULL;
 		return lookup_stored_bitmap(bitmap);

base-commit: 1b3d6e17fe83eb6f79ffbac2f2c61bbf1eaef5f8
-- 
gitgitgadget
Next: Junio C Hamano
Message 1 of 10 in “pack-bitmap: remove trace2 region from hot path”
  1. pack-bitmap: remove trace2 region from hot pathDerrick Stolee via GitGitGadget, Sep 23, 2022
  2. Junio C HamanoSep 23, 2022
  3. Junio C HamanoSep 23, 2022
  4. Junio C HamanoSep 23, 2022
  5. Derrick StoleeSep 23, 2022
  6. Taylor BlauSep 23, 2022
  7. pack-bitmap: remove trace2 region from hot pathDerrick Stolee via GitGitGadget, Sep 26, 2022
  8. Ævar Arnfjörð BjarmasonSep 26, 2022
  9. Derrick StoleeSep 26, 2022
  10. Jeff HostetlerSep 27, 2022

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.