Re: [PATCH v2 0/4] Speed up unpack_trees()

From: Duy Nguyen <pclouds@gmail.com>
Date: Jul 31, 2018, 15:31 UTC
Message-ID: <CACsJy8BUBjPngHz=icHomor-LJOkMLwZ9bQ6YJDxnoXGg++vjg@mail.gmail.com>
In-Reply-To: <9a9a309c-7143-e642-cfd8-6df76e77995a@gmail.com>

On Mon, Jul 30, 2018 at 8:10 PM Ben Peart <peartben@gmail.com> wrote:

Show 22 quoted lines

> I ran "git checkout" on a large repo and averaged the results of 3 runs.
>   This clearly demonstrates the benefit of the optimized unpack_trees()
> as even the final "diff-index" is essentially a 3rd call to unpack_trees().
>
> baseline        new
> ----------------------------------------------------------------------
> 0.535510167     0.556558733     s: read cache .git/index
> 0.3057373       0.3147105       s: initialize name hash
> 0.0184082       0.023558433     s: preload index
> 0.086910967     0.089085967     s: refresh index
> 7.889590767     2.191554433     s: unpack trees
> 0.120760833     0.131941267     s: update worktree after a merge
> 2.2583504       2.572663167     s: repair cache-tree
> 0.8916137       0.959495233     s: write index, changed mask = 28
> 3.405199233     0.2710663       s: unpack trees
> 0.000999667     0.0021554       s: update worktree after a merge
> 3.4063306       0.273318333     s: diff-index
> 16.9524923      9.462943133     s: git command:
> 'c:\git-sdk-64\usr\src\git\git.exe' checkout
>
> The first call to unpack_trees() saves 72%
> The 2nd and 3rd call save 92%

By the 3rd I guess you meant "diff-index" line. I think it's the same with the second call. diff-index triggers the second unpack-trees but there's no indent here and it's misleading to read this as diff-index and unpack-trees execute one after the other.

> Total time savings for the entire command was 44%

Wow.. I guess you have more trees since I could only save 30% on gcc.git.

> In the performance game of whack-a-mole, that call to repair cache-tree
> is now looking quite expensive...

Yeah and I think we can whack that mole too. I did some measurement. Best case possible, we just need to scan through two indexes (one with many good cache-tree, one with no cache-tree), compare and copy cache-tree over. The scanning takes like 1% time of current repair step and I suspect it's the hashing that takes most of the time. Of course real world won't have such nice numbers, but I guess we could maybe half cache-tree update/repair time.

-- 
Duy