Re: [PATCH v4 5/5] unpack-trees: reuse (still valid) cache-tree from src_index
- From
Ben Peart <peartben@gmail.com>
- Date
- Aug 13, 2018, 17:15 UTC
- Message-ID
- <d3faf3e0-5526-8613-9c7e-796f627eaef3@gmail.com>
- In-Reply-To
- <CACsJy8A2L-WX_RLP37fF-f=YBLkCY7+iZLo+Uz=H9EFOWMVnoA@mail.gmail.com>
On 8/13/2018 12:25 PM, Duy Nguyen wrote:
Show 27 quoted lines
> On Mon, Aug 13, 2018 at 6:05 PM Ben Peart <peartben@gmail.com> wrote: >> I was part way through writing a patch that would copy the valid parts >> of the cache-tree from the source index to the dest index > > Yeah sorry about that. I make bad judgements all the time, unfortunately. > > If it's sort of working though, please post to the list anyway to > archive it. Who knows, some time down the road we might actually need > it again. > >> I run some tests on a large repo and the results look very promising. >> >> base new diff % saved >> 0.55 0.52 0.02 4.32% s: read cache .git/index >> 0.31 0.30 0.01 2.98% s: initialize name hash >> 0.03 0.02 0.00 9.98% s: preload index >> 0.09 0.09 0.00 4.86% s: refresh index >> 5.93 1.19 4.74 79.95% s: traverse_trees >> 0.12 0.13 -0.01 -4.15% s: check_updates >> 2.14 0.00 2.14 100.00% s: cache_tree_update >> 10.63 4.29 6.33 59.59% s: unpack_trees > > There's a big gap here, I think. unpack_trees() takes 4s but the sum > of traverse_trees, check_updates and cache_tree_update is 1.5s top. I > guess that's sparse checkout and stuff? It's either that or there's > another big hidden thing we should pay attention to ;-) >
Yes, there are additional costs associated with the sparse-checkout and excludes logic. We've sped that up significantly by converting it to a hashmap but it still has measurable cost as we have to compute the hash of the cache entry name before looking it up.
Name Inc % Inc + git!unpack_trees 50.9 4,575 |+ git!clear_ce_flags_1 16.5 1,479 ||+ git!is_included_in_virtualfilesystem 15.9 1,430 || + git!check_includes_hashmap 15.8 1,418 |+ git!traverse_trees 16.3 1,468 |+ git!cache_tree_fully_valid 11.7 1,055 |+ git!check_updates 1.9 169 |+ git!discard_index 1.8 162 |+ git!apply_sparse_checkout 0.2 15 |+ git!next_cache_entry 0.0 3