Re: Watchman support for git

From: Duy Nguyen <pclouds@gmail.com>
Date: May 3, 2014, 00:52 UTC
Message-ID: <CACsJy8C72QQZd4v+p4bkFFKHnN2Uj1zD-BELwGtYB7sx1P6q6g@mail.gmail.com>
In-Reply-To: <1399072451-15561-1-git-send-email-dturner@twopensource.com>

On Sat, May 3, 2014 at 6:14 AM,  <dturner@twopensource.com> wrote:

> The index format change might be less important with the split index;
> I haven't investigated that since at the time I wrote these patches,
> it didn't exist.

This is the worst case scenario of "git status" on webkit.git (182k files, path name 74 byte long on average), hot cache, no SSD

   366.379ms gitmodules_config:199 if (read_cache() < 0) die("index file
     0.004ms cmd_status:1294 read_cache_preload(&s.pathspec);
   488.433ms cmd_status:1295 refresh_index(&the_index, REFRESH_QUIE
   456.495ms cmd_status:1299 update_index_if_able(&the_index, &inde
    13.088ms wt_status_collect:616 wt_status_collect_changes_worktree(s)
   706.926ms wt_status_collect:621 wt_status_collect_changes_index(s)
   100.495ms lazy_init_name_hash:136 { int nr; if (istate->name_hash_initia
   921.185ms wt_status_collect:622 wt_status_collect_untracked(s)

real 0m2.969s user 0m1.943s sys 0m1.021s

Index v4 and split index (and the following read-cache daemon, hopefully) should help reduce numbers of the 1st and 4th lines, I expect to less than 50ms each line. lazy_init_name_hash could be taken away with read-cache daemon also.

core.preloadindex can cut the total number of 2nd and 3rd lines by half. Watchman should help in these two lines, but it should do better than core.preloadindex.

wt_status_collect_changes_index() depends on how damaged cache-tree is (in this case, totally scraped). watchman does not help this either. We need to try to "heal" cache-tree as much as possible to reduce the number.

The last line could be a competition between watchman and my coming "untracked cache" series. I expect to cut the number in that line at least in half without external dependency.

Patch 2/3 did not seem to make it to the list by the way.. initial reaction is storing the list of all paths seems too much, but I'll need to play with it a bit to understand it.

-- 
Duy