Updated based on comments from Dscho and Ben. Thanks for those! - Alex
[PATCH v2 1/4] fsmonitor: Set the PWD to the top of the working tree
The fsmonitor command inherits the PWD of its caller, which may be anywhere in the working copy. This makes is difficult for the fsmonitor command to operate on the whole repository. Specifically, for the watchman integration, this causes each subdirectory to get its own watch entry.
Set the CWD to the top of the working directory, for consistency.
Signed-off-by: Alex Vandiver <alexmv@dropbox.com> --- fsmonitor.c | 1 + 1 file changed, 1 insertion(+)
Show changes to fsmonitor.c +1 −0
diff --git a/fsmonitor.c b/fsmonitor.c index 7c1540c05..0d26ff34f 100644 --- a/fsmonitor.c +++ b/fsmonitor.c @@ -121,6 +121,7 @@ static int query_fsmonitor(int version, uint64_t last_update, struct strbuf *que argv[3] = NULL; cp.argv = argv; cp.use_shell = 1; + cp.dir = get_git_work_tree(); return capture_command(&cp, query_result, 1024); }
-- 2.15.0.rc1.413.g76aedb451
Re: [PATCH v2 1/4] fsmonitor: Set the PWD to the top of the working tree
On Wed, 25 Oct 2017, Alex Vandiver wrote:
Show 22 quoted lines
> The fsmonitor command inherits the PWD of its caller, which may be > anywhere in the working copy. This makes is difficult for the > fsmonitor command to operate on the whole repository. Specifically, > for the watchman integration, this causes each subdirectory to get its > own watch entry. > > Set the CWD to the top of the working directory, for consistency. > > Signed-off-by: Alex Vandiver <alexmv@dropbox.com> > --- > fsmonitor.c | 1 + > 1 file changed, 1 insertion(+) > > diff --git a/fsmonitor.c b/fsmonitor.c > index 7c1540c05..0d26ff34f 100644 > --- a/fsmonitor.c > +++ b/fsmonitor.c > @@ -121,6 +121,7 @@ static int query_fsmonitor(int version, uint64_t last_update, struct strbuf *que > argv[3] = NULL; > cp.argv = argv; > cp.use_shell = 1; > + cp.dir = get_git_work_tree();
Looks like my editor swapped out a tab on me. I'll hold off on sending a revised version to collect any other comments. - Alex
Ben PeartRe: [PATCH v2 1/4] fsmonitor: Set the PWD to the top of the working tree
On 10/25/2017 9:31 PM, Alex Vandiver wrote:
Show 23 quoted lines
> The fsmonitor command inherits the PWD of its caller, which may be > anywhere in the working copy. This makes is difficult for the > fsmonitor command to operate on the whole repository. Specifically, > for the watchman integration, this causes each subdirectory to get its > own watch entry. > > Set the CWD to the top of the working directory, for consistency. > > Signed-off-by: Alex Vandiver <alexmv@dropbox.com> > --- > fsmonitor.c | 1 + > 1 file changed, 1 insertion(+) > > diff --git a/fsmonitor.c b/fsmonitor.c > index 7c1540c05..0d26ff34f 100644 > --- a/fsmonitor.c > +++ b/fsmonitor.c > @@ -121,6 +121,7 @@ static int query_fsmonitor(int version, uint64_t last_update, struct strbuf *que > argv[3] = NULL; > cp.argv = argv; > cp.use_shell = 1; > + cp.dir = get_git_work_tree(); >
I'm not sure what would trigger this problem but I don't doubt that it is possible. Better to be safe than sorry. This is a better/faster solution than you're previous patch. Thank you!
> return capture_command(&cp, query_result, 1024); > } >
Re: [PATCH v2 1/4] fsmonitor: Set the PWD to the top of the working tree
On Thu, 26 Oct 2017, Ben Peart wrote:
Show 16 quoted lines
> On 10/25/2017 9:31 PM, Alex Vandiver wrote: > > diff --git a/fsmonitor.c b/fsmonitor.c > > index 7c1540c05..0d26ff34f 100644 > > --- a/fsmonitor.c > > +++ b/fsmonitor.c > > @@ -121,6 +121,7 @@ static int query_fsmonitor(int version, uint64_t > > last_update, struct strbuf *que > > argv[3] = NULL; > > cp.argv = argv; > > cp.use_shell = 1; > > + cp.dir = get_git_work_tree(); > > > > I'm not sure what would trigger this problem but I don't doubt that it is > possible. Better to be safe than sorry. This is a better/faster solution than > you're previous patch. Thank you!
See my response on the v1 of this series -- I'm curious how you're _not_ seeing it, actually. Can you not replicate just by running `git status` from differing parts of the working tree? - Alex
Ben PeartRe: [PATCH v2 1/4] fsmonitor: Set the PWD to the top of the working tree
On 10/26/2017 5:27 PM, Alex Vandiver wrote:
Show 23 quoted lines
> On Thu, 26 Oct 2017, Ben Peart wrote: >> On 10/25/2017 9:31 PM, Alex Vandiver wrote: >>> diff --git a/fsmonitor.c b/fsmonitor.c >>> index 7c1540c05..0d26ff34f 100644 >>> --- a/fsmonitor.c >>> +++ b/fsmonitor.c >>> @@ -121,6 +121,7 @@ static int query_fsmonitor(int version, uint64_t >>> last_update, struct strbuf *que >>> argv[3] = NULL; >>> cp.argv = argv; >>> cp.use_shell = 1; >>> + cp.dir = get_git_work_tree(); >>> >> >> I'm not sure what would trigger this problem but I don't doubt that it is >> possible. Better to be safe than sorry. This is a better/faster solution than >> you're previous patch. Thank you! > > See my response on the v1 of this series -- I'm curious how you're > _not_ seeing it, actually. Can you not replicate just by running > `git status` from differing parts of the working tree? > - Alex >
I saw your response but actually can't replicate it locally. I've been running with Watchman integration on all my repos for months and my "watchman watch-list" command only shows the root of the various working directories - no subdirectories.
Re: [PATCH v2 1/4] fsmonitor: Set the PWD to the top of the working tree
On Thu, 26 Oct 2017, Ben Peart wrote:
> I saw your response but actually can't replicate it locally. I've been > running with Watchman integration on all my repos for months and my "watchman > watch-list" command only shows the root of the various working directories - > no subdirectories.
Weird. I double-checked and I see the same behavior with watchman 4.9.0 as with 4.7.0 that I had been using previously. I wonder if something's different between `git` in `next` from wherever your branch was based. - Alex
Johannes SchindelinRe: [PATCH v2 1/4] fsmonitor: Set the PWD to the top of the working tree
Hi,
On Thu, 26 Oct 2017, Ben Peart wrote:
Show 30 quoted lines
> On 10/26/2017 5:27 PM, Alex Vandiver wrote: > > On Thu, 26 Oct 2017, Ben Peart wrote: > > > On 10/25/2017 9:31 PM, Alex Vandiver wrote: > > > > diff --git a/fsmonitor.c b/fsmonitor.c > > > > index 7c1540c05..0d26ff34f 100644 > > > > --- a/fsmonitor.c > > > > +++ b/fsmonitor.c > > > > @@ -121,6 +121,7 @@ static int query_fsmonitor(int version, uint64_t > > > > last_update, struct strbuf *que > > > > argv[3] = NULL; > > > > cp.argv = argv; > > > > cp.use_shell = 1; > > > > + cp.dir = get_git_work_tree(); > > > > > > > > > > I'm not sure what would trigger this problem but I don't doubt that it is > > > possible. Better to be safe than sorry. This is a better/faster solution > > > than > > > you're previous patch. Thank you! > > > > See my response on the v1 of this series -- I'm curious how you're > > _not_ seeing it, actually. Can you not replicate just by running > > `git status` from differing parts of the working tree? > > - Alex > > > > I saw your response but actually can't replicate it locally. I've been > running with Watchman integration on all my repos for months and my "watchman > watch-list" command only shows the root of the various working directories - > no subdirectories.
Indeed, I cannot replicate either. The thing is that "status" is marked with GIT_SETUP in git.c:
https://github.com/git-for-windows/git/blob/v2.14.3.windows.1/git.c#L465
That means that the setup_git_directory() is run, which sets the current working directory to the top-level directory.
So there is your explanation why neither Ben nor I saw this.
And I agree with Ben that it is safer to do it the way you suggested, just in case that the call path comes from a Git command that was not marked with GIT_SETUP.
Ciao, Johannes
[PATCH v2 2/4] fsmonitor: Don't bother pretty-printing JSON from watchman
This provides modest performance savings. Benchmarking with the following program, with and without `--no-pretty`, we find savings of 23% (0.316s -> 0.242s) in the git repository, and savings of 8% (5.24s -> 4.86s) on a large repository with 580k files in the working copy.
#!/usr/bin/perl
use strict;
use warnings;
use IPC::Open2; my $pid = open2(\*CHLD_OUT, \*CHLD_IN, "watchman -j @ARGV")
or die "open2() failed: $!\n" .
"Falling back to scanning...\n"; my $query = qq|["query", "$ENV{PWD}", {}]|; print CHLD_IN $query;
close CHLD_IN;
my $response = do {local $/; <CHLD_OUT>}; my $json_pkg;
eval {
require JSON::XSomething;
$json_pkg = "JSON::XSomething";
1;
} or do {
require JSON::PP;
$json_pkg = "JSON::PP";
};my $o = $json_pkg->new->utf8->decode($response);
Signed-off-by: Alex Vandiver <alexmv@dropbox.com> --- t/t7519/fsmonitor-watchman | 2 +- templates/hooks--fsmonitor-watchman.sample | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-)
Show changes to 2 files +2 −2
t/t7519/fsmonitor-watchman, templates/hooks--fsmonitor-watchman.sample
diff --git a/t/t7519/fsmonitor-watchman b/t/t7519/fsmonitor-watchman index a3e30bf54..79f24325c 100755 --- a/t/t7519/fsmonitor-watchman +++ b/t/t7519/fsmonitor-watchman @@ -50,7 +50,7 @@ launch_watchman(); sub launch_watchman { - my $pid = open2(\*CHLD_OUT, \*CHLD_IN, 'watchman -j') + my $pid = open2(\*CHLD_OUT, \*CHLD_IN, 'watchman -j --no-pretty') or die "open2() failed: $!\n" . "Falling back to scanning...\n"; diff --git a/templates/hooks--fsmonitor-watchman.sample b/templates/hooks--fsmonitor-watchman.sample index 9eba8a740..9a082f278 100755 --- a/templates/hooks--fsmonitor-watchman.sample +++ b/templates/hooks--fsmonitor-watchman.sample @@ -49,7 +49,7 @@ launch_watchman(); sub launch_watchman { - my $pid = open2(\*CHLD_OUT, \*CHLD_IN, 'watchman -j') + my $pid = open2(\*CHLD_OUT, \*CHLD_IN, 'watchman -j --no-pretty') or die "open2() failed: $!\n" . "Falling back to scanning...\n";
-- 2.15.0.rc1.413.g76aedb451
Ben PeartRe: [PATCH v2 2/4] fsmonitor: Don't bother pretty-printing JSON from watchman
On 10/25/2017 9:31 PM, Alex Vandiver wrote:
Show 5 quoted lines
> This provides modest performance savings. Benchmarking with the > following program, with and without `--no-pretty`, we find savings of > 23% (0.316s -> 0.242s) in the git repository, and savings of 8% (5.24s > -> 4.86s) on a large repository with 580k files in the working copy. >
Given this patch series is all about speed, it's good to see *any* wins - especially those that don't impact functionality at all. The performance win of --no-pretty is greater than I expected.
Show 44 quoted lines
> #!/usr/bin/perl
>
> use strict;
> use warnings;
> use IPC::Open2;
>
> my $pid = open2(\*CHLD_OUT, \*CHLD_IN, "watchman -j @ARGV")
> or die "open2() failed: $!\n" .
> "Falling back to scanning...\n";
>
> my $query = qq|["query", "$ENV{PWD}", {}]|;
>
> print CHLD_IN $query;
> close CHLD_IN;
> my $response = do {local $/; <CHLD_OUT>};
>
> my $json_pkg;
> eval {
> require JSON::XSomething;
> $json_pkg = "JSON::XSomething";
> 1;
> } or do {
> require JSON::PP;
> $json_pkg = "JSON::PP";
> };
>
> my $o = $json_pkg->new->utf8->decode($response);
>
> Signed-off-by: Alex Vandiver <alexmv@dropbox.com>
> ---
> t/t7519/fsmonitor-watchman | 2 +-
> templates/hooks--fsmonitor-watchman.sample | 2 +-
> 2 files changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/t/t7519/fsmonitor-watchman b/t/t7519/fsmonitor-watchman
> index a3e30bf54..79f24325c 100755
> --- a/t/t7519/fsmonitor-watchman
> +++ b/t/t7519/fsmonitor-watchman
> @@ -50,7 +50,7 @@ launch_watchman();
>
> sub launch_watchman {
>
> - my $pid = open2(\*CHLD_OUT, \*CHLD_IN, 'watchman -j')
> + my $pid = open2(\*CHLD_OUT, \*CHLD_IN, 'watchman -j --no-pretty')Since this is a test script performance isn't critical. This version of the integration script logs the response to a file in .git/watchman-response.json and is much more human readable without the "--no-pretty." As such, I'd leave this one pretty.
Show 13 quoted lines
> or die "open2() failed: $!\n" .
> "Falling back to scanning...\n";
>
> diff --git a/templates/hooks--fsmonitor-watchman.sample b/templates/hooks--fsmonitor-watchman.sample
> index 9eba8a740..9a082f278 100755
> --- a/templates/hooks--fsmonitor-watchman.sample
> +++ b/templates/hooks--fsmonitor-watchman.sample
> @@ -49,7 +49,7 @@ launch_watchman();
>
> sub launch_watchman {
>
> - my $pid = open2(\*CHLD_OUT, \*CHLD_IN, 'watchman -j')
> + my $pid = open2(\*CHLD_OUT, \*CHLD_IN, 'watchman -j --no-pretty')No human will see this response so the faster --no-pretty option makes sense.
> or die "open2() failed: $!\n" . > "Falling back to scanning...\n"; > >
Ben PeartRe: [PATCH v2 2/4] fsmonitor: Don't bother pretty-printing JSON from watchman
On 10/26/2017 4:05 PM, Ben Peart wrote:
Show 60 quoted lines
>
>
> On 10/25/2017 9:31 PM, Alex Vandiver wrote:
>> This provides modest performance savings. Benchmarking with the
>> following program, with and without `--no-pretty`, we find savings of
>> 23% (0.316s -> 0.242s) in the git repository, and savings of 8% (5.24s
>> -> 4.86s) on a large repository with 580k files in the working copy.
>>
>
> Given this patch series is all about speed, it's good to see *any* wins
> - especially those that don't impact functionality at all. The
> performance win of --no-pretty is greater than I expected.
>
>> #!/usr/bin/perl
>>
>> use strict;
>> use warnings;
>> use IPC::Open2;
>>
>> my $pid = open2(\*CHLD_OUT, \*CHLD_IN, "watchman -j @ARGV")
>> or die "open2() failed: $!\n" .
>> "Falling back to scanning...\n";
>>
>> my $query = qq|["query", "$ENV{PWD}", {}]|;
>>
>> print CHLD_IN $query;
>> close CHLD_IN;
>> my $response = do {local $/; <CHLD_OUT>};
>>
>> my $json_pkg;
>> eval {
>> require JSON::XSomething;
>> $json_pkg = "JSON::XSomething";
>> 1;
>> } or do {
>> require JSON::PP;
>> $json_pkg = "JSON::PP";
>> };
>>
>> my $o = $json_pkg->new->utf8->decode($response);
>>
>> Signed-off-by: Alex Vandiver <alexmv@dropbox.com>
>> ---
>> t/t7519/fsmonitor-watchman | 2 +-
>> templates/hooks--fsmonitor-watchman.sample | 2 +-
>> 2 files changed, 2 insertions(+), 2 deletions(-)
>>
>> diff --git a/t/t7519/fsmonitor-watchman b/t/t7519/fsmonitor-watchman
>> index a3e30bf54..79f24325c 100755
>> --- a/t/t7519/fsmonitor-watchman
>> +++ b/t/t7519/fsmonitor-watchman
>> @@ -50,7 +50,7 @@ launch_watchman();
>> sub launch_watchman {
>> - my $pid = open2(\*CHLD_OUT, \*CHLD_IN, 'watchman -j')
>> + my $pid = open2(\*CHLD_OUT, \*CHLD_IN, 'watchman -j --no-pretty')
>
> Since this is a test script performance isn't critical. This version of
> the integration script logs the response to a file in
> .git/watchman-response.json and is much more human readable without the
> "--no-pretty." As such, I'd leave this one pretty.I didn't see anything (including this) worth another roll so only address it if something else comes up.
Show 19 quoted lines
>
>> or die "open2() failed: $!\n" .
>> "Falling back to scanning...\n";
>> diff --git a/templates/hooks--fsmonitor-watchman.sample
>> b/templates/hooks--fsmonitor-watchman.sample
>> index 9eba8a740..9a082f278 100755
>> --- a/templates/hooks--fsmonitor-watchman.sample
>> +++ b/templates/hooks--fsmonitor-watchman.sample
>> @@ -49,7 +49,7 @@ launch_watchman();
>> sub launch_watchman {
>> - my $pid = open2(\*CHLD_OUT, \*CHLD_IN, 'watchman -j')
>> + my $pid = open2(\*CHLD_OUT, \*CHLD_IN, 'watchman -j --no-pretty')
>
> No human will see this response so the faster --no-pretty option makes
> sense.
>
>> or die "open2() failed: $!\n" .
>> "Falling back to scanning...\n";
>>Re: [PATCH v2 2/4] fsmonitor: Don't bother pretty-printing JSON from watchman
On Thu, 26 Oct 2017, Ben Peart wrote:
Show 14 quoted lines
> On 10/25/2017 9:31 PM, Alex Vandiver wrote:
> > diff --git a/t/t7519/fsmonitor-watchman b/t/t7519/fsmonitor-watchman
> > index a3e30bf54..79f24325c 100755
> > --- a/t/t7519/fsmonitor-watchman
> > +++ b/t/t7519/fsmonitor-watchman
> > @@ -50,7 +50,7 @@ launch_watchman();
> > sub launch_watchman {
> > - my $pid = open2(\*CHLD_OUT, \*CHLD_IN, 'watchman -j')
> > + my $pid = open2(\*CHLD_OUT, \*CHLD_IN, 'watchman -j --no-pretty')
>
> Since this is a test script performance isn't critical. This version of the
> integration script logs the response to a file in .git/watchman-response.json
> and is much more human readable without the "--no-pretty." As such, I'd leave
> this one pretty.This would be the first delta between the test file and the template file. It seems quite important to me to attempt to ensure that we're testing the _same_ contents that we're suggesting users set up. In fact, it makes more sense to me to just turn this into a symlink to the sample template. - Alex
Ben PeartRe: [PATCH v2 2/4] fsmonitor: Don't bother pretty-printing JSON from watchman
On 10/26/2017 5:29 PM, Alex Vandiver wrote:
Show 23 quoted lines
> On Thu, 26 Oct 2017, Ben Peart wrote:
>> On 10/25/2017 9:31 PM, Alex Vandiver wrote:
>>> diff --git a/t/t7519/fsmonitor-watchman b/t/t7519/fsmonitor-watchman
>>> index a3e30bf54..79f24325c 100755
>>> --- a/t/t7519/fsmonitor-watchman
>>> +++ b/t/t7519/fsmonitor-watchman
>>> @@ -50,7 +50,7 @@ launch_watchman();
>>> sub launch_watchman {
>>> - my $pid = open2(\*CHLD_OUT, \*CHLD_IN, 'watchman -j')
>>> + my $pid = open2(\*CHLD_OUT, \*CHLD_IN, 'watchman -j --no-pretty')
>>
>> Since this is a test script performance isn't critical. This version of the
>> integration script logs the response to a file in .git/watchman-response.json
>> and is much more human readable without the "--no-pretty." As such, I'd leave
>> this one pretty.
>
> This would be the first delta between the test file and the template
> file. It seems quite important to me to attempt to ensure that we're
> testing the _same_ contents that we're suggesting users set up. In
> fact, it makes more sense to me to just turn this into a symlink to the
> sample template.
> - Alex
> If you look closer (actually diff the two files) you will see that the test version includes logging that the sample doesn't include. I found the logging very helpful during testing of the feature and debugging Watchman issues but don't want end users to pay the performance penalty of the logging.
t/t7519/fsmonitor-watchman isn't actually used by any of the automated tests as they can't assume Watchman is available. It is just provided as a convenience to enable manual testing and debugging.
[PATCH v2 3/4] fsmonitor: Document GIT_TRACE_FSMONITOR
Signed-off-by: Alex Vandiver <alexmv@dropbox.com> --- Documentation/git.txt | 4 ++++ 1 file changed, 4 insertions(+)
Show changes to Documentation/git.txt +4 −0
diff --git a/Documentation/git.txt b/Documentation/git.txt index 1fca63634..720db196e 100644 --- a/Documentation/git.txt +++ b/Documentation/git.txt @@ -594,6 +594,10 @@ into it. Unsetting the variable, or setting it to empty, "0" or "false" (case insensitive) disables trace messages. +`GIT_TRACE_FSMONITOR`:: + Enables trace messages for the filesystem monitor extension. + See `GIT_TRACE` for available trace output options. + `GIT_TRACE_PACK_ACCESS`:: Enables trace messages for all accesses to any packs. For each access, the pack file name and an offset in the pack is
-- 2.15.0.rc1.413.g76aedb451
[PATCH v2 4/4] fsmonitor: Delay updating state until after split index is merged
If the fsmonitor extension is used in conjunction with the split index extension, the set of entries in the index when it is first loaded is only a subset of the real index. This leads to only the non-"base" index being marked as CE_FSMONITOR_VALID.
Delay the expansion of the ewah bitmap until after tweak_split_index has been called to merge in the base index as well.
The new fsmonitor_dirty is kept from being leaked by dint of being cleaned up in post_read_index_from, which is guaranteed to be called after do_read_index in read_index_from.
Signed-off-by: Alex Vandiver <alexmv@dropbox.com> --- cache.h | 1 + fsmonitor.c | 39 ++++++++++++++++++++++++--------------- 2 files changed, 25 insertions(+), 15 deletions(-)
Show changes to 2 files +25 −15
cache.h, fsmonitor.c
diff --git a/cache.h b/cache.h index 25adcf681..0a4f43ec2 100644 --- a/cache.h +++ b/cache.h @@ -348,6 +348,7 @@ struct index_state { unsigned char sha1[20]; struct untracked_cache *untracked; uint64_t fsmonitor_last_update; + struct ewah_bitmap *fsmonitor_dirty; }; extern struct index_state the_index; diff --git a/fsmonitor.c b/fsmonitor.c index 0d26ff34f..fad9c6b13 100644 --- a/fsmonitor.c +++ b/fsmonitor.c @@ -49,20 +49,7 @@ int read_fsmonitor_extension(struct index_state *istate, const void *data, ewah_free(fsmonitor_dirty); return error("failed to parse ewah bitmap reading fsmonitor index extension"); } - - if (git_config_get_fsmonitor()) { - /* Mark all entries valid */ - for (i = 0; i < istate->cache_nr; i++) - istate->cache[i]->ce_flags |= CE_FSMONITOR_VALID; - - /* Mark all previously saved entries as dirty */ - ewah_each_bit(fsmonitor_dirty, fsmonitor_ewah_callback, istate); - - /* Now mark the untracked cache for fsmonitor usage */ - if (istate->untracked) - istate->untracked->use_fsmonitor = 1; - } - ewah_free(fsmonitor_dirty); + istate->fsmonitor_dirty = fsmonitor_dirty; trace_printf_key(&trace_fsmonitor, "read fsmonitor extension successful"); return 0; @@ -239,7 +226,29 @@ void remove_fsmonitor(struct index_state *istate) void tweak_fsmonitor(struct index_state *istate) { - switch (git_config_get_fsmonitor()) { + int i; + int fsmonitor_enabled = git_config_get_fsmonitor(); + + if (istate->fsmonitor_dirty) { + if (fsmonitor_enabled) { + /* Mark all entries valid */ + for (i = 0; i < istate->cache_nr; i++) { + istate->cache[i]->ce_flags |= CE_FSMONITOR_VALID; + } + + /* Mark all previously saved entries as dirty */ + ewah_each_bit(istate->fsmonitor_dirty, fsmonitor_ewah_callback, istate); + + /* Now mark the untracked cache for fsmonitor usage */ + if (istate->untracked) + istate->untracked->use_fsmonitor = 1; + } + + ewah_free(istate->fsmonitor_dirty); + istate->fsmonitor_dirty = NULL; + } + + switch (fsmonitor_enabled) { case -1: /* keep: do nothing */ break; case 0: /* false */
-- 2.15.0.rc1.413.g76aedb451
Ben PeartRe: [PATCH v2 4/4] fsmonitor: Delay updating state until after split index is merged
On 10/25/2017 9:31 PM, Alex Vandiver wrote:
Show 64 quoted lines
> If the fsmonitor extension is used in conjunction with the split index
> extension, the set of entries in the index when it is first loaded is
> only a subset of the real index. This leads to only the non-"base"
> index being marked as CE_FSMONITOR_VALID.
>
> Delay the expansion of the ewah bitmap until after tweak_split_index
> has been called to merge in the base index as well.
>
> The new fsmonitor_dirty is kept from being leaked by dint of being
> cleaned up in post_read_index_from, which is guaranteed to be called
> after do_read_index in read_index_from.
>
> Signed-off-by: Alex Vandiver <alexmv@dropbox.com>
> ---
> cache.h | 1 +
> fsmonitor.c | 39 ++++++++++++++++++++++++---------------
> 2 files changed, 25 insertions(+), 15 deletions(-)
>
> diff --git a/cache.h b/cache.h
> index 25adcf681..0a4f43ec2 100644
> --- a/cache.h
> +++ b/cache.h
> @@ -348,6 +348,7 @@ struct index_state {
> unsigned char sha1[20];
> struct untracked_cache *untracked;
> uint64_t fsmonitor_last_update;
> + struct ewah_bitmap *fsmonitor_dirty;
> };
>
> extern struct index_state the_index;
> diff --git a/fsmonitor.c b/fsmonitor.c
> index 0d26ff34f..fad9c6b13 100644
> --- a/fsmonitor.c
> +++ b/fsmonitor.c
> @@ -49,20 +49,7 @@ int read_fsmonitor_extension(struct index_state *istate, const void *data,
> ewah_free(fsmonitor_dirty);
> return error("failed to parse ewah bitmap reading fsmonitor index extension");
> }
> -
> - if (git_config_get_fsmonitor()) {
> - /* Mark all entries valid */
> - for (i = 0; i < istate->cache_nr; i++)
> - istate->cache[i]->ce_flags |= CE_FSMONITOR_VALID;
> -
> - /* Mark all previously saved entries as dirty */
> - ewah_each_bit(fsmonitor_dirty, fsmonitor_ewah_callback, istate);
> -
> - /* Now mark the untracked cache for fsmonitor usage */
> - if (istate->untracked)
> - istate->untracked->use_fsmonitor = 1;
> - }
> - ewah_free(fsmonitor_dirty);
> + istate->fsmonitor_dirty = fsmonitor_dirty;
>
> trace_printf_key(&trace_fsmonitor, "read fsmonitor extension successful");
> return 0;
> @@ -239,7 +226,29 @@ void remove_fsmonitor(struct index_state *istate)
>
> void tweak_fsmonitor(struct index_state *istate)
> {
> - switch (git_config_get_fsmonitor()) {
> + int i;
> + int fsmonitor_enabled = git_config_get_fsmonitor();
> +The logic looks good this time. It is nice to know this will now be optimal when split index is also turned on. Thank you.
Show 24 quoted lines
> + if (istate->fsmonitor_dirty) {
> + if (fsmonitor_enabled) {
> + /* Mark all entries valid */
> + for (i = 0; i < istate->cache_nr; i++) {
> + istate->cache[i]->ce_flags |= CE_FSMONITOR_VALID;
> + }
> +
> + /* Mark all previously saved entries as dirty */
> + ewah_each_bit(istate->fsmonitor_dirty, fsmonitor_ewah_callback, istate);
> +
> + /* Now mark the untracked cache for fsmonitor usage */
> + if (istate->untracked)
> + istate->untracked->use_fsmonitor = 1;
> + }
> +
> + ewah_free(istate->fsmonitor_dirty);
> + istate->fsmonitor_dirty = NULL;
> + }
> +
> + switch (fsmonitor_enabled) {
> case -1: /* keep: do nothing */
> break;
> case 0: /* false */
> Johannes SchindelinRe: [PATCH v2 0/4] fsmonitor fixes
Hi Alex,
On Wed, 25 Oct 2017, Alex Vandiver wrote:
> Updated based on comments from Dscho and Ben. Thanks for those!
Thank you for this excellent improvement.
Ciao, Dscho