{"thread":{"id":"58091","subject":"Option to allow fsmonitor to run against repos on network file systems","startedAt":"2022-06-30T17:11:21Z","lastAt":"2022-08-08T21:58:58Z","messageCount":6,"participants":["Eric D","Jeff Hostetler","Junio C Hamano"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"458210","messageId":"CAMxJVdH+o+H56tJ4UmD8YcsNsLuutiUXpOP=euQbomBe1kLkMw@mail.gmail.com","threadId":"58091","inReplyTo":null,"subject":"Option to allow fsmonitor to run against repos on network file systems","fromName":"Eric D","fromEmail":"eric.decosta@gmail.com","sentAt":"2022-06-30T17:11:02Z","receivedAt":"2022-06-30T17:11:21Z","isPatch":false,"sender":{"key":"eric.decosta@gmail.com","avatar":null},"body":"I can appreciate the concerns expressed here:\nhttps://github.com/git/git/commit/d989b266c1a7ef47f27cec75e90f3dfefbfa0200\n\nHowever, in my environment, our file servers are very capable and have\nthe requisite support. It would be great if there was an option to\noverride this check and allow fsmonitor to operate against network\nfilesystems.\n\n-Eric\n"},{"id":"458384","messageId":"16832f8a-c582-23bb-dda9-b7b2597a42eb@jeffhostetler.com","threadId":"58091","inReplyTo":"CAMxJVdH+o+H56tJ4UmD8YcsNsLuutiUXpOP=euQbomBe1kLkMw@mail.gmail.com","subject":"Re: Option to allow fsmonitor to run against repos on network file systems","fromName":"Jeff Hostetler","fromEmail":"git@jeffhostetler.com","sentAt":"2022-07-01T13:32:37Z","receivedAt":"2022-07-01T13:32:41Z","isPatch":false,"sender":{"key":"git@jeffhostetler.com","avatar":null},"body":"\n\nOn 6/30/22 1:11 PM, Eric D wrote:\n> I can appreciate the concerns expressed here:\n> https://github.com/git/git/commit/d989b266c1a7ef47f27cec75e90f3dfefbfa0200\n> \n> However, in my environment, our file servers are very capable and have\n> the requisite support. It would be great if there was an option to\n> override this check and allow fsmonitor to operate against network\n> filesystems.\n\nYeah, I was just being cautious. I probably should have also added\nconcerns on the remote system being an actual Windows server or a\nnon-Windows host running SAMBA. There were just too many combinations\nfor me to be comfortable enabling it by default (on the initial\nrelease, at least).\n\nAlso, the ReadDirectoryChangesW() API limits the buffer size to 64k\nfor remote handles (because of protocol limitations), so there _may_\nbe more of an opportunity for dropped events on very busy remote file\nsystems. (I never saw any dropped events in my testing (without\nintentionally breaking things), but it is a possible concern, so again,\ncaution and safety...) And I do handle dropped events and force a\nresync and send the client a \"trivial\" response (so it must do a regular\nscan), so output is still correct, but slower.\n\n\nHaving said all of that, I did do lots of testing and never had an\nissue with remote drives actually working correctly, so I think it'd\nbe fine allow a config setting to optionally allow it. I just didn't\nwant to clutter up things in advance if no one actually wanted to\nuse it on remote file systems.\n\n\nI think it would be fine to have a \"fsmonitor.allowRemote\" or\n\"fsmonitor.allowWindowsRemote\" config setting and default them to false\nfor now. Or until we learn which combinations of remote mounts are\nsafe and/or problematic.\n\nJeff\n"},{"id":"458406","messageId":"xmqqmtds8ylz.fsf@gitster.g","threadId":"58091","inReplyTo":"16832f8a-c582-23bb-dda9-b7b2597a42eb@jeffhostetler.com","subject":"Re: Option to allow fsmonitor to run against repos on network file systems","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-07-01T18:41:12Z","receivedAt":"2022-07-01T18:41:20Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jeff Hostetler writes:\n\n> Having said all of that, I did do lots of testing and never had an\n> issue with remote drives actually working correctly, so I think it'd\n> be fine allow a config setting to optionally allow it. I just didn't\n> want to clutter up things in advance if no one actually wanted to\n> use it on remote file systems.\n>\n> I think it would be fine to have a \"fsmonitor.allowRemote\" or\n> \"fsmonitor.allowWindowsRemote\" config setting and default them to false\n> for now. Or until we learn which combinations of remote mounts are\n> safe and/or problematic.\n\nHow about getting rid of \"is this remote?\" check altogether (which\npresumably would simplify the logic) and make it totally up to the\nuser of the repository? fsmonitor.disableInRepository that is set\nin ~/.gitignore and lists the paths to the repositories (like\nsafe.directory does), for which fsmonitor gets disabled, may be a\nhandy mechanism to set up the default (and it can be re-enabled with\nper-repository core.fsmonitor).\n\n\n"},{"id":"458409","messageId":"CAMxJVdG5OCgWMN+0aymdVTorrkeViGNy=f49fu7GJE6trwyoWw@mail.gmail.com","threadId":"58091","inReplyTo":"xmqqmtds8ylz.fsf@gitster.g","subject":"Re: Option to allow fsmonitor to run against repos on network file systems","fromName":"Eric D","fromEmail":"eric.decosta@gmail.com","sentAt":"2022-07-01T19:15:02Z","receivedAt":"2022-07-01T19:15:16Z","isPatch":false,"sender":{"key":"eric.decosta@gmail.com","avatar":null},"body":"One more possibility:\n\nLeave the check, but make it a warning if there are still concerns\nabout running fsmonitor against network file systems. Maybe also\nprovide an option to suppress the warning? Not that much different\nfrom having \"fsmonitor.allowRemote\" I suppose other than by default\nfsmonitor would \"just work\" for network mounts.\n\nOn Fri, Jul 1, 2022 at 2:41 PM Junio C Hamano wrote:\n>\n> Jeff Hostetler writes:\n>\n> > Having said all of that, I did do lots of testing and never had an\n> > issue with remote drives actually working correctly, so I think it'd\n> > be fine allow a config setting to optionally allow it. I just didn't\n> > want to clutter up things in advance if no one actually wanted to\n> > use it on remote file systems.\n> >\n> > I think it would be fine to have a \"fsmonitor.allowRemote\" or\n> > \"fsmonitor.allowWindowsRemote\" config setting and default them to false\n> > for now. Or until we learn which combinations of remote mounts are\n> > safe and/or problematic.\n>\n> How about getting rid of \"is this remote?\" check altogether (which\n> presumably would simplify the logic) and make it totally up to the\n> user of the repository? fsmonitor.disableInRepository that is set\n> in ~/.gitignore and lists the paths to the repositories (like\n> safe.directory does), for which fsmonitor gets disabled, may be a\n> handy mechanism to set up the default (and it can be re-enabled with\n> per-repository core.fsmonitor).\n>\n>\n"},{"id":"460368","messageId":"CAMxJVdH9EJCtg_J=HQtH9ghSWS1EOUqmU8rdAVTP+mL3gH6h=w@mail.gmail.com","threadId":"58091","inReplyTo":"CAMxJVdG5OCgWMN+0aymdVTorrkeViGNy=f49fu7GJE6trwyoWw@mail.gmail.com","subject":"Re: Option to allow fsmonitor to run against repos on network file systems","fromName":"Eric D","fromEmail":"eric.decosta@gmail.com","sentAt":"2022-08-01T18:35:28Z","receivedAt":"2022-08-01T18:35:51Z","isPatch":false,"sender":{"key":"eric.decosta@gmail.com","avatar":null},"body":"On Fri, Jul 1, 2022 at 3:15 PM Eric D wrote:\n>\n> One more possibility:\n>\n> Leave the check, but make it a warning if there are still concerns\n> about running fsmonitor against network file systems. Maybe also\n> provide an option to suppress the warning? Not that much different\n> from having \"fsmonitor.allowRemote\" I suppose other than by default\n> fsmonitor would \"just work\" for network mounts.\n>\n> On Fri, Jul 1, 2022 at 2:41 PM Junio C Hamano wrote:\n> >\n> > Jeff Hostetler writes:\n> >\n> > > Having said all of that, I did do lots of testing and never had an\n> > > issue with remote drives actually working correctly, so I think it'd\n> > > be fine allow a config setting to optionally allow it. I just didn't\n> > > want to clutter up things in advance if no one actually wanted to\n> > > use it on remote file systems.\n> > >\n> > > I think it would be fine to have a \"fsmonitor.allowRemote\" or\n> > > \"fsmonitor.allowWindowsRemote\" config setting and default them to false\n> > > for now. Or until we learn which combinations of remote mounts are\n> > > safe and/or problematic.\n> >\n> > How about getting rid of \"is this remote?\" check altogether (which\n> > presumably would simplify the logic) and make it totally up to the\n> > user of the repository? fsmonitor.disableInRepository that is set\n> > in ~/.gitignore and lists the paths to the repositories (like\n> > safe.directory does), for which fsmonitor gets disabled, may be a\n> > handy mechanism to set up the default (and it can be re-enabled with\n> > per-repository core.fsmonitor).\n> >\n> >\n\nAfter modifying the code and playing with it for a bit, I think simply\nremoving the check (but still being able to report on it if tracing is\nenabled) is the way to go. Keep it simple and if new use cases arrive,\ndeal with them then.\n\n-Eric\n"},{"id":"460854","messageId":"CAMxJVdH+_xJc0VmRkaepww+gQH_==7wf3F8tX4hi99gWVTXCnQ@mail.gmail.com","threadId":"58091","inReplyTo":"16832f8a-c582-23bb-dda9-b7b2597a42eb@jeffhostetler.com","subject":"Re: Option to allow fsmonitor to run against repos on network file systems","fromName":"Eric D","fromEmail":"eric.decosta@gmail.com","sentAt":"2022-08-08T21:58:40Z","receivedAt":"2022-08-08T21:58:58Z","isPatch":false,"sender":{"key":"eric.decosta@gmail.com","avatar":null},"body":"On Fri, Jul 1, 2022 at 9:32 AM Jeff Hostetler wrote:\n>\n>\n>\n> On 6/30/22 1:11 PM, Eric D wrote:\n> > I can appreciate the concerns expressed here:\n> > https://github.com/git/git/commit/d989b266c1a7ef47f27cec75e90f3dfefbfa0200\n> >\n> > However, in my environment, our file servers are very capable and have\n> > the requisite support. It would be great if there was an option to\n> > override this check and allow fsmonitor to operate against network\n> > filesystems.\n>\n> Yeah, I was just being cautious. I probably should have also added\n> concerns on the remote system being an actual Windows server or a\n> non-Windows host running SAMBA. There were just too many combinations\n> for me to be comfortable enabling it by default (on the initial\n> release, at least).\n>\n> Also, the ReadDirectoryChangesW() API limits the buffer size to 64k\n> for remote handles (because of protocol limitations), so there _may_\n> be more of an opportunity for dropped events on very busy remote file\n> systems. (I never saw any dropped events in my testing (without\n> intentionally breaking things), but it is a possible concern, so again,\n> caution and safety...) And I do handle dropped events and force a\n> resync and send the client a \"trivial\" response (so it must do a regular\n> scan), so output is still correct, but slower.\n>\n>\n> Having said all of that, I did do lots of testing and never had an\n> issue with remote drives actually working correctly, so I think it'd\n> be fine allow a config setting to optionally allow it. I just didn't\n> want to clutter up things in advance if no one actually wanted to\n> use it on remote file systems.\n>\n>\n> I think it would be fine to have a \"fsmonitor.allowRemote\" or\n> \"fsmonitor.allowWindowsRemote\" config setting and default them to false\n> for now. Or until we learn which combinations of remote mounts are\n> safe and/or problematic.\n>\n> Jeff\n\nOK, based on this and other conversations, I have implemented the following:\n\n1. Introduced a new config setting, \"fsmonitor.allowRemote\"\n\"fsmonitor.allowRemote\" has a default value of false. Setting it to\ntrue overrides\nfsmonitor's default behavior of rejecting network-mounted repos.\n\n2. Restricted allowing remote repos to Windows clients using SMB\nIf the client is not using SMB and using a network path, then\nfsmonitor will reject\nthe repo path regardless of the value of \"fsmonitor.allowRemote\"\n\n-Eric\n"}]}