{"thread":{"id":"62136","subject":"Permission issue in Git in DrvFs-mounted network drives","startedAt":"2024-09-18T16:16:39Z","lastAt":"2024-09-21T11:20:52Z","messageCount":9,"participants":["Marcos Del Sol Vives","rsbecker@nexbridge.com","brian m. carlson","Junio C Hamano"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"503004","messageId":"19205ebb4b9.c2a2da5a2387912.3559118454287459572@orca.pet","threadId":"62136","inReplyTo":null,"subject":"Permission issue in Git in DrvFs-mounted network drives","fromName":"Marcos Del Sol Vives","fromEmail":"marcos@orca.pet","sentAt":"2024-09-18T16:16:26Z","receivedAt":"2024-09-18T16:16:39Z","isPatch":false,"sender":{"key":"marcos@orca.pet","avatar":null},"body":"Under WSL1 (Windows Subsystem for Linux), when using a network share\nmounted via DrvFs, Git fails to add any files to a new or an existing\nrepository.\n\nThe reason is that Git tries to open a temporary file as with RW\npermissions but mode 0444, which causes WSL1 (or Samba, unsure who's here\nto blame) to create first an file empty with the read-only DOS attribute set\nthat prevents any writes, and then actually trying to opening it in write\nmode, which of course fails.\n\nSeems to be a pretty common issue that nobody has yet reported officially,\njudging by the amount of posts on Stackoverflow, impacting not only WSL\nbut also CIFS under Linux (hence why sending to this mailing list and not\nthe Windows-specific one):\n\n - https://superuser.com/questions/681196/debugging-git-repo-permissions-on-samba-share\n - https://superuser.com/questions/1450094/git-on-wsl-commands-fail-despite-permissions-seeming-fine\n - https://superuser.com/questions/1491499/use-git-on-a-shared-drive-within-wsl\n\nAs a workaround, opening the file with permissions 0600 and then using a\nfchmod with the final desired mode works, which is a very small change that\nshould cause no issues under neither real Linux nor WSL:\n\n--- git-2.39.5.orig/wrapper.c\n+++ git-2.39.5/wrapper.c\n@@ -484,9 +484,11 @@ int git_mkstemps_mode(char *pattern, int\n \t\t\tv /= num_letters;\n \t\t}\n \n-\t\tfd = open(pattern, O_CREAT | O_EXCL | O_RDWR, mode);\n-\t\tif (fd >= 0)\n+\t\tfd = open(pattern, O_CREAT | O_EXCL | O_RDWR, 0600);\n+\t\tif (fd >= 0) {\n+\t\t\tfchmod(fd, mode);\n \t\t\treturn fd;\n+\t\t}\n \t\t/*\n \t\t * Fatal error (EPERM, ENOSPC etc).\n \t\t * It doesn't make sense to loop.\n\nThe WSL team at Microsoft has been already informed as well:\nhttps://github.com/microsoft/WSL/issues/12051\n\n"},{"id":"503005","messageId":"03ed01db09e8$080cf090$1826d1b0$@nexbridge.com","threadId":"62136","inReplyTo":"19205ebb4b9.c2a2da5a2387912.3559118454287459572@orca.pet","subject":"RE: Permission issue in Git in DrvFs-mounted network drives","fromName":"","fromEmail":"rsbecker@nexbridge.com","sentAt":"2024-09-18T16:30:05Z","receivedAt":"2024-09-18T16:30:21Z","isPatch":false,"sender":{"key":"randall.becker@nexbridge.ca","avatar":"https://avatars.githubusercontent.com/u/28956764?v=4"},"body":"On September 18, 2024 12:16 PM, Marcos Del Sol Vives wrote:\n>Under WSL1 (Windows Subsystem for Linux), when using a network share\n>mounted via DrvFs, Git fails to add any files to a new or an existing repository.\n>\n>The reason is that Git tries to open a temporary file as with RW permissions but\n>mode 0444, which causes WSL1 (or Samba, unsure who's here to blame) to create\n>first an file empty with the read-only DOS attribute set that prevents any writes,\n>and then actually trying to opening it in write mode, which of course fails.\n>\n>Seems to be a pretty common issue that nobody has yet reported officially, judging\n>by the amount of posts on Stackoverflow, impacting not only WSL but also CIFS\n>under Linux (hence why sending to this mailing list and not the Windows-specific\n>one):\n>\n> - https://superuser.com/questions/681196/debugging-git-repo-permissions-on-\n>samba-share\n> - https://superuser.com/questions/1450094/git-on-wsl-commands-fail-despite-\n>permissions-seeming-fine\n> - https://superuser.com/questions/1491499/use-git-on-a-shared-drive-within-\n>wsl\n>\n>As a workaround, opening the file with permissions 0600 and then using a fchmod\n>with the final desired mode works, which is a very small change that should cause\n>no issues under neither real Linux nor WSL:\n>\n>--- git-2.39.5.orig/wrapper.c\n>+++ git-2.39.5/wrapper.c\n>@@ -484,9 +484,11 @@ int git_mkstemps_mode(char *pattern, int\n> \t\t\tv /= num_letters;\n> \t\t}\n>\n>-\t\tfd = open(pattern, O_CREAT | O_EXCL | O_RDWR, mode);\n>-\t\tif (fd >= 0)\n>+\t\tfd = open(pattern, O_CREAT | O_EXCL | O_RDWR, 0600);\n>+\t\tif (fd >= 0) {\n>+\t\t\tfchmod(fd, mode);\n> \t\t\treturn fd;\n>+\t\t}\n\nI am not certain this is either necessary or important to platforms other than Windows.\nThe /tmp directory is often, and properly set with the sticky bit +t. This ensures that only\nThe creating user has access to the temp file regardless of create ownership or security.\nI would prefer that this be put into a compat layer rather than made general change.\n\n> \t\t/*\n> \t\t * Fatal error (EPERM, ENOSPC etc).\n> \t\t * It doesn't make sense to loop.\n>\n>The WSL team at Microsoft has been already informed as well:\n>https://github.com/microsoft/WSL/issues/12051\n\n"},{"id":"503007","messageId":"19206003b1a.120df84912389761.6123103044544344593@orca.pet","threadId":"62136","inReplyTo":"03ed01db09e8$080cf090$1826d1b0$@nexbridge.com","subject":"RE: Permission issue in Git in DrvFs-mounted network drives","fromName":"Marcos Del Sol Vives","fromEmail":"marcos@orca.pet","sentAt":"2024-09-18T16:38:51Z","receivedAt":"2024-09-18T16:38:59Z","isPatch":false,"sender":{"key":"marcos@orca.pet","avatar":null},"body":"\n\n\n ---- El Wed, 18 Sep 2024 18:30:05 +0200,    escribió ---- \n > On September 18, 2024 12:16 PM, Marcos Del Sol Vives wrote:\n > >Under WSL1 (Windows Subsystem for Linux), when using a network share\n > >mounted via DrvFs, Git fails to add any files to a new or an existing repository.\n > >\n > >The reason is that Git tries to open a temporary file as with RW permissions but\n > >mode 0444, which causes WSL1 (or Samba, unsure who's here to blame) to create\n > >first an file empty with the read-only DOS attribute set that prevents any writes,\n > >and then actually trying to opening it in write mode, which of course fails.\n > >\n > >Seems to be a pretty common issue that nobody has yet reported officially, judging\n > >by the amount of posts on Stackoverflow, impacting not only WSL but also CIFS\n > >under Linux (hence why sending to this mailing list and not the Windows-specific\n > >one):\n > >\n > > - https://superuser.com/questions/681196/debugging-git-repo-permissions-on-\n > >samba-share\n > > - https://superuser.com/questions/1450094/git-on-wsl-commands-fail-despite-\n > >permissions-seeming-fine\n > > - https://superuser.com/questions/1491499/use-git-on-a-shared-drive-within-\n > >wsl\n > >\n > >As a workaround, opening the file with permissions 0600 and then using a fchmod\n > >with the final desired mode works, which is a very small change that should cause\n > >no issues under neither real Linux nor WSL:\n > >\n > >--- git-2.39.5.orig/wrapper.c\n > >+++ git-2.39.5/wrapper.c\n > >@@ -484,9 +484,11 @@ int git_mkstemps_mode(char *pattern, int\n > >             v /= num_letters;\n > >         }\n > >\n > >-        fd = open(pattern, O_CREAT | O_EXCL | O_RDWR, mode);\n > >-        if (fd >= 0)\n > >+        fd = open(pattern, O_CREAT | O_EXCL | O_RDWR, 0600);\n > >+        if (fd >= 0) {\n > >+            fchmod(fd, mode);\n > >             return fd;\n > >+        }\n > \n > I am not certain this is either necessary or important to platforms other than Windows.\n > The /tmp directory is often, and properly set with the sticky bit +t. This ensures that only\n > The creating user has access to the temp file regardless of create ownership or security.\n > I would prefer that this be put into a compat layer rather than made general change.\n \nThe temporary file is created in .git/objects/XX/tmp_obj_YYYY, not in /tmp.\nAs shown in the first StackExchange link, this also impacts Linux machines.\n\n > >         /*\n > >          * Fatal error (EPERM, ENOSPC etc).\n > >          * It doesn't make sense to loop.\n > >\n > >The WSL team at Microsoft has been already informed as well:\n > >https://github.com/microsoft/WSL/issues/12051\n > \n > \n"},{"id":"503140","messageId":"Zu2aHdaw_oDv_dp7@tapette.crustytoothpaste.net","threadId":"62136","inReplyTo":"19205ebb4b9.c2a2da5a2387912.3559118454287459572@orca.pet","subject":"Re: Permission issue in Git in DrvFs-mounted network drives","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2024-09-20T15:51:57Z","receivedAt":"2024-09-20T15:52:07Z","isPatch":false,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On 2024-09-18 at 16:16:26, Marcos Del Sol Vives wrote:\n> Under WSL1 (Windows Subsystem for Linux), when using a network share\n> mounted via DrvFs, Git fails to add any files to a new or an existing\n> repository.\n> \n> The reason is that Git tries to open a temporary file as with RW\n> permissions but mode 0444, which causes WSL1 (or Samba, unsure who's here\n> to blame) to create first an file empty with the read-only DOS attribute set\n> that prevents any writes, and then actually trying to opening it in write\n> mode, which of course fails.\n> \n> Seems to be a pretty common issue that nobody has yet reported officially,\n> judging by the amount of posts on Stackoverflow, impacting not only WSL\n> but also CIFS under Linux (hence why sending to this mailing list and not\n> the Windows-specific one):\n> \n>  - https://superuser.com/questions/681196/debugging-git-repo-permissions-on-samba-share\n>  - https://superuser.com/questions/1450094/git-on-wsl-commands-fail-despite-permissions-seeming-fine\n>  - https://superuser.com/questions/1491499/use-git-on-a-shared-drive-within-wsl\n> \n> As a workaround, opening the file with permissions 0600 and then using a\n> fchmod with the final desired mode works, which is a very small change that\n> should cause no issues under neither real Linux nor WSL:\n\nThis behaviour also occurs on some broken NFS mounts.  It is, however,\nexplicitly permitted by POSIX to open a file 0444 with O_WRONLY or\nO_RDWR, and we've traditionally not fixed it for that reason.  I would\nsay that WSL1 behaviour is incorrect here.  Here's what POSIX says[0]:\n\n  The argument following the oflag argument does not affect whether the\n  file is open for reading, writing, or for both.\n\nThat is, POSIX forbids failing to open a file for writing because the\npermissions are 0444, and as such this is not POSIX compliant.  Note\nthat the operation must also be atomic; that is, the server can't\nemulate this by setting the permissions to 0600 and then doing an fchmod\ninternally unless nobody can ever detect the intermediate state.\n\nI think we also probably have a few other codepaths that are affected in\nthe same way and this is maybe just the most noticeable.\n\nOther folks may feel differently about fixing this, but I would suggest\nagainst it.\n\n> The WSL team at Microsoft has been already informed as well:\n> https://github.com/microsoft/WSL/issues/12051\n\nI've provided some comments in the issue there.\n\n[0] https://pubs.opengroup.org/onlinepubs/9799919799/functions/open.html\n-- \nbrian m. carlson (they/them or he/him)\nToronto, Ontario, CA\n"},{"id":"503153","messageId":"xmqqsetugpip.fsf@gitster.g","threadId":"62136","inReplyTo":"Zu2aHdaw_oDv_dp7@tapette.crustytoothpaste.net","subject":"Re: Permission issue in Git in DrvFs-mounted network drives","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2024-09-20T18:15:42Z","receivedAt":"2024-09-20T18:15:45Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"brian m. carlson\" <sandals@crustytoothpaste.net> writes:\n\n> Other folks may feel differently about fixing this, but I would suggest\n> against it.\n>\n>> The WSL team at Microsoft has been already informed as well:\n>> https://github.com/microsoft/WSL/issues/12051\n>\n> I've provided some comments in the issue there.\n>\n> [0] https://pubs.opengroup.org/onlinepubs/9799919799/functions/open.html\n\nIt depends on how hard it would be for filesystem providers to fix\ntheir ware, but (1) leaving our code to require open semantics of\nPOSIX may serve as an incentive to encourage them to do so, and (2)\nI am not fundamentally opposed to have an option to help users of\nsuch flawed filesystem, provided if such a change is done cleanly.\n\nThanks.\n"},{"id":"503155","messageId":"19210b887c6.f59622352625372.1022723129771458212@orca.pet","threadId":"62136","inReplyTo":"xmqqsetugpip.fsf@gitster.g","subject":"Re: Permission issue in Git in DrvFs-mounted network drives","fromName":"Marcos Del Sol Vives","fromEmail":"marcos@orca.pet","sentAt":"2024-09-20T18:36:21Z","receivedAt":"2024-09-20T18:36:47Z","isPatch":false,"sender":{"key":"marcos@orca.pet","avatar":null},"body":"---- El Fri, 20 Sep 2024 17:51:57 +0200,  brian m. carlson  escribió ---- \n> That is, POSIX forbids failing to open a file for writing because the\n> permissions are 0444, and as such this is not POSIX compliant.  Note\n> that the operation must also be atomic; that is, the server can't\n> emulate this by setting the permissions to 0600 and then doing an fchmod\n> internally unless nobody can ever detect the intermediate state.\n\nI'm not sure why atomicity would be an issue here?\n\nThe file already exists between the open and the fchmod, so another\nconcurrent thread would be unable to open it regardless due to O_EXCL and\nO_CREAT.\n\nIn fact, Git is *already* using this method of opening with 0600 for\nwriting and then chmoding to 0444 when writing pack files: in\nbuiltin/index-pack.c, method open_pack_file opens with 0600 to write, and\nthen chmods to 444 in rename_tmp_packfile.\n\n> Other folks may feel differently about fixing this, but I would suggest\n> against it.\n\nI understand that it's a workaround required due to non-POSIX compliance,\nbut considering it should have no impact on any other OS and will actually\nas you said make buggy NFS mount work, IMHO it'd be still nice to have\nthe changes made.\n\n---- El Fri, 20 Sep 2024 20:15:42 +0200,  Junio C Hamano  escribió ---- \n> It depends on how hard it would be for filesystem providers to fix\n> their ware, but (1) leaving our code to require open semantics of\n> POSIX may serve as an incentive to encourage them to do so, and (2)\n> I am not fundamentally opposed to have an option to help users of\n> such flawed filesystem, provided if such a change is done cleanly.\n\nYou mean adding a Git setting to, for example, disable using 444 mode\naltogether? Or do it in two steps (open + fchmod)?\n\n"},{"id":"503165","messageId":"Zu3ec1mDj9JD1Bbj@tapette.crustytoothpaste.net","threadId":"62136","inReplyTo":"19210b887c6.f59622352625372.1022723129771458212@orca.pet","subject":"Re: Permission issue in Git in DrvFs-mounted network drives","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2024-09-20T20:43:31Z","receivedAt":"2024-09-20T20:43:36Z","isPatch":false,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On 2024-09-20 at 18:36:21, Marcos Del Sol Vives wrote:\n> ---- El Fri, 20 Sep 2024 17:51:57 +0200,  brian m. carlson  escribió ----\n> > Other folks may feel differently about fixing this, but I would suggest\n> > against it.\n> \n> I understand that it's a workaround required due to non-POSIX compliance,\n> but considering it should have no impact on any other OS and will actually\n> as you said make buggy NFS mount work, IMHO it'd be still nice to have\n> the changes made.\n\nGit could, in theory, accept a patch here.  However, you're also going\nto have lots of other software that breaks in this case, not just Git.\nSo it's less useful to patch Git and hundreds of other packages on Linux\ndistributions that have relied on the POSIX standard and more useful to\nfix your OS (or maybe switch to WSL2, if that doesn't have the problem).\nMost Linux distros will generally not be interested in fixing this class\nof problem, in my experience.\n\nIn addition, chmod doesn't always work under WSL.  I believe it _does_\nwork if the drive is mounted with metadata, but some people don't have\nthat enabled and I don't know if it works for all drives.  For those\npeople, the current code will work, since it doesn't call chmod or\nfchmod, but it will fail with your patch.\n-- \nbrian m. carlson (they/them or he/him)\nToronto, Ontario, CA\n"},{"id":"503172","messageId":"1921197742f.d7b8c9992630384.1644253106538527208@orca.pet","threadId":"62136","inReplyTo":"Zu3ec1mDj9JD1Bbj@tapette.crustytoothpaste.net","subject":"Re: Permission issue in Git in DrvFs-mounted network drives","fromName":"Marcos Del Sol Vives","fromEmail":"marcos@orca.pet","sentAt":"2024-09-20T22:39:51Z","receivedAt":"2024-09-20T22:40:14Z","isPatch":false,"sender":{"key":"marcos@orca.pet","avatar":null},"body":"---- El Fri, 20 Sep 2024 22:43:31 +0200,  brian m. carlson  escribió ---- \n> Git could, in theory, accept a patch here.  However, you're also going\n> to have lots of other software that breaks in this case, not just Git.\n> So it's less useful to patch Git and hundreds of other packages on Linux\n> distributions that have relied on the POSIX standard and more useful to\n> fix your OS (or maybe switch to WSL2, if that doesn't have the problem).\n> Most Linux distros will generally not be interested in fixing this class\n> of problem, in my experience.\n\nThere is not a significant amount of software that opens files for writing\nin read-only mode. Despite using WSL extensively for development, so far\nGit has been the only that failed to work.\n\nWSL2 has poor support for accessing local devices (such as serial ports,\nsomething really useful for embedded development like the Proxmark project\nwhere I contribute to) or pipes (which I use to sign commits using a\nTPM-backed SSH key), so it's not really an option.\n\n> In addition, chmod doesn't always work under WSL.  I believe it _does_\n> work if the drive is mounted with metadata, but some people don't have\n> that enabled and I don't know if it works for all drives.  For those\n> people, the current code will work, since it doesn't call chmod or\n> fchmod, but it will fail with your patch.\n\nI can guarantee you it'll work under WSL, even without metadata, because\nI've patched Git on my WSL1 Debian machine by doing the aforementioned\ntrick and it totally works. Metadata is not even supported on anything but\nlocal NTFS drives.\n\nWSL1, when mounted without metadata, chmod and fchmod calls will not fail\nbut would instead just ignore the call, *except* when setting mode 444,\n400 or any other read-only mode where it will set the FAT/NTFS read-only\nattribute:\n\nmarcos@desk:/mnt/nas/marcos$ mount | grep nas\n\\\\192.168.0.245\\marcos on /mnt/nas/marcos type drvfs (rw,relatime,uid=1000,gid=1000,case=off)\nmarcos@desk:/mnt/nas/marcos$ touch readonly\nmarcos@desk:/mnt/nas/marcos$ ls -l readonly\n-rwxrwxrwx 1 marcos marcos 0 Sep 21 00:33 readonly\nmarcos@desk:/mnt/nas/marcos$ chmod 444 readonly\nmarcos@desk:/mnt/nas/marcos$ ls -l readonly\n-r-xr-xr-x 1 marcos marcos 0 Sep 21 00:33 readonly\n\nThat is exactly what is causing the issue - two SMB requests are being\nissued: one to create the file in read-only mode, and then another to\nopen it.\n\n"},{"id":"503187","messageId":"Zu6sDlL65s6NNS_o@tapette.crustytoothpaste.net","threadId":"62136","inReplyTo":"1921197742f.d7b8c9992630384.1644253106538527208@orca.pet","subject":"Re: Permission issue in Git in DrvFs-mounted network drives","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2024-09-21T11:20:46Z","receivedAt":"2024-09-21T11:20:52Z","isPatch":false,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On 2024-09-20 at 22:39:51, Marcos Del Sol Vives wrote:\n> ---- El Fri, 20 Sep 2024 22:43:31 +0200,  brian m. carlson  escribió ---- \n> There is not a significant amount of software that opens files for writing\n> in read-only mode. Despite using WSL extensively for development, so far\n> Git has been the only that failed to work.\n\nI have been using Debian for many years and have contributed to several\nof the packages.  I assure you that using what Git is doing is not, in\nfact, uncommon.  It may be that you haven't run into it, but it is\ndefinitely not rare.\n\nFor example, I just found an example of doing exactly the same thing in\nzsh, which is a major project that people use across distros.  There's\napparently another in the elm MUA.  I found yet another one in some m4\n(autoconf) files used in building Emacs.  And there are a lot of other\nexamples you can find by searching GitHub's code search for \"open\nO_WRONLY 0444\".\n\nPerhaps you don't use any of that software, but they're all substantial\nprojects that are in common use across Linux distros and Unix systems in\ngeneral.  I am a zsh user, for instance.\n\n> > In addition, chmod doesn't always work under WSL.  I believe it _does_\n> > work if the drive is mounted with metadata, but some people don't have\n> > that enabled and I don't know if it works for all drives.  For those\n> > people, the current code will work, since it doesn't call chmod or\n> > fchmod, but it will fail with your patch.\n> \n> I can guarantee you it'll work under WSL, even without metadata, because\n> I've patched Git on my WSL1 Debian machine by doing the aforementioned\n> trick and it totally works. Metadata is not even supported on anything but\n> local NTFS drives.\n\nI have actually answered questions about chmod failing to work in some\ncases on WSL, which causes operations to fail.  Here's a different\nquestion on StackOverflow that demonstrates a particular problem that's\ncommon where chmod fails to work in Git on certain WSL drives[0].  The\nerror looks like this:\n\n  Cloning into '<repo>'...\n  error: chmod on /mnt/c/Users/User/Code/<repo>/.git/config.lock failed: Operation not permitted\n  fatal: could not set 'core.filemode' to 'false'\n\nIf this situation occurs, your patch will actually leave the file\nwritable, which we don't want, since the fchmod will silently fail.\n\nIt may be that WSL has changed since that was written, but I've seen\nreports of that happening relatively recently, and, in any event, we\ndon't want to require that people use the absolute latest WSL to get\nthings working.\n\nAgain, maybe Junio will accept a patch here, but I don't think the\ncurrent one is going to work correctly, even on WSL.  It may not be\nobvious that it's broken because the patch ignores fchmod's exit status.\nIf you want to submit a patch that uses an option to either use the old\nbehaviour or one that does an fchmod after the fact, that would probably\nbe accepted, as long as you added tests (including the read-only bit)\nfor both cases and sufficient documentation.\n\n[0] https://askubuntu.com/questions/1115564/wsl-ubuntu-distro-how-to-solve-operation-not-permitted-on-cloning-repository\n-- \nbrian m. carlson (they/them or he/him)\nToronto, Ontario, CA\n"}]}