{"thread":{"id":"59009","subject":"[BUG] fatal: transport 'file' not allowed during submodule add","startedAt":"2022-12-27T23:00:44Z","lastAt":"2023-01-03T08:57:24Z","messageCount":12,"participants":["rsbecker@nexbridge.com","Junio C Hamano","Jonathan Nieder","Taylor Blau","Jeff King"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"469566","messageId":"00f901d91a47$09400110$1bc00330$@nexbridge.com","threadId":"59009","inReplyTo":null,"subject":"[BUG] fatal: transport 'file' not allowed during submodule add","fromName":"","fromEmail":"rsbecker@nexbridge.com","sentAt":"2022-12-27T23:00:32Z","receivedAt":"2022-12-27T23:00:44Z","isPatch":false,"sender":{"key":"randall.becker@nexbridge.ca","avatar":"https://avatars.githubusercontent.com/u/28956764?v=4"},"body":"As of 2.39.0, I am now getting fatal: transport 'file' not allowed when\nperforming a submodule add after a clone -l. The simple reproduce of this\nis:\n\n1. Start with an empty bare repository, src.git.\n2. Create an empty non-bare repository and set the upstream remote to the\nbare repo.\n3. Populate the non-bare repository with:\n\ttouch .gitignore &&\n\tgit add .gitignore &&\n\ttouch file1 &&\n\tgit add file1 &&\n\tgit commit -m initial &&\n\tgit remote add origin ../src.git &&\n\tgit push --set-upstream origin master\n4. Create another empty bare repository to be used as the submodule,\nsubsrc.git.\n5. Create another empty non-bare repository and set the upstream remote to\nthe bare repo for the submodule.\n6. Populate the non-bare submodule repository with\n\ttouch .gitignore &&\n\tgit add .gitignore &&\n\tgit commit -m initial &&\n\tgit add .gitignore &&\n\ttouch file2 &&\n\tgit add file2 &&\n\tgit commit -m initial &&\n\tgit remote add origin ../subsrc.git &&\n\tgit push --set-upstream origin master\n7. Clone the main repo using -l or without it (makes no difference):\n\tgit clone -l src.git dest\n8. Attempt to add the submodule:\n\tcd dest &&\n\tgit submodule add -- ../subsrc.git subsrc\n\nThis results in:\nCloning into 'dest'...\ndone.\nCloning into '/home/randall/dest/subsrc'...\nfatal: transport 'file' not allowed\nfatal: clone of '/home/randall/subsrc.git' into submodule path\n'/home/randall/dest/subsrc' failed\n\nThis happens for any submodule add on the same system. Some online research\nindicates that there was a security patch to git causing this, but I can't\nfind it. This does not seem correct to me or how this improves security.\nHelp please - this is causing some of my workflows to break.\n\nThanks,\nRandall\n--\nBrief whoami: NonStop&UNIX developer since approximately\nUNIX(421664400)\nNonStop(211288444200000000)\n-- In real life, I talk too much.\n\n\n\n"},{"id":"469574","messageId":"xmqqilhwp5g4.fsf@gitster.g","threadId":"59009","inReplyTo":"00f901d91a47$09400110$1bc00330$@nexbridge.com","subject":"Re: [BUG] fatal: transport 'file' not allowed during submodule add","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-12-28T03:34:03Z","receivedAt":"2022-12-28T03:34:12Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"<rsbecker@nexbridge.com> writes:\n\n> As of 2.39.0, I am now getting fatal: transport 'file' not allowed when\n> performing a submodule add after a clone -l. The simple reproduce of this\n> is:\n> ...\n> This happens for any submodule add on the same system. Some online research\n> indicates that there was a security patch to git causing this, but I can't\n> find it. This does not seem correct to me or how this improves security.\n> Help please - this is causing some of my workflows to break.\n\nThanks for reporting, Randall.\n\nThis suspiciously sounds like what a1d4f67c (transport: make\n`protocol.file.allow` be \"user\" by default, 2022-07-29) is doing\ndeliberately.  Taylor, does this look like a corner case the 2.30.6\nupdates forgot to consider?\n\nThanks.\n"},{"id":"469584","messageId":"011201d91aca$a5db7800$f1926800$@nexbridge.com","threadId":"59009","inReplyTo":"xmqqilhwp5g4.fsf@gitster.g","subject":"RE: [BUG] fatal: transport 'file' not allowed during submodule add","fromName":"","fromEmail":"rsbecker@nexbridge.com","sentAt":"2022-12-28T14:42:39Z","receivedAt":"2022-12-28T14:42:51Z","isPatch":false,"sender":{"key":"randall.becker@nexbridge.ca","avatar":"https://avatars.githubusercontent.com/u/28956764?v=4"},"body":"\n\n>-----Original Message-----\n>From: Junio C Hamano <jch2355@gmail.com> On Behalf Of Junio C Hamano\nOn December 27, 2022 10:34 PM, Junio C Hamano wrote:\n><rsbecker@nexbridge.com> writes:\n>\n>> As of 2.39.0, I am now getting fatal: transport 'file' not allowed\n>> when performing a submodule add after a clone -l. The simple reproduce\n>> of this\n>> is:\n>> ...\n>> This happens for any submodule add on the same system. Some online\n>> research indicates that there was a security patch to git causing\n>> this, but I can't find it. This does not seem correct to me or how this\nimproves\n>security.\n>> Help please - this is causing some of my workflows to break.\n>\n>Thanks for reporting, Randall.\n>\n>This suspiciously sounds like what a1d4f67c (transport: make\n`protocol.file.allow`\n>be \"user\" by default, 2022-07-29) is doing deliberately.  Taylor, does this\nlook like a\n>corner case the 2.30.6 updates forgot to consider?\n\nI have tried using 'git config --local protocol.file.allow always' and/or\n'git config --local protocol.allow always' to get past this, without\nsuccess. \n\n--Randall\n\n"},{"id":"469614","messageId":"Y6y+zkUsPhknTYH/@google.com","threadId":"59009","inReplyTo":"011201d91aca$a5db7800$f1926800$@nexbridge.com","subject":"Re: [BUG] fatal: transport 'file' not allowed during submodule add","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2022-12-28T22:10:42Z","receivedAt":"2022-12-28T22:12:05Z","isPatch":false,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Hi Randall,\n\nrsbecker@nexbridge.com wrote:\n> Junio C Hamano wrote:\n\n>> This suspiciously sounds like what a1d4f67c (transport: make `protocol.file.allow`\n>> be \"user\" by default, 2022-07-29) is doing deliberately.\n>\n> I have tried using 'git config --local protocol.file.allow always' and/or\n> 'git config --local protocol.allow always' to get past this, without\n> success.\n\nDoes `git config --global protocol.file.allow always` do the trick?\n\n>>                                                           Taylor, does this look like a\n>> corner case the 2.30.6 updates forgot to consider?\n\nI think it's the intended effect (preventing file:// submodules), but\nI wonder if this hints that we'd want that protection to be more\ntargeted.  A file:// submodule (as opposed to a bare path without URL\nscheme) wouldn't trigger the \"git clone --local\" behavior that that\ncommit mentions wanting to protect against, so at first glance it\nwould appear to be no more or less dangerous than cloning from a\nremote repository.\n\nOne thing I'd be curious about is whether --local happening\nautomatically is actually worth it nowadays.  \"git worktree\" does a\nbetter job of sharing with an existing local repository, since the\nsharing continues even after the worktree has been created, after any\n\"git gc\" operations, and so on.  Meanwhile, the distinction between\nfile:// and bare paths is subtle enough that I regularly encounter\npeople not being aware of it (for example when wanting a way to test\nprotocol code locally and not understanding why a bare-path clone\ndoesn't do that).  Would it be more in the spirit of secure defaults\nto require --local when someone wants to request the hardlinking trick\nof local clone?\n\nThanks,\nJonathan\n"},{"id":"469616","messageId":"013501d91b0b$3cd4ceb0$b67e6c10$@nexbridge.com","threadId":"59009","inReplyTo":"Y6y+zkUsPhknTYH/@google.com","subject":"RE: [BUG] fatal: transport 'file' not allowed during submodule add","fromName":"","fromEmail":"rsbecker@nexbridge.com","sentAt":"2022-12-28T22:25:00Z","receivedAt":"2022-12-28T22:25:13Z","isPatch":false,"sender":{"key":"randall.becker@nexbridge.ca","avatar":"https://avatars.githubusercontent.com/u/28956764?v=4"},"body":"On December 28, 2022 5:11 PM, Jonathan Nieder wrote:\n>Hi Randall,\n>\n>rsbecker@nexbridge.com wrote:\n>> Junio C Hamano wrote:\n>\n>>> This suspiciously sounds like what a1d4f67c (transport: make\n>>> `protocol.file.allow` be \"user\" by default, 2022-07-29) is doing\ndeliberately.\n>>\n>> I have tried using 'git config --local protocol.file.allow always'\n>> and/or 'git config --local protocol.allow always' to get past this,\n>> without success.\n>\n>Does `git config --global protocol.file.allow always` do the trick?\n\nI tried git config --local protocol.file.allow always after the initial\nclone. This should work but does not.\nI also tried git config --global protocol.file.allow always before the\ninitial clone.  This also did not work.\n\n>>>                                                           Taylor,\n>>> does this look like a corner case the 2.30.6 updates forgot to consider?\n>\n>I think it's the intended effect (preventing file:// submodules), but I\nwonder if this\n>hints that we'd want that protection to be more targeted.  A file://\nsubmodule (as\n>opposed to a bare path without URL\n>scheme) wouldn't trigger the \"git clone --local\" behavior that that commit\n>mentions wanting to protect against, so at first glance it would appear to\nbe no\n>more or less dangerous than cloning from a remote repository.\n>\n>One thing I'd be curious about is whether --local happening automatically\nis\n>actually worth it nowadays.  \"git worktree\" does a better job of sharing\nwith an\n>existing local repository, since the sharing continues even after the\nworktree has\n>been created, after any \"git gc\" operations, and so on.  Meanwhile, the\ndistinction\n>between file:// and bare paths is subtle enough that I regularly encounter\npeople\n>not being aware of it (for example when wanting a way to test protocol code\n>locally and not understanding why a bare-path clone doesn't do that).\nWould it be\n>more in the spirit of secure defaults to require --local when someone wants\nto\n>request the hardlinking trick of local clone?\n\nI think the risk of someone hacking a hardlink is less risky than someone\nmisdirecting a remote site not under a user's direct control.\n\nThe tests I did show the same behaviour no matter which combination of the\nabove. --local appears to be implied, at least there is no apparent\nbehavioural difference between specifying the argument and not.\n\n--Randall\n\n\n"},{"id":"469685","messageId":"000001d91c8b$6a26cd60$3e746820$@nexbridge.com","threadId":"59009","inReplyTo":"xmqqilhwp5g4.fsf@gitster.g","subject":"RE: [BUG] fatal: transport 'file' not allowed during submodule add","fromName":"","fromEmail":"rsbecker@nexbridge.com","sentAt":"2022-12-30T20:15:03Z","receivedAt":"2022-12-30T20:15:19Z","isPatch":false,"sender":{"key":"randall.becker@nexbridge.ca","avatar":"https://avatars.githubusercontent.com/u/28956764?v=4"},"body":"On December 27, 2022 10:34 PM, Junio C Hamano wrote:\n><rsbecker@nexbridge.com> writes:\n>\n>> As of 2.39.0, I am now getting fatal: transport 'file' not allowed\n>> when performing a submodule add after a clone -l. The simple reproduce\n>> of this\n>> is:\n>> ...\n>> This happens for any submodule add on the same system. Some online\n>> research indicates that there was a security patch to git causing\n>> this, but I can't find it. This does not seem correct to me or how this\nimproves\n>security.\n>> Help please - this is causing some of my workflows to break.\n>\n>Thanks for reporting, Randall.\n>\n>This suspiciously sounds like what a1d4f67c (transport: make\n`protocol.file.allow`\n>be \"user\" by default, 2022-07-29) is doing deliberately.  Taylor, does this\nlook like a\n>corner case the 2.30.6 updates forgot to consider?\n\nAny updates on this? Neither protocol.file.allow=always nor\nprotocol.allow=always gets past the error condition.\n\nThanks,\nRandall\n\n"},{"id":"469686","messageId":"Y69SRs9ifDPagOUo@nand.local","threadId":"59009","inReplyTo":"011201d91aca$a5db7800$f1926800$@nexbridge.com","subject":"Re: [BUG] fatal: transport 'file' not allowed during submodule add","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2022-12-30T21:04:06Z","receivedAt":"2022-12-30T21:04:11Z","isPatch":false,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"On Wed, Dec 28, 2022 at 09:42:39AM -0500, rsbecker@nexbridge.com wrote:\n>\n>\n> >-----Original Message-----\n> >From: Junio C Hamano <jch2355@gmail.com> On Behalf Of Junio C Hamano\n> On December 27, 2022 10:34 PM, Junio C Hamano wrote:\n> ><rsbecker@nexbridge.com> writes:\n> >\n> >> As of 2.39.0, I am now getting fatal: transport 'file' not allowed\n> >> when performing a submodule add after a clone -l. The simple reproduce\n> >> of this\n> >> is:\n> >> ...\n> >> This happens for any submodule add on the same system. Some online\n> >> research indicates that there was a security patch to git causing\n> >> this, but I can't find it. This does not seem correct to me or how this\n> improves\n> >security.\n> >> Help please - this is causing some of my workflows to break.\n> >\n> >Thanks for reporting, Randall.\n> >\n> >This suspiciously sounds like what a1d4f67c (transport: make\n> `protocol.file.allow`\n> >be \"user\" by default, 2022-07-29) is doing deliberately.  Taylor, does this\n> look like a\n> >corner case the 2.30.6 updates forgot to consider?\n>\n> I have tried using 'git config --local protocol.file.allow always' and/or\n> 'git config --local protocol.allow always' to get past this, without\n> success.\n\nI couldn't reproduce the symptom you described. Indeed, the behavior of\nnot allowing local-submodules to be cloned without explicitly opting in\nvia the `protocol.file.allow` configuration is intentional.\n\nThe patch Junio mentioned, a1d4f67c12 (transport: make\n`protocol.file.allow` be \"user\" by default, 2022-07-29) has some\nexamples of why this behavior was changed in the 2.30.6 update.\n\nIf you run either `git config --global protocol.file.allow always`, or\nreplace your last submodule add with:\n\n  $ git -c protocol.file.allow=always submodule add /path/to/subsrc.git\n\nit should work as expected.\n\nThanks,\nTaylor\n"},{"id":"469687","messageId":"Y69TMzIf/bdsZe6/@nand.local","threadId":"59009","inReplyTo":"Y6y+zkUsPhknTYH/@google.com","subject":"Re: [BUG] fatal: transport 'file' not allowed during submodule add","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2022-12-30T21:08:03Z","receivedAt":"2022-12-30T21:08:09Z","isPatch":false,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"Hi Jonathan,\n\nOn Wed, Dec 28, 2022 at 02:10:42PM -0800, Jonathan Nieder wrote:\n> Hi Randall,\n>\n> rsbecker@nexbridge.com wrote:\n> > Junio C Hamano wrote:\n>\n> >> This suspiciously sounds like what a1d4f67c (transport: make `protocol.file.allow`\n> >> be \"user\" by default, 2022-07-29) is doing deliberately.\n> >\n> > I have tried using 'git config --local protocol.file.allow always' and/or\n> > 'git config --local protocol.allow always' to get past this, without\n> > success.\n>\n> Does `git config --global protocol.file.allow always` do the trick?\n>\n> >>                                                           Taylor, does this look like a\n> >> corner case the 2.30.6 updates forgot to consider?\n>\n> I think it's the intended effect (preventing file:// submodules), but\n> I wonder if this hints that we'd want that protection to be more\n> targeted.  A file:// submodule (as opposed to a bare path without URL\n> scheme) wouldn't trigger the \"git clone --local\" behavior that that\n> commit mentions wanting to protect against, so at first glance it\n> would appear to be no more or less dangerous than cloning from a\n> remote repository.\n\nChanging the default value of 'protocol.file.allow' isn't solely about whether\nor not we use the `file://` scheme and transport or not. Instead, it's\nabout preventing the user from accidentally cloning local repositories\ncontaining sensitive data into the working copy of a malicious\nrepository.\n\nOne example might be that I convince you to clone my malicious\nrepository, which has a Dockerfile that uploads everything in the\ncontainer filesystem to some data harvesting server. Since 'docker run'\nautomatically puts everything in '.' into the volume mount, anything in\nthe working copy of my malicious repository will get exfiltrated.\n\nThe worry that I wrote about in a1d4f67c was that if I knew that you\nstored, say, your SSH private key material in a repository that is at\n`$HOME/.git` (as is sometimes common practice), then I could add a\nsubmodule at /home/jrnieder/.git, and extract any sensitive data\ntherein.\n\nSo I think our new default is sensible here if we are concerned with\npreventing such a case.\n\nThanks,\nTaylor\n"},{"id":"469688","messageId":"000701d91c97$cc35fd30$64a1f790$@nexbridge.com","threadId":"59009","inReplyTo":"Y69SRs9ifDPagOUo@nand.local","subject":"RE: [BUG] fatal: transport 'file' not allowed during submodule add","fromName":"","fromEmail":"rsbecker@nexbridge.com","sentAt":"2022-12-30T21:43:41Z","receivedAt":"2022-12-30T21:43:52Z","isPatch":false,"sender":{"key":"randall.becker@nexbridge.ca","avatar":"https://avatars.githubusercontent.com/u/28956764?v=4"},"body":"On December 30, 2022 4:04 PM, Taylor Blau wrote:\n>On Wed, Dec 28, 2022 at 09:42:39AM -0500, rsbecker@nexbridge.com wrote:\n>> >-----Original Message-----\n>> >From: Junio C Hamano <jch2355@gmail.com> On Behalf Of Junio C Hamano\n>> On December 27, 2022 10:34 PM, Junio C Hamano wrote:\n>> ><rsbecker@nexbridge.com> writes:\n>> >\n>> >> As of 2.39.0, I am now getting fatal: transport 'file' not allowed\n>> >> when performing a submodule add after a clone -l. The simple\n>> >> reproduce of this\n>> >> is:\n>> >> ...\n>> >> This happens for any submodule add on the same system. Some online\n>> >> research indicates that there was a security patch to git causing\n>> >> this, but I can't find it. This does not seem correct to me or how\n>> >> this\n>> improves\n>> >security.\n>> >> Help please - this is causing some of my workflows to break.\n>> >\n>> >Thanks for reporting, Randall.\n>> >\n>> >This suspiciously sounds like what a1d4f67c (transport: make\n>> `protocol.file.allow`\n>> >be \"user\" by default, 2022-07-29) is doing deliberately.  Taylor,\n>> >does this\n>> look like a\n>> >corner case the 2.30.6 updates forgot to consider?\n>>\n>> I have tried using 'git config --local protocol.file.allow always'\n>> and/or 'git config --local protocol.allow always' to get past this,\n>> without success.\n>\n>I couldn't reproduce the symptom you described. Indeed, the behavior of not\n>allowing local-submodules to be cloned without explicitly opting in via the\n>`protocol.file.allow` configuration is intentional.\n>\n>The patch Junio mentioned, a1d4f67c12 (transport: make `protocol.file.allow` be\n>\"user\" by default, 2022-07-29) has some examples of why this behavior was\n>changed in the 2.30.6 update.\n>\n>If you run either `git config --global protocol.file.allow always`, or replace your last\n>submodule add with:\n>\n>  $ git -c protocol.file.allow=always submodule add /path/to/subsrc.git\n>\n>it should work as expected.\n\nI have reproduced this on multiple platforms including NonStop and Cygwin64 on Windows with the same results as earlier. The protocol.file.allowed=always does not appear to even get considered. With some fprintfs in the code, the code in is_transport_allowed falls through to the PROTOCOL_ALLOW_USER_ONLY case and only considers environment variable GIT_PROTOCOL_FROM_USER, which is not passed into the child doing the submodule add. The is_transport_allowed(\"file\",-1) always returns 0 no matter what and 0 is what gets used upwards. There is no difference in the behaviour regardless of the protocol.file.allowed value either in -c, .gitconfig, or on the user environment variable.\n\n"},{"id":"469689","messageId":"000801d91c98$6a8bbdd0$3fa33970$@nexbridge.com","threadId":"59009","inReplyTo":"Y69TMzIf/bdsZe6/@nand.local","subject":"RE: [BUG] fatal: transport 'file' not allowed during submodule add","fromName":"","fromEmail":"rsbecker@nexbridge.com","sentAt":"2022-12-30T21:48:07Z","receivedAt":"2022-12-30T21:48:20Z","isPatch":false,"sender":{"key":"randall.becker@nexbridge.ca","avatar":"https://avatars.githubusercontent.com/u/28956764?v=4"},"body":"On December 30, 2022 4:08 PM, Taylor Blau wrote:\n>On Wed, Dec 28, 2022 at 02:10:42PM -0800, Jonathan Nieder wrote:\n>> Hi Randall,\n>>\n>> rsbecker@nexbridge.com wrote:\n>> > Junio C Hamano wrote:\n>>\n>> >> This suspiciously sounds like what a1d4f67c (transport: make\n>> >> `protocol.file.allow` be \"user\" by default, 2022-07-29) is doing deliberately.\n>> >\n>> > I have tried using 'git config --local protocol.file.allow always'\n>> > and/or 'git config --local protocol.allow always' to get past this,\n>> > without success.\n>>\n>> Does `git config --global protocol.file.allow always` do the trick?\n>>\n>> >>                                                           Taylor,\n>> >> does this look like a corner case the 2.30.6 updates forgot to consider?\n>>\n>> I think it's the intended effect (preventing file:// submodules), but\n>> I wonder if this hints that we'd want that protection to be more\n>> targeted.  A file:// submodule (as opposed to a bare path without URL\n>> scheme) wouldn't trigger the \"git clone --local\" behavior that that\n>> commit mentions wanting to protect against, so at first glance it\n>> would appear to be no more or less dangerous than cloning from a\n>> remote repository.\n>\n>Changing the default value of 'protocol.file.allow' isn't solely about whether or not\n>we use the `file://` scheme and transport or not. Instead, it's about preventing the\n>user from accidentally cloning local repositories containing sensitive data into the\n>working copy of a malicious repository.\n>\n>One example might be that I convince you to clone my malicious repository, which\n>has a Dockerfile that uploads everything in the container filesystem to some data\n>harvesting server. Since 'docker run'\n>automatically puts everything in '.' into the volume mount, anything in the working\n>copy of my malicious repository will get exfiltrated.\n>\n>The worry that I wrote about in a1d4f67c was that if I knew that you stored, say,\n>your SSH private key material in a repository that is at `$HOME/.git` (as is\n>sometimes common practice), then I could add a submodule at\n>/home/jrnieder/.git, and extract any sensitive data therein.\n>\n>So I think our new default is sensible here if we are concerned with preventing\n>such a case.\n\nI think the new default is reasonable but this did catch me by surprise as it broke our workflows. I guess I need to look at the release notes in more depth - that's my bad. With the caveat that I do not think this is working as intended, which I am finding, because changing the configuration does not make any behavioural difference on any platform I can test on.\n\n"},{"id":"469697","messageId":"000201d91ca4$b868caf0$293a60d0$@nexbridge.com","threadId":"59009","inReplyTo":"Y69SRs9ifDPagOUo@nand.local","subject":"RE: [BUG] fatal: transport 'file' not allowed during submodule add","fromName":"","fromEmail":"rsbecker@nexbridge.com","sentAt":"2022-12-30T23:16:11Z","receivedAt":"2022-12-30T23:16:24Z","isPatch":false,"sender":{"key":"randall.becker@nexbridge.ca","avatar":"https://avatars.githubusercontent.com/u/28956764?v=4"},"body":"On December 30, 2022 4:04 PM, Taylor Blau wrote:\n>On Wed, Dec 28, 2022 at 09:42:39AM -0500, rsbecker@nexbridge.com wrote:\n>> >From: Junio C Hamano <jch2355@gmail.com> On Behalf Of Junio C Hamano\n>> On December 27, 2022 10:34 PM, Junio C Hamano wrote:\n>> ><rsbecker@nexbridge.com> writes:\n>> >\n>> >> As of 2.39.0, I am now getting fatal: transport 'file' not allowed\n>> >> when performing a submodule add after a clone -l. The simple\n>> >> reproduce of this\n>> >> is:\n>> >> ...\n>> >> This happens for any submodule add on the same system. Some online\n>> >> research indicates that there was a security patch to git causing\n>> >> this, but I can't find it. This does not seem correct to me or how\n>> >> this\n>> improves\n>> >security.\n>> >> Help please - this is causing some of my workflows to break.\n>> >\n>> >Thanks for reporting, Randall.\n>> >\n>> >This suspiciously sounds like what a1d4f67c (transport: make\n>> `protocol.file.allow`\n>> >be \"user\" by default, 2022-07-29) is doing deliberately.  Taylor,\n>> >does this\n>> look like a\n>> >corner case the 2.30.6 updates forgot to consider?\n>>\n>> I have tried using 'git config --local protocol.file.allow always'\n>> and/or 'git config --local protocol.allow always' to get past this,\n>> without success.\n>\n>I couldn't reproduce the symptom you described. Indeed, the behavior of not\n>allowing local-submodules to be cloned without explicitly opting in via the\n>`protocol.file.allow` configuration is intentional.\n>\n>The patch Junio mentioned, a1d4f67c12 (transport: make `protocol.file.allow` be\n>\"user\" by default, 2022-07-29) has some examples of why this behavior was\n>changed in the 2.30.6 update.\n>\n>If you run either `git config --global protocol.file.allow always`, or replace your last\n>submodule add with:\n>\n>  $ git -c protocol.file.allow=always submodule add /path/to/subsrc.git\n>\n>it should work as expected.\n\nOk, operator error. This does work as expected if you run the test slightly different. If the repositories are all cloned, the upstream remotes are set up properly and the submodule add works. In my test case, I used git remote add with a relative path. This seems to be an edge condition that triggers the situation. When avoiding the upstream remote command (implied by clone), the submodule add works if the protocol.file.allow = always, no matter how it is set.\n\n--Randall\n\n"},{"id":"469759","messageId":"Y7Pt7R0VX3kGI5Dc@coredump.intra.peff.net","threadId":"59009","inReplyTo":"Y69TMzIf/bdsZe6/@nand.local","subject":"Re: [BUG] fatal: transport 'file' not allowed during submodule add","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2023-01-03T08:57:17Z","receivedAt":"2023-01-03T08:57:24Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Fri, Dec 30, 2022 at 04:08:03PM -0500, Taylor Blau wrote:\n\n> Changing the default value of 'protocol.file.allow' isn't solely about whether\n> or not we use the `file://` scheme and transport or not. Instead, it's\n> about preventing the user from accidentally cloning local repositories\n> containing sensitive data into the working copy of a malicious\n> repository.\n> \n> One example might be that I convince you to clone my malicious\n> repository, which has a Dockerfile that uploads everything in the\n> container filesystem to some data harvesting server. Since 'docker run'\n> automatically puts everything in '.' into the volume mount, anything in\n> the working copy of my malicious repository will get exfiltrated.\n> \n> The worry that I wrote about in a1d4f67c was that if I knew that you\n> stored, say, your SSH private key material in a repository that is at\n> `$HOME/.git` (as is sometimes common practice), then I could add a\n> submodule at /home/jrnieder/.git, and extract any sensitive data\n> therein.\n> \n> So I think our new default is sensible here if we are concerned with\n> preventing such a case.\n\nOne of the justifications for disallowing filesystem URLs in submodules\nis that they're generally a bad idea anyway. If you're cloning over the\nnetwork, there's nothing to guarantee that the cloner's filesystem looks\nanything like the original committer's. So it's an accident waiting to\nhappen.\n\nBut one case where it's not _completely_ unreasonable is when the\nsuperproject clone itself is happening via the local filesystem. That\nmatches Randall's example, though I'm not sure if that's his \"real\"\nworkflow.\n\nSo one thing we could do to soften the change is to allow a submodule to\nuse a filesystem URL if and only if the immediate parent clone also did\n(and naturally, allow \"submodule add\" on anything). But:\n\n  - it's not clear to me if that just re-opens the Docker problem you\n    were trying to fix (I think not, because it should be forbidding\n    file:// remotes for the initial clone, too; but...)\n\n  - as a security rule, it's complicated and confusing, which means it\n    is likely to have loopholes or be mis-used. As a user I now have to\n    be aware that copying an untrusted .git to my filesystem and cloning\n    from it has a different trust level than cloning it over https, with\n    respect to submodule protocols.\n\n  - in a distributed system, the notion of \"the parent clone\" is vague\n    anyway. During \"git clone --recurse-submodules\", sure, you can use\n    the top-level URL as your basis. But later if I run \"git submodule\n    update\", what's the \"original\" protocol? I can guess at it by\n    looking at remote.origin.url, but of course it could change, there\n    could be multiple remotes, etc.\n\nSo I think it's probably not worth doing. People with that workflow\nshould set protocol.file.allow as appropriate.\n\nI also wondered if you could get around this with url.*.insteadOf. That\nis, it would seem reasonable to use a unique URL in .gitmodules (even if\nit's not a real functional URL!), and then let local developers override\nit to point to their filesystem with the insteadOf mechanism. That gives\ntighter permissions, and provides more options for local redirection.\n\nUnfortunately, it doesn't work here. We check the protocol permissions\nas we're about to use them (which is good, because we catch all paths\nthat get there). But at that point we have no idea about the rewrite.\nLooks like there was some discussion in this thread:\n\n  https://lore.kernel.org/git/CAPZ477MCsBsfbqKzp69MT_brwz-0aes6twJofQrhizUBV7ZoeA@mail.gmail.com/\n\nabout having a rewrite make the URL \"from the user\". But ultimately it\nseems scary. And anyway, it doesn't help this case much anyway (people\nstill need to set up config, so they might as well just set the protocol\nconfig).\n\n-Peff\n"}]}