{"thread":{"id":"38860","subject":"Sparse checkout not working as expected (colons in filenames on Windows)","startedAt":"2015-03-19T23:03:43Z","lastAt":"2015-03-25T08:17:43Z","messageCount":7,"participants":["Philip Oakley","Duy Nguyen","Johannes Schindelin"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"258074","messageId":"BE2660C974C9415E997F20A49ABC766E@PhilipOakley","threadId":"38860","inReplyTo":null,"subject":"Sparse checkout not working as expected (colons in filenames on Windows)","fromName":"Philip Oakley","fromEmail":"philipoakley@iee.org","sentAt":null,"receivedAt":"2015-03-19T23:03:43Z","isPatch":false,"sender":{"key":"philipoakley@iee.email","avatar":"https://avatars.githubusercontent.com/u/914343?v=4"},"body":"Hi, I was expecting that sparse checkout could be used to avoid the \nchecking out, by git, of files which have colons in their name into the \nworktree when on Windows.\n\nYue Lin Ho reported on the Msygit list [1] that he had a repo where \nthere was already committed a file with a colon in it's name, which was \na needed file and had been committed by a Linux user. The problem was \nhow to work with that repo on a Windows box where such a file is \nprohibited to exist on the FS (hence the expectation that a sparse \ncheckout would suffice). Yue has created a short test repo [2]\n\nEven after getting the pathspec escaping right, I still haven't been \nable to make this expected behaviour work [3].\n\nAm I wrong to expect that sparse checkout (and the skip-worktree bit) \ncan be used to avoid files with undesirable filenames hitting the OS's \nfile system?\n\nIf it should be OK, what's the correct recipe?\n\n--\nPhilip\n\n[1] \nhttps://groups.google.com/forum/?hl=en_US?hl%3Den#!topic/msysgit/D4HcHRpxPgU \n\"How to play around with the filename with colon on Windows?\"\n[2] Test repo https://github.com/t-pascal/tortoisegit-colons\n\n[3] test sequence::\n$ mkdir colons && cd colons\n$ git clone -n https://github.com/t-pascal/tortoisegit-colons\n$ cd tortoisegit-colons/\n$ git config core.sparseCheckout true\n$ cat .git/info/sparse-checkout # external editor\n/*\n!ifcfg-eth0\\:0\n$ git update-index --skip-worktree -- ifcfg-eth0\\:0\nIgnoring path ifcfg-eth0:0\n$ git checkout -b test 7f35d34bc6160cc # tip commit, we are still \nunborn!\nerror: Invalid path 'ifcfg-eth0:0\nD       ifcfg-eth0:0\nSwitched to a new branch 'test'\n"},{"id":"258394","messageId":"CACsJy8BOuhohQzaGzD+4cthqOaL4sz1ADRD-ekhFhKt4wMcg7A@mail.gmail.com","threadId":"38860","inReplyTo":"BE2660C974C9415E997F20A49ABC766E@PhilipOakley","subject":"Re: Sparse checkout not working as expected (colons in filenames on Windows)","fromName":"Duy Nguyen","fromEmail":"pclouds@gmail.com","sentAt":"2015-03-24T01:21:12Z","receivedAt":"2015-03-24T01:21:12Z","isPatch":false,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Fri, Mar 20, 2015 at 6:07 AM, Philip Oakley <philipoakley@iee.org> wrote:\n> Hi, I was expecting that sparse checkout could be used to avoid the checking\n> out, by git, of files which have colons in their name into the worktree when\n> on Windows.\n>\n> Yue Lin Ho reported on the Msygit list [1] that he had a repo where there\n> was already committed a file with a colon in it's name, which was a needed\n> file and had been committed by a Linux user. The problem was how to work\n> with that repo on a Windows box where such a file is prohibited to exist on\n> the FS (hence the expectation that a sparse checkout would suffice). Yue has\n> created a short test repo [2]\n>\n> Even after getting the pathspec escaping right, I still haven't been able to\n> make this expected behaviour work [3].\n>\n> Am I wrong to expect that sparse checkout (and the skip-worktree bit) can be\n> used to avoid files with undesirable filenames hitting the OS's file system?\n>\n> If it should be OK, what's the correct recipe?\n>\n> --\n> Philip\n>\n> [1]\n> https://groups.google.com/forum/?hl=en_US?hl%3Den#!topic/msysgit/D4HcHRpxPgU\n> \"How to play around with the filename with colon on Windows?\"\n> [2] Test repo https://github.com/t-pascal/tortoisegit-colons\n>\n> [3] test sequence::\n> $ mkdir colons && cd colons\n> $ git clone -n https://github.com/t-pascal/tortoisegit-colons\n> $ cd tortoisegit-colons/\n> $ git config core.sparseCheckout true\n> $ cat .git/info/sparse-checkout # external editor\n> /*\n> !ifcfg-eth0\\:0\n\nColons have no special meaning in gitignore rules and therefore need\nnot be escaped. The backslash is considered a literal character in\nthis case, probably not what you want.\n\n> $ git update-index --skip-worktree -- ifcfg-eth0\\:0\n> Ignoring path ifcfg-eth0:0\n> $ git checkout -b test 7f35d34bc6160cc # tip commit, we are still unborn!\n> error: Invalid path 'ifcfg-eth0:0\n> D       ifcfg-eth0:0\n> Switched to a new branch 'test'\n-- \nDuy\n"},{"id":"258462","messageId":"90D5D4B5F2ED41A6AB07E291004AA615@PhilipOakley","threadId":"38860","inReplyTo":"CACsJy8BOuhohQzaGzD+4cthqOaL4sz1ADRD-ekhFhKt4wMcg7A@mail.gmail.com","subject":"Re: Sparse checkout not working as expected (colons in filenames on Windows)","fromName":"Philip Oakley","fromEmail":"philipoakley@iee.org","sentAt":null,"receivedAt":"2015-03-24T23:43:34Z","isPatch":false,"sender":{"key":"philipoakley@iee.email","avatar":"https://avatars.githubusercontent.com/u/914343?v=4"},"body":"From: \"Duy Nguyen\" <pclouds@gmail.com>\n> On Fri, Mar 20, 2015 at 6:07 AM, Philip Oakley <philipoakley@iee.org> \n> wrote:\n>> Hi, I was expecting that sparse checkout could be used to avoid the \n>> checking\n>> out, by git, of files which have colons in their name into the \n>> worktree when\n>> on Windows.\n>>\n>> Yue Lin Ho reported on the Msygit list [1] that he had a repo where \n>> there\n>> was already committed a file with a colon in it's name, which was a \n>> needed\n>> file and had been committed by a Linux user. The problem was how to \n>> work\n>> with that repo on a Windows box where such a file is prohibited to \n>> exist on\n>> the FS (hence the expectation that a sparse checkout would suffice). \n>> Yue has\n>> created a short test repo [2]\n>>\n>> Even after getting the pathspec escaping right, I still haven't been \n>> able to\n>> make this expected behaviour work [3].\n>>\n>> Am I wrong to expect that sparse checkout (and the skip-worktree bit) \n>> can be\n>> used to avoid files with undesirable filenames hitting the OS's file \n>> system?\n>>\n>> If it should be OK, what's the correct recipe?\n>>\n>> --\n>> Philip\n>>\n>> [1]\n>> https://groups.google.com/forum/?hl=en_US?hl%3Den#!topic/msysgit/D4HcHRpxPgU\n>> \"How to play around with the filename with colon on Windows?\"\n>> [2] Test repo https://github.com/t-pascal/tortoisegit-colons\n>>\n>> [3] test sequence::\n>> $ mkdir colons && cd colons\n>> $ git clone -n https://github.com/t-pascal/tortoisegit-colons\n>> $ cd tortoisegit-colons/\n>> $ git config core.sparseCheckout true\n>> $ cat .git/info/sparse-checkout # external editor\n>> /*\n>> !ifcfg-eth0\\:0\n>\n> Colons have no special meaning in gitignore rules and therefore need\n> not be escaped. The backslash is considered a literal character in\n> this case, probably not what you want.\n>\n>> $ git update-index --skip-worktree -- ifcfg-eth0\\:0\n>> Ignoring path ifcfg-eth0:0\n>> $ git checkout -b test 7f35d34bc6160cc # tip commit, we are still \n>> unborn!\n>> error: Invalid path 'ifcfg-eth0:0\n>> D       ifcfg-eth0:0\n>> Switched to a new branch 'test'\n> -- \nI've corrected the sparse-checkout, but won't the command line 'git \nupdate-index --skip-worktree' will still need it? (demo commands below)\n\nThat said, the final error (which I'd missed in the earlier post) is:\nfatal: make_cache_entry failed for path 'ifcfg-eth0:0'\n\nThis is on the Windows (pre-compiled msysgit at v1.9.5) Git bash, so \ncould be a catch path in that code for make_cache_entry (I've not \nchecked the code yet). So at the moment it doesn't look like sparse \ncheckout can be used to avoid colons in windows on-disk files based on \nthe current code.\n--\nPhilip\n\nPhilip@PHILIPOAKLEY /d/Git_repos/colons\n$ cd tortoisegit-colons/\n\nPhilip@PHILIPOAKLEY /d/Git_repos/colons/tortoisegit-colons (test)\n$ git update-index --skip-worktree -- ifcfg-eth0\\:0\nIgnoring path ifcfg-eth0:0\n\nPhilip@PHILIPOAKLEY /d/Git_repos/colons/tortoisegit-colons (test)\n$ git reset\nerror: Invalid path 'ifcfg-eth0:0'\nfatal: make_cache_entry failed for path 'ifcfg-eth0:0'\n"},{"id":"258466","messageId":"CACsJy8AACwDPSEWdGiLLK3PTY=hLm0XOvN4_y4PbCFYrMzrnHg@mail.gmail.com","threadId":"38860","inReplyTo":"90D5D4B5F2ED41A6AB07E291004AA615@PhilipOakley","subject":"Re: Sparse checkout not working as expected (colons in filenames on Windows)","fromName":"Duy Nguyen","fromEmail":"pclouds@gmail.com","sentAt":"2015-03-25T00:46:04Z","receivedAt":"2015-03-25T00:46:04Z","isPatch":false,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Wed, Mar 25, 2015 at 6:50 AM, Philip Oakley <philipoakley@iee.org> wrote:\n> I've corrected the sparse-checkout, but won't the command line 'git\n> update-index --skip-worktree' will still need it? (demo commands below)\n\nA \"git checkout\" (without arguments) or \"read-trree -mu\" should attach\nskip-worktree properly. You don't need to do it yourself.\n\n> That said, the final error (which I'd missed in the earlier post) is:\n> fatal: make_cache_entry failed for path 'ifcfg-eth0:0'\n>\n> This is on the Windows (pre-compiled msysgit at v1.9.5) Git bash, so could\n> be a catch path in that code for make_cache_entry (I've not checked the code\n> yet). So at the moment it doesn't look like sparse checkout can be used to\n> avoid colons in windows on-disk files based on the current code.\n\nBoth of your commands below fail by the same function, verify_path()\nbecause of this msysgit commit 2e2a2d1 (NTFS: Prevent problematic\npaths from being checked out - 2014-12-10). I guess that check is a\nbit too strong, it should apply when new index entries are created\nfrom worktree (not from a tree)..\n\n> --\n> Philip\n>\n> Philip@PHILIPOAKLEY /d/Git_repos/colons\n> $ cd tortoisegit-colons/\n>\n> Philip@PHILIPOAKLEY /d/Git_repos/colons/tortoisegit-colons (test)\n> $ git update-index --skip-worktree -- ifcfg-eth0\\:0\n> Ignoring path ifcfg-eth0:0\n>\n> Philip@PHILIPOAKLEY /d/Git_repos/colons/tortoisegit-colons (test)\n> $ git reset\n> error: Invalid path 'ifcfg-eth0:0'\n> fatal: make_cache_entry failed for path 'ifcfg-eth0:0'\n-- \nDuy\n\n-- \n-- \n*** Please reply-to-all at all times ***\n*** (do not pretend to know who is subscribed and who is not) ***\n*** Please avoid top-posting. ***\nThe msysGit Wiki is here: https://github.com/msysgit/msysgit/wiki - Github accounts are free.\n\nYou received this message because you are subscribed to the Google\nGroups \"msysGit\" group.\nTo post to this group, send email to msysgit@googlegroups.com\nTo unsubscribe from this group, send email to\nmsysgit+unsubscribe@googlegroups.com\nFor more options, and view previous threads, visit this group at\nhttp://groups.google.com/group/msysgit?hl=en_US?hl=en\n\n--- \nYou received this message because you are subscribed to the Google Groups \"Git for Windows\" group.\nTo unsubscribe from this group and stop receiving emails from it, send an email to msysgit+unsubscribe@googlegroups.com.\nFor more options, visit https://groups.google.com/d/optout.\n"},{"id":"258486","messageId":"61604ca6e1c2c114f5b536be420ba73a@www.dscho.org","threadId":"38860","inReplyTo":"CACsJy8AACwDPSEWdGiLLK3PTY=hLm0XOvN4_y4PbCFYrMzrnHg@mail.gmail.com","subject":"Re: Re: Sparse checkout not working as expected (colons in filenames on Windows)","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2015-03-25T06:39:49Z","receivedAt":"2015-03-25T06:39:49Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Duy,\n\nOn 2015-03-25 01:46, Duy Nguyen wrote:\n> On Wed, Mar 25, 2015 at 6:50 AM, Philip Oakley <philipoakley@iee.org> wrote:\n>\n>> That said, the final error (which I'd missed in the earlier post) is:\n>> fatal: make_cache_entry failed for path 'ifcfg-eth0:0'\n>>\n>> This is on the Windows (pre-compiled msysgit at v1.9.5) Git bash, so could\n>> be a catch path in that code for make_cache_entry (I've not checked the code\n>> yet). So at the moment it doesn't look like sparse checkout can be used to\n>> avoid colons in windows on-disk files based on the current code.\n> \n> Both of your commands below fail by the same function, verify_path()\n> because of this msysgit commit 2e2a2d1 (NTFS: Prevent problematic\n> paths from being checked out - 2014-12-10). I guess that check is a\n> bit too strong, it should apply when new index entries are created\n> from worktree (not from a tree)..\n\nOh, right, that check needs some relaxing. But certainly in a different way: you *definitely* want to prevent attacks from the outside, and those originate from the *tree*.\n\nWhat we need to do instead is to relax the check to apply only if we are really going to write out that file, not when it is skipped.\n\nThanks,\nJohannes\n\n-- \n-- \n*** Please reply-to-all at all times ***\n*** (do not pretend to know who is subscribed and who is not) ***\n*** Please avoid top-posting. ***\nThe msysGit Wiki is here: https://github.com/msysgit/msysgit/wiki - Github accounts are free.\n\nYou received this message because you are subscribed to the Google\nGroups \"msysGit\" group.\nTo post to this group, send email to msysgit@googlegroups.com\nTo unsubscribe from this group, send email to\nmsysgit+unsubscribe@googlegroups.com\nFor more options, and view previous threads, visit this group at\nhttp://groups.google.com/group/msysgit?hl=en_US?hl=en\n\n--- \nYou received this message because you are subscribed to the Google Groups \"Git for Windows\" group.\nTo unsubscribe from this group and stop receiving emails from it, send an email to msysgit+unsubscribe@googlegroups.com.\nFor more options, visit https://groups.google.com/d/optout.\n"},{"id":"258488","messageId":"CACsJy8CZPufnyjdx0Z+SuUNhQ58gvKQprY98V7EJnodHgeyK5A@mail.gmail.com","threadId":"38860","inReplyTo":"61604ca6e1c2c114f5b536be420ba73a@www.dscho.org","subject":"Re: [msysGit] Re: Sparse checkout not working as expected (colons in filenames on Windows)","fromName":"Duy Nguyen","fromEmail":"pclouds@gmail.com","sentAt":"2015-03-25T06:44:30Z","receivedAt":"2015-03-25T06:44:30Z","isPatch":false,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Wed, Mar 25, 2015 at 1:39 PM, Johannes Schindelin\n<johannes.schindelin@gmx.de> wrote:\n> Hi Duy,\n>\n> On 2015-03-25 01:46, Duy Nguyen wrote:\n>> On Wed, Mar 25, 2015 at 6:50 AM, Philip Oakley <philipoakley@iee.org> wrote:\n>>\n>>> That said, the final error (which I'd missed in the earlier post) is:\n>>> fatal: make_cache_entry failed for path 'ifcfg-eth0:0'\n>>>\n>>> This is on the Windows (pre-compiled msysgit at v1.9.5) Git bash, so could\n>>> be a catch path in that code for make_cache_entry (I've not checked the code\n>>> yet). So at the moment it doesn't look like sparse checkout can be used to\n>>> avoid colons in windows on-disk files based on the current code.\n>>\n>> Both of your commands below fail by the same function, verify_path()\n>> because of this msysgit commit 2e2a2d1 (NTFS: Prevent problematic\n>> paths from being checked out - 2014-12-10). I guess that check is a\n>> bit too strong, it should apply when new index entries are created\n>> from worktree (not from a tree)..\n>\n> Oh, right, that check needs some relaxing. But certainly in a different way: you *definitely* want to prevent attacks from the outside, and those originate from the *tree*.\n>\n> What we need to do instead is to relax the check to apply only if we are really going to write out that file, not when it is skipped.\n\nYeah. In fact if you do this, not checking out  files that can't be or\nnot safe to be checked out, then the ifcfg-eth0:0 problem that Phillip\nwanted to avoid using sparse checkout is also gone.\n-- \nDuy\n"},{"id":"258495","messageId":"576a56072f8f30818b7af016e0ead628@www.dscho.org","threadId":"38860","inReplyTo":"CACsJy8CZPufnyjdx0Z+SuUNhQ58gvKQprY98V7EJnodHgeyK5A@mail.gmail.com","subject":"Re: Re: Sparse checkout not working as expected (colons in filenames on Windows)","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2015-03-25T08:17:43Z","receivedAt":"2015-03-25T08:17:43Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Duy,\n\nOn 2015-03-25 07:44, Duy Nguyen wrote:\n> On Wed, Mar 25, 2015 at 1:39 PM, Johannes Schindelin\n> <johannes.schindelin@gmx.de> wrote:\n>>\n>> On 2015-03-25 01:46, Duy Nguyen wrote:\n>>> On Wed, Mar 25, 2015 at 6:50 AM, Philip Oakley <philipoakley@iee.org> wrote:\n>>>\n>>>> That said, the final error (which I'd missed in the earlier post) is:\n>>>> fatal: make_cache_entry failed for path 'ifcfg-eth0:0'\n>>>>\n>>>> This is on the Windows (pre-compiled msysgit at v1.9.5) Git bash, so could\n>>>> be a catch path in that code for make_cache_entry (I've not checked the code\n>>>> yet). So at the moment it doesn't look like sparse checkout can be used to\n>>>> avoid colons in windows on-disk files based on the current code.\n>>>\n>>> Both of your commands below fail by the same function, verify_path()\n>>> because of this msysgit commit 2e2a2d1 (NTFS: Prevent problematic\n>>> paths from being checked out - 2014-12-10). I guess that check is a\n>>> bit too strong, it should apply when new index entries are created\n>>> from worktree (not from a tree)..\n>>\n>> Oh, right, that check needs some relaxing. But certainly in a different way: you *definitely* want to prevent attacks from the outside, and those originate from the *tree*.\n>>\n>> What we need to do instead is to relax the check to apply only if we are really going to write out that file, not when it is skipped.\n> \n> Yeah. In fact if you do this, not checking out  files that can't be or\n> not safe to be checked out, then the ifcfg-eth0:0 problem that Phillip\n> wanted to avoid using sparse checkout is also gone.\n\nI had a quick look at the `verify_path()` function and have to say: it is more complicated than this. At the point when this function is called, we are only concerned with the index. So we have no idea whether we are going to write this to disk or not.\n\nI guess there is a much easier solution to this: set `core.protectntfs` to `false` in this specific repository, before checking out any files.\n\nCiao,\nJohannes\n\n-- \n-- \n*** Please reply-to-all at all times ***\n*** (do not pretend to know who is subscribed and who is not) ***\n*** Please avoid top-posting. ***\nThe msysGit Wiki is here: https://github.com/msysgit/msysgit/wiki - Github accounts are free.\n\nYou received this message because you are subscribed to the Google\nGroups \"msysGit\" group.\nTo post to this group, send email to msysgit@googlegroups.com\nTo unsubscribe from this group, send email to\nmsysgit+unsubscribe@googlegroups.com\nFor more options, and view previous threads, visit this group at\nhttp://groups.google.com/group/msysgit?hl=en_US?hl=en\n\n--- \nYou received this message because you are subscribed to the Google Groups \"Git for Windows\" group.\nTo unsubscribe from this group and stop receiving emails from it, send an email to msysgit+unsubscribe@googlegroups.com.\nFor more options, visit https://groups.google.com/d/optout.\n"}]}