{"thread":{"id":"47205","subject":"Changing encoding of a file : What should happen to CRLF in file ?","startedAt":"2017-11-14T12:31:36Z","lastAt":"2017-12-08T18:51:08Z","messageCount":23,"participants":["Ashish Negi","Torsten Bögershausen","tboegi@web.de","Eric Sunshine","Junio C Hamano"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"332535","messageId":"CAJ_+vJ6FXXda4fe7=1YxtDGR2d8CqP4KXN+YR6+mdQ+5jQQXug@mail.gmail.com","threadId":"47205","inReplyTo":null,"subject":"Changing encoding of a file : What should happen to CRLF in file ?","fromName":"Ashish Negi","fromEmail":"ashishnegi33@gmail.com","sentAt":"2017-11-14T12:31:30Z","receivedAt":"2017-11-14T12:31:36Z","isPatch":false,"sender":{"key":"ashishnegi33@gmail.com","avatar":null},"body":"Hello\n\nI have a cross platform project. I have a utf-16 file in it.\nI changed its encoding to urf-8 and committed. When i pulled the file\nin Linux, it shows that file is modified. This means that the commit\nwhich changed the encoding does not convert crlf to lf, when new\nformat is text based (utf-8).\n\nSteps to reproduce:\n\n   In windows :\n\n    Change encoding of file from utf-16 to utf-8.\n    Commit the change.\n\n   In linux:\n\n    Pull your branch.\n    You will see the issue of file being shown as modified even though\nyou have not used it.\n\n\nIf i change the file encoding in linux and commit. Then if i do git\npull in windows, i don't see the issue.\nIn linux, during committing i get warning : warning: CRLF will be\nreplaced by LF in …file_name..\n\nHere are my configuration :\n\n\n> git config --global --get core.autocrlf\n\nfalse\n\n\n> git config  --get core.autocrlf\n\nfalse\n\n\n\n> cat E:\\work\\WindowsFabric\\.gitattributes\n\n\n# Set the default behavior, in case people don't have core.autocrlf set.\n\n* text=auto\n*.vcxproj eol=crlf\n*.sh      eol=lf\n\n# Denote all files that are truly binary and should not be modified.\n*.exe binary\n*.dll binary\n*.pdb binary\n*.ico binary\n*.png binary\n*.jpg binary\n\n\n> git --version\ngit version 2.14.2.windows.2\n\n\nI played around with core.autocrlf values like true and native, but\nthat didn't help.\n\nThe behavior is inconsistent across platforms, and windows version is\ngiving me problem.\n\nCan someone suggest right settings for this ? Or is this a bug in Git.\n\nThanks\n"},{"id":"332544","messageId":"8b3225ce-a4aa-56ee-5296-6cc7528556d1@web.de","threadId":"47205","inReplyTo":"CAJ_+vJ6FXXda4fe7=1YxtDGR2d8CqP4KXN+YR6+mdQ+5jQQXug@mail.gmail.com","subject":"Re: Changing encoding of a file : What should happen to CRLF in file ?","fromName":"Torsten Bögershausen","fromEmail":"tboegi@web.de","sentAt":"2017-11-14T15:20:20Z","receivedAt":"2017-11-14T15:20:40Z","isPatch":false,"sender":{"key":"tboegi@web.de","avatar":"https://avatars.githubusercontent.com/u/7138363?v=4"},"body":"On 2017-11-14 13:31, Ashish Negi wrote:\n> Hello\n> \n> I have a cross platform project. I have a utf-16 file in it.\n> I changed its encoding to urf-8 and committed. When i pulled the file\n> in Linux, it shows that file is modified. This means that the commit\n> which changed the encoding does not convert crlf to lf, when new\n> format is text based (utf-8).\n> \n> Steps to reproduce:\n> \n>    In windows :\n> \n>     Change encoding of file from utf-16 to utf-8.\n>     Commit the change.\n> \n>    In linux:\n> \n>     Pull your branch.\n>     You will see the issue of file being shown as modified even though\n> you have not used it.\n> \n> \n> If i change the file encoding in linux and commit. Then if i do git\n> pull in windows, i don't see the issue.\n> In linux, during committing i get warning : warning: CRLF will be\n> replaced by LF in …file_name..\n> \n> Here are my configuration :\n> \n> \n>> git config --global --get core.autocrlf\n> \n> false\n> \n> \n>> git config  --get core.autocrlf\n> \n> false\n> \n> \n> \n>> cat E:\\work\\WindowsFabric\\.gitattributes\n> \n> \n> # Set the default behavior, in case people don't have core.autocrlf set.\n> \n> * text=auto\n> *.vcxproj eol=crlf\n> *.sh      eol=lf\n> \n> # Denote all files that are truly binary and should not be modified.\n> *.exe binary\n> *.dll binary\n> *.pdb binary\n> *.ico binary\n> *.png binary\n> *.jpg binary\n> \n> \n>> git --version\n> git version 2.14.2.windows.2\n> \n> \n> I played around with core.autocrlf values like true and native, but\n> that didn't help.\n> \n> The behavior is inconsistent across platforms, and windows version is\n> giving me problem.\n> \n> Can someone suggest right settings for this ? Or is this a bug in Git.\n> \n\nI don't think it is a bug :-)\nWhat does\ngit ls-files --eol …file_name\ngive you under Windows ?\n\n\n\n"},{"id":"332549","messageId":"CAJ_+vJ5J250CtzVg4QwEusddviDSYuJhubsbTJyv5Nc2conAfA@mail.gmail.com","threadId":"47205","inReplyTo":"8b3225ce-a4aa-56ee-5296-6cc7528556d1@web.de","subject":"Re: Changing encoding of a file : What should happen to CRLF in file ?","fromName":"Ashish Negi","fromEmail":"ashishnegi33@gmail.com","sentAt":"2017-11-14T16:13:41Z","receivedAt":"2017-11-14T16:13:46Z","isPatch":false,"sender":{"key":"ashishnegi33@gmail.com","avatar":null},"body":"Running the command gives me :\n\n      git ls-files --eol file_name\n      i/-text w/-text attr/text=auto          file_name\n"},{"id":"332550","messageId":"CAJ_+vJ7Yfcpz5252M4XJnDmEDCANp+eJ7RLJJF8TCcTxexZEUA@mail.gmail.com","threadId":"47205","inReplyTo":"CAJ_+vJ5J250CtzVg4QwEusddviDSYuJhubsbTJyv5Nc2conAfA@mail.gmail.com","subject":"Re: Changing encoding of a file : What should happen to CRLF in file ?","fromName":"Ashish Negi","fromEmail":"ashishnegi33@gmail.com","sentAt":"2017-11-14T16:15:12Z","receivedAt":"2017-11-14T16:15:17Z","isPatch":false,"sender":{"key":"ashishnegi33@gmail.com","avatar":null},"body":"After changing the encoding of file to utf-8, same command gives:\n\ngit ls-files --eol file_name\ni/lf    w/crlf  attr/text=auto          ApplicationManifest.xml\n\nOn Tue, Nov 14, 2017 at 9:43 PM, Ashish Negi <ashishnegi33@gmail.com> wrote:\n> Running the command gives me :\n>\n>       git ls-files --eol file_name\n>       i/-text w/-text attr/text=auto          file_name\n"},{"id":"332552","messageId":"9d1f6549-fcf7-e96f-a15d-ca19c51c7fbd@web.de","threadId":"47205","inReplyTo":"CAJ_+vJ5J250CtzVg4QwEusddviDSYuJhubsbTJyv5Nc2conAfA@mail.gmail.com","subject":"Re: Changing encoding of a file : What should happen to CRLF in file ?","fromName":"Torsten Bögershausen","fromEmail":"tboegi@web.de","sentAt":"2017-11-14T16:45:49Z","receivedAt":"2017-11-14T16:46:00Z","isPatch":false,"sender":{"key":"tboegi@web.de","avatar":"https://avatars.githubusercontent.com/u/7138363?v=4"},"body":"On 2017-11-14 17:13, Ashish Negi wrote:\n> Running the command gives me :\n> \n>       git ls-files --eol file_name\n>       i/-text w/-text attr/text=auto          file_name\n> \n\nThat is strange to me:\nAccording to that, Git would treat the file as text=auto.\nAnd the content is \"not next\", so there is no need to convert.\nDo you have configured any filters ?\n\nIs this a public repo ?\nif not: Is there any chance that you provide a public example,\nso that we can have a look ?\n\n\n"},{"id":"332554","messageId":"f9dc6482-587d-50a9-d649-aed63be18fad@web.de","threadId":"47205","inReplyTo":"CAJ_+vJ7Yfcpz5252M4XJnDmEDCANp+eJ7RLJJF8TCcTxexZEUA@mail.gmail.com","subject":"Re: Changing encoding of a file : What should happen to CRLF in file ?","fromName":"Torsten Bögershausen","fromEmail":"tboegi@web.de","sentAt":"2017-11-14T17:09:57Z","receivedAt":"2017-11-14T17:10:04Z","isPatch":false,"sender":{"key":"tboegi@web.de","avatar":"https://avatars.githubusercontent.com/u/7138363?v=4"},"body":"(Back to the beginning)\n\nYou have a file ApplicationManifest.xml\nIt is encoded in UTF-16 (and has CRLF)\n\nYou convert it into UTF-8\nThe file has still CRLF (in the worktree)\n\nNow you add it and make a commit.\nUnder both Linux and Windows you have \"text=auto\".\n\nI assume that you have efficiently core.eol=lf under Linux\nand core.eol=crlf on Windows.\n\n(That is the default, when you don't change anything)\n\nNow, what happens to the CRLF?\nIf you commit the file, it will be stored with LF in the index,\non both systems.\nOn checkout, Windows will convert them into CRLF, but Linux will not.\n\nThat why you see\n>On linux, during committing i get warning : warning: CRLF will be\n>replaced by LF in …file_name..\n\nAll in all there is nothing wrong, at least as I see it.\n\nThe question remains:\nDo you need CRLF in Linux ?\nProbably not, but if yes, plase add a line\n\n*.xml text eol=crlf\n\nto your\n.gitattributes\n\nOtherwise your .gitconfig looks good to  me.\n\n\n\n\n\n\n"},{"id":"332597","messageId":"CAJ_+vJ6SxLOmZfG79Sa-vskBR-XG=C97--PB-vpijjUKym=jYw@mail.gmail.com","threadId":"47205","inReplyTo":"f9dc6482-587d-50a9-d649-aed63be18fad@web.de","subject":"Re: Changing encoding of a file : What should happen to CRLF in file ?","fromName":"Ashish Negi","fromEmail":"ashishnegi33@gmail.com","sentAt":"2017-11-15T08:11:42Z","receivedAt":"2017-11-15T08:11:49Z","isPatch":false,"sender":{"key":"ashishnegi33@gmail.com","avatar":null},"body":"> If you commit the file, it will be stored with LF in the index,\nThis is what i believe is not happening.\n\nLets do this with a public repository and steps which are reproducible.\nI have created a repo : https://github.com/ashishnegi/git_encoding\n\nIf you clone this repo in linux and run `git status`, you will find\nthat file is modified.\n\nAbout repo :\nRepo have 2 commits, done on windows machine.\nFirst one check in a utf-16le encoded file which has crlf. crlf will\nnot be converted to lf in index as git treats it as binary file.\n2nd commit changes encoding to utf-8 and commits.\nThis commit does not change crlf to lf in index, even though new\nformat is utf-8 which is text based for git. This is the crux of\nproblem.\n\nI have attached all commands i ran on windows while creating the repo.\nI tried to capture all information that i could give.\nPlease have a look. It might be useful.\n\nFinally, thank you Torsten for giving your time to the problem. Really\nappreciate it.\n\nOn Tue, Nov 14, 2017 at 10:39 PM, Torsten Bögershausen <tboegi@web.de> wrote:\n> (Back to the beginning)\n>\n> You have a file ApplicationManifest.xml\n> It is encoded in UTF-16 (and has CRLF)\n>\n> You convert it into UTF-8\n> The file has still CRLF (in the worktree)\n>\n> Now you add it and make a commit.\n> Under both Linux and Windows you have \"text=auto\".\n>\n> I assume that you have efficiently core.eol=lf under Linux\n> and core.eol=crlf on Windows.\n>\n> (That is the default, when you don't change anything)\n>\n> Now, what happens to the CRLF?\n> If you commit the file, it will be stored with LF in the index,\n> on both systems.\n> On checkout, Windows will convert them into CRLF, but Linux will not.\n>\n> That why you see\n>>On linux, during committing i get warning : warning: CRLF will be\n>>replaced by LF in …file_name..\n>\n> All in all there is nothing wrong, at least as I see it.\n>\n> The question remains:\n> Do you need CRLF in Linux ?\n> Probably not, but if yes, plase add a line\n>\n> *.xml text eol=crlf\n>\n> to your\n> .gitattributes\n>\n> Otherwise your .gitconfig looks good to  me.\n>\n>\n>\n>\n>\n>\n\n\ngit_encoding_repo>git config --global core.safecrlf\ntrue\n\ngit_encoding_repo>git config  core.autocrlf\nfalse\n\ngit_encoding_repo>git config --get core.autocrlf\nfalse\n\ngit_encoding_repo>cat .gitattributes\n# Set the default behavior, in case people don't have core.autocrlf set.\n* text=auto\n\n*.vcxproj eol=crlf\n*.sh      eol=lf\n\n# Denote all files that are truly binary and should not be modified.\n*.exe binary\n*.dll binary\n*.pdb binary\n*.ico binary\n*.png binary\n*.jpg binary\n\ngit_encoding_repo>git status\nOn branch master\n\nNo commits yet\n\nUntracked files:\n  (use \"git add <file>...\" to include in what will be committed)\n\n        .gitattributes\n        file_name.txt\n\nnothing added to commit but untracked files present (use \"git add\" to track)\n\ngit_encoding_repo>git ls-files --eol file_name.txt\n\ngit_encoding_repo>git add .\n\ngit_encoding_repo>git status\nOn branch master\n\nNo commits yet\n\nChanges to be committed:\n  (use \"git rm --cached <file>...\" to unstage)\n\n        new file:   .gitattributes\n        new file:   file_name.txt\n\n\ngit_encoding_repo>git ls-files --eol file_name.txt\ni/-text w/-text attr/text=auto          file_name.txt\n\ngit_encoding_repo>git commit -m \"Commit utf-16le encoded file which has crlf.\"\n[master (root-commit) 91fe3bd] Commit utf-16le encoded file which has crlf.\n 2 files changed, 13 insertions(+)\n create mode 100644 .gitattributes\n create mode 100644 file_name.txt\n\n## At this time, i changed file encoding to utf-8.\n\ngit_encoding_repo>git status\nOn branch master\nnothing to commit, working tree clean\n\ngit_encoding_repo>git add -p\nOnly binary files changed.\n\ngit_encoding_repo>git status\nOn branch master\nChanges not staged for commit:\n  (use \"git add <file>...\" to update what will be committed)\n  (use \"git checkout -- <file>...\" to discard changes in working directory)\n\n        modified:   file_name.txt\n\nno changes added to commit (use \"git add\" and/or \"git commit -a\")\n\ngit_encoding_repo>git add file_name.txt\n\ngit_encoding_repo>git status\nOn branch master\nChanges to be committed:\n  (use \"git reset HEAD <file>...\" to unstage)\n\n        modified:   file_name.txt\n\n\ngit_encoding_repo>git commit -m \"Change encoding of file to utf-8\"\n[master 179c27b] Change encoding of file to utf-8\n 1 file changed, 0 insertions(+), 0 deletions(-)\n rewrite file_name.txt (100%)\n\ngit_encoding_repo>git remote add origin https://github.com/ashishnegi/git_encoding.git\n\ngit_encoding_repo>git push -u origin master\nCounting objects: 7, done.\nDelta compression using up to 12 threads.\nCompressing objects: 100% (7/7), done.\nWriting objects: 100% (7/7), 837 bytes | 837.00 KiB/s, done.\nTotal 7 (delta 0), reused 0 (delta 0)\nTo https://github.com/ashishnegi/git_encoding.git\n * [new branch]      master -> master\nBranch master set up to track remote branch master from origin.\n\ngit_encoding_repo>git ls-files --eol file_name.txt\ni/crlf  w/crlf  attr/text=auto          file_name.txt\n\ngit_encoding_repo>"},{"id":"332608","messageId":"20171115171258.GA12963@tor.lan","threadId":"47205","inReplyTo":"CAJ_+vJ6SxLOmZfG79Sa-vskBR-XG=C97--PB-vpijjUKym=jYw@mail.gmail.com","subject":"Re: Changing encoding of a file : What should happen to CRLF in file ?","fromName":"Torsten Bögershausen","fromEmail":"tboegi@web.de","sentAt":"2017-11-15T17:12:58Z","receivedAt":"2017-11-15T17:13:10Z","isPatch":false,"sender":{"key":"tboegi@web.de","avatar":"https://avatars.githubusercontent.com/u/7138363?v=4"},"body":"On Wed, Nov 15, 2017 at 01:41:42PM +0530, Ashish Negi wrote:\n> > If you commit the file, it will be stored with LF in the index,\n> This is what i believe is not happening.\n> \n> Lets do this with a public repository and steps which are reproducible.\n> I have created a repo : https://github.com/ashishnegi/git_encoding\n> \n> If you clone this repo in linux and run `git status`, you will find\n> that file is modified.\n\nThis is what what I get:\n\n git ls-files --eol\ni/lf    w/lf    attr/text=auto          .gitattributes\ni/crlf  w/crlf  attr/text=auto          file_name.txt\n\n(And you get the same, I think)\nNewer versions of Git (>=2.10) will -not- treat file_name.txt as changed,\nolder versions do.\nWhat does \"git --version\" say on your Linux machine ?\n\nHowever, if you want to fix it, you want to either end up with\n\nA)\ngit ls-files --eol\ni/lf    w/lf    attr/text=auto          .gitattributes\ni/lf    w/crlf  attr/text=auto          file_name.txt\n\nor\nB)\ngit ls-files --eol\ni/lf    w/lf    attr/text=auto          .gitattributes\ni/crlf  w/crlf  attr/-text              file_name.txt\n\n(The \"attr/-text\" means \"don't change the line endings\")\n\nBoth A) or B) will work, typically A) is preferred.\n\nYou should be able to achive A) :\n git rm --cached file_name.txt  && git add file_name.txt \nrm 'file_name.txt'\nwarning: CRLF will be replaced by LF in file_name.txt.\nThe file will have its original line endings in your working directory.\n\ngit ls-files --eol\ni/lf    w/lf    attr/text=auto          .gitattributes\ni/lf    w/crlf  attr/text=auto          file_name.txt\n\n[snip the rest]\n"},{"id":"332614","messageId":"CAJ_+vJ7am7dU9B0qTYVtxpkHoZdV45ciZN5ifbv9W8xRO2o=EQ@mail.gmail.com","threadId":"47205","inReplyTo":"20171115171258.GA12963@tor.lan","subject":"Re: Changing encoding of a file : What should happen to CRLF in file ?","fromName":"Ashish Negi","fromEmail":"ashishnegi33@gmail.com","sentAt":"2017-11-15T19:05:33Z","receivedAt":"2017-11-15T19:05:40Z","isPatch":false,"sender":{"key":"ashishnegi33@gmail.com","avatar":null},"body":"On windows :\n> git --version\ngit version 2.14.2.windows.2\n\nOn linux :\n> git --version\ngit version 2.7.4\n\nI would like to understand the solution :\nIf i understood it correctly : it removes file_name.txt from index, so\ngit forgets about it.\nwe then add the file again after changing encoding. This time, git\ntakes it as utf-8 file and converts crlf to lf when storing it\ninternally.\nRight ?\n\nThank you for the support.\n\n\nOn Wed, Nov 15, 2017 at 10:42 PM, Torsten Bögershausen <tboegi@web.de> wrote:\n> On Wed, Nov 15, 2017 at 01:41:42PM +0530, Ashish Negi wrote:\n>> > If you commit the file, it will be stored with LF in the index,\n>> This is what i believe is not happening.\n>>\n>> Lets do this with a public repository and steps which are reproducible.\n>> I have created a repo : https://github.com/ashishnegi/git_encoding\n>>\n>> If you clone this repo in linux and run `git status`, you will find\n>> that file is modified.\n>\n> This is what what I get:\n>\n>  git ls-files --eol\n> i/lf    w/lf    attr/text=auto          .gitattributes\n> i/crlf  w/crlf  attr/text=auto          file_name.txt\n>\n> (And you get the same, I think)\n> Newer versions of Git (>=2.10) will -not- treat file_name.txt as changed,\n> older versions do.\n> What does \"git --version\" say on your Linux machine ?\n>\n> However, if you want to fix it, you want to either end up with\n>\n> A)\n> git ls-files --eol\n> i/lf    w/lf    attr/text=auto          .gitattributes\n> i/lf    w/crlf  attr/text=auto          file_name.txt\n>\n> or\n> B)\n> git ls-files --eol\n> i/lf    w/lf    attr/text=auto          .gitattributes\n> i/crlf  w/crlf  attr/-text              file_name.txt\n>\n> (The \"attr/-text\" means \"don't change the line endings\")\n>\n> Both A) or B) will work, typically A) is preferred.\n>\n> You should be able to achive A) :\n>  git rm --cached file_name.txt  && git add file_name.txt\n> rm 'file_name.txt'\n> warning: CRLF will be replaced by LF in file_name.txt.\n> The file will have its original line endings in your working directory.\n>\n> git ls-files --eol\n> i/lf    w/lf    attr/text=auto          .gitattributes\n> i/lf    w/crlf  attr/text=auto          file_name.txt\n>\n> [snip the rest]\n"},{"id":"332669","messageId":"20171116161506.GA20809@tor.lan","threadId":"47205","inReplyTo":"CAJ_+vJ7am7dU9B0qTYVtxpkHoZdV45ciZN5ifbv9W8xRO2o=EQ@mail.gmail.com","subject":"Re: Changing encoding of a file : What should happen to CRLF in file ?","fromName":"Torsten Bögershausen","fromEmail":"tboegi@web.de","sentAt":"2017-11-16T16:15:06Z","receivedAt":"2017-11-16T16:15:14Z","isPatch":false,"sender":{"key":"tboegi@web.de","avatar":"https://avatars.githubusercontent.com/u/7138363?v=4"},"body":"On Thu, Nov 16, 2017 at 12:35:33AM +0530, Ashish Negi wrote:\n> On windows :\n> > git --version\n> git version 2.14.2.windows.2\n> \n> On linux :\n> > git --version\n> git version 2.7.4\n> \n> I would like to understand the solution :\n> If i understood it correctly : it removes file_name.txt from index, so\n> git forgets about it.\n> we then add the file again after changing encoding. This time, git\n> takes it as utf-8 file and converts crlf to lf when storing it\n> internally.\n> Right ?\n\nYes, exactly.\n(In a coming release of Git there will be a \"git add --renormalize <pathspec>\" )\n\n> \n> Thank you for the support.\n> \n\nThanks for a clean bug report.\nActually it is a bug, I put it on my to do list\n\n\n"},{"id":"333387","messageId":"CAJ_+vJ74mocjicpQ2BGUT+TrqrWqidp0bUJUGSz-B40-bC-c6w@mail.gmail.com","threadId":"47205","inReplyTo":"20171116161506.GA20809@tor.lan","subject":"Re: Changing encoding of a file : What should happen to CRLF in file ?","fromName":"Ashish Negi","fromEmail":"ashishnegi33@gmail.com","sentAt":"2017-11-23T16:31:40Z","receivedAt":"2017-11-23T16:31:47Z","isPatch":false,"sender":{"key":"ashishnegi33@gmail.com","avatar":null},"body":"Thanks for confirming.\nIs it possible to track this via a bug number ?\nIt will help me to try out the fix when its available.\n\n\nOn Thu, Nov 16, 2017 at 9:45 PM, Torsten Bögershausen <tboegi@web.de> wrote:\n> On Thu, Nov 16, 2017 at 12:35:33AM +0530, Ashish Negi wrote:\n>> On windows :\n>> > git --version\n>> git version 2.14.2.windows.2\n>>\n>> On linux :\n>> > git --version\n>> git version 2.7.4\n>>\n>> I would like to understand the solution :\n>> If i understood it correctly : it removes file_name.txt from index, so\n>> git forgets about it.\n>> we then add the file again after changing encoding. This time, git\n>> takes it as utf-8 file and converts crlf to lf when storing it\n>> internally.\n>> Right ?\n>\n> Yes, exactly.\n> (In a coming release of Git there will be a \"git add --renormalize <pathspec>\" )\n>\n>>\n>> Thank you for the support.\n>>\n>\n> Thanks for a clean bug report.\n> Actually it is a bug, I put it on my to do list\n>\n>\n"},{"id":"333391","messageId":"20171123202548.GA30957@tor.lan","threadId":"47205","inReplyTo":"CAJ_+vJ74mocjicpQ2BGUT+TrqrWqidp0bUJUGSz-B40-bC-c6w@mail.gmail.com","subject":"Re: Changing encoding of a file : What should happen to CRLF in file ?","fromName":"Torsten Bögershausen","fromEmail":"tboegi@web.de","sentAt":"2017-11-23T20:25:48Z","receivedAt":"2017-11-23T20:25:55Z","isPatch":false,"sender":{"key":"tboegi@web.de","avatar":"https://avatars.githubusercontent.com/u/7138363?v=4"},"body":"On Thu, Nov 23, 2017 at 10:01:40PM +0530, Ashish Negi wrote:\n> Thanks for confirming.\n> Is it possible to track this via a bug number ?\n> It will help me to try out the fix when its available.\n> \n\nNo worry, the fix is nearly complete and will come out in a couple of days.\n"},{"id":"333414","messageId":"CAJ_+vJ7-M2fek8nFQgqYPs9pcAnYgenuvG50GC3ZRuPCc=R2EQ@mail.gmail.com","threadId":"47205","inReplyTo":"20171123202548.GA30957@tor.lan","subject":"Re: Changing encoding of a file : What should happen to CRLF in file ?","fromName":"Ashish Negi","fromEmail":"ashishnegi33@gmail.com","sentAt":"2017-11-24T06:37:16Z","receivedAt":"2017-11-24T06:37:23Z","isPatch":false,"sender":{"key":"ashishnegi33@gmail.com","avatar":null},"body":"Great work !!!\nThanks\n\nOn Fri, Nov 24, 2017 at 1:55 AM, Torsten Bögershausen <tboegi@web.de> wrote:\n> On Thu, Nov 23, 2017 at 10:01:40PM +0530, Ashish Negi wrote:\n>> Thanks for confirming.\n>> Is it possible to track this via a bug number ?\n>> It will help me to try out the fix when its available.\n>>\n>\n> No worry, the fix is nearly complete and will come out in a couple of days.\n"},{"id":"333446","messageId":"20171124161407.30698-1-tboegi@web.de","threadId":"47205","inReplyTo":"CAJ_+vJ6FXXda4fe7=1YxtDGR2d8CqP4KXN+YR6+mdQ+5jQQXug@mail.gmail.com","subject":"[PATCH 1/1] convert: tighten the safe autocrlf handling","fromName":"","fromEmail":"tboegi@web.de","sentAt":"2017-11-24T16:14:07Z","receivedAt":"2017-11-24T16:14:19Z","isPatch":true,"sender":{"key":"tboegi@web.de","avatar":"https://avatars.githubusercontent.com/u/7138363?v=4"},"body":"From: Torsten Bögershausen <tboegi@web.de>\n\nWhen a text file had been commited with CRLF and the file is commited\nagain, the CRLF are kept if .gitattributs has \"text=auto\".\nThis is done by analyzing the content of the blob stored in the index:\nIf a '\\r' is found, Git assumes that the blob was commited with CRLF.\n\nThe simple search for a '\\r' does not always work as expected:\nA file is encoded in UTF-16 with CRLF and commited. Git treats it as binary.\nNow the content is converted into UTF-8. At the next commit Git treats the\nfile as text, the CRLF should be converted into LF, but isn't.\n\nSolution:\nReplace has_cr_in_index() with has_crlf_in_index(). When no '\\r' is found,\n0 is returned directly, this is the most common case.\nIf a '\\r' is found, the content is analyzed more deeply.\n\nReported-By: Ashish Negi <ashishnegi33@gmail.com>\nSigned-off-by: Torsten Bögershausen <tboegi@web.de>\n---\n convert.c            | 19 +++++++++----\n t/t0027-auto-crlf.sh | 76 ++++++++++++++++++++++++++++++++++++++++++++++++----\n 2 files changed, 85 insertions(+), 10 deletions(-)\n\ndiff --git a/convert.c b/convert.c\nindex 20d7ab67bd..63ef799239 100644\n--- a/convert.c\n+++ b/convert.c\n@@ -220,18 +220,27 @@ static void check_safe_crlf(const char *path, enum crlf_action crlf_action,\n \t}\n }\n \n-static int has_cr_in_index(const struct index_state *istate, const char *path)\n+static int has_crlf_in_index(const struct index_state *istate, const char *path)\n {\n \tunsigned long sz;\n \tvoid *data;\n-\tint has_cr;\n+\tconst char *crp;\n+\tint has_crlf = 0;\n \n \tdata = read_blob_data_from_index(istate, path, &sz);\n \tif (!data)\n \t\treturn 0;\n-\thas_cr = memchr(data, '\\r', sz) != NULL;\n+\n+\tcrp = memchr(data, '\\r', sz);\n+\tif (crp && (crp[1] == '\\n')) {\n+\t\tunsigned int ret_stats;\n+\t\tret_stats = gather_convert_stats(data, sz);\n+\t\tif (!(ret_stats & CONVERT_STAT_BITS_BIN) &&\n+\t\t    (ret_stats & CONVERT_STAT_BITS_TXT_CRLF))\n+\t\t\thas_crlf = 1;\n+\t}\n \tfree(data);\n-\treturn has_cr;\n+\treturn has_crlf;\n }\n \n static int will_convert_lf_to_crlf(size_t len, struct text_stat *stats,\n@@ -290,7 +299,7 @@ static int crlf_to_git(const struct index_state *istate,\n \t\t * cherry-pick.\n \t\t */\n \t\tif ((checksafe != SAFE_CRLF_RENORMALIZE) &&\n-\t\t    has_cr_in_index(istate, path))\n+\t\t    has_crlf_in_index(istate, path))\n \t\t\tconvert_crlf_into_lf = 0;\n \t}\n \tif ((checksafe == SAFE_CRLF_WARN ||\ndiff --git a/t/t0027-auto-crlf.sh b/t/t0027-auto-crlf.sh\nindex 68108d956a..0af35cfb1f 100755\n--- a/t/t0027-auto-crlf.sh\n+++ b/t/t0027-auto-crlf.sh\n@@ -43,19 +43,31 @@ create_gitattributes () {\n \t} >.gitattributes\n }\n \n-create_NNO_files () {\n+# Create 2 sets of files:\n+# The NNO files are \"Not NOrmalized in the repo. We use CRLF_mix_LF and store\n+#   it under different names for the different test cases, see ${pfx}\n+#   Depending on .gitattributes they are normalized at the next commit (or not)\n+# The MIX files have different contents in the repo.\n+#   Depending on its contents, the \"new safer autocrlf\" may kick in.\n+create_NNO_MIX_files () {\n \tfor crlf in false true input\n \tdo\n \t\tfor attr in \"\" auto text -text\n \t\tdo\n \t\t\tfor aeol in \"\" lf crlf\n \t\t\tdo\n-\t\t\t\tpfx=NNO_attr_${attr}_aeol_${aeol}_${crlf}\n+\t\t\t\tpfx=NNO_attr_${attr}_aeol_${aeol}_${crlf} &&\n \t\t\t\tcp CRLF_mix_LF ${pfx}_LF.txt &&\n \t\t\t\tcp CRLF_mix_LF ${pfx}_CRLF.txt &&\n \t\t\t\tcp CRLF_mix_LF ${pfx}_CRLF_mix_LF.txt &&\n \t\t\t\tcp CRLF_mix_LF ${pfx}_LF_mix_CR.txt &&\n-\t\t\t\tcp CRLF_mix_LF ${pfx}_CRLF_nul.txt\n+\t\t\t\tcp CRLF_mix_LF ${pfx}_CRLF_nul.txt &&\n+\t\t\t\tpfx=MIX_attr_${attr}_aeol_${aeol}_${crlf} &&\n+\t\t\t\tcp LF          ${pfx}_LF.txt &&\n+\t\t\t\tcp CRLF        ${pfx}_CRLF.txt &&\n+\t\t\t\tcp CRLF_mix_LF ${pfx}_CRLF_mix_LF.txt &&\n+\t\t\t\tcp LF_mix_CR   ${pfx}_LF_mix_CR.txt &&\n+\t\t\t\tcp CRLF_nul    ${pfx}_CRLF_nul.txt\n \t\t\tdone\n \t\tdone\n \tdone\n@@ -136,6 +148,49 @@ commit_chk_wrnNNO () {\n \t'\n }\n \n+# Commit a file with mixed line endings on top of different files\n+# in the index. Check for warnings\n+commit_MIX_chkwrn () {\n+\tattr=$1 ; shift\n+\taeol=$1 ; shift\n+\tcrlf=$1 ; shift\n+\tlfwarn=$1 ; shift\n+\tcrlfwarn=$1 ; shift\n+\tlfmixcrlf=$1 ; shift\n+\tlfmixcr=$1 ; shift\n+\tcrlfnul=$1 ; shift\n+\tpfx=MIX_attr_${attr}_aeol_${aeol}_${crlf}\n+\t#Commit file with CLRF_mix_LF on top of existing file\n+\tcreate_gitattributes \"$attr\" $aeol &&\n+\tfor f in LF CRLF CRLF_mix_LF LF_mix_CR CRLF_nul\n+\tdo\n+\t\tfname=${pfx}_$f.txt &&\n+\t\tcp CRLF_mix_LF $fname &&\n+\t\tprintf Z >>\"$fname\" &&\n+\t\tgit -c core.autocrlf=$crlf add $fname 2>\"${pfx}_$f.err\"\n+\tdone\n+\n+\ttest_expect_success \"commit file with mixed EOL crlf=$crlf attr=$attr LF\" '\n+\t\tcheck_warning \"$lfwarn\" ${pfx}_LF.err\n+\t'\n+\ttest_expect_success \"commit file with mixed EOL attr=$attr aeol=$aeol crlf=$crlf CRLF\" '\n+\t\tcheck_warning \"$crlfwarn\" ${pfx}_CRLF.err\n+\t'\n+\n+\ttest_expect_success \"commit file with mixed EOL attr=$attr aeol=$aeol crlf=$crlf CRLF_mix_LF\" '\n+\t\tcheck_warning \"$lfmixcrlf\" ${pfx}_CRLF_mix_LF.err\n+\t'\n+\n+\ttest_expect_success \"commit file with mixed EOL attr=$attr aeol=$aeol crlf=$crlf LF_mix_cr\" '\n+\t\tcheck_warning \"$lfmixcr\" ${pfx}_LF_mix_CR.err\n+\t'\n+\n+\ttest_expect_success \"commit file with mixed EOL attr=$attr aeol=$aeol crlf=$crlf CRLF_nul\" '\n+\t\tcheck_warning \"$crlfnul\" ${pfx}_CRLF_nul.err\n+\t'\n+}\n+\n+\n stats_ascii () {\n \tcase \"$1\" in\n \tLF)\n@@ -323,8 +378,8 @@ test_expect_success 'setup master' '\n \tprintf \"\\$Id: 0000000000000000000000000000000000000000 \\$\\r\\nLINEONE\\r\\nLINETWO\\rLINETHREE\"   >CRLF_mix_CR &&\n \tprintf \"\\$Id: 0000000000000000000000000000000000000000 \\$\\r\\nLINEONEQ\\r\\nLINETWO\\r\\nLINETHREE\" | q_to_nul >CRLF_nul &&\n \tprintf \"\\$Id: 0000000000000000000000000000000000000000 \\$\\nLINEONEQ\\nLINETWO\\nLINETHREE\" | q_to_nul >LF_nul &&\n-\tcreate_NNO_files CRLF_mix_LF CRLF_mix_LF CRLF_mix_LF CRLF_mix_LF CRLF_mix_LF &&\n-\tgit -c core.autocrlf=false add NNO_*.txt &&\n+\tcreate_NNO_MIX_files CRLF_mix_LF CRLF_mix_LF CRLF_mix_LF CRLF_mix_LF CRLF_mix_LF &&\n+\tgit -c core.autocrlf=false add NNO_*.txt MIX_*.txt &&\n \tgit commit -m \"mixed line endings\" &&\n \ttest_tick\n '\n@@ -385,6 +440,17 @@ test_expect_success 'commit files attr=crlf' '\n \tcommit_check_warn input \"crlf\" \"LF_CRLF\" \"\"        \"LF_CRLF\" \"LF_CRLF\" \"\"\n '\n \n+# Commit \"CRLFmixLF\" on top of these files already in the repo:\n+# LF, CRLF, CRLFmixLF LF_mix_CR CRLFNULL\n+#                 attr                    LF        CRLF      CRLFmixLF   LF_mix_CR   CRLFNUL\n+commit_MIX_chkwrn \"\"      \"\"      false   \"\"        \"\"        \"\"          \"\"          \"\"\n+commit_MIX_chkwrn \"\"      \"\"      true    \"LF_CRLF\" \"\"        \"\"          \"LF_CRLF\"   \"LF_CRLF\"\n+commit_MIX_chkwrn \"\"      \"\"      input   \"CRLF_LF\" \"\"        \"\"          \"CRLF_LF\"   \"CRLF_LF\"\n+\n+commit_MIX_chkwrn \"auto\"  \"\"      false   \"CRLF_LF\" \"\"        \"\"          \"CRLF_LF\"   \"CRLF_LF\"\n+commit_MIX_chkwrn \"auto\"  \"\"      true    \"LF_CRLF\" \"\"        \"\"          \"LF_CRLF\"   \"LF_CRLF\"\n+commit_MIX_chkwrn \"auto\"  \"\"      input   \"CRLF_LF\" \"\"        \"\"          \"CRLF_LF\"   \"CRLF_LF\"\n+\n #                 attr                    LF        CRLF      CRLFmixLF   LF_mix_CR   CRLFNUL\n commit_chk_wrnNNO \"\"      \"\"      false   \"\"        \"\"        \"\"          \"\"          \"\"\n commit_chk_wrnNNO \"\"      \"\"      true    LF_CRLF   \"\"        \"\"          \"\"          \"\"\n-- \n2.15.0.278.gbd97f72a1b.dirty\n\n"},{"id":"333449","messageId":"CAPig+cT7=yLUVpmtutmTep5NBbSRNOL17dsOuVvn_Scu7_+p_w@mail.gmail.com","threadId":"47205","inReplyTo":"20171124161407.30698-1-tboegi@web.de","subject":"Re: [PATCH 1/1] convert: tighten the safe autocrlf handling","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2017-11-24T17:24:48Z","receivedAt":"2017-11-24T17:33:55Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Fri, Nov 24, 2017 at 11:14 AM,  <tboegi@web.de> wrote:\n> When a text file had been commited with CRLF and the file is commited\n> again, the CRLF are kept if .gitattributs has \"text=auto\".\n> This is done by analyzing the content of the blob stored in the index:\n> If a '\\r' is found, Git assumes that the blob was commited with CRLF.\n>\n> The simple search for a '\\r' does not always work as expected:\n> A file is encoded in UTF-16 with CRLF and commited. Git treats it as binary.\n> Now the content is converted into UTF-8. At the next commit Git treats the\n> file as text, the CRLF should be converted into LF, but isn't.\n>\n> Solution:\n> Replace has_cr_in_index() with has_crlf_in_index(). When no '\\r' is found,\n> 0 is returned directly, this is the most common case.\n> If a '\\r' is found, the content is analyzed more deeply.\n>\n> Signed-off-by: Torsten Bögershausen <tboegi@web.de>\n> ---\n> diff --git a/convert.c b/convert.c\n> @@ -220,18 +220,27 @@ static void check_safe_crlf(const char *path, enum crlf_action crlf_action,\n> -static int has_cr_in_index(const struct index_state *istate, const char *path)\n> +static int has_crlf_in_index(const struct index_state *istate, const char *path)\n>  {\n>         unsigned long sz;\n>         void *data;\n> -       int has_cr;\n> +       const char *crp;\n> +       int has_crlf = 0;\n>\n>         data = read_blob_data_from_index(istate, path, &sz);\n>         if (!data)\n>                 return 0;\n> -       has_cr = memchr(data, '\\r', sz) != NULL;\n> +\n> +       crp = memchr(data, '\\r', sz);\n> +       if (crp && (crp[1] == '\\n')) {\n\nIf I understand correctly, this isn't a NUL-terminated string and it\nmight be a binary blob, so if the lone CR in a file resides at the end\nof the file, won't this try looking for LF out-of-bounds? I would have\nexpected the conditional to be:\n\n    if (crp && crp - data + 1 < sz && crp[1] == '\\n') {\n\nor any equivalent variation.\n\n> +               unsigned int ret_stats;\n> +               ret_stats = gather_convert_stats(data, sz);\n> +               if (!(ret_stats & CONVERT_STAT_BITS_BIN) &&\n> +                   (ret_stats & CONVERT_STAT_BITS_TXT_CRLF))\n> +                       has_crlf = 1;\n> +       }\n>         free(data);\n> -       return has_cr;\n> +       return has_crlf;\n>  }\n"},{"id":"333454","messageId":"20171124185905.GA9736@tor.lan","threadId":"47205","inReplyTo":"CAPig+cT7=yLUVpmtutmTep5NBbSRNOL17dsOuVvn_Scu7_+p_w@mail.gmail.com","subject":"Re: [PATCH 1/1] convert: tighten the safe autocrlf handling","fromName":"Torsten Bögershausen","fromEmail":"tboegi@web.de","sentAt":"2017-11-24T18:59:05Z","receivedAt":"2017-11-24T18:59:34Z","isPatch":true,"sender":{"key":"tboegi@web.de","avatar":"https://avatars.githubusercontent.com/u/7138363?v=4"},"body":"On Fri, Nov 24, 2017 at 12:24:48PM -0500, Eric Sunshine wrote:\n> On Fri, Nov 24, 2017 at 11:14 AM,  <tboegi@web.de> wrote:\n> > When a text file had been commited with CRLF and the file is commited\n> > again, the CRLF are kept if .gitattributs has \"text=auto\".\n> > This is done by analyzing the content of the blob stored in the index:\n> > If a '\\r' is found, Git assumes that the blob was commited with CRLF.\n> >\n> > The simple search for a '\\r' does not always work as expected:\n> > A file is encoded in UTF-16 with CRLF and commited. Git treats it as binary.\n> > Now the content is converted into UTF-8. At the next commit Git treats the\n> > file as text, the CRLF should be converted into LF, but isn't.\n> >\n> > Solution:\n> > Replace has_cr_in_index() with has_crlf_in_index(). When no '\\r' is found,\n> > 0 is returned directly, this is the most common case.\n> > If a '\\r' is found, the content is analyzed more deeply.\n> >\n> > Signed-off-by: Torsten Bögershausen <tboegi@web.de>\n> > ---\n> > diff --git a/convert.c b/convert.c\n> > @@ -220,18 +220,27 @@ static void check_safe_crlf(const char *path, enum crlf_action crlf_action,\n> > -static int has_cr_in_index(const struct index_state *istate, const char *path)\n> > +static int has_crlf_in_index(const struct index_state *istate, const char *path)\n> >  {\n> >         unsigned long sz;\n> >         void *data;\n> > -       int has_cr;\n> > +       const char *crp;\n> > +       int has_crlf = 0;\n> >\n> >         data = read_blob_data_from_index(istate, path, &sz);\n> >         if (!data)\n> >                 return 0;\n> > -       has_cr = memchr(data, '\\r', sz) != NULL;\n> > +\n> > +       crp = memchr(data, '\\r', sz);\n> > +       if (crp && (crp[1] == '\\n')) {\n> \n> If I understand correctly, this isn't a NUL-terminated string and it\n> might be a binary blob, so if the lone CR in a file resides at the end\n> of the file, won't this try looking for LF out-of-bounds? I would have\n> expected the conditional to be:\n> \n>     if (crp && crp - data + 1 < sz && crp[1] == '\\n') {\n> \n> or any equivalent variation.\n> \n\nThe read_blob_data_from_index() function should always append a '\\0',\nregardless if the blob is binary or not.\n"},{"id":"333468","messageId":"xmqqlgivkpzg.fsf@gitster.mtv.corp.google.com","threadId":"47205","inReplyTo":"20171124161407.30698-1-tboegi@web.de","subject":"Re: [PATCH 1/1] convert: tighten the safe autocrlf handling","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-11-25T03:16:03Z","receivedAt":"2017-11-25T03:16:12Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"tboegi@web.de writes:\n\n> From: Torsten Bögershausen <tboegi@web.de>\n>\n> When a text file had been commited with CRLF and the file is commited\n> again, the CRLF are kept if .gitattributs has \"text=auto\".\n> This is done by analyzing the content of the blob stored in the index:\n> If a '\\r' is found, Git assumes that the blob was commited with CRLF.\n>\n> The simple search for a '\\r' does not always work as expected:\n> A file is encoded in UTF-16 with CRLF and commited. Git treats it as binary.\n> Now the content is converted into UTF-8. At the next commit Git treats the\n> file as text, the CRLF should be converted into LF, but isn't.\n>\n> Solution:\n\nRemove this line.\n\n> Replace has_cr_in_index() with has_crlf_in_index(). When no '\\r' is found,\n> 0 is returned directly, this is the most common case.\n> If a '\\r' is found, the content is analyzed more deeply.\n\nI may be recalling things incorrectly, but didn't an old version of\nthe code check CRLF explicitly, unlike the current implementation\nthat only check CRs?\n\nIn any case, I think we have accumulated enough cruft only to work\naround the issues caused by \"safe\" crlf.  I moderately strongly\nwonder if we should go back and think if that \"feature\" is adding\nmuch value, and remove it if it is not.\n\nIn the meantime, let's queue this fix on top of the \"safe crlf\nworkaround\" pile.\n\nThanks.\n\n>\n> Reported-By: Ashish Negi <ashishnegi33@gmail.com>\n> Signed-off-by: Torsten Bögershausen <tboegi@web.de>\n> ---\n>  convert.c            | 19 +++++++++----\n>  t/t0027-auto-crlf.sh | 76 ++++++++++++++++++++++++++++++++++++++++++++++++----\n>  2 files changed, 85 insertions(+), 10 deletions(-)\n>\n> diff --git a/convert.c b/convert.c\n> index 20d7ab67bd..63ef799239 100644\n> --- a/convert.c\n> +++ b/convert.c\n> @@ -220,18 +220,27 @@ static void check_safe_crlf(const char *path, enum crlf_action crlf_action,\n>  \t}\n>  }\n>  \n> -static int has_cr_in_index(const struct index_state *istate, const char *path)\n> +static int has_crlf_in_index(const struct index_state *istate, const char *path)\n>  {\n>  \tunsigned long sz;\n>  \tvoid *data;\n> -\tint has_cr;\n> +\tconst char *crp;\n> +\tint has_crlf = 0;\n>  \n>  \tdata = read_blob_data_from_index(istate, path, &sz);\n>  \tif (!data)\n>  \t\treturn 0;\n> -\thas_cr = memchr(data, '\\r', sz) != NULL;\n> +\n> +\tcrp = memchr(data, '\\r', sz);\n> +\tif (crp && (crp[1] == '\\n')) {\n> +\t\tunsigned int ret_stats;\n> +\t\tret_stats = gather_convert_stats(data, sz);\n> +\t\tif (!(ret_stats & CONVERT_STAT_BITS_BIN) &&\n> +\t\t    (ret_stats & CONVERT_STAT_BITS_TXT_CRLF))\n> +\t\t\thas_crlf = 1;\n> +\t}\n>  \tfree(data);\n> -\treturn has_cr;\n> +\treturn has_crlf;\n>  }\n>  \n>  static int will_convert_lf_to_crlf(size_t len, struct text_stat *stats,\n> @@ -290,7 +299,7 @@ static int crlf_to_git(const struct index_state *istate,\n>  \t\t * cherry-pick.\n>  \t\t */\n>  \t\tif ((checksafe != SAFE_CRLF_RENORMALIZE) &&\n> -\t\t    has_cr_in_index(istate, path))\n> +\t\t    has_crlf_in_index(istate, path))\n>  \t\t\tconvert_crlf_into_lf = 0;\n>  \t}\n>  \tif ((checksafe == SAFE_CRLF_WARN ||\n> diff --git a/t/t0027-auto-crlf.sh b/t/t0027-auto-crlf.sh\n> index 68108d956a..0af35cfb1f 100755\n> --- a/t/t0027-auto-crlf.sh\n> +++ b/t/t0027-auto-crlf.sh\n> @@ -43,19 +43,31 @@ create_gitattributes () {\n>  \t} >.gitattributes\n>  }\n>  \n> -create_NNO_files () {\n> +# Create 2 sets of files:\n> +# The NNO files are \"Not NOrmalized in the repo. We use CRLF_mix_LF and store\n> +#   it under different names for the different test cases, see ${pfx}\n> +#   Depending on .gitattributes they are normalized at the next commit (or not)\n> +# The MIX files have different contents in the repo.\n> +#   Depending on its contents, the \"new safer autocrlf\" may kick in.\n> +create_NNO_MIX_files () {\n>  \tfor crlf in false true input\n>  \tdo\n>  \t\tfor attr in \"\" auto text -text\n>  \t\tdo\n>  \t\t\tfor aeol in \"\" lf crlf\n>  \t\t\tdo\n> -\t\t\t\tpfx=NNO_attr_${attr}_aeol_${aeol}_${crlf}\n> +\t\t\t\tpfx=NNO_attr_${attr}_aeol_${aeol}_${crlf} &&\n>  \t\t\t\tcp CRLF_mix_LF ${pfx}_LF.txt &&\n>  \t\t\t\tcp CRLF_mix_LF ${pfx}_CRLF.txt &&\n>  \t\t\t\tcp CRLF_mix_LF ${pfx}_CRLF_mix_LF.txt &&\n>  \t\t\t\tcp CRLF_mix_LF ${pfx}_LF_mix_CR.txt &&\n> -\t\t\t\tcp CRLF_mix_LF ${pfx}_CRLF_nul.txt\n> +\t\t\t\tcp CRLF_mix_LF ${pfx}_CRLF_nul.txt &&\n> +\t\t\t\tpfx=MIX_attr_${attr}_aeol_${aeol}_${crlf} &&\n> +\t\t\t\tcp LF          ${pfx}_LF.txt &&\n> +\t\t\t\tcp CRLF        ${pfx}_CRLF.txt &&\n> +\t\t\t\tcp CRLF_mix_LF ${pfx}_CRLF_mix_LF.txt &&\n> +\t\t\t\tcp LF_mix_CR   ${pfx}_LF_mix_CR.txt &&\n> +\t\t\t\tcp CRLF_nul    ${pfx}_CRLF_nul.txt\n>  \t\t\tdone\n>  \t\tdone\n>  \tdone\n> @@ -136,6 +148,49 @@ commit_chk_wrnNNO () {\n>  \t'\n>  }\n>  \n> +# Commit a file with mixed line endings on top of different files\n> +# in the index. Check for warnings\n> +commit_MIX_chkwrn () {\n> +\tattr=$1 ; shift\n> +\taeol=$1 ; shift\n> +\tcrlf=$1 ; shift\n> +\tlfwarn=$1 ; shift\n> +\tcrlfwarn=$1 ; shift\n> +\tlfmixcrlf=$1 ; shift\n> +\tlfmixcr=$1 ; shift\n> +\tcrlfnul=$1 ; shift\n> +\tpfx=MIX_attr_${attr}_aeol_${aeol}_${crlf}\n> +\t#Commit file with CLRF_mix_LF on top of existing file\n> +\tcreate_gitattributes \"$attr\" $aeol &&\n> +\tfor f in LF CRLF CRLF_mix_LF LF_mix_CR CRLF_nul\n> +\tdo\n> +\t\tfname=${pfx}_$f.txt &&\n> +\t\tcp CRLF_mix_LF $fname &&\n> +\t\tprintf Z >>\"$fname\" &&\n> +\t\tgit -c core.autocrlf=$crlf add $fname 2>\"${pfx}_$f.err\"\n> +\tdone\n> +\n> +\ttest_expect_success \"commit file with mixed EOL crlf=$crlf attr=$attr LF\" '\n> +\t\tcheck_warning \"$lfwarn\" ${pfx}_LF.err\n> +\t'\n> +\ttest_expect_success \"commit file with mixed EOL attr=$attr aeol=$aeol crlf=$crlf CRLF\" '\n> +\t\tcheck_warning \"$crlfwarn\" ${pfx}_CRLF.err\n> +\t'\n> +\n> +\ttest_expect_success \"commit file with mixed EOL attr=$attr aeol=$aeol crlf=$crlf CRLF_mix_LF\" '\n> +\t\tcheck_warning \"$lfmixcrlf\" ${pfx}_CRLF_mix_LF.err\n> +\t'\n> +\n> +\ttest_expect_success \"commit file with mixed EOL attr=$attr aeol=$aeol crlf=$crlf LF_mix_cr\" '\n> +\t\tcheck_warning \"$lfmixcr\" ${pfx}_LF_mix_CR.err\n> +\t'\n> +\n> +\ttest_expect_success \"commit file with mixed EOL attr=$attr aeol=$aeol crlf=$crlf CRLF_nul\" '\n> +\t\tcheck_warning \"$crlfnul\" ${pfx}_CRLF_nul.err\n> +\t'\n> +}\n> +\n> +\n>  stats_ascii () {\n>  \tcase \"$1\" in\n>  \tLF)\n> @@ -323,8 +378,8 @@ test_expect_success 'setup master' '\n>  \tprintf \"\\$Id: 0000000000000000000000000000000000000000 \\$\\r\\nLINEONE\\r\\nLINETWO\\rLINETHREE\"   >CRLF_mix_CR &&\n>  \tprintf \"\\$Id: 0000000000000000000000000000000000000000 \\$\\r\\nLINEONEQ\\r\\nLINETWO\\r\\nLINETHREE\" | q_to_nul >CRLF_nul &&\n>  \tprintf \"\\$Id: 0000000000000000000000000000000000000000 \\$\\nLINEONEQ\\nLINETWO\\nLINETHREE\" | q_to_nul >LF_nul &&\n> -\tcreate_NNO_files CRLF_mix_LF CRLF_mix_LF CRLF_mix_LF CRLF_mix_LF CRLF_mix_LF &&\n> -\tgit -c core.autocrlf=false add NNO_*.txt &&\n> +\tcreate_NNO_MIX_files CRLF_mix_LF CRLF_mix_LF CRLF_mix_LF CRLF_mix_LF CRLF_mix_LF &&\n> +\tgit -c core.autocrlf=false add NNO_*.txt MIX_*.txt &&\n>  \tgit commit -m \"mixed line endings\" &&\n>  \ttest_tick\n>  '\n> @@ -385,6 +440,17 @@ test_expect_success 'commit files attr=crlf' '\n>  \tcommit_check_warn input \"crlf\" \"LF_CRLF\" \"\"        \"LF_CRLF\" \"LF_CRLF\" \"\"\n>  '\n>  \n> +# Commit \"CRLFmixLF\" on top of these files already in the repo:\n> +# LF, CRLF, CRLFmixLF LF_mix_CR CRLFNULL\n> +#                 attr                    LF        CRLF      CRLFmixLF   LF_mix_CR   CRLFNUL\n> +commit_MIX_chkwrn \"\"      \"\"      false   \"\"        \"\"        \"\"          \"\"          \"\"\n> +commit_MIX_chkwrn \"\"      \"\"      true    \"LF_CRLF\" \"\"        \"\"          \"LF_CRLF\"   \"LF_CRLF\"\n> +commit_MIX_chkwrn \"\"      \"\"      input   \"CRLF_LF\" \"\"        \"\"          \"CRLF_LF\"   \"CRLF_LF\"\n> +\n> +commit_MIX_chkwrn \"auto\"  \"\"      false   \"CRLF_LF\" \"\"        \"\"          \"CRLF_LF\"   \"CRLF_LF\"\n> +commit_MIX_chkwrn \"auto\"  \"\"      true    \"LF_CRLF\" \"\"        \"\"          \"LF_CRLF\"   \"LF_CRLF\"\n> +commit_MIX_chkwrn \"auto\"  \"\"      input   \"CRLF_LF\" \"\"        \"\"          \"CRLF_LF\"   \"CRLF_LF\"\n> +\n>  #                 attr                    LF        CRLF      CRLFmixLF   LF_mix_CR   CRLFNUL\n>  commit_chk_wrnNNO \"\"      \"\"      false   \"\"        \"\"        \"\"          \"\"          \"\"\n>  commit_chk_wrnNNO \"\"      \"\"      true    LF_CRLF   \"\"        \"\"          \"\"          \"\"\n"},{"id":"333528","messageId":"20171126122052.13810-1-tboegi@web.de","threadId":"47205","inReplyTo":"CAJ_+vJ6FXXda4fe7=1YxtDGR2d8CqP4KXN+YR6+mdQ+5jQQXug@mail.gmail.com","subject":"[PATCH v2 1/1] convert: tighten the safe autocrlf handling","fromName":"","fromEmail":"tboegi@web.de","sentAt":"2017-11-26T12:20:52Z","receivedAt":"2017-11-26T12:21:01Z","isPatch":true,"sender":{"key":"tboegi@web.de","avatar":"https://avatars.githubusercontent.com/u/7138363?v=4"},"body":"From: Torsten Bögershausen <tboegi@web.de>\n\nWhen a text file had been commited with CRLF and the file is commited\nagain, the CRLF are kept if .gitattributs has \"text=auto\".\nThis is done by analyzing the content of the blob stored in the index:\nIf a '\\r' is found, Git assumes that the blob was commited with CRLF.\n\nThe simple search for a '\\r' does not always work as expected:\nA file is encoded in UTF-16 with CRLF and commited. Git treats it as binary.\nNow the content is converted into UTF-8. At the next commit Git treats the\nfile as text, the CRLF should be converted into LF, but isn't.\n\nReplace has_cr_in_index() with has_crlf_in_index(). When no '\\r' is found,\n0 is returned directly, this is the most common case.\nIf a '\\r' is found, the content is analyzed more deeply.\n\nReported-By: Ashish Negi <ashishnegi33@gmail.com>\nSigned-off-by: Torsten Bögershausen <tboegi@web.de>\n---\n\nChanges against v1:\n- change \"if (crp && (crp[1] == '\\n'))\" to \"if (crp)\"\n  (Thanks Eric. The new patch is more straightforward, and no risk to\n   read out of data)\n- Remove the \"Solution:\" in the commit message\n\nNote:\n  The original function has_cr_in_index() is from this commit:\n    c4805393d73 (Finn Arne Gangstad   2010-05-12 00:37:57 +0200  225)\n  And has this info:\n    >Change autocrlf to not do any conversions to files that in the\n    >repository already contain a CR. git with autocrlf set will never\n    >create such a file, or change a LF only file to contain CRs, so the\n    >(new) assumption is that if a file contains a CR, it is intentional,\n    >and autocrlf should not change that.\n  So the original assumption was slightly optimistic (but did work in 7 years)\n\n\n\nconvert.c            | 19 +++++++++----\n t/t0027-auto-crlf.sh | 76 ++++++++++++++++++++++++++++++++++++++++++++++++----\n 2 files changed, 85 insertions(+), 10 deletions(-)\n\ndiff --git a/convert.c b/convert.c\nindex 20d7ab67bd..1a41a48e15 100644\n--- a/convert.c\n+++ b/convert.c\n@@ -220,18 +220,27 @@ static void check_safe_crlf(const char *path, enum crlf_action crlf_action,\n \t}\n }\n \n-static int has_cr_in_index(const struct index_state *istate, const char *path)\n+static int has_crlf_in_index(const struct index_state *istate, const char *path)\n {\n \tunsigned long sz;\n \tvoid *data;\n-\tint has_cr;\n+\tconst char *crp;\n+\tint has_crlf = 0;\n \n \tdata = read_blob_data_from_index(istate, path, &sz);\n \tif (!data)\n \t\treturn 0;\n-\thas_cr = memchr(data, '\\r', sz) != NULL;\n+\n+\tcrp = memchr(data, '\\r', sz);\n+\tif (crp) {\n+\t\tunsigned int ret_stats;\n+\t\tret_stats = gather_convert_stats(data, sz);\n+\t\tif (!(ret_stats & CONVERT_STAT_BITS_BIN) &&\n+\t\t    (ret_stats & CONVERT_STAT_BITS_TXT_CRLF))\n+\t\t\thas_crlf = 1;\n+\t}\n \tfree(data);\n-\treturn has_cr;\n+\treturn has_crlf;\n }\n \n static int will_convert_lf_to_crlf(size_t len, struct text_stat *stats,\n@@ -290,7 +299,7 @@ static int crlf_to_git(const struct index_state *istate,\n \t\t * cherry-pick.\n \t\t */\n \t\tif ((checksafe != SAFE_CRLF_RENORMALIZE) &&\n-\t\t    has_cr_in_index(istate, path))\n+\t\t    has_crlf_in_index(istate, path))\n \t\t\tconvert_crlf_into_lf = 0;\n \t}\n \tif ((checksafe == SAFE_CRLF_WARN ||\ndiff --git a/t/t0027-auto-crlf.sh b/t/t0027-auto-crlf.sh\nindex 68108d956a..0af35cfb1f 100755\n--- a/t/t0027-auto-crlf.sh\n+++ b/t/t0027-auto-crlf.sh\n@@ -43,19 +43,31 @@ create_gitattributes () {\n \t} >.gitattributes\n }\n \n-create_NNO_files () {\n+# Create 2 sets of files:\n+# The NNO files are \"Not NOrmalized in the repo. We use CRLF_mix_LF and store\n+#   it under different names for the different test cases, see ${pfx}\n+#   Depending on .gitattributes they are normalized at the next commit (or not)\n+# The MIX files have different contents in the repo.\n+#   Depending on its contents, the \"new safer autocrlf\" may kick in.\n+create_NNO_MIX_files () {\n \tfor crlf in false true input\n \tdo\n \t\tfor attr in \"\" auto text -text\n \t\tdo\n \t\t\tfor aeol in \"\" lf crlf\n \t\t\tdo\n-\t\t\t\tpfx=NNO_attr_${attr}_aeol_${aeol}_${crlf}\n+\t\t\t\tpfx=NNO_attr_${attr}_aeol_${aeol}_${crlf} &&\n \t\t\t\tcp CRLF_mix_LF ${pfx}_LF.txt &&\n \t\t\t\tcp CRLF_mix_LF ${pfx}_CRLF.txt &&\n \t\t\t\tcp CRLF_mix_LF ${pfx}_CRLF_mix_LF.txt &&\n \t\t\t\tcp CRLF_mix_LF ${pfx}_LF_mix_CR.txt &&\n-\t\t\t\tcp CRLF_mix_LF ${pfx}_CRLF_nul.txt\n+\t\t\t\tcp CRLF_mix_LF ${pfx}_CRLF_nul.txt &&\n+\t\t\t\tpfx=MIX_attr_${attr}_aeol_${aeol}_${crlf} &&\n+\t\t\t\tcp LF          ${pfx}_LF.txt &&\n+\t\t\t\tcp CRLF        ${pfx}_CRLF.txt &&\n+\t\t\t\tcp CRLF_mix_LF ${pfx}_CRLF_mix_LF.txt &&\n+\t\t\t\tcp LF_mix_CR   ${pfx}_LF_mix_CR.txt &&\n+\t\t\t\tcp CRLF_nul    ${pfx}_CRLF_nul.txt\n \t\t\tdone\n \t\tdone\n \tdone\n@@ -136,6 +148,49 @@ commit_chk_wrnNNO () {\n \t'\n }\n \n+# Commit a file with mixed line endings on top of different files\n+# in the index. Check for warnings\n+commit_MIX_chkwrn () {\n+\tattr=$1 ; shift\n+\taeol=$1 ; shift\n+\tcrlf=$1 ; shift\n+\tlfwarn=$1 ; shift\n+\tcrlfwarn=$1 ; shift\n+\tlfmixcrlf=$1 ; shift\n+\tlfmixcr=$1 ; shift\n+\tcrlfnul=$1 ; shift\n+\tpfx=MIX_attr_${attr}_aeol_${aeol}_${crlf}\n+\t#Commit file with CLRF_mix_LF on top of existing file\n+\tcreate_gitattributes \"$attr\" $aeol &&\n+\tfor f in LF CRLF CRLF_mix_LF LF_mix_CR CRLF_nul\n+\tdo\n+\t\tfname=${pfx}_$f.txt &&\n+\t\tcp CRLF_mix_LF $fname &&\n+\t\tprintf Z >>\"$fname\" &&\n+\t\tgit -c core.autocrlf=$crlf add $fname 2>\"${pfx}_$f.err\"\n+\tdone\n+\n+\ttest_expect_success \"commit file with mixed EOL crlf=$crlf attr=$attr LF\" '\n+\t\tcheck_warning \"$lfwarn\" ${pfx}_LF.err\n+\t'\n+\ttest_expect_success \"commit file with mixed EOL attr=$attr aeol=$aeol crlf=$crlf CRLF\" '\n+\t\tcheck_warning \"$crlfwarn\" ${pfx}_CRLF.err\n+\t'\n+\n+\ttest_expect_success \"commit file with mixed EOL attr=$attr aeol=$aeol crlf=$crlf CRLF_mix_LF\" '\n+\t\tcheck_warning \"$lfmixcrlf\" ${pfx}_CRLF_mix_LF.err\n+\t'\n+\n+\ttest_expect_success \"commit file with mixed EOL attr=$attr aeol=$aeol crlf=$crlf LF_mix_cr\" '\n+\t\tcheck_warning \"$lfmixcr\" ${pfx}_LF_mix_CR.err\n+\t'\n+\n+\ttest_expect_success \"commit file with mixed EOL attr=$attr aeol=$aeol crlf=$crlf CRLF_nul\" '\n+\t\tcheck_warning \"$crlfnul\" ${pfx}_CRLF_nul.err\n+\t'\n+}\n+\n+\n stats_ascii () {\n \tcase \"$1\" in\n \tLF)\n@@ -323,8 +378,8 @@ test_expect_success 'setup master' '\n \tprintf \"\\$Id: 0000000000000000000000000000000000000000 \\$\\r\\nLINEONE\\r\\nLINETWO\\rLINETHREE\"   >CRLF_mix_CR &&\n \tprintf \"\\$Id: 0000000000000000000000000000000000000000 \\$\\r\\nLINEONEQ\\r\\nLINETWO\\r\\nLINETHREE\" | q_to_nul >CRLF_nul &&\n \tprintf \"\\$Id: 0000000000000000000000000000000000000000 \\$\\nLINEONEQ\\nLINETWO\\nLINETHREE\" | q_to_nul >LF_nul &&\n-\tcreate_NNO_files CRLF_mix_LF CRLF_mix_LF CRLF_mix_LF CRLF_mix_LF CRLF_mix_LF &&\n-\tgit -c core.autocrlf=false add NNO_*.txt &&\n+\tcreate_NNO_MIX_files CRLF_mix_LF CRLF_mix_LF CRLF_mix_LF CRLF_mix_LF CRLF_mix_LF &&\n+\tgit -c core.autocrlf=false add NNO_*.txt MIX_*.txt &&\n \tgit commit -m \"mixed line endings\" &&\n \ttest_tick\n '\n@@ -385,6 +440,17 @@ test_expect_success 'commit files attr=crlf' '\n \tcommit_check_warn input \"crlf\" \"LF_CRLF\" \"\"        \"LF_CRLF\" \"LF_CRLF\" \"\"\n '\n \n+# Commit \"CRLFmixLF\" on top of these files already in the repo:\n+# LF, CRLF, CRLFmixLF LF_mix_CR CRLFNULL\n+#                 attr                    LF        CRLF      CRLFmixLF   LF_mix_CR   CRLFNUL\n+commit_MIX_chkwrn \"\"      \"\"      false   \"\"        \"\"        \"\"          \"\"          \"\"\n+commit_MIX_chkwrn \"\"      \"\"      true    \"LF_CRLF\" \"\"        \"\"          \"LF_CRLF\"   \"LF_CRLF\"\n+commit_MIX_chkwrn \"\"      \"\"      input   \"CRLF_LF\" \"\"        \"\"          \"CRLF_LF\"   \"CRLF_LF\"\n+\n+commit_MIX_chkwrn \"auto\"  \"\"      false   \"CRLF_LF\" \"\"        \"\"          \"CRLF_LF\"   \"CRLF_LF\"\n+commit_MIX_chkwrn \"auto\"  \"\"      true    \"LF_CRLF\" \"\"        \"\"          \"LF_CRLF\"   \"LF_CRLF\"\n+commit_MIX_chkwrn \"auto\"  \"\"      input   \"CRLF_LF\" \"\"        \"\"          \"CRLF_LF\"   \"CRLF_LF\"\n+\n #                 attr                    LF        CRLF      CRLFmixLF   LF_mix_CR   CRLFNUL\n commit_chk_wrnNNO \"\"      \"\"      false   \"\"        \"\"        \"\"          \"\"          \"\"\n commit_chk_wrnNNO \"\"      \"\"      true    LF_CRLF   \"\"        \"\"          \"\"          \"\"\n-- \n2.15.0.278.gbd97f72a1b.dirty\n\n"},{"id":"334466","messageId":"20171208174633.9903-1-tboegi@web.de","threadId":"47205","inReplyTo":"CAJ_+vJ6FXXda4fe7=1YxtDGR2d8CqP4KXN+YR6+mdQ+5jQQXug@mail.gmail.com","subject":"[PATCH v1 1/2] t0027: Don't use git commit <empty-pathspec>","fromName":"","fromEmail":"tboegi@web.de","sentAt":"2017-12-08T17:46:33Z","receivedAt":"2017-12-08T17:46:42Z","isPatch":true,"sender":{"key":"tboegi@web.de","avatar":"https://avatars.githubusercontent.com/u/7138363?v=4"},"body":"From: Torsten Bögershausen <tboegi@web.de>\n\nReplace `git commit -m \"comment\" \"\"` with `git commit -m \"comment\"` to\nremove the empty path spec.\n\nSigned-off-by: Torsten Bögershausen <tboegi@web.de>\n---\n t/t0027-auto-crlf.sh | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)\n\ndiff --git a/t/t0027-auto-crlf.sh b/t/t0027-auto-crlf.sh\nindex c2c208fdcd..97154f5c79 100755\n--- a/t/t0027-auto-crlf.sh\n+++ b/t/t0027-auto-crlf.sh\n@@ -370,7 +370,7 @@ test_expect_success 'setup master' '\n \techo >.gitattributes &&\n \tgit checkout -b master &&\n \tgit add .gitattributes &&\n-\tgit commit -m \"add .gitattributes\" \"\" &&\n+\tgit commit -m \"add .gitattributes\" &&\n \tprintf \"\\$Id: 0000000000000000000000000000000000000000 \\$\\nLINEONE\\nLINETWO\\nLINETHREE\"     >LF &&\n \tprintf \"\\$Id: 0000000000000000000000000000000000000000 \\$\\r\\nLINEONE\\r\\nLINETWO\\r\\nLINETHREE\" >CRLF &&\n \tprintf \"\\$Id: 0000000000000000000000000000000000000000 \\$\\nLINEONE\\r\\nLINETWO\\nLINETHREE\"   >CRLF_mix_LF &&\n-- \n2.15.1.271.g1a4e40aa5d\n\n"},{"id":"334467","messageId":"20171208174636.9946-1-tboegi@web.de","threadId":"47205","inReplyTo":"CAJ_+vJ6FXXda4fe7=1YxtDGR2d8CqP4KXN+YR6+mdQ+5jQQXug@mail.gmail.com","subject":"[PATCH v1 2/2] t0027: Adapt the new MIX tests to Windows","fromName":"","fromEmail":"tboegi@web.de","sentAt":"2017-12-08T17:46:36Z","receivedAt":"2017-12-08T17:46:45Z","isPatch":true,"sender":{"key":"tboegi@web.de","avatar":"https://avatars.githubusercontent.com/u/7138363?v=4"},"body":"From: Torsten Bögershausen <tboegi@web.de>\n\nThe new MIX tests don't pass under Windows, adapt them\nto use the correct native line ending.\n\nSigned-off-by: Torsten Bögershausen <tboegi@web.de>\n---\n\n Sorry for the breakage.\n This needs to go on top of tb/check-crlf-for-safe-crlf\n \n t/t0027-auto-crlf.sh | 17 +++++++++--------\n 1 file changed, 9 insertions(+), 8 deletions(-)\n\ndiff --git a/t/t0027-auto-crlf.sh b/t/t0027-auto-crlf.sh\nindex 97154f5c79..8d929b76dc 100755\n--- a/t/t0027-auto-crlf.sh\n+++ b/t/t0027-auto-crlf.sh\n@@ -170,22 +170,22 @@ commit_MIX_chkwrn () {\n \t\tgit -c core.autocrlf=$crlf add $fname 2>\"${pfx}_$f.err\"\n \tdone\n \n-\ttest_expect_success \"commit file with mixed EOL crlf=$crlf attr=$attr LF\" '\n+\ttest_expect_success \"commit file with mixed EOL onto LF crlf=$crlf attr=$attr\" '\n \t\tcheck_warning \"$lfwarn\" ${pfx}_LF.err\n \t'\n-\ttest_expect_success \"commit file with mixed EOL attr=$attr aeol=$aeol crlf=$crlf CRLF\" '\n+\ttest_expect_success \"commit file with mixed EOL onto CLRF attr=$attr aeol=$aeol crlf=$crlf\" '\n \t\tcheck_warning \"$crlfwarn\" ${pfx}_CRLF.err\n \t'\n \n-\ttest_expect_success \"commit file with mixed EOL attr=$attr aeol=$aeol crlf=$crlf CRLF_mix_LF\" '\n+\ttest_expect_success \"commit file with mixed EOL onto CRLF_mix_LF attr=$attr aeol=$aeol crlf=$crlf\" '\n \t\tcheck_warning \"$lfmixcrlf\" ${pfx}_CRLF_mix_LF.err\n \t'\n \n-\ttest_expect_success \"commit file with mixed EOL attr=$attr aeol=$aeol crlf=$crlf LF_mix_cr\" '\n+\ttest_expect_success \"commit file with mixed EOL onto LF_mix_cr attr=$attr aeol=$aeol crlf=$crlf \" '\n \t\tcheck_warning \"$lfmixcr\" ${pfx}_LF_mix_CR.err\n \t'\n \n-\ttest_expect_success \"commit file with mixed EOL attr=$attr aeol=$aeol crlf=$crlf CRLF_nul\" '\n+\ttest_expect_success \"commit file with mixed EOL onto CRLF_nul attr=$attr aeol=$aeol crlf=$crlf\" '\n \t\tcheck_warning \"$crlfnul\" ${pfx}_CRLF_nul.err\n \t'\n }\n@@ -378,7 +378,7 @@ test_expect_success 'setup master' '\n \tprintf \"\\$Id: 0000000000000000000000000000000000000000 \\$\\r\\nLINEONE\\r\\nLINETWO\\rLINETHREE\"   >CRLF_mix_CR &&\n \tprintf \"\\$Id: 0000000000000000000000000000000000000000 \\$\\r\\nLINEONEQ\\r\\nLINETWO\\r\\nLINETHREE\" | q_to_nul >CRLF_nul &&\n \tprintf \"\\$Id: 0000000000000000000000000000000000000000 \\$\\nLINEONEQ\\nLINETWO\\nLINETHREE\" | q_to_nul >LF_nul &&\n-\tcreate_NNO_MIX_files CRLF_mix_LF CRLF_mix_LF CRLF_mix_LF CRLF_mix_LF CRLF_mix_LF &&\n+\tcreate_NNO_MIX_files &&\n \tgit -c core.autocrlf=false add NNO_*.txt MIX_*.txt &&\n \tgit commit -m \"mixed line endings\" &&\n \ttest_tick\n@@ -441,13 +441,14 @@ test_expect_success 'commit files attr=crlf' '\n '\n \n # Commit \"CRLFmixLF\" on top of these files already in the repo:\n-# LF, CRLF, CRLFmixLF LF_mix_CR CRLFNULL\n+#                                         mixed     mixed     mixed       mixed       mixed\n+#                                         onto      onto      onto        onto        onto\n #                 attr                    LF        CRLF      CRLFmixLF   LF_mix_CR   CRLFNUL\n commit_MIX_chkwrn \"\"      \"\"      false   \"\"        \"\"        \"\"          \"\"          \"\"\n commit_MIX_chkwrn \"\"      \"\"      true    \"LF_CRLF\" \"\"        \"\"          \"LF_CRLF\"   \"LF_CRLF\"\n commit_MIX_chkwrn \"\"      \"\"      input   \"CRLF_LF\" \"\"        \"\"          \"CRLF_LF\"   \"CRLF_LF\"\n \n-commit_MIX_chkwrn \"auto\"  \"\"      false   \"CRLF_LF\" \"\"        \"\"          \"CRLF_LF\"   \"CRLF_LF\"\n+commit_MIX_chkwrn \"auto\"  \"\"      false   \"$WAMIX\"  \"\"        \"\"          \"$WAMIX\"    \"$WAMIX\"\n commit_MIX_chkwrn \"auto\"  \"\"      true    \"LF_CRLF\" \"\"        \"\"          \"LF_CRLF\"   \"LF_CRLF\"\n commit_MIX_chkwrn \"auto\"  \"\"      input   \"CRLF_LF\" \"\"        \"\"          \"CRLF_LF\"   \"CRLF_LF\"\n \n-- \n2.15.1.271.g1a4e40aa5d\n\n"},{"id":"334475","messageId":"xmqqpo7p84vz.fsf@gitster.mtv.corp.google.com","threadId":"47205","inReplyTo":"20171208174633.9903-1-tboegi@web.de","subject":"Re: [PATCH v1 1/2] t0027: Don't use git commit <empty-pathspec>","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-12-08T18:13:20Z","receivedAt":"2017-12-08T18:13:27Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"tboegi@web.de writes:\n\n> From: Torsten Bögershausen <tboegi@web.de>\n>\n> Replace `git commit -m \"comment\" \"\"` with `git commit -m \"comment\"` to\n> remove the empty path spec.\n>\n> Signed-off-by: Torsten Bögershausen <tboegi@web.de>\n> ---\n>  t/t0027-auto-crlf.sh | 2 +-\n>  1 file changed, 1 insertion(+), 1 deletion(-)\n\nThis looks a bit strange.  The intent seems to commit all changes\nmade in the working tree, so I'd understand it if it replaced the\nempty string with a single dot.\n\nI also thought that we deprecated use of an empty string as \"match\nall\" pathspec recently, and expected that this to break.\n\n    ... goes and looks ...\n\nIndeed, 229a95aa (\"t0027: do not use an empty string as a pathspec\nelement\", 2017-06-23) does exactly that.\n\n\n> diff --git a/t/t0027-auto-crlf.sh b/t/t0027-auto-crlf.sh\n> index c2c208fdcd..97154f5c79 100755\n> --- a/t/t0027-auto-crlf.sh\n> +++ b/t/t0027-auto-crlf.sh\n> @@ -370,7 +370,7 @@ test_expect_success 'setup master' '\n>  \techo >.gitattributes &&\n>  \tgit checkout -b master &&\n>  \tgit add .gitattributes &&\n> -\tgit commit -m \"add .gitattributes\" \"\" &&\n> +\tgit commit -m \"add .gitattributes\" &&\n>  \tprintf \"\\$Id: 0000000000000000000000000000000000000000 \\$\\nLINEONE\\nLINETWO\\nLINETHREE\"     >LF &&\n>  \tprintf \"\\$Id: 0000000000000000000000000000000000000000 \\$\\r\\nLINEONE\\r\\nLINETWO\\r\\nLINETHREE\" >CRLF &&\n>  \tprintf \"\\$Id: 0000000000000000000000000000000000000000 \\$\\nLINEONE\\r\\nLINETWO\\nLINETHREE\"   >CRLF_mix_LF &&\n"},{"id":"334476","messageId":"xmqqlgid84io.fsf@gitster.mtv.corp.google.com","threadId":"47205","inReplyTo":"xmqqpo7p84vz.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH v1 1/2] t0027: Don't use git commit <empty-pathspec>","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-12-08T18:21:19Z","receivedAt":"2017-12-08T18:21:27Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Junio C Hamano <gitster@pobox.com> writes:\n\n> tboegi@web.de writes:\n>\n>> From: Torsten Bögershausen <tboegi@web.de>\n>>\n>> Replace `git commit -m \"comment\" \"\"` with `git commit -m \"comment\"` to\n>> remove the empty path spec.\n>>\n>> Signed-off-by: Torsten Bögershausen <tboegi@web.de>\n>> ---\n>>  t/t0027-auto-crlf.sh | 2 +-\n>>  1 file changed, 1 insertion(+), 1 deletion(-)\n>\n> This looks a bit strange.  The intent seems to commit all changes\n> made in the working tree, so I'd understand it if it replaced the\n> empty string with a single dot.\n>\n> I also thought that we deprecated use of an empty string as \"match\n> all\" pathspec recently, and expected that this to break.\n>\n>     ... goes and looks ...\n>\n> Indeed, 229a95aa (\"t0027: do not use an empty string as a pathspec\n> element\", 2017-06-23) does exactly that.\n\nOK, I think I can safely omit this patch, because\n\n (1) when 2/2 is queued on top of tb/check-crlf-for-safe-crlf,\n     because ex/deprecate-empty-pathspec-as-match-all is not yet in\n     the topic, an empty pathspec still means \"match all\" and\n     nothing breaks; and\n\n (2) when tb/check-crlf-for-safe-crlf plus 2/2 is merged to any\n     branch with ex/deprecate-empty-pathspec-as-match-all, the topic\n     already fixes what this 1/2 tries to\n\nThanks for being thorough, though.\n\n"},{"id":"334480","messageId":"20171208185058.GA13088@tor.lan","threadId":"47205","inReplyTo":"xmqqlgid84io.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH v1 1/2] t0027: Don't use git commit <empty-pathspec>","fromName":"Torsten Bögershausen","fromEmail":"tboegi@web.de","sentAt":"2017-12-08T18:50:58Z","receivedAt":"2017-12-08T18:51:08Z","isPatch":true,"sender":{"key":"tboegi@web.de","avatar":"https://avatars.githubusercontent.com/u/7138363?v=4"},"body":"On Fri, Dec 08, 2017 at 10:21:19AM -0800, Junio C Hamano wrote:\n> Junio C Hamano <gitster@pobox.com> writes:\n> \n> > tboegi@web.de writes:\n> >\n> >> From: Torsten Bögershausen <tboegi@web.de>\n> >>\n> >> Replace `git commit -m \"comment\" \"\"` with `git commit -m \"comment\"` to\n> >> remove the empty path spec.\n> >>\n> >> Signed-off-by: Torsten Bögershausen <tboegi@web.de>\n> >> ---\n> >>  t/t0027-auto-crlf.sh | 2 +-\n> >>  1 file changed, 1 insertion(+), 1 deletion(-)\n> >\n> > This looks a bit strange.  The intent seems to commit all changes\n> > made in the working tree, so I'd understand it if it replaced the\n> > empty string with a single dot.\n> >\n> > I also thought that we deprecated use of an empty string as \"match\n> > all\" pathspec recently, and expected that this to break.\n> >\n> >     ... goes and looks ...\n> >\n> > Indeed, 229a95aa (\"t0027: do not use an empty string as a pathspec\n> > element\", 2017-06-23) does exactly that.\n> \n> OK, I think I can safely omit this patch, because\n> \n>  (1) when 2/2 is queued on top of tb/check-crlf-for-safe-crlf,\n>      because ex/deprecate-empty-pathspec-as-match-all is not yet in\n>      the topic, an empty pathspec still means \"match all\" and\n>      nothing breaks; and\n> \n>  (2) when tb/check-crlf-for-safe-crlf plus 2/2 is merged to any\n>      branch with ex/deprecate-empty-pathspec-as-match-all, the topic\n>      already fixes what this 1/2 tries to\n> \n> Thanks for being thorough, though.\n> \n\nSure, the credit goes 100% to you:\n\ncommit 229a95aafa77b583b46a3156b4fad469c264ddfd\nAuthor: Junio C Hamano <gitster@pobox.com>\nDate:   Fri Jun 23 11:04:14 2017 -0700\n\n    t0027: do not use an empty string as a pathspec element\n\nMy brain did just assume that this had mad it to master, sorry for the noise\n\n"}]}