{"thread":{"id":"44257","subject":"A head scratcher, clone results in modified files (tested linux and cygwin) - .gitattributes file?","startedAt":"2016-10-09T06:48:59Z","lastAt":"2016-10-09T12:48:09Z","messageCount":3,"participants":["Jason Pyeron","Torsten Bögershausen"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"303657","messageId":"7E06D61EACFB462AB96DD60720930968@black7","threadId":"44257","inReplyTo":null,"subject":"A head scratcher, clone results in modified files (tested linux and cygwin) - .gitattributes file?","fromName":"Jason Pyeron","fromEmail":"jpyeron@pdinc.us","sentAt":"2016-10-09T06:48:49Z","receivedAt":"2016-10-09T06:48:59Z","isPatch":false,"sender":{"key":"jpyeron@pdinc.us","avatar":"https://gravatar.com/avatar/c2e53452caa53d940768a1ffc9cf76196d851b9b534b7a39cd39852a70a0508f?d=mp&s=160"},"body":"Does anyone have any ideas as to why this clone resulted in modified files and how to prevent it?\n\nThere is a .gitattributes in the tree, which says:\n\n*        text=auto\n*.java   text diff=java\n*.html   text diff=html\n*.css    text\n*.js     text\n*.sql    text\n\nBut even then the *.bin files full of non-ascii garbage are impacted?! But I cannot find a difference:\n\nroot@YYYYYYY /projects/commons-io\n$ git cat-file -p 7c9cd59c8a00e0bd3f18da42b32cd40024ad1505 | hexdump -C\n00000000  a9 fa bf e9 a4 6c a8 ca  0d 0a c1 63 c5 e9 a4 a4  |.....l.....c....|\n00000010  a4 e5 0d 0a                                       |....|\n00000014\n\nroot@YYYYYYY /projects/commons-io\n$ git cat-file -p 7c9cd59c8a00e0bd3f18da42b32cd40024ad1505 | sha1sum.exe\nd69820e1282801ccd627e35fb213e8832949c6ac *-\n\nroot@YYYYYYY /projects/commons-io\n$ hexdump.exe -C src/test/resources/test-file-x-windows-950.bin\n00000000  a9 fa bf e9 a4 6c a8 ca  0d 0a c1 63 c5 e9 a4 a4  |.....l.....c....|\n00000010  a4 e5 0d 0a                                       |....|\n00000014\n\nroot@YYYYYYY /projects/commons-io\n$ sha1sum.exe src/test/resources/test-file-x-windows-950.bin\nd69820e1282801ccd627e35fb213e8832949c6ac *src/test/resources/test-file-x-windows-950.bin\n\nDeleting the .gitattributes and the checkout -- did not help. No luck deleting the file then restoring it either.\n\nNot even git clone git://git.apache.org/commons-io.git --config core.attributesFile=/dev/null fixed it. Details below.\n\n-Jason\n\nCygwin test:\n\nroot@YYYYYYY /projects\n$ git clone git://git.apache.org/commons-io.git\nCloning into 'commons-io'...\nremote: Counting objects: 21203, done.\nremote: Compressing objects: 100% (3454/3454), done.\nremote: Total 21203 (delta 10822), reused 21129 (delta 10794)\nReceiving objects: 100% (21203/21203), 2.51 MiB | 607.00 KiB/s, done.\nResolving deltas: 100% (10822/10822), done.\nChecking connectivity... done.\n\nroot@YYYYYYY /projects\n$ cd commons-io/\n\nroot@YYYYYYY /projects/commons-io\n$ git status\nOn branch master\nYour branch is up-to-date with 'origin/master'.\nChanges not staged for commit:\n  (use \"git add <file>...\" to update what will be committed)\n  (use \"git checkout -- <file>...\" to discard changes in working directory)\n\n        modified:   src/main/java/org/apache/commons/io/serialization/package.html\n        modified:   src/site/xdoc/upgradeto2_3.xml\n        modified:   src/test/resources/org/apache/commons/io/FileUtilsTestDataCRLF.dat\n        modified:   src/test/resources/test-file-gbk.bin\n        modified:   src/test/resources/test-file-iso8859-1-shortlines-win-linebr.bin\n        modified:   src/test/resources/test-file-utf8-win-linebr.bin\n        modified:   src/test/resources/test-file-windows-31j.bin\n        modified:   src/test/resources/test-file-x-windows-949.bin\n        modified:   src/test/resources/test-file-x-windows-950.bin\n\nno changes added to commit (use \"git add\" and/or \"git commit -a\")\n\nroot@YYYYYYY /projects/commons-io\n$ git rev-parse HEAD\nc5f2e40e7a8234fe48e08d451d3152ba58a03ac6\n\nroot@YYYYYYY /projects/commons-io\n$ git version\ngit version 2.8.3\n\nroot@YYYYYYY /projects/commons-io\n$ git config --list\nuser.email=jpyeron@pdinc.us\nuser.name=Jason Pyeron\ncredential.helper=cache --timeout=9999999999\npush.default=simple\ncore.repositoryformatversion=0\ncore.filemode=true\ncore.bare=false\ncore.logallrefupdates=true\ncore.ignorecase=true\nremote.origin.url=git://git.apache.org/commons-io.git\nremote.origin.fetch=+refs/heads/*:refs/remotes/origin/*\nbranch.master.remote=origin\nbranch.master.merge=refs/heads/master\n\n$ uname -a\nCYGWIN_NT-6.1-WOW YYYYYYY 2.5.2(0.297/5/3) 2016-06-23 14:27 i686 Cygwin\n\nroot@YYYYYYY /projects/commons-io\n$\n\nLinux test:\n\nroot@XXXXXX /tmp\n# git clone git://git.apache.org/commons-io.git\nCloning into 'commons-io'...\nremote: Counting objects: 21203, done.\nremote: Compressing objects: 100% (3454/3454), done.\nremote: Total 21203 (delta 10822), reused 21129 (delta 10794)\nReceiving objects: 100% (21203/21203), 2.51 MiB | 176 KiB/s, done.\nResolving deltas: 100% (10822/10822), done.\n\nroot@XXXXXX /tmp\n# cd commons-io/\n\nroot@XXXXXX /tmp/commons-io\n# git status\n# On branch master\n# Changes not staged for commit:\n#   (use \"git add <file>...\" to update what will be committed)\n#   (use \"git checkout -- <file>...\" to discard changes in working directory)\n#\n#       modified:   src/main/java/org/apache/commons/io/serialization/package.html\n#       modified:   src/site/xdoc/upgradeto2_3.xml\n#       modified:   src/test/resources/org/apache/commons/io/FileUtilsTestDataCRLF.dat\n#       modified:   src/test/resources/test-file-gbk.bin\n#       modified:   src/test/resources/test-file-iso8859-1-shortlines-win-linebr.bin\n#       modified:   src/test/resources/test-file-utf8-win-linebr.bin\n#       modified:   src/test/resources/test-file-windows-31j.bin\n#       modified:   src/test/resources/test-file-x-windows-949.bin\n#       modified:   src/test/resources/test-file-x-windows-950.bin\n#\nno changes added to commit (use \"git add\" and/or \"git commit -a\")\n\nroot@XXXXXX /tmp/commons-io\n# git rev-parse HEAD\nc5f2e40e7a8234fe48e08d451d3152ba58a03ac6\n\nroot@XXXXXX /tmp/commons-io\n# git version\ngit version 1.7.9.6\n\nroot@XXXXXX /tmp/commons-io\n# git config --list\ncore.repositoryformatversion=0\ncore.filemode=true\ncore.bare=false\ncore.logallrefupdates=true\nremote.origin.fetch=+refs/heads/*:refs/remotes/origin/*\nremote.origin.url=git://git.apache.org/commons-io.git\nbranch.master.remote=origin\nbranch.master.merge=refs/heads/master\n\nroot@XXXXXX /tmp/commons-io\n# uname -a\nLinux XXXXXX.internal.pdinc.us 2.6.9-103.ELsmp #1 SMP Fri Dec 9 04:43:08 EST 2011 x86_64 x86_64 x86_64 GNU/Linux\n\nroot@XXXXXX /tmp/commons-io\n#\n\n"},{"id":"303665","messageId":"c9c90c24-3bf7-cd57-cad6-db28db16f323@web.de","threadId":"44257","inReplyTo":"7E06D61EACFB462AB96DD60720930968@black7","subject":"Re: A head scratcher, clone results in modified files (tested linux and cygwin) - .gitattributes file?","fromName":"Torsten Bögershausen","fromEmail":"tboegi@web.de","sentAt":"2016-10-09T10:27:11Z","receivedAt":"2016-10-09T10:31:18Z","isPatch":false,"sender":{"key":"tboegi@web.de","avatar":"https://avatars.githubusercontent.com/u/7138363?v=4"},"body":"\n\nOn 09/10/16 08:48, Jason Pyeron wrote:\n\n\nThe whole .gitattributes needs to be adopted, I think\n\nGit 2.10 or higher has \"git ls-files --eol\":\n\ngit ls-files --eol   | grep \"i/crlf.*auto\"\ni/crlf  w/crlf  attr/text=auto src/site/xdoc/upgradeto2_3.xml\ni/crlf  w/crlf  attr/text=auto \nsrc/test/resources/org/apache/commons/io/FileUtilsTestDataCRLF.dat\ni/crlf  w/crlf  attr/text=auto src/test/resources/test-file-gbk.bin\ni/crlf  w/crlf  attr/text=auto \nsrc/test/resources/test-file-iso8859-1-shortlines-win-linebr.bin\ni/crlf  w/crlf  attr/text=auto src/test/resources/test-file-utf8-win-linebr.bin\ni/crlf  w/crlf  attr/text=auto src/test/resources/test-file-windows-31j.bin\ni/crlf  w/crlf  attr/text=auto src/test/resources/test-file-x-windows-949.bin\ni/crlf  w/crlf  attr/text=auto src/test/resources/test-file-x-windows-950.bin\n############\nProblem:\nxml file had been commited  with CRLF : either normalize it or declare \"-text\".\n\nThe same is valid for the other files as well.\nThey are identified by auto as text, and commited with CRLF.\nMy feeling is that they should be declared as \"-text\".\nOr, to be more compatible, with \"-crlf\":\n\nSolution:\nMake up your mind about the xml file and the html files.\nIf they are text, they need to be normalized.\n\n\nQuestion:\nWhat happens, if you do this:\n# Auto detect text files and perform LF normalization\n*        crlf=auto\n\n*.bin    -crlf\n*.dat    -crlf\n*.java   crlf diff=java\n*.html   crlf diff=html\n*.css    crlf\n*.js     crlf\n*.sql    crlf\n\n\n\n"},{"id":"303675","messageId":"B539E7023E19454A80C564A234BA9014@black7","threadId":"44257","inReplyTo":"c9c90c24-3bf7-cd57-cad6-db28db16f323@web.de","subject":"RE: A head scratcher, clone results in modified files (tested linux and cygwin) - .gitattributes file?","fromName":"Jason Pyeron","fromEmail":"jpyeron@pdinc.us","sentAt":"2016-10-09T12:48:00Z","receivedAt":"2016-10-09T12:48:09Z","isPatch":false,"sender":{"key":"jpyeron@pdinc.us","avatar":"https://gravatar.com/avatar/c2e53452caa53d940768a1ffc9cf76196d851b9b534b7a39cd39852a70a0508f?d=mp&s=160"},"body":"> -----Original Message-----\n> From: Torsten Bögershausen\n> Sent: Sunday, October 09, 2016 06:27\n> \n> On 09/10/16 08:48, Jason Pyeron wrote:\n> \n> \n> The whole .gitattributes needs to be adopted, I think\n> \n> Git 2.10 or higher has \"git ls-files --eol\":\n> \n> git ls-files --eol   | grep \"i/crlf.*auto\"\n> i/crlf  w/crlf  attr/text=auto src/site/xdoc/upgradeto2_3.xml\n> i/crlf  w/crlf  attr/text=auto \n> src/test/resources/org/apache/commons/io/FileUtilsTestDataCRLF.dat\n> i/crlf  w/crlf  attr/text=auto src/test/resources/test-file-gbk.bin\n> i/crlf  w/crlf  attr/text=auto \n> src/test/resources/test-file-iso8859-1-shortlines-win-linebr.bin\n> i/crlf  w/crlf  attr/text=auto \n> src/test/resources/test-file-utf8-win-linebr.bin\n> i/crlf  w/crlf  attr/text=auto \n> src/test/resources/test-file-windows-31j.bin\n> i/crlf  w/crlf  attr/text=auto \n> src/test/resources/test-file-x-windows-949.bin\n> i/crlf  w/crlf  attr/text=auto \n> src/test/resources/test-file-x-windows-950.bin\n> ############\n> Problem:\n> xml file had been commited  with CRLF : either normalize it \n> or declare \"-text\".\n> \n> The same is valid for the other files as well.\n> They are identified by auto as text, and commited with CRLF.\n> My feeling is that they should be declared as \"-text\".\n> Or, to be more compatible, with \"-crlf\":\n> \n\nGood call.\n\n> Solution:\n> Make up your mind about the xml file and the html files.\n> If they are text, they need to be normalized.\n> \n> \n> Question:\n> What happens, if you do this:\n> # Auto detect text files and perform LF normalization\n> *        crlf=auto\n> \n> *.bin    -crlf\n> *.dat    -crlf\n\n*.bin     -text\n*.dat     -text\n\n#fix that issue\n\n> *.java   crlf diff=java\n> *.html   crlf diff=html\n> *.css    crlf\n> *.js     crlf\n> *.sql    crlf\n> \n\nOr create a subordinate\n\nsrc/test/resources/.gitattributes:\n*        -text\n\nSince these are \"test\" resources, some with text extensions from above.\n\nThanks!\n\n-Jason\n\n"}]}