{"thread":{"id":"38542","subject":"EOL handling (EGit/svn/Windows)","startedAt":"2015-02-09T22:22:19Z","lastAt":"2015-02-11T15:51:27Z","messageCount":6,"participants":["Piotr Krukowiecki","Torsten Bögershausen"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"255864","messageId":"CAA01Csrp0ZOouM5zhVuHQW9sWCnbMMLRkmcbRtgsU3k8wNtSqw@mail.gmail.com","threadId":"38542","inReplyTo":null,"subject":"EOL handling (EGit/svn/Windows)","fromName":"Piotr Krukowiecki","fromEmail":"piotr.krukowiecki@gmail.com","sentAt":"2015-02-09T22:22:19Z","receivedAt":"2015-02-09T22:22:19Z","isPatch":false,"sender":{"key":"piotr.krukowiecki@gmail.com","avatar":"https://avatars.githubusercontent.com/u/3259959?v=4"},"body":"Hi,\n\nin short: how to do svn->git conversion, or how to configure git/EGit\nso that Windows/Linux users are happy with EOLs?\n\n\nLong story:\n\nrecently I converted (with git-svn) an svn repository to git. This\ncreated git repository with Windows (CRLF) end of line files. I\nsuppose the original svn repository had Windows EOLs (I did the\nconversion on linux). The development is done in Eclipse (mostly on\nWindows), so the EGit is a natural choice for git client.\n\nBut the problem appears: what should be the core.eol and core.autocrlf settings?\n\nEGit seems to have problems with core.autocrlf=true - it always\nconverts files to LF on commit. So if someone modifies one line in a\nfile which was not modified yet since the git->svn conversion, each\nline is seen as modified (CRLF changed to LF). It seems like a bug in\nEGit - git-config man page says \"files that contain CRLF in the\nrepository will not be touched\" for core.autocrlf=true. \"Normal\"\ncommand line git works better - if a file has CRLF in it in\nrepository, it will not convert it (it will convert new files from\nCRLF to LF though - that's OK).\n\nI could also set core.autocrlf=false which should avoid any conversion\nto LF on check-in. This will move the responsibility of using correct\nEOL to users. Everyone, both Windows and Linux users, will have to use\nthe same EOLs. Probably core.eol will have to be set to crlf, the same\nas in repo. But I suppose this will not work - new files will be\ncreated with native EOLs so the files will end up being checked in\nlike that. I also think it might create some problems with programs\nnot accepting non-native EOLs. On the other hand, it would prevent\nproblems caused by CRLF->LF conversion on \"binary\" files incorrectly\ndetected as \"text\" files...\n\nSo maybe I should re-write the repository (all the history) so that\nthe files are converted to LF? (examples:\nhttp://superuser.com/questions/293941/rewrite-git-history-to-replace-all-crlf-to-lf\nhttp://blog.gyoshev.net/2013/08/normalizing-line-endings-in-git-repositories/).This\nway I should be able to use core.autocrlf for both git and EGit.\nThat's what I'm favoring currently. The downside is that, if I assume\ncorrectly that SVN repo kept files in CRLF format, the conversion to\nLF might break something (not sure what - some UTF files? or some\nunexpected stuff...).\n\nThis re-writing of repo would not be needed if EGit did work like\nnormal git... should I drop EGit? I would prefer not to :(\n\nAny other suggestions?\n\nBTW, core.autocrlf documentation in git-config is a bit hard to\nunderstand... it's not obvious what it affects - working dir? commits?\nDoes it convert to CRLF or LF? What is the \"input\" setting (what is\n\"output conversion\")?  I found some explanation here:\nhttp://thread.gmane.org/gmane.comp.version-control.git/174413\n\n\nThanks,\n-- \nPiotr Krukowiecki\n"},{"id":"255877","messageId":"54D99BED.8010005@web.de","threadId":"38542","inReplyTo":"CAA01Csrp0ZOouM5zhVuHQW9sWCnbMMLRkmcbRtgsU3k8wNtSqw@mail.gmail.com","subject":"Re: EOL handling (EGit/svn/Windows)","fromName":"Torsten Bögershausen","fromEmail":"tboegi@web.de","sentAt":"2015-02-10T05:49:33Z","receivedAt":"2015-02-10T05:49:33Z","isPatch":false,"sender":{"key":"tboegi@web.de","avatar":"https://avatars.githubusercontent.com/u/7138363?v=4"},"body":"On 02/09/2015 11:22 PM, Piotr Krukowiecki wrote:\n >Any other suggestions?\n\nMy, somewhat personally, suggestion:\nIf there is more than one developer, don't use core.autocrlf at all-\nit is a local setting, which doesn't travel through the repo, and\nis slightly different in Git Egit, depending on the version.\n\nGit for Windows has it enabled by default, Cygwin has not.\n\nWhich Git versions are you using ?\nHow many people are there involved, how many on Windows, how many on Linux ?\nDo you want to commit to svn, or is this a one-time conversion ?\n\nIf it is a one-time conversion, and you continue to work in Git only,\nthen the cleanest, most portable and future proof way is to use the \n.gitattributes file,\nadd that to the repo, do the normalization  and push.\nA line like this:\n* text=auto\nis the easiest way.\n\nHave a look at\nhttps://www.kernel.org/pub/software/scm/git/docs/gitattributes.html\ntake a tee or coffee, do some experiments first with a dummy repo,\nbut all the client OS/Gits involved.\nPlease let us know the result (or feel free to ask more questions)\n"},{"id":"255879","messageId":"CAA01CspP-C5ZVSR6Vyxe0X6UxpxGPwEM=P_7QwcfPFpUBGEzPQ@mail.gmail.com","threadId":"38542","inReplyTo":"54D99BED.8010005@web.de","subject":"Re: EOL handling (EGit/svn/Windows)","fromName":"Piotr Krukowiecki","fromEmail":"piotr.krukowiecki@gmail.com","sentAt":"2015-02-10T10:52:39Z","receivedAt":"2015-02-10T10:52:39Z","isPatch":false,"sender":{"key":"piotr.krukowiecki@gmail.com","avatar":"https://avatars.githubusercontent.com/u/3259959?v=4"},"body":"On Tue, Feb 10, 2015 at 6:49 AM, Torsten Bögershausen <tboegi@web.de> wrote:\n> Which Git versions are you using ?\n\nThe one I'm testing currently:\n\ngit version 1.7.9 (cygwin)\ngit version 1.9.0.msysgit.0 (msys)\nEGit from Eclipse Luna\n\nCygwin git is a bit old, as I see now. Will try to update later.\n\n\n> How many people are there involved, how many on Windows, how many on Linux ?\n\nLess than 10 actively, most on Windows.\n\n\n> Do you want to commit to svn, or is this a one-time conversion ?\n\nOne-time.\n\n\n> If it is a one-time conversion, and you continue to work in Git only,\n> then the cleanest, most portable and future proof way is to use the\n> .gitattributes file,\n\nI'm not sure if EGit supports .gitattributes:\nhttps://bugs.eclipse.org/bugs/show_bug.cgi?id=342372\n\n\n> add that to the repo, do the normalization  and push.\n> A line like this:\n> * text=auto\n> is the easiest way.\n\nI'm trying it.\n\n\n> Have a look at\n> https://www.kernel.org/pub/software/scm/git/docs/gitattributes.html\n> take a tee or coffee, do some experiments first with a dummy repo,\n> but all the client OS/Gits involved.\n\nThat's one step we did not do carefully enough :(\n\n\n> Please let us know the result (or feel free to ask more questions)\n\nFor testing, I've converted all files to LF and commited it, also\nadded the .gitattribute file.\n\nSo far:\n1. msysgit can't checkout a one file (saying filename too long, the\nrelative path has 215 bytes) - probably not related to EOL issue.\nCygwin git works ok. So I did not check how msysgit works yet.\n2. maybe due to old cygwin git, I have a problem of not displaying\nchanges, if the changed line has LF eol (and the file was checked out\non Windows with CRLF eols). Will try later with newer git.\n2a. EGit handles such files gracefuly, but OTOH if the file is simple\ndos2unix'ed, it shows diffs showing all lines changed, and when you\ncommit the files, it will create empty commit.\n\n\n$ git status\n# On branch master\n#\nnothing to commit (working directory clean)\n\n$ file master/settings.gradle\nmaster/settings.gradle: ASCII text, with CRLF line terminators\n\n$ dos2unix.exe master/settings.gradle\ndos2unix: converting file master/settings.gradle to Unix format ...\n\n$ git status\n# On branch master\n#\n# Changes not staged for commit:\n#   (use \"git add <file>...\" to update what will be committed)\n#   (use \"git checkout -- <file>...\" to discard changes in working directory)\n#\n#       modified:   master/settings.gradle\n#\nno changes added to commit (use \"git add\" and/or \"git commit -a\")\n\n$ git diff\nfatal: LF would be replaced by CRLF in master/settings.gradle\n\n$ echo \"hi\" >> master/settings.gradle\n\n$ file master/settings.gradle\nmaster/settings.gradle: ASCII text\n\n############### diff does not show changes! ###################\n$ git diff\nfatal: LF would be replaced by CRLF in master/settings.gradle\n\n$ git diff -- master/settings.gradle\nfatal: LF would be replaced by CRLF in master/settings.gradle\n\n$ cat master/settings.gradle\n[the changes are there]\n\n$ unix2dos.exe master/settings.gradle\nunix2dos: converting file master/settings.gradle to DOS format ...\n\n$ git diff\ndiff --git a/master/settings.gradle b/master/settings.gradle\nindex a8d6609..7aa9e6b 100755\n[changes are shown]\n\n$ vim -b master/settings.gradle\n[remove CR from the changed line]\n\n$ git status\n# On branch master\n#\n# Changes not staged for commit:\n#   (use \"git add <file>...\" to update what will be committed)\n#   (use \"git checkout -- <file>...\" to discard changes in working directory)\n#\n#       modified:   master/settings.gradle\n#\nno changes added to commit (use \"git add\" and/or \"git commit -a\")\n\n$ git diff\nfatal: LF would be replaced by CRLF in master/settings.gradle\n\n\n\n-- \nPiotr Krukowiecki\n"},{"id":"255887","messageId":"54DA248E.4080100@web.de","threadId":"38542","inReplyTo":"CAA01CspP-C5ZVSR6Vyxe0X6UxpxGPwEM=P_7QwcfPFpUBGEzPQ@mail.gmail.com","subject":"Re: EOL handling (EGit/svn/Windows)","fromName":"Torsten Bögershausen","fromEmail":"tboegi@web.de","sentAt":"2015-02-10T15:32:30Z","receivedAt":"2015-02-10T15:32:30Z","isPatch":false,"sender":{"key":"tboegi@web.de","avatar":"https://avatars.githubusercontent.com/u/7138363?v=4"},"body":"On 2015-02-10 11.52, Piotr Krukowiecki wrote:\n> On Tue, Feb 10, 2015 at 6:49 AM, Torsten Bögershausen <tboegi@web.de> wrote:\n>> Which Git versions are you using ?\n> \n> The one I'm testing currently:\n> \n> git version 1.7.9 (cygwin)\n> git version 1.9.0.msysgit.0 (msys)\n> EGit from Eclipse Luna\n> \n> Cygwin git is a bit old, as I see now. Will try to update later.\n> \n> \n>> How many people are there involved, how many on Windows, how many on Linux ?\n> \n> Less than 10 actively, most on Windows.\n> \n> \n>> Do you want to commit to svn, or is this a one-time conversion ?\n> \n> One-time.\n> \n> \n>> If it is a one-time conversion, and you continue to work in Git only,\n>> then the cleanest, most portable and future proof way is to use the\n>> .gitattributes file,\n> \n> I'm not sure if EGit supports .gitattributes:\n> https://bugs.eclipse.org/bugs/show_bug.cgi?id=342372\n> \n> \n>> add that to the repo, do the normalization  and push.\n>> A line like this:\n>> * text=auto\n>> is the easiest way.\n> \n> I'm trying it.\n> \n> \n>> Have a look at\n>> https://www.kernel.org/pub/software/scm/git/docs/gitattributes.html\n>> take a tee or coffee, do some experiments first with a dummy repo,\n>> but all the client OS/Gits involved.\n> \n> That's one step we did not do carefully enough :(\n> \n> \n>> Please let us know the result (or feel free to ask more questions)\n> \n> For testing, I've converted all files to LF and commited it, also\n> added the .gitattribute file.\n> \n> So far:\n> 1. msysgit can't checkout a one file (saying filename too long, the\n> relative path has 215 bytes) - probably not related to EOL issue.\nPlease have a look here:\nhttps://github.com/msysgit/msysgit/releases/Git-1.9.5-preview20141217\nI think we have support for long path names (Haven't tested it myself)\n> Cygwin git works ok. So I did not check how msysgit works yet.\n> 2. maybe due to old cygwin git, I have a problem of not displaying\n> changes, if the changed line has LF eol (and the file was checked out\n> on Windows with CRLF eols). Will try later with newer git.\nNormally you will not see any changes, and \"git diff\" will not show\nanything either.\n> 2a. EGit handles such files gracefuly, but OTOH if the file is simple\n> dos2unix'ed, it shows diffs showing all lines changed, and when you\n> commit the files, it will create empty commit.\nWhy this dos2unix ?\nIs there a special reason ?\nBy the way, when people only use Egit, I assume they use Eclipse,\nand you don't use Notepad.exe or so at all.\nThen you don't need CRLF in the worktree at all, as Eclipse handle\nLF well.\n\nand in this case you should be able to set\ngit config core.autocrlf input\non all repos, just in case someone sneaks in a CRLF somewhere.\n(And after the normalizing of course)\n\nhttps://www.kernel.org/pub/software/scm/git/docs/git-config.html\n\n(and don't ask me if Egit supports that)\n\n> \n> \n> $ git status\n> # On branch master\n> #\n> nothing to commit (working directory clean)\n> \n> $ file master/settings.gradle\n> master/settings.gradle: ASCII text, with CRLF line terminators\nThat is under msysgit ?\n(Side note: Msysgit is called Git for Windows these days)\n> \n> $ dos2unix.exe master/settings.gradle\nIs this under Msysgit ?\n> dos2unix: converting file master/settings.gradle to Unix format ...\n> \n> $ git status\n> # On branch master\n> #\n> # Changes not staged for commit:\n> #   (use \"git add <file>...\" to update what will be committed)\n> #   (use \"git checkout -- <file>...\" to discard changes in working directory)\n> #\n> #       modified:   master/settings.gradle\n> #\n> no changes added to commit (use \"git add\" and/or \"git commit -a\")\n> \n> $ git diff\n> fatal: LF would be replaced by CRLF in master/settings.gradle\nThat's interesting.\n\nWhat does \ngit config -l | grep core\ngive ?\n"},{"id":"255925","messageId":"CAA01CsoDV3FQfrCJ8Zs6gwSwwn3YOBhioG5QGxXFnnsVK8Z+Tw@mail.gmail.com","threadId":"38542","inReplyTo":"54DA248E.4080100@web.de","subject":"Re: EOL handling (EGit/svn/Windows)","fromName":"Piotr Krukowiecki","fromEmail":"piotr.krukowiecki@gmail.com","sentAt":"2015-02-11T11:46:08Z","receivedAt":"2015-02-11T11:46:08Z","isPatch":false,"sender":{"key":"piotr.krukowiecki@gmail.com","avatar":"https://avatars.githubusercontent.com/u/3259959?v=4"},"body":"On Tue, Feb 10, 2015 at 4:32 PM, Torsten Bögershausen <tboegi@web.de> wrote:\n> On 2015-02-10 11.52, Piotr Krukowiecki wrote:\n>>\n>> So far:\n>> 1. msysgit can't checkout a one file (saying filename too long, the\n>> relative path has 215 bytes) - probably not related to EOL issue.\n>> Cygwin git works ok. So I did not check how msysgit works yet.\n>\n> Please have a look here:\n> https://github.com/msysgit/msysgit/releases/Git-1.9.5-preview20141217\n> I think we have support for long path names (Haven't tested it myself)\n\nThanks, I'll have a look later if I have some free time. Since\nCygwin's git is more recent I'd prefer to use it instead of \"msys\",\nunless there are some other advantages of msys version.\n\n\n>> 2. maybe due to old cygwin git, I have a problem of not displaying\n>> changes, if the changed line has LF eol (and the file was checked out\n>> on Windows with CRLF eols). Will try later with newer git.\n> Normally you will not see any changes, and \"git diff\" will not show\n> anything either.\n\nI meant that there are some actual changes in the file (not only\nwhitespaces), but the line with the changes also has LF eol instead of\nCRLF, and the actual changes are now shown. That's what the examples\nbelow show.\n\n\n>> 2a. EGit handles such files gracefuly, but OTOH if the file is simple\n>> dos2unix'ed, it shows diffs showing all lines changed, and when you\n>> commit the files, it will create empty commit.\n> Why this dos2unix ?\n> Is there a special reason ?\n\nJust an use case - if for some reason someone/something coverts the\nfile to LF eol. Plus I think it would be better if empty commit was\nnot done (since there are no actual changes besides LF<->CRLF).\n\n\n> By the way, when people only use Egit, I assume they use Eclipse,\n> and you don't use Notepad.exe or so at all.\n> Then you don't need CRLF in the worktree at all, as Eclipse handle\n> LF well.\n\nThat's true, but I thought it'd be better to use native EOLs. The reasons:\n- new files will have initially CRLF (and will be converted to LF on\nfirst commit, but I think they will be left with CRLF in workspace\neven after commit), so workspace might have CRLF files anyway\n- some files are c# code/projects and are developer in VS, so they\nshould probably have CRLF eols; not checked how VS will work yet.\n- some external tools might not work with non-native EOLs (not that I\nknow of any specific tool we use - just a precaution)\n- when in Rome, do as Romans do, generally ;)\n\n\n> and in this case you should be able to set\n> git config core.autocrlf input\n> on all repos, just in case someone sneaks in a CRLF somewhere.\n> (And after the normalizing of course)\n\nI'm aware of that option, we might change to it - but I still don't\nsee what's the advantage, except lack of CRLF<->LF conversion (that's\na valid reason, but won't core.safecrlf help here?)\n\n\n>> $ file master/settings.gradle\n>> master/settings.gradle: ASCII text, with CRLF line terminators\n> That is under msysgit ?\n> (Side note: Msysgit is called Git for Windows these days)\n\nNope - both repository and client was cygwin git.\n\n\n>> $ dos2unix.exe master/settings.gradle\n> Is this under Msysgit ?\n\nCygwin.\n\n\n>> dos2unix: converting file master/settings.gradle to Unix format ...\n>>\n>> $ git status\n>> # On branch master\n>> #\n>> # Changes not staged for commit:\n>> #   (use \"git add <file>...\" to update what will be committed)\n>> #   (use \"git checkout -- <file>...\" to discard changes in working directory)\n>> #\n>> #       modified:   master/settings.gradle\n>> #\n>> no changes added to commit (use \"git add\" and/or \"git commit -a\")\n>>\n>> $ git diff\n>> fatal: LF would be replaced by CRLF in master/settings.gradle\n> That's interesting.\n>\n> What does\n> git config -l | grep core\n> give ?\n\ncore.autocrlf=true\ncore.safecrlf=true\ncore.eol=native\ncore.repositoryformatversion=0\ncore.filemode=true\ncore.bare=false\ncore.logallrefupdates=true\ncore.ignorecase=true\n\nPlus, \"*.gradle text\" in .gitattributes.\n\nAs I wrote, it might be old Cygwin git - 1.7.9. Will try to update.\n\n\n-- \nPiotr Krukowiecki\n"},{"id":"255932","messageId":"CAA01CsrJv8HmtNG6XAHZm5Hk8x8S_vFCYPz8vCtwBwd9-GAykg@mail.gmail.com","threadId":"38542","inReplyTo":"54D99BED.8010005@web.de","subject":"Re: EOL handling (EGit/svn/Windows)","fromName":"Piotr Krukowiecki","fromEmail":"piotr.krukowiecki@gmail.com","sentAt":"2015-02-11T15:51:27Z","receivedAt":"2015-02-11T15:51:27Z","isPatch":false,"sender":{"key":"piotr.krukowiecki@gmail.com","avatar":"https://avatars.githubusercontent.com/u/3259959?v=4"},"body":"On Tue, Feb 10, 2015 at 6:49 AM, Torsten Bögershausen <tboegi@web.de> wrote:\n> add that to the repo, do the normalization  and push.\n\nI did LF normalization over history. Looks like everything worked\ncorrectly, just wondering, what does the WARNING mean?\n\n$ git filter-branch --tree-filter '~/fix-eol.sh' -d /dev/shm/repo-tmp -- --all\nRewrite 4eab51b65af61d2b0e56f9990cc14be1e1372194 (13349/13349)\nRef 'refs/heads/master' was rewritten\nRef 'refs/remotes/origin/master' was rewritten\nWARNING: Ref 'refs/remotes/origin/master' is unchanged\n\n$ cat ~/fix-eol.sh\nfind . -type f \\! -path ./.git/\\* -print0 | xargs -0 dos2unix\n--keepdate 2>/dev/null\n\n\n-- \nPiotr Krukowiecki\n"}]}