{"thread":{"id":"13142","subject":"crlf with git-svn driving me nuts...","startedAt":"2008-04-16T19:10:26Z","lastAt":"2008-04-17T22:42:17Z","messageCount":19,"participants":["Nigel Magnay","Avery Pennarun","Dmitry Potapov","Martin Langhoff","Steffen Prohaska"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"74573","messageId":"320075ff0804161210m46f3e83bpf7bf9d1d5816d914@mail.gmail.com","threadId":"13142","inReplyTo":null,"subject":"crlf with git-svn driving me nuts...","fromName":"Nigel Magnay","fromEmail":"nigel.magnay@gmail.com","sentAt":"2008-04-16T19:10:26Z","receivedAt":"2008-04-16T19:10:26Z","isPatch":false,"sender":{"key":"nigel.magnay@gmail.com","avatar":"https://gravatar.com/avatar/d85cf38287bef3a8e4fa02358d2756d7589f8676c5eeb881ce2f6d731e4526c3?d=mp&s=160"},"body":"We've got projects with a mixed userbase of windows / *nix; I'm trying\nto migrate some users onto git, whilst everyone else stays happy in\ntheir SVN repo.\n\nHowever, there's one issue that has been driving me slowly insane.\nThis is best illustrated thusly (on windows) :\n\n  $ git init\n  $ git config core.autocrlf false\n\n-->Create a file with some text content on a few lines\n  $ notepad file.txt\n\n  $ git add file.txt\n  $ git commit -m \"initial checkin\"\n\n  $ git status\n# On branch master\nnothing to commit (working directory clean)\n--> Yarp, what I wanted\n\n  $ git config core.autocrlf true\n  $ git status\n\n# On branch master\nnothing to commit (working directory clean)\n--> Yarp, still all good\n\n--> Simulate non-change happened by an editor opening file...\n  $ touch file.txt\n  $ git status\n# On branch master\n# Changed but not updated:\n#   (use \"git add <file>...\" to update what will be committed)\n#\n#       modified:   file.txt\n#\nno changes added to commit (use \"git add\" and/or \"git commit -a\")\n\n--> Oh Noes! I wonder what it could be\n  $ git diff file.txt\ndiff --git a/file.txt b/file.txt\nindex 7a2051f..31ca3a0 100644\n--- a/file.txt\n+++ b/file.txt\n@@ -1,3 +1,3 @@\n-<xml>\n-       wooot\n-</xml>\n+<xml>\n+       wooot\n+</xml>\n\n--> Huh? ...\n  $ git diff -b file.txt\ndiff --git a/file.txt b/file.txt\nindex 7a2051f..31ca3a0 100644\n\n--> Bah... don't care! get me back to the start...\n  $ git reset --hard\n\nHEAD is now at 4762c31... initial checkin\n\n  $ git status\n# On branch master\n# Changed but not updated:\n#   (use \"git add <file>...\" to update what will be committed)\n#\n#       modified:   file.txt\n#\nno changes added to commit (use \"git add\" and/or \"git commit -a\")\n\n--> ARGH!\n  $ git config core.autocrlf false\n  $ git status\n# On branch master\nnothing to commit (working directory clean)\n\n  $ git config core.autocrlf true\n  $ git status\n# On branch master\nnothing to commit (working directory clean)\n\n--> WtF?\n\nWhy does it think in this instance that there is a change? It's CRLF\nin the repo, it's CRLF in the working tree, and the checkout in either\nmode ought to be identical ??\n\nNow this is further compounded by the fact that users then typically\ntend to do a 'CRLF->LF conversion' checkin - *BUT* this will cause\nmerge conflicts if another user actually made a genuine change (I.E.\nthe removal of CR and the change are both treated as significant).\n\nAdditional fun is caused because some editors 'touching' files that\nthey actually haven't modified, leading to all these 'null' changes.\n\nThis is a bigger deal for us than it ought to be, because we're\npulling changes from a windows-based svn repo, which is always CRLF.\nShould I set core.autocrlf=input when doing 'git svn fetch' (and would\nit pay any attention)? Also is it possible to tell the diff / merge\nmachinery that it ought to just ignore text file line endings when\nmerging ?\n\nSorry if some of this is stupid-user territory, but there's probably a\nfew people out there also looking at trying to migrate away from\nWindows+SVN that are likely to hit the same things...\n"},{"id":"74587","messageId":"20080416200107.GG3133@dpotapov.dyndns.org","threadId":"13142","inReplyTo":"320075ff0804161210m46f3e83bpf7bf9d1d5816d914@mail.gmail.com","subject":"Re: crlf with git-svn driving me nuts...","fromName":"Dmitry Potapov","fromEmail":"dpotapov@gmail.com","sentAt":"2008-04-16T20:01:07Z","receivedAt":"2008-04-16T20:01:07Z","isPatch":false,"sender":{"key":"dpotapov@gmail.com","avatar":"https://avatars.githubusercontent.com/u/6568595?v=4"},"body":"On Wed, Apr 16, 2008 at 08:10:26PM +0100, Nigel Magnay wrote:\n> We've got projects with a mixed userbase of windows / *nix; I'm trying\n> to migrate some users onto git, whilst everyone else stays happy in\n> their SVN repo.\n> \n> However, there's one issue that has been driving me slowly insane.\n> This is best illustrated thusly (on windows) :\n> \n>   $ git init\n>   $ git config core.autocrlf false\n\ncore.autocrlf=false is a bad choice for Windows.\n\n> \n> -->Create a file with some text content on a few lines\n>   $ notepad file.txt\n> \n>   $ git add file.txt\n>   $ git commit -m \"initial checkin\"\n\nYou added a file with the CRLF ending in the repository!\nYou are going to have problems now...\n\n> \n>   $ git status\n> # On branch master\n> nothing to commit (working directory clean)\n> --> Yarp, what I wanted\n> \n>   $ git config core.autocrlf true\n>   $ git status\n\nYou should not change core.autocrlf during your work, or you\nare going to have some funny problems. If you really need to\nchange it, it should be followed by \"git reset --hard\".\n\nIn this case, you already have a file with the wrong ending,\nso file.txt will be shown as changed now, because if you commit\nit again then it will be commited with <LF>, which should have\nbeen done in the first place.\n\n> \n> # On branch master\n> nothing to commit (working directory clean)\n> --> Yarp, still all good\n> \n> --> Simulate non-change happened by an editor opening file...\n>   $ touch file.txt\n>   $ git status\n> # On branch master\n> # Changed but not updated:\n> #   (use \"git add <file>...\" to update what will be committed)\n> #\n> #       modified:   file.txt\n> #\n> no changes added to commit (use \"git add\" and/or \"git commit -a\")\n> \n> --> Oh Noes! I wonder what it could be\n>   $ git diff file.txt\n> diff --git a/file.txt b/file.txt\n> index 7a2051f..31ca3a0 100644\n> --- a/file.txt\n> +++ b/file.txt\n> @@ -1,3 +1,3 @@\n> -<xml>\n> -       wooot\n> -</xml>\n> +<xml>\n> +       wooot\n> +</xml>\n> \n> --> Huh? ...\n\nActually, it is\n\n@@ -1,3 +1,3 @@\n-<xml>^M\n-       wooot^M\n-</xml>^M\n+<xml>\n+       wooot\n+</xml>\n\nwhere ^M is <CR>\n\n> \n> --> WtF?\n> \n> Why does it think in this instance that there is a change? It's CRLF\n> in the repo, it's CRLF in the working tree, and the checkout in either\n> mode ought to be identical ??\n\nIf you do not want problems, you should use core.autocrlf=true\non Windows. Then all text files will be stored in the repository\nwith <LF>, but they will have <CR><LF> in your work tree.\nUsers on *nix should set core.autocrlf=input or false, so they\nwill have <LF> in their work tree.\n\nDmitry\n"},{"id":"74585","messageId":"32541b130804161303i74a4dd3bw5ef72105ebf5485b@mail.gmail.com","threadId":"13142","inReplyTo":"320075ff0804161210m46f3e83bpf7bf9d1d5816d914@mail.gmail.com","subject":"Re: crlf with git-svn driving me nuts...","fromName":"Avery Pennarun","fromEmail":"apenwarr@gmail.com","sentAt":"2008-04-16T20:03:34Z","receivedAt":"2008-04-16T20:03:34Z","isPatch":false,"sender":{"key":"apenwarr@gmail.com","avatar":"https://avatars.githubusercontent.com/u/20592?v=4"},"body":"On 4/16/08, Nigel Magnay <nigel.magnay@gmail.com> wrote:\n>  Why does it think in this instance that there is a change? It's CRLF\n>  in the repo, it's CRLF in the working tree, and the checkout in either\n>  mode ought to be identical ??\n\nWe got quite confused by this here too.  I'm pretty sure git's\nautocrlf feature is buggy, as you've noticed.  Combined with that, svn\nhas its *own* kind of autocrlf feature (svn:eol-style property on each\nfile) that acts completely differently.\n\nAs an added bonus, I don't know if you've run into this yet, but\ncygwin's \"patch\" command seems to unconditionally strip CR from\npatches *before* trying to apply them at all, *even if* the target\nfile is CRLF, so patches just never apply to CRLF files ever.  Ha ha!\n\nI managed to make the two systems stop stomping on each other, in our\ncase, by using svn:eol-style of \"native\" (which means when git-svn\nchecks out the file, it gets only LF, since it seems to always claim\nto be Unix) and not using git's autocrlf at all.  However, this isn't\noptimal since then Windows git users end up with LF instead of CRLF in\ntheir files, which confuses them.\n\nOn the other hand, the conflicts and the random-newline-changing diffs\ngo away, as svn fixes things up at checkin time no matter how badly\nthey got mangled by the windows user (most commonly, they run a\nprogram that resaves the whole file as CRLF).\n\nObviously a working git autocrlf feature would be better, but I\nhaven't looked into it closely enough to say where the problem\nactually lies.\n\nHave fun,\n\nAvery\n"},{"id":"74592","messageId":"32541b130804161320w42c68bc2p9d838dc127a57027@mail.gmail.com","threadId":"13142","inReplyTo":"20080416200107.GG3133@dpotapov.dyndns.org","subject":"Re: crlf with git-svn driving me nuts...","fromName":"Avery Pennarun","fromEmail":"apenwarr@gmail.com","sentAt":"2008-04-16T20:20:27Z","receivedAt":"2008-04-16T20:20:27Z","isPatch":false,"sender":{"key":"apenwarr@gmail.com","avatar":"https://avatars.githubusercontent.com/u/20592?v=4"},"body":"On 4/16/08, Dmitry Potapov <dpotapov@gmail.com> wrote:\n>  In this case, you already have a file with the wrong ending,\n>  so file.txt will be shown as changed now, because if you commit\n>  it again then it will be commited with <LF>, which should have\n>  been done in the first place.\n[...]\n> If you do not want problems, you should use core.autocrlf=true\n>  on Windows. Then all text files will be stored in the repository\n>  with <LF>, but they will have <CR><LF> in your work tree.\n>  Users on *nix should set core.autocrlf=input or false, so they\n>  will have <LF> in their work tree.\n\nAlas, the subject of this thread involves git-svn, and the typical\ngit-svn user is someone who has no way of rewriting the existing\nhistory in their svn repositories.  Thus, files *will* be in the\nrepository that have the wrong line endings, and (as you noted) git\njust gets totally confused in that case.\n\nNigel's example showed a few situations where git *thought* the file\nhad changed when it hadn't, and yet is incapable of checking in the\nchanges.\n\nIf all I had to do was checkout (thus converting everything to LF),\nand then \"git commit -a\" to check in all the corrected files, then\ngit-svn would make one giant, very rude checkin to svn, and my\nproblems would be largely solved.  However, this does not seem to be\npossible due to the problems you noted (\"you are going to have\nproblems now\").\n\nHave fun,\n\nAvery\n"},{"id":"74594","messageId":"20080416203923.GH3133@dpotapov.dyndns.org","threadId":"13142","inReplyTo":"32541b130804161320w42c68bc2p9d838dc127a57027@mail.gmail.com","subject":"Re: crlf with git-svn driving me nuts...","fromName":"Dmitry Potapov","fromEmail":"dpotapov@gmail.com","sentAt":"2008-04-16T20:39:23Z","receivedAt":"2008-04-16T20:39:23Z","isPatch":false,"sender":{"key":"dpotapov@gmail.com","avatar":"https://avatars.githubusercontent.com/u/6568595?v=4"},"body":"On Wed, Apr 16, 2008 at 04:20:27PM -0400, Avery Pennarun wrote:\n> On 4/16/08, Dmitry Potapov <dpotapov@gmail.com> wrote:\n> >  In this case, you already have a file with the wrong ending,\n> >  so file.txt will be shown as changed now, because if you commit\n> >  it again then it will be commited with <LF>, which should have\n> >  been done in the first place.\n> [...]\n> > If you do not want problems, you should use core.autocrlf=true\n> >  on Windows. Then all text files will be stored in the repository\n> >  with <LF>, but they will have <CR><LF> in your work tree.\n> >  Users on *nix should set core.autocrlf=input or false, so they\n> >  will have <LF> in their work tree.\n> \n> Alas, the subject of this thread involves git-svn, and the typical\n> git-svn user is someone who has no way of rewriting the existing\n> history in their svn repositories.  Thus, files *will* be in the\n> repository that have the wrong line endings, and (as you noted) git\n> just gets totally confused in that case.\n\nActually, what matters in what format files are in _Git_ repository.\nMaybe, there is a problem with git-svn and how it imports SVN commits\nto Git, but I have not encountered it.\n\n> Nigel's example showed a few situations where git *thought* the file\n> had changed when it hadn't, and yet is incapable of checking in the\n> changes.\n\nIncapable of checking in? I have not found a single example in\nhis mail where it was impossible. The only quirk with autocrlf\nis that you need to re-checkout your work tree after changing\nit. There is no other problems with it as far as I know.\n\nDmitry\n"},{"id":"74598","messageId":"46a038f90804161356o7518ec72j3bfb4e9fe4e48852@mail.gmail.com","threadId":"13142","inReplyTo":"20080416200107.GG3133@dpotapov.dyndns.org","subject":"Re: crlf with git-svn driving me nuts...","fromName":"Martin Langhoff","fromEmail":"martin.langhoff@gmail.com","sentAt":"2008-04-16T20:56:18Z","receivedAt":"2008-04-16T20:56:18Z","isPatch":false,"sender":{"key":"martin.langhoff@gmail.com","avatar":"https://gravatar.com/avatar/1e3f311b6c4c15836501901ca58f8c0b0667246488084ba524d8bc9867e22fd9?d=mp&s=160"},"body":"On Wed, Apr 16, 2008 at 3:01 PM, Dmitry Potapov <dpotapov@gmail.com> wrote:\n>  core.autocrlf=false is a bad choice for Windows.\n...\n>  If you do not want problems, you should use core.autocrlf=true\n>  on Windows.\n\nIf you are making the above statements in generally about git, I\ndisagree. I have used msysgit a lot with unix-newlines projects, and\nit works fantastic. I am careful to work with newline-smart editors\nbut any half-decent editor will cope. The general hint is: avoid any\ncontent-mangling options if possible, and git will do the right thing.\n\nOTOH, you might be referring to git-svn on Windows, which I have no\nexperience with :-)\n\ncheers,\n\n\n\nmartin\n-- \n martin.langhoff@gmail.com\n martin@laptop.org -- School Server Architect\n - ask interesting questions\n - don't get distracted with shiny stuff - working code first\n - http://wiki.laptop.org/go/User:Martinlanghoff\n"},{"id":"74600","messageId":"32541b130804161402u736f4c02wa1c2b977d64e98ec@mail.gmail.com","threadId":"13142","inReplyTo":"46a038f90804161356o7518ec72j3bfb4e9fe4e48852@mail.gmail.com","subject":"Re: crlf with git-svn driving me nuts...","fromName":"Avery Pennarun","fromEmail":"apenwarr@gmail.com","sentAt":"2008-04-16T21:02:20Z","receivedAt":"2008-04-16T21:02:20Z","isPatch":false,"sender":{"key":"apenwarr@gmail.com","avatar":"https://avatars.githubusercontent.com/u/20592?v=4"},"body":"On 4/16/08, Martin Langhoff <martin.langhoff@gmail.com> wrote:\n> On Wed, Apr 16, 2008 at 3:01 PM, Dmitry Potapov <dpotapov@gmail.com> wrote:\n> >  If you do not want problems, you should use core.autocrlf=true\n> >  on Windows.\n>\n> If you are making the above statements in generally about git, I\n>  disagree. I have used msysgit a lot with unix-newlines projects, and\n>  it works fantastic. I am careful to work with newline-smart editors\n>  but any half-decent editor will cope. The general hint is: avoid any\n>  content-mangling options if possible, and git will do the right thing.\n\nVarious Windows IDEs (notably Delphi... and notepad :)) get confused\nby non-CRLF files and either do random things to the file, fail to\ncompile, or \"helpfully\" change all the line endings back to CRLF.  I\nagree that any program that does any such thing is braindead, but\nunfortunately, some people are stuck with such programs.\n\nAvery\n"},{"id":"74602","messageId":"20080416211723.GI3133@dpotapov.dyndns.org","threadId":"13142","inReplyTo":"46a038f90804161356o7518ec72j3bfb4e9fe4e48852@mail.gmail.com","subject":"Re: crlf with git-svn driving me nuts...","fromName":"Dmitry Potapov","fromEmail":"dpotapov@gmail.com","sentAt":"2008-04-16T21:17:23Z","receivedAt":"2008-04-16T21:17:23Z","isPatch":false,"sender":{"key":"dpotapov@gmail.com","avatar":"https://avatars.githubusercontent.com/u/6568595?v=4"},"body":"On Wed, Apr 16, 2008 at 03:56:18PM -0500, Martin Langhoff wrote:\n> On Wed, Apr 16, 2008 at 3:01 PM, Dmitry Potapov <dpotapov@gmail.com> wrote:\n> >  core.autocrlf=false is a bad choice for Windows.\n> ...\n> >  If you do not want problems, you should use core.autocrlf=true\n> >  on Windows.\n> \n> If you are making the above statements in generally about git, I\n> disagree.\n\nI stand corrected. It should be either core.autocrlf=true is you\nlike DOS ending or core.autocrlf=input if you prefer unix-newlines.\nIn both cases, your Git repository will have only LF, which is the\nRight Thing. The only argument for core.autocrlf=false was that\nautomatic heuristic may incorrectly detect some binary as text and\nthen your tile will be corrupted. So, core.safecrlf option was\nintroduced to warn a user if a irreversable change happens. In fact,\nthere are two possibilities of irreversable changes -- mixed line-ending\nin text file, in this normalization is desirable, so this warning can be\nignored, or (very unlikely) that Git incorrectly detected your binary\nfile as text. Then you need to use attributes to tell Git that this file\nis binary.\n\nI have not used git-svn on Windows for some time now, because now I have\na mirror running on Linux, so I clone directly from it.\n\nDmitry\n"},{"id":"74605","messageId":"320075ff0804161456h1a6fd4e8qdd185f4540a1259a@mail.gmail.com","threadId":"13142","inReplyTo":"20080416203923.GH3133@dpotapov.dyndns.org","subject":"Re: crlf with git-svn driving me nuts...","fromName":"Nigel Magnay","fromEmail":"nigel.magnay@gmail.com","sentAt":"2008-04-16T21:56:54Z","receivedAt":"2008-04-16T21:56:54Z","isPatch":false,"sender":{"key":"nigel.magnay@gmail.com","avatar":"https://gravatar.com/avatar/d85cf38287bef3a8e4fa02358d2756d7589f8676c5eeb881ce2f6d731e4526c3?d=mp&s=160"},"body":">  > Nigel's example showed a few situations where git *thought* the file\n>  > had changed when it hadn't, and yet is incapable of checking in the\n>  > changes.\n>\n>  Incapable of checking in? I have not found a single example in\n>  his mail where it was impossible. The only quirk with autocrlf\n>  is that you need to re-checkout your work tree after changing\n>  it. There is no other problems with it as far as I know.\n>\n\nMy (initial) setting of core.autocrlf to false was because that's what\nit was on all the windows clients (I know the default has now changed)\nand to make the later parts of the script obvious that the file in the\nrepo had a CRLF ending, rather than have being converted to LF. That's\nthe situation we have, because they've all come from SVN.\n\nThe bit I really don't understand is why git thinks a file that has\njust been touched has chnaged when it hasn't, and doing a 'git reset\n--hard' actually doesn't help at all (but, bizzarely, git config\ncore.autocrlf false & git config core.autocrlf true *does* !). The\nrepo copy is CRLF, the working copy is CRLF, but git thinks it's\nchanged...\n"},{"id":"74608","messageId":"320075ff0804161607p3f9e983ehb75aae4e0bfe8837@mail.gmail.com","threadId":"13142","inReplyTo":"20080416223739.GJ3133@dpotapov.dyndns.org","subject":"Re: crlf with git-svn driving me nuts...","fromName":"Nigel Magnay","fromEmail":"nigel.magnay@gmail.com","sentAt":"2008-04-16T23:07:27Z","receivedAt":"2008-04-16T23:07:27Z","isPatch":false,"sender":{"key":"nigel.magnay@gmail.com","avatar":"https://gravatar.com/avatar/d85cf38287bef3a8e4fa02358d2756d7589f8676c5eeb881ce2f6d731e4526c3?d=mp&s=160"},"body":">  > The bit I really don't understand is why git thinks a file that has\n>  > just been touched has chnaged when it hasn't,\n>\n>  Actually, it did change in the sense that if you try to commit this\n>  file now into the repository, you will have a different file in Git!\n>  So, it is more correct to say that Git did not notice this change until\n>  you touch this file, because this change is indirect (autocrlf causes\n>  a different interpretation of the file).\n>\n\nOkay - at the very least this behaviour is really, really confusing.\nAnd I think there's actually a bug (it should *always* report that the\nfile is different), not magically after it's been touched.\n\nBut fixing that minor bug still leads to badness for the user. Doing\n(on a core.autocrlf=true machine) a checkout of any revision\ncontaining a file that is (currently) CRLF in the repository, and your\nWC is *immediately* dirty. However technically correct that is, it\ndoesn't fit most people's user model of an SCM, because they haven't\nmade any modification. And if 1 person makes a change along with their\nconversion, and the other 'just' does a CRLF->LF conversion, their\nrevisions will conflict at merge time. Blech. And because the svn is\nmastered crlf (well, strictly speaking, it's ignorant of line endings)\nthis is gonna happen a lot.\n\nCan't git be taught that if the WC is byte-identical to the revision\nin the repository (regardless of autocrlf) then that ought not to be\nregarded as a change?\nIs there a way I can persuade the diff / merge mechanisms to normalise\nbefore they operate? (e.g if core.autocrlf does lf->crlf/crlf->lf,\nthen an equivalent that does crlf->lf/crlf->lf before doing the merge\n)?\n\nIn a perfect world I'd be able to switch all files int he repo to LF,\nbut that's not going to happen any time soon because of the majority\nof developers, still on svn, still on windows.\n"},{"id":"74610","messageId":"20080417004645.GK3133@dpotapov.dyndns.org","threadId":"13142","inReplyTo":"320075ff0804161607p3f9e983ehb75aae4e0bfe8837@mail.gmail.com","subject":"Re: crlf with git-svn driving me nuts...","fromName":"Dmitry Potapov","fromEmail":"dpotapov@gmail.com","sentAt":"2008-04-17T00:46:45Z","receivedAt":"2008-04-17T00:46:45Z","isPatch":false,"sender":{"key":"dpotapov@gmail.com","avatar":"https://avatars.githubusercontent.com/u/6568595?v=4"},"body":"On Thu, Apr 17, 2008 at 12:07:27AM +0100, Nigel Magnay wrote:\n> >  > The bit I really don't understand is why git thinks a file that has\n> >  > just been touched has chnaged when it hasn't,\n> >\n> >  Actually, it did change in the sense that if you try to commit this\n> >  file now into the repository, you will have a different file in Git!\n> >  So, it is more correct to say that Git did not notice this change until\n> >  you touch this file, because this change is indirect (autocrlf causes\n> >  a different interpretation of the file).\n> >\n> \n> Okay - at the very least this behaviour is really, really confusing.\n> And I think there's actually a bug (it should *always* report that the\n> file is different), not magically after it's been touched.\n\nI don't think there is a simple way to correct that without penalizing\nnormal use cases. Usually, people do not change autocrlf during their\nnormal work. Besides, you can have your own input filters and they may\ncause the same effect. So, Git works in the assumption that input filters\nalways produce the same results...\n\n> \n> But fixing that minor bug still leads to badness for the user. Doing\n> (on a core.autocrlf=true machine) a checkout of any revision\n> containing a file that is (currently) CRLF in the repository, and your\n> WC is *immediately* dirty. However technically correct that is, it\n> doesn't fit most people's user model of an SCM, because they haven't\n> made any modification.\n\nIMHO, the only sane way is never store CRLF in the Git repository.\nYou can have whatever ending you like in your work tree, but inside\nof Git, LF is the actually marker of the end-of-line.\n\n> And if 1 person makes a change along with their\n> conversion, and the other 'just' does a CRLF->LF conversion,\n\nIf you imported correctly in Git, it should not have CRLF for text\nfiles. So, there is no conversion that a user does expliciltly.\n\n> And because the svn is\n> mastered crlf (well, strictly speaking, it's ignorant of line endings)\n> this is gonna happen a lot.\n\nNot really. SVN has its own setting for EOL conversion. If you have\n'svn:eol-style' set to 'native' for any text file then SVN will\ncheckout text files accordingly to your native EOL (you can specify\nyour native EOL using the --native-eol option when it is necessary).\n\n> Can't git be taught that if the WC is byte-identical to the revision\n> in the repository (regardless of autocrlf) then that ought not to be\n> regarded as a change?\n\nWhy should not it? If a file is different as long as Git repository is\nconcern then then it *is* a change. Git binary compare files _after_\napplying all specified filters (and you can have your own filters, not\nonly autocrlf).\n\n> Is there a way I can persuade the diff / merge mechanisms to normalise\n> before they operate? (e.g if core.autocrlf does lf->crlf/crlf->lf,\n> then an equivalent that does crlf->lf/crlf->lf before doing the merge\n> )?\n\nI am not sure if there is a standard option for that, but it is\ncertainly possible to define your own merge strategy.\n\n> \n> In a perfect world I'd be able to switch all files int he repo to LF,\n> but that's not going to happen any time soon because of the majority\n> of developers, still on svn, still on windows.\n\nWell, I don't see any problem here if everything is configured properly.\nHow files are stored inside and what you have in your work tree does\nnot have to be the same. So, storing everything inside with LF is\ncertainly possible. Actually, I believe it is exactly what CVS does\n(unless you added a file with '-kb'), and people use CVS on Windows.\nImporting files with CRLF in Git, it is like putting files as _binary_\nin CVS.\n\nDmitry\n"},{"id":"74611","messageId":"32541b130804161844g9c7b321u70bbd36a01494185@mail.gmail.com","threadId":"13142","inReplyTo":"20080417004645.GK3133@dpotapov.dyndns.org","subject":"Re: crlf with git-svn driving me nuts...","fromName":"Avery Pennarun","fromEmail":"apenwarr@gmail.com","sentAt":"2008-04-17T01:44:39Z","receivedAt":"2008-04-17T01:44:39Z","isPatch":false,"sender":{"key":"apenwarr@gmail.com","avatar":"https://avatars.githubusercontent.com/u/20592?v=4"},"body":"On 4/16/08, Dmitry Potapov <dpotapov@gmail.com> wrote:\n> On Thu, Apr 17, 2008 at 12:07:27AM +0100, Nigel Magnay wrote:\n>  > Okay - at the very least this behaviour is really, really confusing.\n>  > And I think there's actually a bug (it should *always* report that the\n>  > file is different), not magically after it's been touched.\n>\n> I don't think there is a simple way to correct that without penalizing\n>  normal use cases. Usually, people do not change autocrlf during their\n>  normal work. Besides, you can have your own input filters and they may\n>  cause the same effect. So, Git works in the assumption that input filters\n>  always produce the same results...\n\nHowever, it doesn't check that before it marks the file as unmodified\nright after checkout.  That is, the problem is hidden until the file's\nmtime changes.\n\nIs there a way to quickly check that every file in the repo is \"sane\",\nie. the input filter is the proper inverse of the output filter and\nwill put each file back in the repo?  This is pretty important for\nanyone designing any kind of input filter, or bugs will go undetected\nuntil some later time when they're confusing.\n\n> If you imported correctly in Git, it should not have CRLF for text\n>  files. So, there is no conversion that a user does expliciltly.\n\nCan you give a set of steps for how to import \"correctly\" using git-svn?\n\nRemember that a given svn repository might have long ago been\nconfigured to store CRLF (actually, to store files without changing\ntheir line endings), since that is the svn default.  Also remember\nthat the svn:eol-style flag may be set differently on various files in\nsvn, and may have changed in different svn revisions over time.\n\nThanks,\n\nAvery\n"},{"id":"74614","messageId":"A9C84864-8F7C-4B0D-A304-A95399559149@zib.de","threadId":"13142","inReplyTo":"320075ff0804161607p3f9e983ehb75aae4e0bfe8837@mail.gmail.com","subject":"Re: crlf with git-svn driving me nuts...","fromName":"Steffen Prohaska","fromEmail":"prohaska@zib.de","sentAt":"2008-04-17T05:43:49Z","receivedAt":"2008-04-17T05:43:49Z","isPatch":false,"sender":{"key":"prohaska@zib.de","avatar":"https://avatars.githubusercontent.com/u/217580?v=4"},"body":"\nOn Apr 17, 2008, at 1:07 AM, Nigel Magnay wrote:\n\n> In a perfect world I'd be able to switch all files int the repo to LF,\n> but that's not going to happen any time soon because of the majority\n> of developers, still on svn, still on windows.\n\n\nIf you want Git's autocrlf to convert to the native line endings on  \nWindows and Unix, you need to convert everything to LF in the repo.   \nThis is what we did and now everything runs smoothly.\n\nI have no recommendation, though, how to use svn and git together.  I  \ndo not use git-svn.\n\n\tSteffen\n"},{"id":"74618","messageId":"320075ff0804170007k5171eb72n68882679f62fa238@mail.gmail.com","threadId":"13142","inReplyTo":"20080417004645.GK3133@dpotapov.dyndns.org","subject":"Re: crlf with git-svn driving me nuts...","fromName":"Nigel Magnay","fromEmail":"nigel.magnay@gmail.com","sentAt":"2008-04-17T07:07:27Z","receivedAt":"2008-04-17T07:07:27Z","isPatch":false,"sender":{"key":"nigel.magnay@gmail.com","avatar":"https://gravatar.com/avatar/d85cf38287bef3a8e4fa02358d2756d7589f8676c5eeb881ce2f6d731e4526c3?d=mp&s=160"},"body":"On Thu, Apr 17, 2008 at 1:46 AM, Dmitry Potapov <dpotapov@gmail.com> wrote:\n> On Thu, Apr 17, 2008 at 12:07:27AM +0100, Nigel Magnay wrote:\n>  > >  > The bit I really don't understand is why git thinks a file that has\n>  > >  > just been touched has chnaged when it hasn't,\n>  > >\n>  > >  Actually, it did change in the sense that if you try to commit this\n>  > >  file now into the repository, you will have a different file in Git!\n>  > >  So, it is more correct to say that Git did not notice this change until\n>  > >  you touch this file, because this change is indirect (autocrlf causes\n>  > >  a different interpretation of the file).\n>  > >\n>  >\n>  > Okay - at the very least this behaviour is really, really confusing.\n>  > And I think there's actually a bug (it should *always* report that the\n>  > file is different), not magically after it's been touched.\n>\n>  I don't think there is a simple way to correct that without penalizing\n>  normal use cases. Usually, people do not change autocrlf during their\n>  normal work. Besides, you can have your own input filters and they may\n>  cause the same effect. So, Git works in the assumption that input filters\n>  always produce the same results...\n\nThis has nothing to do with changing core.autocrlf after checkout -\nit's a problem with *any* repo with CRLF files, being checked out on a\ncore.autocrlf=true machine, which basically is any windows machine.\n\nThe current 'isDirty' check seems to be something like\n\nisDirty = ( wc.file.mtime > someValue ) && ( repository.file !=\nfilter(wc.file) )\n\nI'm saying it ought to be something like\n\nisDirty = ( wc.file.mtime > someValue ) && (sha1(repository.file) !=\nsha1(wc.file) ) && ( repository.file != filter(wc.file) )\n\n\n>\n>\n>  >\n>  > But fixing that minor bug still leads to badness for the user. Doing\n>  > (on a core.autocrlf=true machine) a checkout of any revision\n>  > containing a file that is (currently) CRLF in the repository, and your\n>  > WC is *immediately* dirty. However technically correct that is, it\n>  > doesn't fit most people's user model of an SCM, because they haven't\n>  > made any modification.\n>\n>  IMHO, the only sane way is never store CRLF in the Git repository.\n>  You can have whatever ending you like in your work tree, but inside\n>  of Git, LF is the actually marker of the end-of-line.\n>\n\nGreat. I'll go and argue with the team using svn, who don't even\n*notice* this issue, and try to get them to adjust the metadata on\nevery single file in the repository.\n\nThen, for a bonus, I'll try the same with every OSS project that I'm\ntracking with git-svn. :-(\n\nI get that things are horribly broken if you get CRLF in your\nrepository. But it's unreasonable to expect the ability to bend the\nrest of the world to what's convenient for me! Some of our windows\ncoders probably even *like* svn:eol-style=CRLF !\n\n>\n>  > And if 1 person makes a change along with their\n>  > conversion, and the other 'just' does a CRLF->LF conversion,\n>\n>  If you imported correctly in Git, it should not have CRLF for text\n>  files. So, there is no conversion that a user does expliciltly.\n>\n>\n>  > And because the svn is\n>  > mastered crlf (well, strictly speaking, it's ignorant of line endings)\n>  > this is gonna happen a lot.\n>\n>  Not really. SVN has its own setting for EOL conversion. If you have\n>  'svn:eol-style' set to 'native' for any text file then SVN will\n>  checkout text files accordingly to your native EOL (you can specify\n>  your native EOL using the --native-eol option when it is necessary).\n>\n\nCan I set this personally, without affecting the svn repo? If so, why\nisn't git-svn doing this anyway, and can I tell it to do so?\n\n>\n>  > Can't git be taught that if the WC is byte-identical to the revision\n>  > in the repository (regardless of autocrlf) then that ought not to be\n>  > regarded as a change?\n>\n>  Why should not it? If a file is different as long as Git repository is\n>  concern then then it *is* a change. Git binary compare files _after_\n>  applying all specified filters (and you can have your own filters, not\n>  only autocrlf).\n>\n\nSee above. Unchanged (on disk, byte identical) files, if touched, get\n(sometimes) marked as dirty.\n\n>\n>  > Is there a way I can persuade the diff / merge mechanisms to normalise\n>  > before they operate? (e.g if core.autocrlf does lf->crlf/crlf->lf,\n>  > then an equivalent that does crlf->lf/crlf->lf before doing the merge\n>  > )?\n>\n>  I am not sure if there is a standard option for that, but it is\n>  certainly possible to define your own merge strategy.\n>\nOk - I'll have a look into this - just a filter on each file before\nmerging would be sufficient. Presumably people that do things like\n$Id$ expansion need something similar to avoid constant merge\nconflicts..\n\n>\n>  >\n>  > In a perfect world I'd be able to switch all files int he repo to LF,\n>  > but that's not going to happen any time soon because of the majority\n>  > of developers, still on svn, still on windows.\n>\n>  Well, I don't see any problem here if everything is configured properly.\n>  How files are stored inside and what you have in your work tree does\n>  not have to be the same. So, storing everything inside with LF is\n>  certainly possible. Actually, I believe it is exactly what CVS does\n>  (unless you added a file with '-kb'), and people use CVS on Windows.\n>  Importing files with CRLF in Git, it is like putting files as _binary_\n>  in CVS.\n>\n>  Dmitry\n>\n"},{"id":"74626","messageId":"20080417094342.GM3133@dpotapov.dyndns.org","threadId":"13142","inReplyTo":"320075ff0804170007k5171eb72n68882679f62fa238@mail.gmail.com","subject":"Re: crlf with git-svn driving me nuts...","fromName":"Dmitry Potapov","fromEmail":"dpotapov@gmail.com","sentAt":"2008-04-17T09:43:42Z","receivedAt":"2008-04-17T09:43:42Z","isPatch":false,"sender":{"key":"dpotapov@gmail.com","avatar":"https://avatars.githubusercontent.com/u/6568595?v=4"},"body":"On Thu, Apr 17, 2008 at 08:07:27AM +0100, Nigel Magnay wrote:\n> \n> This has nothing to do with changing core.autocrlf after checkout -\n> it's a problem with *any* repo with CRLF files, being checked out on a\n> core.autocrlf=true machine, which basically is any windows machine.\n> \n> The current 'isDirty' check seems to be something like\n> \n> isDirty = ( wc.file.mtime > someValue ) && ( repository.file !=\n> filter(wc.file) )\n\nBasically, yes.\n\n> \n> I'm saying it ought to be something like\n> \n> isDirty = ( wc.file.mtime > someValue ) && (sha1(repository.file) !=\n> sha1(wc.file) ) && ( repository.file != filter(wc.file) )\n\nI don't think it is reasonable. Files inside of the repository and\nin the work are not meant to be the same. What if I have $Id$ expansion\nor something else. What could make sense is to add an additional check:\n && convert_to_work_tree(repository.file) != wc.file\nbut it should be optional, so it will not penalize those who do need\nor do not want this extra check.\n\n> >  >\n> >  > But fixing that minor bug still leads to badness for the user. Doing\n> >  > (on a core.autocrlf=true machine) a checkout of any revision\n> >  > containing a file that is (currently) CRLF in the repository, and your\n> >  > WC is *immediately* dirty. However technically correct that is, it\n> >  > doesn't fit most people's user model of an SCM, because they haven't\n> >  > made any modification.\n> >\n> >  IMHO, the only sane way is never store CRLF in the Git repository.\n> >  You can have whatever ending you like in your work tree, but inside\n> >  of Git, LF is the actually marker of the end-of-line.\n> >\n> \n> Great. I'll go and argue with the team using svn, who don't even\n> *notice* this issue, and try to get them to adjust the metadata on\n> every single file in the repository.\n\nMaybe, you can teach git-svn to be smarter... I mean storing text files\nin Git repo with CRLF is stupid, so, perhaps, git-svn can do a better\njob converting CRLF<->LF when it exports and imports from/to SVN.\n\n> \n> Then, for a bonus, I'll try the same with every OSS project that I'm\n> tracking with git-svn. :-(\n> \n> I get that things are horribly broken if you get CRLF in your\n> repository. But it's unreasonable to expect the ability to bend the\n> rest of the world to what's convenient for me! Some of our windows\n> coders probably even *like* svn:eol-style=CRLF !\n\nYou can use Git and have CRLF in your work tree. You just need to\nhave autocrlf=true for that. _Inside_ of Git, only LF is the end\nof line. How you store in SVN, it is a separate issue with git-svn.\nI guess, git-svn needs improvement in this area...\n\nDmitry\n"},{"id":"74628","messageId":"320075ff0804170309h4232463fk984f362e6cf0a259@mail.gmail.com","threadId":"13142","inReplyTo":"20080417094342.GM3133@dpotapov.dyndns.org","subject":"Re: crlf with git-svn driving me nuts...","fromName":"Nigel Magnay","fromEmail":"nigel.magnay@gmail.com","sentAt":"2008-04-17T10:09:12Z","receivedAt":"2008-04-17T10:09:12Z","isPatch":false,"sender":{"key":"nigel.magnay@gmail.com","avatar":"https://gravatar.com/avatar/d85cf38287bef3a8e4fa02358d2756d7589f8676c5eeb881ce2f6d731e4526c3?d=mp&s=160"},"body":">  >\n>  > I'm saying it ought to be something like\n>  >\n>  > isDirty = ( wc.file.mtime > someValue ) && (sha1(repository.file) !=\n>  > sha1(wc.file) ) && ( repository.file != filter(wc.file) )\n>\n>  I don't think it is reasonable. Files inside of the repository and\n>  in the work are not meant to be the same. What if I have $Id$ expansion\n>  or something else. What could make sense is to add an additional check:\n>   && convert_to_work_tree(repository.file) != wc.file\n>  but it should be optional, so it will not penalize those who do need\n>  or do not want this extra check.\n>\n\nAh, yes - you're right (I was only thinking about check-in filters,\nnot check-out).\n\nI agree it ought to be optional; I suggest it ought to be turned on\n(be default) in the $Id$ expansion and the core.autocrlf=true\nscenarios (I.E when there's some filter in place).\n\n>\n>  > >  > ...\n>  Maybe, you can teach git-svn to be smarter... I mean storing text files\n>  in Git repo with CRLF is stupid, so, perhaps, git-svn can do a better\n>  job converting CRLF<->LF when it exports and imports from/to SVN.\n>\n\nYar - maybe there's some options there. Maybe it isn't so bad - all\nsvn projects probably *ought* to be using eol=native, but it isn't\ndefault; so maybe it's just easier to coax those projects into fixing\ntheir svn repos (but of course it's not really an issue for them, so\nit might be a bit of a hard sell).\n\nI may add some detail to the wiki docs to point this out - if I'd done\nit up front to our local projects, my life would be easier!\n\n>  ...\n>  You can use Git and have CRLF in your work tree. You just need to\n>  have autocrlf=true for that. _Inside_ of Git, only LF is the end\n>  of line. How you store in SVN, it is a separate issue with git-svn.\n>  I guess, git-svn needs improvement in this area...\n>\n\nYes, in the sense that git is primarily a *nix tool, so it treats LF\nas canon and CRLF as somehow 'stupid' (I.E you could make an equally\nvalid argument for the reverse position, it just depends on your\nperspective ;-)) ; but then again, it's only an issue because I'm now\nmerging in git *waaay* more often and it's uncovering a problem that\nmight actually be there already (modulo the fact that svn merging may\nignore line endings anyway - but I don't know because all merges there\nseem to inevitably end up in conflicts anyway..).\n"},{"id":"74656","messageId":"20080417185357.GO3133@dpotapov.dyndns.org","threadId":"13142","inReplyTo":"320075ff0804170309h4232463fk984f362e6cf0a259@mail.gmail.com","subject":"Re: crlf with git-svn driving me nuts...","fromName":"Dmitry Potapov","fromEmail":"dpotapov@gmail.com","sentAt":"2008-04-17T18:53:57Z","receivedAt":"2008-04-17T18:53:57Z","isPatch":false,"sender":{"key":"dpotapov@gmail.com","avatar":"https://avatars.githubusercontent.com/u/6568595?v=4"},"body":"On Thu, Apr 17, 2008 at 11:09:12AM +0100, Nigel Magnay wrote:\n>\n> Maybe it isn't so bad - all\n> svn projects probably *ought* to be using eol=native, but it isn't\n> default; \n\nIf you want to have native EOL for each platform then you have to do\nthis conversion, but it should be applied to only to text files.  So,\nthe question is how can a VCS know what file is text and what is not.\nCVS considers everything what you check-in as text by default. If you\nwant to put a binary file, you have to use -kb flag, otherwise your file\nmay be damaged. People tend to be forgetful and some lose their data in\nthis way. So team SVN team decided to stay on the safe side and put\neverything as is, because if you forget to set eol=native, you do not\nlose anything and you can set eol=native later. Unfortunately, now SVN\nusers forget to set eol=native a way too often. So, IMHO, Git approach\nbased on heuristic is much better when most of stored files are text.\n\n> so maybe it's just easier to coax those projects into fixing\n> their svn repos (but of course it's not really an issue for them, so\n> it might be a bit of a hard sell).\n\nIf they care about support different platforms then it _is_ issue\nfor them too. On the other hand, if everyone uses Windows with CRLF,\nyou can do that with Git too just by setting autocrlf=false.\n\n> \n> Yes, in the sense that git is primarily a *nix tool, so it treats LF\n> as canon\n\nand perhaps even more important, it is written in C and where LF has\nalways been considered as EOL since the first Hello-World program was\nwritten in C:\n\n   printf(\"Hello world!\\n\");\n-----------------------^^\n\nSo, naturally LF is considered as EOL inside of Git. Actually, CVS does\nso, and even SVN does if you set eol=native.\n\n> and CRLF as somehow 'stupid' (I.E you could make an equally\n> valid argument for the reverse position, it just depends on your\n> perspective ;-)) ;\n\nThere is no good technical reason to have two symbols as the end-of-line\nmarker instead of one. Most programs on Windows just remove CR when read\nfrom a file and then adding it back before LF when writing it back. So,\nCR is clearly redundant.\n\nDmitry\n"},{"id":"74685","messageId":"320075ff0804171503g6dc04aa5pb7c9c362e31ce942@mail.gmail.com","threadId":"13142","inReplyTo":"20080417185357.GO3133@dpotapov.dyndns.org","subject":"Re: crlf with git-svn driving me nuts...","fromName":"Nigel Magnay","fromEmail":"nigel.magnay@gmail.com","sentAt":"2008-04-17T22:03:10Z","receivedAt":"2008-04-17T22:03:10Z","isPatch":false,"sender":{"key":"nigel.magnay@gmail.com","avatar":"https://gravatar.com/avatar/d85cf38287bef3a8e4fa02358d2756d7589f8676c5eeb881ce2f6d731e4526c3?d=mp&s=160"},"body":">  lose anything and you can set eol=native later. Unfortunately, now SVN\n>  users forget to set eol=native a way too often. So, IMHO, Git approach\n>  based on heuristic is much better when most of stored files are text.\n>\n\nI agree - since the forgetful users includes us!\n\n>\n>  > so maybe it's just easier to coax those projects into fixing\n>  > their svn repos (but of course it's not really an issue for them, so\n>  > it might be a bit of a hard sell).\n>\n>  If they care about support different platforms then it _is_ issue\n>  for them too. On the other hand, if everyone uses Windows with CRLF,\n>  you can do that with Git too just by setting autocrlf=false.\n>\n\nActually it seems to be less of an everyday issue- but I think it's\nbecause the diff tools in use by programs downstream are probably\nstripping CRs anyway before presenting diffs, so it all 'appears' to\nbe right. Certainly I've been sharing via a svn repo through Eclipse\nwith windows users for ages without it being a problem. Either way,\nthe problem in touched/untouched-files was the majority of my\nconfusion as I wasn't expecting to find a bug and was assuming I was\ndoing something wrong...\n\n>\n>  >\n>  > Yes, in the sense that git is primarily a *nix tool, so it treats LF\n>  > as canon\n>\n>  and perhaps even more important, it is written in C and where LF has\n>  always been considered as EOL since the first Hello-World program was\n>  written in C:\n>\n>    printf(\"Hello world!\\n\");\n>  -----------------------^^\n>\n>  So, naturally LF is considered as EOL inside of Git. Actually, CVS does\n>  so, and even SVN does if you set eol=native.\n>\n>\n>  > and CRLF as somehow 'stupid' (I.E you could make an equally\n>  > valid argument for the reverse position, it just depends on your\n>  > perspective ;-)) ;\n>\n>  There is no good technical reason to have two symbols as the end-of-line\n>  marker instead of one. Most programs on Windows just remove CR when read\n>  from a file and then adding it back before LF when writing it back. So,\n>  CR is clearly redundant.\n>\n\nWell.... Newline = LF vs CRLF (vs CR for early mac.. erk) dates to\nwell before C and UNIX; back into the days of baudot codes and\nteletype printers that couldn't physically newline in the time taken\nfor 1 character to be processed; LF is meant to mean Line Feed and CR\nis meant to mean \"Carriage Return\", so CRLF is in that sense quite\nlogical. But that's standards committees and backwards compatibility\nfor you :-/\n"},{"id":"74689","messageId":"20080417224217.GQ3133@dpotapov.dyndns.org","threadId":"13142","inReplyTo":"320075ff0804171503g6dc04aa5pb7c9c362e31ce942@mail.gmail.com","subject":"Re: crlf with git-svn driving me nuts...","fromName":"Dmitry Potapov","fromEmail":"dpotapov@gmail.com","sentAt":"2008-04-17T22:42:17Z","receivedAt":"2008-04-17T22:42:17Z","isPatch":false,"sender":{"key":"dpotapov@gmail.com","avatar":"https://avatars.githubusercontent.com/u/6568595?v=4"},"body":"On Thu, Apr 17, 2008 at 11:03:10PM +0100, Nigel Magnay wrote:\n> \n> Well.... Newline = LF vs CRLF (vs CR for early mac.. erk) dates to\n> well before C and UNIX; back into the days of baudot codes and\n> teletype printers that couldn't physically newline in the time taken\n> for 1 character to be processed; LF is meant to mean Line Feed and CR\n> is meant to mean \"Carriage Return\", so CRLF is in that sense quite\n> logical. But that's standards committees and backwards compatibility\n> for you :-/\n\nCRLF is logical from the point of you of teletype printers, but when\nwe speak about text files then it is more logical to consider them as\na list of lines. What particular symbol is used as line-separator does\nnot really matter, but IMHO it is stupid to have two symbols for that.\nSo, LF vs CR is matter of preferences, but CRLF is just stupid -;)\n\nDmitry\n"}]}