{"thread":{"id":"52479","subject":"Ability to ignore EOL changes for certain projects","startedAt":"2019-12-18T11:10:41Z","lastAt":"2019-12-20T10:59:02Z","messageCount":9,"participants":["Scott Richmond","Torsten Bögershausen","Philip Oakley","brian m. carlson"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"388422","messageId":"CAB1T5w2GyfERoaCyFZeKaui_xuXd0r6J+Zvq4pecstBJ9UyRzw@mail.gmail.com","threadId":"52479","inReplyTo":null,"subject":"Ability to ignore EOL changes for certain projects","fromName":"Scott Richmond","fromEmail":"scott@brightrockgames.com","sentAt":"2019-12-18T11:10:27Z","receivedAt":"2019-12-18T11:10:41Z","isPatch":false,"sender":{"key":"scott@brightrockgames.com","avatar":null},"body":"The Problem Domain\nIn certain dev environments (Unity3D projects) there is (AFAIK) an\nunsolvable problem where some files are often modified with line\nendings that aren't the native system or not the committed line\nendings for that file. Secondarily, in this case line endings don't\nmatter - Nothing in the dev environment \"cares\" which kind of line\nending is used.\n\nThe Problem\nGit always cares about EOL. Git has options to transparently modify\nEOLs when files are checked in or out. However it is not possible to\ntell Git to ignore EOLs in other commands:\nGit status shows the file modified.\nMerging/Pulling has to care because it can't merge with a modified\nworking tree. Which means the user has to care - They have to either\nstash the EOL changes or wipe them out. Sometimes, if the user has a\nparticular app running, it may automatically reload that file and\nrecreate the modified EOL changes, causing an endless loop. This\nproblem is often made unclear to the user how to solve, especially if\nthey aren't domain experts.\n\nTo be clear, in this particular dev environment, I can say with\nconfidence that this particular issue is a major and common pain point\nfor users. It is made worse as many users in this environment aren't\nprogrammers by trade and aren't domain experts in version control. I\nalso believe this environment is becoming a non-trivial portion of the\nGit userbase and it would be worthwhile looking into resolving.\n\nSolution Request\nIt would be fantastic if we could tell Git to stop caring about EOL\nchanges on a per-repo basis, with the effective output being that git\nstatus shouldn't show changes for files with differing EOLs.\n\nI'm experienced with Git, though I am not expert enough to consider\ncreating such a change myself - It is unclear to me just how\ncomplicated a change may be. However maybe I could look into it if it\nwas made clear that this improvement is possible and has no serious\nside effects.\n\nRegards,\n\nScott Richmond.\n  Director, Producer, Programmer\n  Brightrock Games\n  T: 07480795661\n"},{"id":"388457","messageId":"20191218192741.ybi3xpvax7lrrubz@tb-raspi4","threadId":"52479","inReplyTo":"CAB1T5w2GyfERoaCyFZeKaui_xuXd0r6J+Zvq4pecstBJ9UyRzw@mail.gmail.com","subject":"Re: Ability to ignore EOL changes for certain projects","fromName":"Torsten Bögershausen","fromEmail":"tboegi@web.de","sentAt":"2019-12-18T19:27:41Z","receivedAt":"2019-12-18T19:27:45Z","isPatch":false,"sender":{"key":"tboegi@web.de","avatar":"https://avatars.githubusercontent.com/u/7138363?v=4"},"body":"On Wed, Dec 18, 2019 at 11:10:27AM +0000, Scott Richmond wrote:\n> The Problem Domain\n> In certain dev environments (Unity3D projects) there is (AFAIK) an\n> unsolvable problem where some files are often modified with line\n> endings that aren't the native system or not the committed line\n> endings for that file. Secondarily, in this case line endings don't\n> matter - Nothing in the dev environment \"cares\" which kind of line\n> ending is used.\n>\n> The Problem\n> Git always cares about EOL. Git has options to transparently modify\n> EOLs when files are checked in or out. However it is not possible to\n> tell Git to ignore EOLs in other commands:\n> Git status shows the file modified.\n> Merging/Pulling has to care because it can't merge with a modified\n> working tree. Which means the user has to care - They have to either\n> stash the EOL changes or wipe them out. Sometimes, if the user has a\n> particular app running, it may automatically reload that file and\n> recreate the modified EOL changes, causing an endless loop. This\n> problem is often made unclear to the user how to solve, especially if\n> they aren't domain experts.\n>\n> To be clear, in this particular dev environment, I can say with\n> confidence that this particular issue is a major and common pain point\n> for users. It is made worse as many users in this environment aren't\n> programmers by trade and aren't domain experts in version control. I\n> also believe this environment is becoming a non-trivial portion of the\n> Git userbase and it would be worthwhile looking into resolving.\n>\n> Solution Request\n> It would be fantastic if we could tell Git to stop caring about EOL\n> changes on a per-repo basis, with the effective output being that git\n> status shouldn't show changes for files with differing EOLs.\n>\n> I'm experienced with Git, though I am not expert enough to consider\n> creating such a change myself - It is unclear to me just how\n> complicated a change may be. However maybe I could look into it if it\n> was made clear that this improvement is possible and has no serious\n> side effects.\n\nHej Scott,\nI think that you problem can be solved.\nFor each repository, you can tell Git to ignore the line endings,\nCRLF vs LF.\n\nIf you start with a fresh repo,\nyou can do something like:\n\necho \"* text=auto\" >.gitattributes\ngit add .gitattributes\ngit commit -m \"Add .gitattributes\"\n\nFor existing repos, we need to take another step:\n\necho \"* text=auto\" >.gitattributes\ngit add .gitattributes\ngit add  --renormlize .\ngit commit -m \"Add .gitattributes\"\n\nMore information can be found e.g. here:\nhttps://git-scm.com/docs/git-add\n\nOnce you done that, you can merge branches\ninto the master branch with help of the renormalize\n\ngit merge -X renormalze branch-name\n\nSee even here:\nhttps://git-scm.com/docs/git-merge\n\n\nThis is just a (too) short introduction, I hope that it\nhelps and that you find the time to dig somewhat deeper into\nthe documentation.\n\nOther developers have that problem as well, you are not alone.\n\nIf you have a public repo, I could help with one example.\n\n\n>\n> Regards,\n>\n> Scott Richmond.\n>   Director, Producer, Programmer\n>   Brightrock Games\n>   T: 07480795661\n"},{"id":"388476","messageId":"CAB1T5w1Ct7_D7kiUypRuoK+zeiocyPJn0SindXfs6M5wUkVavw@mail.gmail.com","threadId":"52479","inReplyTo":"20191218192741.ybi3xpvax7lrrubz@tb-raspi4","subject":"Re: Ability to ignore EOL changes for certain projects","fromName":"Scott Richmond","fromEmail":"scott@brightrockgames.com","sentAt":"2019-12-18T21:33:32Z","receivedAt":"2019-12-18T21:33:45Z","isPatch":false,"sender":{"key":"scott@brightrockgames.com","avatar":null},"body":"Hey Torsten,\n\nThanks for the reply!\n\nCorrect me if am wrong, but those steps don't tell git to \"ignore\"\nline endings. That just causes git to check all text files in and out\nwith a specific EOL type (Either automatically chosen, or not). If an\napp in the dev env changes a files' EOL to something else, git will\nnotice the change locally.\n\nRegards,\n\nScott Richmond.\n  Director, Producer, Programmer\n  Brightrock Games\n  T: 07480795661\n\nOn Wed, Dec 18, 2019 at 7:27 PM Torsten Bögershausen <tboegi@web.de> wrote:\n>\n> On Wed, Dec 18, 2019 at 11:10:27AM +0000, Scott Richmond wrote:\n> > The Problem Domain\n> > In certain dev environments (Unity3D projects) there is (AFAIK) an\n> > unsolvable problem where some files are often modified with line\n> > endings that aren't the native system or not the committed line\n> > endings for that file. Secondarily, in this case line endings don't\n> > matter - Nothing in the dev environment \"cares\" which kind of line\n> > ending is used.\n> >\n> > The Problem\n> > Git always cares about EOL. Git has options to transparently modify\n> > EOLs when files are checked in or out. However it is not possible to\n> > tell Git to ignore EOLs in other commands:\n> > Git status shows the file modified.\n> > Merging/Pulling has to care because it can't merge with a modified\n> > working tree. Which means the user has to care - They have to either\n> > stash the EOL changes or wipe them out. Sometimes, if the user has a\n> > particular app running, it may automatically reload that file and\n> > recreate the modified EOL changes, causing an endless loop. This\n> > problem is often made unclear to the user how to solve, especially if\n> > they aren't domain experts.\n> >\n> > To be clear, in this particular dev environment, I can say with\n> > confidence that this particular issue is a major and common pain point\n> > for users. It is made worse as many users in this environment aren't\n> > programmers by trade and aren't domain experts in version control. I\n> > also believe this environment is becoming a non-trivial portion of the\n> > Git userbase and it would be worthwhile looking into resolving.\n> >\n> > Solution Request\n> > It would be fantastic if we could tell Git to stop caring about EOL\n> > changes on a per-repo basis, with the effective output being that git\n> > status shouldn't show changes for files with differing EOLs.\n> >\n> > I'm experienced with Git, though I am not expert enough to consider\n> > creating such a change myself - It is unclear to me just how\n> > complicated a change may be. However maybe I could look into it if it\n> > was made clear that this improvement is possible and has no serious\n> > side effects.\n>\n> Hej Scott,\n> I think that you problem can be solved.\n> For each repository, you can tell Git to ignore the line endings,\n> CRLF vs LF.\n>\n> If you start with a fresh repo,\n> you can do something like:\n>\n> echo \"* text=auto\" >.gitattributes\n> git add .gitattributes\n> git commit -m \"Add .gitattributes\"\n>\n> For existing repos, we need to take another step:\n>\n> echo \"* text=auto\" >.gitattributes\n> git add .gitattributes\n> git add  --renormlize .\n> git commit -m \"Add .gitattributes\"\n>\n> More information can be found e.g. here:\n> https://git-scm.com/docs/git-add\n>\n> Once you done that, you can merge branches\n> into the master branch with help of the renormalize\n>\n> git merge -X renormalze branch-name\n>\n> See even here:\n> https://git-scm.com/docs/git-merge\n>\n>\n> This is just a (too) short introduction, I hope that it\n> helps and that you find the time to dig somewhat deeper into\n> the documentation.\n>\n> Other developers have that problem as well, you are not alone.\n>\n> If you have a public repo, I could help with one example.\n>\n>\n> >\n> > Regards,\n> >\n> > Scott Richmond.\n> >   Director, Producer, Programmer\n> >   Brightrock Games\n> >   T: 07480795661\n"},{"id":"388483","messageId":"b4338168-50c5-8aa1-32e0-2fe96e82e9bf@iee.email","threadId":"52479","inReplyTo":"CAB1T5w1Ct7_D7kiUypRuoK+zeiocyPJn0SindXfs6M5wUkVavw@mail.gmail.com","subject":"Re: Ability to ignore EOL changes for certain projects","fromName":"Philip Oakley","fromEmail":"philipoakley@iee.email","sentAt":"2019-12-18T22:34:59Z","receivedAt":"2019-12-18T22:35:05Z","isPatch":false,"sender":{"key":"philipoakley@iee.email","avatar":"https://avatars.githubusercontent.com/u/914343?v=4"},"body":"On 18/12/2019 21:33, Scott Richmond wrote:\n> Hey Torsten,\n>\n> Thanks for the reply!\n>\n> Correct me if am wrong, but those steps don't tell git to \"ignore\"\n> line endings. That just causes git to check all text files in and out\n> with a specific EOL type (Either automatically chosen, or not). If an\n> app in the dev env changes a files' EOL to something else, git will\n> notice the change locally.\n\nA broader question is \"how does the dev environment handle lone `CR`\ncharacters? are they also considered as EOLs?\"\n\nIf I recall previous discussions, part of the issue is determining what\nthe alternate abuse cases are, such as lone CRs, or alternate character\nsets to the 'pure' utf-8.\n\nYou will still have the difficulty of how `identicality` is determined,\nwhich currently uses the sha1/oid value. In essence you need a way of\nensuring that all checkins are always of one defined EOL, but then need\ngit to have a broader allowance of changed EOL values (including no lone\nCRs that are not EOL, etc).\n\nDoes the environment (on any of the OS's) change the files in the\nbackground, such that the file time stamps will indicate a modification?\n\nPhilip\n\n> Regards,\n>\n> Scott Richmond.\n>   Director, Producer, Programmer\n>   Brightrock Games\n>   T: 07480795661\n>\n> On Wed, Dec 18, 2019 at 7:27 PM Torsten Bögershausen <tboegi@web.de> wrote:\n>> On Wed, Dec 18, 2019 at 11:10:27AM +0000, Scott Richmond wrote:\n>>> The Problem Domain\n>>> In certain dev environments (Unity3D projects) there is (AFAIK) an\n>>> unsolvable problem where some files are often modified with line\n>>> endings that aren't the native system or not the committed line\n>>> endings for that file. Secondarily, in this case line endings don't\n>>> matter - Nothing in the dev environment \"cares\" which kind of line\n>>> ending is used.\n>>>\n>>> The Problem\n>>> Git always cares about EOL. Git has options to transparently modify\n>>> EOLs when files are checked in or out. However it is not possible to\n>>> tell Git to ignore EOLs in other commands:\n>>> Git status shows the file modified.\n>>> Merging/Pulling has to care because it can't merge with a modified\n>>> working tree. Which means the user has to care - They have to either\n>>> stash the EOL changes or wipe them out. Sometimes, if the user has a\n>>> particular app running, it may automatically reload that file and\n>>> recreate the modified EOL changes, causing an endless loop. This\n>>> problem is often made unclear to the user how to solve, especially if\n>>> they aren't domain experts.\n>>>\n>>> To be clear, in this particular dev environment, I can say with\n>>> confidence that this particular issue is a major and common pain point\n>>> for users. It is made worse as many users in this environment aren't\n>>> programmers by trade and aren't domain experts in version control. I\n>>> also believe this environment is becoming a non-trivial portion of the\n>>> Git userbase and it would be worthwhile looking into resolving.\n>>>\n>>> Solution Request\n>>> It would be fantastic if we could tell Git to stop caring about EOL\n>>> changes on a per-repo basis, with the effective output being that git\n>>> status shouldn't show changes for files with differing EOLs.\n>>>\n>>> I'm experienced with Git, though I am not expert enough to consider\n>>> creating such a change myself - It is unclear to me just how\n>>> complicated a change may be. However maybe I could look into it if it\n>>> was made clear that this improvement is possible and has no serious\n>>> side effects.\n>> Hej Scott,\n>> I think that you problem can be solved.\n>> For each repository, you can tell Git to ignore the line endings,\n>> CRLF vs LF.\n>>\n>> If you start with a fresh repo,\n>> you can do something like:\n>>\n>> echo \"* text=auto\" >.gitattributes\n>> git add .gitattributes\n>> git commit -m \"Add .gitattributes\"\n>>\n>> For existing repos, we need to take another step:\n>>\n>> echo \"* text=auto\" >.gitattributes\n>> git add .gitattributes\n>> git add  --renormlize .\n>> git commit -m \"Add .gitattributes\"\n>>\n>> More information can be found e.g. here:\n>> https://git-scm.com/docs/git-add\n>>\n>> Once you done that, you can merge branches\n>> into the master branch with help of the renormalize\n>>\n>> git merge -X renormalze branch-name\n>>\n>> See even here:\n>> https://git-scm.com/docs/git-merge\n>>\n>>\n>> This is just a (too) short introduction, I hope that it\n>> helps and that you find the time to dig somewhat deeper into\n>> the documentation.\n>>\n>> Other developers have that problem as well, you are not alone.\n>>\n>> If you have a public repo, I could help with one example.\n>>\n>>\n>>> Regards,\n>>>\n>>> Scott Richmond.\n>>>   Director, Producer, Programmer\n>>>   Brightrock Games\n>>>   T: 07480795661\n\n"},{"id":"388500","messageId":"20191219031227.xqlv564h3iq5ofq7@tb-raspi4","threadId":"52479","inReplyTo":"CAB1T5w1Ct7_D7kiUypRuoK+zeiocyPJn0SindXfs6M5wUkVavw@mail.gmail.com","subject":"Re: Ability to ignore EOL changes for certain projects","fromName":"Torsten Bögershausen","fromEmail":"tboegi@web.de","sentAt":"2019-12-19T03:12:28Z","receivedAt":"2019-12-19T03:12:33Z","isPatch":false,"sender":{"key":"tboegi@web.de","avatar":"https://avatars.githubusercontent.com/u/7138363?v=4"},"body":"Scott, please avoid top-posting in this list, see my answers at the end.\n\n> On Wed, Dec 18, 2019 at 7:27 PM Torsten Bögershausen <tboegi@web.de> wrote:\n> >\n> > On Wed, Dec 18, 2019 at 11:10:27AM +0000, Scott Richmond wrote:\n> > > The Problem Domain\n> > > In certain dev environments (Unity3D projects) there is (AFAIK) an\n> > > unsolvable problem where some files are often modified with line\n> > > endings that aren't the native system or not the committed line\n> > > endings for that file. Secondarily, in this case line endings don't\n> > > matter - Nothing in the dev environment \"cares\" which kind of line\n> > > ending is used.\n> > >\n> > > The Problem\n> > > Git always cares about EOL. Git has options to transparently modify\n> > > EOLs when files are checked in or out. However it is not possible to\n> > > tell Git to ignore EOLs in other commands:\n> > > Git status shows the file modified.\n> > > Merging/Pulling has to care because it can't merge with a modified\n> > > working tree. Which means the user has to care - They have to either\n> > > stash the EOL changes or wipe them out. Sometimes, if the user has a\n> > > particular app running, it may automatically reload that file and\n> > > recreate the modified EOL changes, causing an endless loop. This\n> > > problem is often made unclear to the user how to solve, especially if\n> > > they aren't domain experts.\n> > >\n> > > To be clear, in this particular dev environment, I can say with\n> > > confidence that this particular issue is a major and common pain point\n> > > for users. It is made worse as many users in this environment aren't\n> > > programmers by trade and aren't domain experts in version control. I\n> > > also believe this environment is becoming a non-trivial portion of the\n> > > Git userbase and it would be worthwhile looking into resolving.\n> > >\n> > > Solution Request\n> > > It would be fantastic if we could tell Git to stop caring about EOL\n> > > changes on a per-repo basis, with the effective output being that git\n> > > status shouldn't show changes for files with differing EOLs.\n> > >\n> > > I'm experienced with Git, though I am not expert enough to consider\n> > > creating such a change myself - It is unclear to me just how\n> > > complicated a change may be. However maybe I could look into it if it\n> > > was made clear that this improvement is possible and has no serious\n> > > side effects.\n> >\n> > Hej Scott,\n> > I think that you problem can be solved.\n> > For each repository, you can tell Git to ignore the line endings,\n> > CRLF vs LF.\n> >\n> > If you start with a fresh repo,\n> > you can do something like:\n> >\n> > echo \"* text=auto\" >.gitattributes\n> > git add .gitattributes\n> > git commit -m \"Add .gitattributes\"\n> >\n> > For existing repos, we need to take another step:\n> >\n> > echo \"* text=auto\" >.gitattributes\n> > git add .gitattributes\n> > git add  --renormlize .\n> > git commit -m \"Add .gitattributes\"\n> >\n> > More information can be found e.g. here:\n> > https://git-scm.com/docs/git-add\n> >\n> > Once you done that, you can merge branches\n> > into the master branch with help of the renormalize\n> >\n> > git merge -X renormalze branch-name\n> >\n> > See even here:\n> > https://git-scm.com/docs/git-merge\n> >\n> >\n> > This is just a (too) short introduction, I hope that it\n> > helps and that you find the time to dig somewhat deeper into\n> > the documentation.\n> >\n> > Other developers have that problem as well, you are not alone.\n> >\n> > If you have a public repo, I could help with one example.\n> >\n> >\n> > >\n> > > Regards,\n> > >\n> > > Scott Richmond.\n> > >   Director, Producer, Programmer\n> > >   Brightrock Games\n> > >   T: 07480795661\nOn Wed, Dec 18, 2019 at 09:33:32PM +0000, Scott Richmond wrote:\n> Hey Torsten,\n>\n> Thanks for the reply!\n>\n> Correct me if am wrong, but those steps don't tell git to \"ignore\"\n> line endings. That just causes git to check all text files in and out\n> with a specific EOL type (Either automatically chosen, or not). If an\n> app in the dev env changes a files' EOL to something else, git will\n> notice the change locally.\n>\n> Regards,\n>\n> Scott Richmond.\n>   Director, Producer, Programmer\n>   Brightrock Games\n>   T: 07480795661\n>\n\nHej Scott,\n\nI am not sure whether I understand your question correctly.\nSo I set up a little test, to illustrate things better.\n\n\nuser@linux:/tmp/EOLtest $ git init\nInitialized empty Git repository in /tmp/EOLtest/.git/\nuser@linux:/tmp/EOLtest $ printf \"Line1\\r\\nLine2\\r\\n\" >file\nuser@linux:/tmp/EOLtest $ git add file\nuser@linux:/tmp/EOLtest $ git commit -m \"add file with CRLF\"\n[master (root-commit) 0e4d1df] add file with CRLF\n 1 file changed, 2 insertions(+)\n  create mode 100644 file\n\nuser@linux:/tmp/EOLtest $ printf \"Line1\\nLine2\\n\" >file\nuser@linux:/tmp/EOLtest $ git status\nOn branch master\nChanges not staged for commit:\n  (use \"git add <file>...\" to update what will be committed)\n    (use \"git checkout -- <file>...\" to discard changes in working directory)\n\n        modified:   file\n\nno changes added to commit (use \"git add\" and/or \"git commit -a\")\nuser@linux:/tmp/EOLtest $ git diff\ndiff --git a/file b/file\nindex 4aa551d..ac2dd81 100644\n--- a/file\n+++ b/file\n@@ -1,2 +1,2 @@\n-Line1\n-Line2\n+Line1\n+Line2\n\nuser@linux:/tmp/EOLtest $ echo \"* text=auto\" >.gitattributes\nuser@linux:/tmp/EOLtest $ git add --renormalize .\nuser@linux:/tmp/EOLtest $ git commit -m \"Normalize the repo\"\n[master b41136d] Normalize the repo\n 1 file changed, 2 insertions(+), 2 deletions(-)\n user@linux:/tmp/EOLtest $\n\nuser@linux:/tmp/EOLtest $ printf \"Line1\\r\\nLine2\\r\\n\" >file\nuser@linux:/tmp/EOLtest $ git status\nOn branch master\nChanges not staged for commit:\n  (use \"git add <file>...\" to update what will be committed)\n    (use \"git checkout -- <file>...\" to discard changes in working directory)\n\n        modified:   file\n\nuser@linux:/tmp/EOLtest $ git diff\nwarning: CRLF will be replaced by LF in file.\nThe file will have its original line endings in your working directory\n\n#############################\nNo, at this point, I am surprised myself.\n\"file\" is reported as \"modified\", but it should not be modified, right?\nIs that the problem, that you have ?\n\n\nWe can fix it, by running:\n\nuser@linux:/tmp/EOLtest $ git add file\nuser@linux:/tmp/EOLtest $ git status\nOn branch master\nnothing to commit, working tree clean\n\nBut I still think, that this \"modified\" is unexpected,\nand a (possible) bug in Git, thanks for reporting.\n\nWhich Git version do you use ?\nIs this test script a description of you problem ?\n\n"},{"id":"388511","messageId":"CAB1T5w3k6RcsyaTFZ77aXrGp19fA2oFkvJU_xaJ3JekJK_aNqw@mail.gmail.com","threadId":"52479","inReplyTo":"20191219031227.xqlv564h3iq5ofq7@tb-raspi4","subject":"Re: Ability to ignore EOL changes for certain projects","fromName":"Scott Richmond","fromEmail":"scott@brightrockgames.com","sentAt":"2019-12-19T13:25:27Z","receivedAt":"2019-12-19T13:25:43Z","isPatch":false,"sender":{"key":"scott@brightrockgames.com","avatar":null},"body":"> Scott, please avoid top-posting in this list, see my answers at the end.\nSure! Sorry I rarely use mail lists.\n\n> But I still think, that this \"modified\" is unexpected,\n> and a (possible) bug in Git, thanks for reporting.\n>\n> Which Git version do you use ?\n> Is this test script a description of you problem ?\n\nYes this appears to be a correct repro of the kind of scenario that is\ncommon in many dev environments like ours.\nI'm not yet sure if this is actually a bug (I'd be happy if it is!).\nReading the docs I think this is intended behaviour - All the text\nCRLF options relate to what Git should do on check in/out. It is\ncorrectly reporting the file change, even if it is just EOL.\nWhat Philip is describing with the sha1/iod check sounds to me what\nthe root cause \"problem\" is - Any file change will show up on git.\nPhilip appears to imply that there is no way to get git to do further\nintrospection on the file at this time. But maybe that can be added?\nRe the Git version - I try to stay on the latest at all times. This\nissue has been present for years, for as long as I can remember.\n\nRegards,\n\nScott Richmond.\n  Director, Producer, Programmer\n  Brightrock Games\n  T: 07480795661\n\nOn Thu, Dec 19, 2019 at 3:12 AM Torsten Bögershausen <tboegi@web.de> wrote:\n>\n> Scott, please avoid top-posting in this list, see my answers at the end.\n>\n> > On Wed, Dec 18, 2019 at 7:27 PM Torsten Bögershausen <tboegi@web.de> wrote:\n> > >\n> > > On Wed, Dec 18, 2019 at 11:10:27AM +0000, Scott Richmond wrote:\n> > > > The Problem Domain\n> > > > In certain dev environments (Unity3D projects) there is (AFAIK) an\n> > > > unsolvable problem where some files are often modified with line\n> > > > endings that aren't the native system or not the committed line\n> > > > endings for that file. Secondarily, in this case line endings don't\n> > > > matter - Nothing in the dev environment \"cares\" which kind of line\n> > > > ending is used.\n> > > >\n> > > > The Problem\n> > > > Git always cares about EOL. Git has options to transparently modify\n> > > > EOLs when files are checked in or out. However it is not possible to\n> > > > tell Git to ignore EOLs in other commands:\n> > > > Git status shows the file modified.\n> > > > Merging/Pulling has to care because it can't merge with a modified\n> > > > working tree. Which means the user has to care - They have to either\n> > > > stash the EOL changes or wipe them out. Sometimes, if the user has a\n> > > > particular app running, it may automatically reload that file and\n> > > > recreate the modified EOL changes, causing an endless loop. This\n> > > > problem is often made unclear to the user how to solve, especially if\n> > > > they aren't domain experts.\n> > > >\n> > > > To be clear, in this particular dev environment, I can say with\n> > > > confidence that this particular issue is a major and common pain point\n> > > > for users. It is made worse as many users in this environment aren't\n> > > > programmers by trade and aren't domain experts in version control. I\n> > > > also believe this environment is becoming a non-trivial portion of the\n> > > > Git userbase and it would be worthwhile looking into resolving.\n> > > >\n> > > > Solution Request\n> > > > It would be fantastic if we could tell Git to stop caring about EOL\n> > > > changes on a per-repo basis, with the effective output being that git\n> > > > status shouldn't show changes for files with differing EOLs.\n> > > >\n> > > > I'm experienced with Git, though I am not expert enough to consider\n> > > > creating such a change myself - It is unclear to me just how\n> > > > complicated a change may be. However maybe I could look into it if it\n> > > > was made clear that this improvement is possible and has no serious\n> > > > side effects.\n> > >\n> > > Hej Scott,\n> > > I think that you problem can be solved.\n> > > For each repository, you can tell Git to ignore the line endings,\n> > > CRLF vs LF.\n> > >\n> > > If you start with a fresh repo,\n> > > you can do something like:\n> > >\n> > > echo \"* text=auto\" >.gitattributes\n> > > git add .gitattributes\n> > > git commit -m \"Add .gitattributes\"\n> > >\n> > > For existing repos, we need to take another step:\n> > >\n> > > echo \"* text=auto\" >.gitattributes\n> > > git add .gitattributes\n> > > git add  --renormlize .\n> > > git commit -m \"Add .gitattributes\"\n> > >\n> > > More information can be found e.g. here:\n> > > https://git-scm.com/docs/git-add\n> > >\n> > > Once you done that, you can merge branches\n> > > into the master branch with help of the renormalize\n> > >\n> > > git merge -X renormalze branch-name\n> > >\n> > > See even here:\n> > > https://git-scm.com/docs/git-merge\n> > >\n> > >\n> > > This is just a (too) short introduction, I hope that it\n> > > helps and that you find the time to dig somewhat deeper into\n> > > the documentation.\n> > >\n> > > Other developers have that problem as well, you are not alone.\n> > >\n> > > If you have a public repo, I could help with one example.\n> > >\n> > >\n> > > >\n> > > > Regards,\n> > > >\n> > > > Scott Richmond.\n> > > >   Director, Producer, Programmer\n> > > >   Brightrock Games\n> > > >   T: 07480795661\n> On Wed, Dec 18, 2019 at 09:33:32PM +0000, Scott Richmond wrote:\n> > Hey Torsten,\n> >\n> > Thanks for the reply!\n> >\n> > Correct me if am wrong, but those steps don't tell git to \"ignore\"\n> > line endings. That just causes git to check all text files in and out\n> > with a specific EOL type (Either automatically chosen, or not). If an\n> > app in the dev env changes a files' EOL to something else, git will\n> > notice the change locally.\n> >\n> > Regards,\n> >\n> > Scott Richmond.\n> >   Director, Producer, Programmer\n> >   Brightrock Games\n> >   T: 07480795661\n> >\n>\n> Hej Scott,\n>\n> I am not sure whether I understand your question correctly.\n> So I set up a little test, to illustrate things better.\n>\n>\n> user@linux:/tmp/EOLtest $ git init\n> Initialized empty Git repository in /tmp/EOLtest/.git/\n> user@linux:/tmp/EOLtest $ printf \"Line1\\r\\nLine2\\r\\n\" >file\n> user@linux:/tmp/EOLtest $ git add file\n> user@linux:/tmp/EOLtest $ git commit -m \"add file with CRLF\"\n> [master (root-commit) 0e4d1df] add file with CRLF\n>  1 file changed, 2 insertions(+)\n>   create mode 100644 file\n>\n> user@linux:/tmp/EOLtest $ printf \"Line1\\nLine2\\n\" >file\n> user@linux:/tmp/EOLtest $ git status\n> On branch master\n> Changes not staged for commit:\n>   (use \"git add <file>...\" to update what will be committed)\n>     (use \"git checkout -- <file>...\" to discard changes in working directory)\n>\n>         modified:   file\n>\n> no changes added to commit (use \"git add\" and/or \"git commit -a\")\n> user@linux:/tmp/EOLtest $ git diff\n> diff --git a/file b/file\n> index 4aa551d..ac2dd81 100644\n> --- a/file\n> +++ b/file\n> @@ -1,2 +1,2 @@\n> -Line1\n> -Line2\n> +Line1\n> +Line2\n>\n> user@linux:/tmp/EOLtest $ echo \"* text=auto\" >.gitattributes\n> user@linux:/tmp/EOLtest $ git add --renormalize .\n> user@linux:/tmp/EOLtest $ git commit -m \"Normalize the repo\"\n> [master b41136d] Normalize the repo\n>  1 file changed, 2 insertions(+), 2 deletions(-)\n>  user@linux:/tmp/EOLtest $\n>\n> user@linux:/tmp/EOLtest $ printf \"Line1\\r\\nLine2\\r\\n\" >file\n> user@linux:/tmp/EOLtest $ git status\n> On branch master\n> Changes not staged for commit:\n>   (use \"git add <file>...\" to update what will be committed)\n>     (use \"git checkout -- <file>...\" to discard changes in working directory)\n>\n>         modified:   file\n>\n> user@linux:/tmp/EOLtest $ git diff\n> warning: CRLF will be replaced by LF in file.\n> The file will have its original line endings in your working directory\n>\n> #############################\n> No, at this point, I am surprised myself.\n> \"file\" is reported as \"modified\", but it should not be modified, right?\n> Is that the problem, that you have ?\n>\n>\n> We can fix it, by running:\n>\n> user@linux:/tmp/EOLtest $ git add file\n> user@linux:/tmp/EOLtest $ git status\n> On branch master\n> nothing to commit, working tree clean\n>\n> But I still think, that this \"modified\" is unexpected,\n> and a (possible) bug in Git, thanks for reporting.\n>\n> Which Git version do you use ?\n> Is this test script a description of you problem ?\n>\n"},{"id":"388512","messageId":"CAB1T5w3ZeLDQcvxuR_j0q3+=V7Bfqh3px01ys_x2f6EYVrnuzA@mail.gmail.com","threadId":"52479","inReplyTo":"b4338168-50c5-8aa1-32e0-2fe96e82e9bf@iee.email","subject":"Re: Ability to ignore EOL changes for certain projects","fromName":"Scott Richmond","fromEmail":"scott@brightrockgames.com","sentAt":"2019-12-19T13:39:40Z","receivedAt":"2019-12-19T13:39:56Z","isPatch":false,"sender":{"key":"scott@brightrockgames.com","avatar":null},"body":"> If I recall previous discussions, part of the issue is determining what\nthe alternate abuse cases are, such as lone CRs, or alternate character\nsets to the 'pure' utf-8.\nIn this dev env, I would be surprised if any CR type makes a\ndifference. But I could be wrong.\nIf that is indeed one of the reasons why previous discussions broke\ndown, maybe that could be a configurable option in the feature\nimprovement.\n\n> Does the environment (on any of the OS's) change the files in the background, such that the file time stamps will indicate a modification?\nI would imagine the file timestamps do get updated yes. Though I'd\nhave to do some research to confirm it.\n\nIf indeed this isn't a bug (As Torsten suspects), then I suppose what\nI am asking for is a feature improvement to perform a further check\nafter sha1/iod to confirm if the changes are only between CR types?\n\nRegards,\n\nScott Richmond.\n  Director, Producer, Programmer\n  Brightrock Games\n  T: 07480795661\nOn Wed, Dec 18, 2019 at 10:35 PM Philip Oakley <philipoakley@iee.email> wrote:\n>\n> On 18/12/2019 21:33, Scott Richmond wrote:\n> > Hey Torsten,\n> >\n> > Thanks for the reply!\n> >\n> > Correct me if am wrong, but those steps don't tell git to \"ignore\"\n> > line endings. That just causes git to check all text files in and out\n> > with a specific EOL type (Either automatically chosen, or not). If an\n> > app in the dev env changes a files' EOL to something else, git will\n> > notice the change locally.\n>\n> A broader question is \"how does the dev environment handle lone `CR`\n> characters? are they also considered as EOLs?\"\n>\n> If I recall previous discussions, part of the issue is determining what\n> the alternate abuse cases are, such as lone CRs, or alternate character\n> sets to the 'pure' utf-8.\n>\n> You will still have the difficulty of how `identicality` is determined,\n> which currently uses the sha1/oid value. In essence you need a way of\n> ensuring that all checkins are always of one defined EOL, but then need\n> git to have a broader allowance of changed EOL values (including no lone\n> CRs that are not EOL, etc).\n>\n> Does the environment (on any of the OS's) change the files in the\n> background, such that the file time stamps will indicate a modification?\n>\n> Philip\n>\n> > Regards,\n> >\n> > Scott Richmond.\n> >   Director, Producer, Programmer\n> >   Brightrock Games\n> >   T: 07480795661\n> >\n> > On Wed, Dec 18, 2019 at 7:27 PM Torsten Bögershausen <tboegi@web.de> wrote:\n> >> On Wed, Dec 18, 2019 at 11:10:27AM +0000, Scott Richmond wrote:\n> >>> The Problem Domain\n> >>> In certain dev environments (Unity3D projects) there is (AFAIK) an\n> >>> unsolvable problem where some files are often modified with line\n> >>> endings that aren't the native system or not the committed line\n> >>> endings for that file. Secondarily, in this case line endings don't\n> >>> matter - Nothing in the dev environment \"cares\" which kind of line\n> >>> ending is used.\n> >>>\n> >>> The Problem\n> >>> Git always cares about EOL. Git has options to transparently modify\n> >>> EOLs when files are checked in or out. However it is not possible to\n> >>> tell Git to ignore EOLs in other commands:\n> >>> Git status shows the file modified.\n> >>> Merging/Pulling has to care because it can't merge with a modified\n> >>> working tree. Which means the user has to care - They have to either\n> >>> stash the EOL changes or wipe them out. Sometimes, if the user has a\n> >>> particular app running, it may automatically reload that file and\n> >>> recreate the modified EOL changes, causing an endless loop. This\n> >>> problem is often made unclear to the user how to solve, especially if\n> >>> they aren't domain experts.\n> >>>\n> >>> To be clear, in this particular dev environment, I can say with\n> >>> confidence that this particular issue is a major and common pain point\n> >>> for users. It is made worse as many users in this environment aren't\n> >>> programmers by trade and aren't domain experts in version control. I\n> >>> also believe this environment is becoming a non-trivial portion of the\n> >>> Git userbase and it would be worthwhile looking into resolving.\n> >>>\n> >>> Solution Request\n> >>> It would be fantastic if we could tell Git to stop caring about EOL\n> >>> changes on a per-repo basis, with the effective output being that git\n> >>> status shouldn't show changes for files with differing EOLs.\n> >>>\n> >>> I'm experienced with Git, though I am not expert enough to consider\n> >>> creating such a change myself - It is unclear to me just how\n> >>> complicated a change may be. However maybe I could look into it if it\n> >>> was made clear that this improvement is possible and has no serious\n> >>> side effects.\n> >> Hej Scott,\n> >> I think that you problem can be solved.\n> >> For each repository, you can tell Git to ignore the line endings,\n> >> CRLF vs LF.\n> >>\n> >> If you start with a fresh repo,\n> >> you can do something like:\n> >>\n> >> echo \"* text=auto\" >.gitattributes\n> >> git add .gitattributes\n> >> git commit -m \"Add .gitattributes\"\n> >>\n> >> For existing repos, we need to take another step:\n> >>\n> >> echo \"* text=auto\" >.gitattributes\n> >> git add .gitattributes\n> >> git add  --renormlize .\n> >> git commit -m \"Add .gitattributes\"\n> >>\n> >> More information can be found e.g. here:\n> >> https://git-scm.com/docs/git-add\n> >>\n> >> Once you done that, you can merge branches\n> >> into the master branch with help of the renormalize\n> >>\n> >> git merge -X renormalze branch-name\n> >>\n> >> See even here:\n> >> https://git-scm.com/docs/git-merge\n> >>\n> >>\n> >> This is just a (too) short introduction, I hope that it\n> >> helps and that you find the time to dig somewhat deeper into\n> >> the documentation.\n> >>\n> >> Other developers have that problem as well, you are not alone.\n> >>\n> >> If you have a public repo, I could help with one example.\n> >>\n> >>\n> >>> Regards,\n> >>>\n> >>> Scott Richmond.\n> >>>   Director, Producer, Programmer\n> >>>   Brightrock Games\n> >>>   T: 07480795661\n>\n"},{"id":"388608","messageId":"20191220030259.GC163225@camp.crustytoothpaste.net","threadId":"52479","inReplyTo":"CAB1T5w2GyfERoaCyFZeKaui_xuXd0r6J+Zvq4pecstBJ9UyRzw@mail.gmail.com","subject":"Re: Ability to ignore EOL changes for certain projects","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2019-12-20T03:02:59Z","receivedAt":"2019-12-20T03:03:09Z","isPatch":false,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On 2019-12-18 at 11:10:27, Scott Richmond wrote:\n> The Problem Domain\n> In certain dev environments (Unity3D projects) there is (AFAIK) an\n> unsolvable problem where some files are often modified with line\n> endings that aren't the native system or not the committed line\n> endings for that file. Secondarily, in this case line endings don't\n> matter - Nothing in the dev environment \"cares\" which kind of line\n> ending is used.\n\nI'm not sure this syncs up with your statements below.  Typically,\nprograms that do editing of text files either (a) write the line endings\nthat the file started out with or (b) write the system line endings.  If\nyou set the appropriate files with the `text` attribute in\n`.gitattributes` (or use `* text=auto` if appropriate), then Git will\nalways write in the system line endings and convert internally to LF,\nand programs that do either (a) or (b) will work.\n\nYet it sounds like you have development tools that don't do either (a)\nor (b): they write in some fixed line ending, and therefore care very\nmuch which line ending is being used.  In such a case, you could\nwrite (for example) `*.yaml text eol=crlf` (or `eol=lf`) if you always\nwant them to do that and Git will convert to those line endings and\nstore in LF.\n\nIf the problem is text editor settings, you can use a `.editorconfig`\nfile, which is a cross-platform text editor configuration that can\nspecify line endings.  Most text editors can be configured to honor such\nsettings, although it may require a plugin.\n\nIf your problem is a shared Windows / Linux environment like WSL, you\ncan set `core.eol` to `crlf` in the repository and things will work.  If\nyou need settings like this, you can even set them up appropriately for\nthe system using a make target or bootstrap script so you don't need to\ndo that by hand on each system.\n\nIf you do this, then your tools will write the same line endings as are\nchecked out, and files won't appear modified.  You can see how Git\nitself uses this to set up files appropriately for different systems.\n\nThe only case this wouldn't work is if the tools wrote some random line\nendings depending on an attribute other than the OS they're on, or if\nyou had multiple tools doing different things.  If that's really your\nproblem, then yes, you'd need a new Git feature.  It is of course\npossible to use a filter to strip out all carriage returns, but that\ndoesn't prevent Git from showing the file as being modified.\n\nWhat I've proposed, of course, requires some setup work and\nconfiguration.  It isn't trivial, but it does work for a lot of projects\nalready.\n\n> Solution Request\n> It would be fantastic if we could tell Git to stop caring about EOL\n> changes on a per-repo basis, with the effective output being that git\n> status shouldn't show changes for files with differing EOLs.\n> \n> I'm experienced with Git, though I am not expert enough to consider\n> creating such a change myself - It is unclear to me just how\n> complicated a change may be. However maybe I could look into it if it\n> was made clear that this improvement is possible and has no serious\n> side effects.\n\nI'm not sure how such a feature would interact with how Git operates\nwhen it re-reads the index.  It isn't 100% clear to me when data is\nfiltered through various text filters such as EOL filters.  All the\nfiltering I've used is one-to-one, and therefore any modification of the\nfile contents necessarily means that the indexed contents have changed.\n\nIf Git does apply such filters when refreshing the index (such as\nhappens before `git status`), then such a feature would be relatively\neasy to implement, although you'd incur a performance penalty when\nchanging the EOL of a file, even if the file were otherwise identical.\nI suspect it would make the most sense as an additional value for\n`core.autocrlf`.\n\nIf Git doesn't apply those filters, then there really isn't a way to do\nwhat you want without fundamentally changing the characteristics of how\nGit operates on the index, since it would still show files as modified.\n\nMaybe someone else can comment on the feasibility of this better than I\ncan.\n-- \nbrian m. carlson: Houston, Texas, US\nOpenPGP: https://keybase.io/bk2204\n"},{"id":"388611","messageId":"CAB1T5w0_Dp5u9JYRqGnQbuaL_pV2VN2ZTpXkDEJrEyHR2WJ3yA@mail.gmail.com","threadId":"52479","inReplyTo":"20191220030259.GC163225@camp.crustytoothpaste.net","subject":"Re: Ability to ignore EOL changes for certain projects","fromName":"Scott Richmond","fromEmail":"scott@brightrockgames.com","sentAt":"2019-12-20T10:58:47Z","receivedAt":"2019-12-20T10:59:02Z","isPatch":false,"sender":{"key":"scott@brightrockgames.com","avatar":null},"body":"> I'm not sure this syncs up with your statements below...[snip]\n\nUnfortunately it is the latter case where certain dev apps, or plugins\nwithin dev apps, that can occasionally write incorrect line endings.\nWe don't have configurable control over these things unfortunately.\nIf I widen the scope to all the scenarios you presented though - My\nfeature request could still have value as another way to alleviate the\npressure EOL changes create on a git repo. I would agree that it would\nbe best to solve the problem at its source where possible, but it is\nsometimes not possible.\n\n> I'm not sure how such a feature would interact with how Git operates when it re-reads the index...\n\nAFAIK git status does not filter the results, as that would push\nwrites to files (I think?) when the user wouldn't expect it to. So it\nmakes sense that actions such as git status are unaffected by\ntext/crlf options.\nI agree the additional cost to run a filter or similar action with git\nstatus would be a cause for concern, but maybe an optional and\nacceptable trade-off for the pain it alleviates elsewhere.\n\nI do not know the underlying way git works either, but I suspect it\nwould have to be a 3rd test after sha1 & oid against the files that\ncome out of git status. I'm guessing that is an area of git that has\nlittle configurability and therefore very difficult to make changes\nto. :/\n\nRegards,\n\nScott Richmond.\n  Director, Producer, Programmer\n  Brightrock Games\n  T: 07480795661\n\nOn Fri, Dec 20, 2019 at 3:03 AM brian m. carlson\n<sandals@crustytoothpaste.net> wrote:\n>\n> On 2019-12-18 at 11:10:27, Scott Richmond wrote:\n> > The Problem Domain\n> > In certain dev environments (Unity3D projects) there is (AFAIK) an\n> > unsolvable problem where some files are often modified with line\n> > endings that aren't the native system or not the committed line\n> > endings for that file. Secondarily, in this case line endings don't\n> > matter - Nothing in the dev environment \"cares\" which kind of line\n> > ending is used.\n>\n> I'm not sure this syncs up with your statements below.  Typically,\n> programs that do editing of text files either (a) write the line endings\n> that the file started out with or (b) write the system line endings.  If\n> you set the appropriate files with the `text` attribute in\n> `.gitattributes` (or use `* text=auto` if appropriate), then Git will\n> always write in the system line endings and convert internally to LF,\n> and programs that do either (a) or (b) will work.\n>\n> Yet it sounds like you have development tools that don't do either (a)\n> or (b): they write in some fixed line ending, and therefore care very\n> much which line ending is being used.  In such a case, you could\n> write (for example) `*.yaml text eol=crlf` (or `eol=lf`) if you always\n> want them to do that and Git will convert to those line endings and\n> store in LF.\n>\n> If the problem is text editor settings, you can use a `.editorconfig`\n> file, which is a cross-platform text editor configuration that can\n> specify line endings.  Most text editors can be configured to honor such\n> settings, although it may require a plugin.\n>\n> If your problem is a shared Windows / Linux environment like WSL, you\n> can set `core.eol` to `crlf` in the repository and things will work.  If\n> you need settings like this, you can even set them up appropriately for\n> the system using a make target or bootstrap script so you don't need to\n> do that by hand on each system.\n>\n> If you do this, then your tools will write the same line endings as are\n> checked out, and files won't appear modified.  You can see how Git\n> itself uses this to set up files appropriately for different systems.\n>\n> The only case this wouldn't work is if the tools wrote some random line\n> endings depending on an attribute other than the OS they're on, or if\n> you had multiple tools doing different things.  If that's really your\n> problem, then yes, you'd need a new Git feature.  It is of course\n> possible to use a filter to strip out all carriage returns, but that\n> doesn't prevent Git from showing the file as being modified.\n>\n> What I've proposed, of course, requires some setup work and\n> configuration.  It isn't trivial, but it does work for a lot of projects\n> already.\n>\n> > Solution Request\n> > It would be fantastic if we could tell Git to stop caring about EOL\n> > changes on a per-repo basis, with the effective output being that git\n> > status shouldn't show changes for files with differing EOLs.\n> >\n> > I'm experienced with Git, though I am not expert enough to consider\n> > creating such a change myself - It is unclear to me just how\n> > complicated a change may be. However maybe I could look into it if it\n> > was made clear that this improvement is possible and has no serious\n> > side effects.\n>\n> I'm not sure how such a feature would interact with how Git operates\n> when it re-reads the index.  It isn't 100% clear to me when data is\n> filtered through various text filters such as EOL filters.  All the\n> filtering I've used is one-to-one, and therefore any modification of the\n> file contents necessarily means that the indexed contents have changed.\n>\n> If Git does apply such filters when refreshing the index (such as\n> happens before `git status`), then such a feature would be relatively\n> easy to implement, although you'd incur a performance penalty when\n> changing the EOL of a file, even if the file were otherwise identical.\n> I suspect it would make the most sense as an additional value for\n> `core.autocrlf`.\n>\n> If Git doesn't apply those filters, then there really isn't a way to do\n> what you want without fundamentally changing the characteristics of how\n> Git operates on the index, since it would still show files as modified.\n>\n> Maybe someone else can comment on the feasibility of this better than I\n> can.\n> --\n> brian m. carlson: Houston, Texas, US\n> OpenPGP: https://keybase.io/bk2204\n"}]}