{"thread":{"id":"21964","subject":"core.autocrlf & Cygwin - files incorrectly flagged as modified","startedAt":"2009-12-15T22:24:15Z","lastAt":"2009-12-18T06:06:46Z","messageCount":6,"participants":["David Antliff","Tay Ray Chuan","Robin Rosenberg"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"129966","messageId":"loom.20091215T225528-115@post.gmane.org","threadId":"21964","inReplyTo":null,"subject":"core.autocrlf & Cygwin - files incorrectly flagged as modified","fromName":"David Antliff","fromEmail":"david.antliff@gmail.com","sentAt":"2009-12-15T22:24:15Z","receivedAt":"2009-12-15T22:24:15Z","isPatch":false,"sender":{"key":"david.antliff@gmail.com","avatar":"https://gravatar.com/avatar/4ab5acd0e81835d0faa8c33939d1ca10947d9b7eb8437ea9afef419ceeaa973a?d=mp&s=160"},"body":"I approached the Cygwin mailing list about this problem, and the package\nmaintainer suggested I ask here as well, in case someone can shed some light on\nthis.\n\nI am using git-1.6.4.2 (as part of Cygwin-1.7 beta) and I've observed some\nbehaviour that seems wrong to me.\n\nWhen we deployed git on Cygwin, we decided to set core.autocrlf=true, as many of\nour users are working with DOS/Windows files (CRLF endings). Some users are\nusing Linux and working on the same code. I'm not actually sure why we chose to\nuse this setting - there's a lot of conflicting advice around. I think the final\ndecision was made so that kdiff3 would operate correctly. However we did go to\nlengths to ensure that everyone is using the same configuration setting.\n\nThe problem is that sometimes, after a git-clone, the output of git-status and\ngit-diff shows entire files as being different. However these files have not\nbeen modified by the user - only git has had a chance to change them (due to\nautocrlf=true). But surely if git has converted the file automatically, it\nshould know that it has to compensate for this when comparing with the local\nrepository?\n\nAnd I think it usually does this and almost all files behave properly. The\nproblem arises when the source-controlled file contains lines with trailing\nwhitespace. In this situation, git-diff clearly indicates a difference and this\nblocks merges and checkouts.\n\nI suspect what is happening is that the line conversion routine in git might be\nstripping trailing whitespace, as well as converting the line endings. This\noperation is not properly accounted for in the reverse direction, and the file\nis flagged as modified.\n\nThe Cygwin git package maintainer has indicated that he thinks this might be a\nbug: http://cygwin.com/ml/cygwin/2009-12/msg00466.html\n\nAlso, as cloned files are converted to DOS-line-endings, by default Cygwin's\nbash cannot run any scripts as they have the wrong line endings. I have to set\nthe 'permanent' bash variable SHELLOPTS to include 'igncr' before bash scripts\ncan run. Perhaps this is wrong and git on Cygwin (with binary mounts) should be\nconverting to UNIX line endings instead?\n\nAt one point I tried switching off core.autocrlf for myself but this caused a\nlot of conflicts due to mismatched line-endings. It seems to me that if we want\nto switch to this, *everyone* has to do it at once.\n\nI'd like to know whether this is a problem worthy of further investigation,\nwhether it should fall in the git or Cygwin camp, and whether I really ought to\nbe using autocrlf=true please.\n"},{"id":"130013","messageId":"be6fef0d0912161705q1ef23634rccdc170d69cf9484@mail.gmail.com","threadId":"21964","inReplyTo":"loom.20091215T225528-115@post.gmane.org","subject":"Re: core.autocrlf & Cygwin - files incorrectly flagged as modified","fromName":"Tay Ray Chuan","fromEmail":"rctay89@gmail.com","sentAt":"2009-12-17T01:05:12Z","receivedAt":"2009-12-17T01:05:12Z","isPatch":false,"sender":{"key":"rctay89@gmail.com","avatar":"https://avatars.githubusercontent.com/u/61553?v=4"},"body":"Hi,\n\nOn Wed, Dec 16, 2009 at 6:24 AM, David Antliff <david.antliff@gmail.com> wrote:\n> I suspect what is happening is that the line conversion routine in git might be\n> stripping trailing whitespace, as well as converting the line endings. This\n> operation is not properly accounted for in the reverse direction, and the file\n> is flagged as modified.\n\nI am not a Git expert, but could it be your editor on Windows?\n\nNot just stripping whitespace; it might be setting the eol to LF\n(instead of CRLF) when saving the file.\n\n> Also, as cloned files are converted to DOS-line-endings, by default Cygwin's\n> bash cannot run any scripts as they have the wrong line endings. I have to set\n> the 'permanent' bash variable SHELLOPTS to include 'igncr' before bash scripts\n> can run. Perhaps this is wrong and git on Cygwin (with binary mounts) should be\n> converting to UNIX line endings instead?\n\nAgain, IANAGE, but according to the manual, this should be expected,\nbecause Git, when writing to the filesystem, converts LF to CRLF:\n\n  If true, makes git convert CRLF at the end of lines in text files to\n  LF when reading from the filesystem, and convert in reverse when\n  writing to the filesystem.\n\n> At one point I tried switching off core.autocrlf for myself but this caused a\n> lot of conflicts due to mismatched line-endings. It seems to me that if we want\n> to switch to this, *everyone* has to do it at once.\n\nJust make this conversion a commit, and depending on your project's\nworkflow, push/pull it.\n\nAlthough the whole file will be marked as changed, you can always\ndouble-check that only whitespace changes have been made by running\ngit-diff with --ignore-space-at-eol or -b.\n\n-- \nCheers,\nRay Chuan\n"},{"id":"130015","messageId":"loom.20091217T024907-524@post.gmane.org","threadId":"21964","inReplyTo":"be6fef0d0912161705q1ef23634rccdc170d69cf9484@mail.gmail.com","subject":"Re: core.autocrlf & Cygwin - files incorrectly flagged as modified","fromName":"David Antliff","fromEmail":"david.antliff@gmail.com","sentAt":"2009-12-17T02:01:14Z","receivedAt":"2009-12-17T02:01:14Z","isPatch":false,"sender":{"key":"david.antliff@gmail.com","avatar":"https://gravatar.com/avatar/4ab5acd0e81835d0faa8c33939d1ca10947d9b7eb8437ea9afef419ceeaa973a?d=mp&s=160"},"body":"Tay Ray Chuan writes:\n> On Wed, Dec 16, 2009 at 6:24 AM, David Antliff wrote:\n> > I suspect what is happening is that the line conversion routine in git\n> > might be\n> > stripping trailing whitespace, as well as converting the line endings. This\n> > operation is not properly accounted for in the reverse direction, and the\n> > file is flagged as modified.\n> \n> I am not a Git expert, but could it be your editor on Windows?\n\nHi Ray, thank you for responding.\n\nThis observation is made immediately after the clone, before any editor (or any\nother program) even touches the files. If you can imagine doing a fresh clone\nand then immediately doing 'git-status' and seeing modified files, you'd have a\npretty good idea of what I'm seeing. Git itself has somehow marked the files as\nmodified on check-out. I am fairly certain it's related to trailing whitespace\non one or more lines within the affected files.\n\nIf there's no trailing whitespace, git converts the file to CRLF on checkout\nand shows no modification (since it knows to convert back when referring to\nlocal repository).\n\n> > Also, as cloned files are converted to DOS-line-endings, by default Cygwins\n> > bash cannot run any scripts as they have the wrong line endings. I have to \n> > set the 'permanent' bash variable SHELLOPTS to include 'igncr' before bash\n> > scripts can run. Perhaps this is wrong and git on Cygwin (with binary \n> > mounts) should be converting to UNIX line endings instead?\n> \n> Again, IANAGE, but according to the manual, this should be expected,\n> because Git, when writing to the filesystem, converts LF to CRLF:\n> \n>   If true, makes git convert CRLF at the end of lines in text files to\n>   LF when reading from the filesystem, and convert in reverse when\n>   writing to the filesystem.\n\nYes, perhaps the advice to use autocrlf=true on Cygwin (in binary mount mode,\nsince in text mode git doesn't work *at all*) is misplaced since Cygwin is not\nexpecting CRLF endings in that mode. Apparently it's required for apps like\nKdiff3 however... all so confusing really...\n\nThe inability to run bash scripts straight out of git when using autocrlf=true\nis almost enough to suggest to me that this mode really shouldn't be used in\nCygwin.\n \n> > At one point I tried switching off core.autocrlf for myself but this\n> > caused a lot of conflicts due to mismatched line-endings. It seems to me\n> > that if we want to switch to this, *everyone* has to do it at once.\n> \n> Just make this conversion a commit, and depending on your project's\n> workflow, push/pull it.\n\nYes, I think that's how it has to be done. Unfortunately I have 20+ projects\neach with many active branches. The entire conversion is possible but it's\ngoing to take a while... :)\n\n> Although the whole file will be marked as changed, you can always\n> double-check that only whitespace changes have been made by running\n> git-diff with --ignore-space-at-eol or -b.\n\nThat's a helpful option - thank you.\n\n\nI'd be quite interested to know if people are successfully using git in Cygwin\nwith autocrlf=true, and if so how they are working around these modified-files\nand bash-compatibility problems.\n\nAlso, is anyone using autocrlf=false in Cygwin successfully?\n\nThanks again for your reply.\n\n-- David.\n"},{"id":"130019","messageId":"200912170800.35752.robin.rosenberg@dewire.com","threadId":"21964","inReplyTo":"loom.20091215T225528-115@post.gmane.org","subject":"Re: core.autocrlf & Cygwin - files incorrectly flagged as modified","fromName":"Robin Rosenberg","fromEmail":"robin.rosenberg@dewire.com","sentAt":"2009-12-17T07:00:35Z","receivedAt":"2009-12-17T07:00:35Z","isPatch":false,"sender":{"key":"robin.rosenberg@dewire.com","avatar":"https://avatars.githubusercontent.com/u/46357?v=4"},"body":"tisdag 15 december 2009 23:24:15 skrev  David Antliff:\n> I approached the Cygwin mailing list about this problem, and the package\n> maintainer suggested I ask here as well, in case someone can shed some\n>  light on this.\n> \n> I am using git-1.6.4.2 (as part of Cygwin-1.7 beta) and I've observed some\n> behaviour that seems wrong to me.\n> \n> When we deployed git on Cygwin, we decided to set core.autocrlf=true, as\n>  many of our users are working with DOS/Windows files (CRLF endings). Some\n>  users are using Linux and working on the same code. I'm not actually sure\n>  why we chose to use this setting - there's a lot of conflicting advice\n>  around. I think the final decision was made so that kdiff3 would operate\n>  correctly. However we did go to lengths to ensure that everyone is using\n>  the same configuration setting.\n> \n> The problem is that sometimes, after a git-clone, the output of git-status\n>  and git-diff shows entire files as being different. However these files\n>  have not been modified by the user - only git has had a chance to change\n>  them (due to autocrlf=true). But surely if git has converted the file\n>  automatically, it should know that it has to compensate for this when\n>  comparing with the local repository?\n\nAFAIK, this happens if you have CRLF line endings in the blobs in the repo.\n\n-- robin\n"},{"id":"130055","messageId":"loom.20091218T033941-445@post.gmane.org","threadId":"21964","inReplyTo":"200912170800.35752.robin.rosenberg@dewire.com","subject":"Re: core.autocrlf & Cygwin - files incorrectly flagged as modified","fromName":"David Antliff","fromEmail":"david.antliff@gmail.com","sentAt":"2009-12-18T02:43:05Z","receivedAt":"2009-12-18T02:43:05Z","isPatch":false,"sender":{"key":"david.antliff@gmail.com","avatar":"https://gravatar.com/avatar/4ab5acd0e81835d0faa8c33939d1ca10947d9b7eb8437ea9afef419ceeaa973a?d=mp&s=160"},"body":"Robin Rosenberg writes:\n> tisdag 15 december 2009 23:24:15 skrev  David Antliff:\n> > The problem is that sometimes, after a git-clone, the output of git-status\n> >  and git-diff shows entire files as being different. However these files\n> >  have not been modified by the user - only git has had a chance to change\n> >  them (due to autocrlf=true). But surely if git has converted the file\n> >  automatically, it should know that it has to compensate for this when\n> >  comparing with the local repository?\n> \n> AFAIK, this happens if you have CRLF line endings in the blobs in the repo.\n\nOh? That could very well be true - how would such line endings get into a blob?\nI'm pretty sure everyone is using autocrlf=true which should convert all line\nendings to LF on commit, but perhaps there's another way to get CRLF line\nendings into a blob?\n\nIf autocrlf=false, files are committed as-is (I believe), so this would be one\nway to end up committing such files. But as far as I know, nobody has turned\nthat option off in our group.\n"},{"id":"130062","messageId":"200912180706.46455.robin.rosenberg@dewire.com","threadId":"21964","inReplyTo":"loom.20091218T033941-445@post.gmane.org","subject":"Re: core.autocrlf & Cygwin - files incorrectly flagged as modified","fromName":"Robin Rosenberg","fromEmail":"robin.rosenberg@dewire.com","sentAt":"2009-12-18T06:06:46Z","receivedAt":"2009-12-18T06:06:46Z","isPatch":false,"sender":{"key":"robin.rosenberg@dewire.com","avatar":"https://avatars.githubusercontent.com/u/46357?v=4"},"body":"fredagen den 18 december 2009 03.43.05 skrev  David Antliff:\n> Robin Rosenberg writes:\n> > tisdag 15 december 2009 23:24:15 skrev  David Antliff:\n> > > The problem is that sometimes, after a git-clone, the output of\n> > > git-status and git-diff shows entire files as being different. However\n> > > these files have not been modified by the user - only git has had a\n> > > chance to change them (due to autocrlf=true). But surely if git has\n> > > converted the file automatically, it should know that it has to\n> > > compensate for this when comparing with the local repository?\n> >\n> > AFAIK, this happens if you have CRLF line endings in the blobs in the\n> > repo.\n> \n> Oh? That could very well be true - how would such line endings get into a\n>  blob? I'm pretty sure everyone is using autocrlf=true which should convert\n>  all line endings to LF on commit, but perhaps there's another way to get\n>  CRLF line endings into a blob?\n> \n> If autocrlf=false, files are committed as-is (I believe), so this would be\n>  one way to end up committing such files. But as far as I know, nobody has\n>  turned that option off in our group.\n\nEGit works this way at the moment. \nConversion tools might do this.\nSome Eclipse plugins produce files with CRLF regardless on platform and on\nnon-Windows the core.autocrlf flags is false by default.\n\n-- robin\n"}]}