{"thread":{"id":"39755","subject":"end-of-line diff checkout direction dependence problem","startedAt":"2015-06-30T14:12:41Z","lastAt":"2015-07-06T06:53:20Z","messageCount":4,"participants":["Thomas Vieten","Torsten Bögershausen"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"265233","messageId":"5592A3D9.3080706@ivision.de","threadId":"39755","inReplyTo":null,"subject":"end-of-line diff checkout direction dependence problem","fromName":"Thomas Vieten","fromEmail":"tvie@ivision.de","sentAt":"2015-06-30T14:12:41Z","receivedAt":"2015-06-30T14:12:41Z","isPatch":false,"sender":{"key":"tvie@ivision.de","avatar":null},"body":"We face a very inconvenient problem with end-of-line diffs which are not \n\"real\".\nWe know the end-of-line problem very well as we thought.\nBut now we found a new phenomenon and nobody mentioning it.\n\nConsider the following repository structure:\n\n           -----------|----|------------->branch1\n         /\nmaster\n         \\\n----------|-------|---------|--------------->branch2\n\nThe branches are based on master/head.\nWe just consider one branch here, e.g. branch1 .\n\nWorking with the head of branch1 gives no problems. No end-of-line diffs.\nAlso going back in the direction of master - no problems.\nOnly in the case if we do a checkout from branch1 to master, then\nall of a sudden end-of-line diffs appear.\nThe files might be changed, but the end-of-line attributes in \ngitattributes are not changed in the branch.\n\nIt seems to be the case that since the last change to the files which \npop up with eol differences, gittattributes where changed and touch \ntheir extensions.\n\nWith the operation\n\ngit ls-files -z | xargs -0 rm -f  # delete all the files of this version \n- here master/head\ngit reset --hard                  # force checkout of master/head and \nreset index\n\nThe problem disappears! No eol problems anymore. Something like a brute \nforce checkout.\n\nAlso checking out versions in the direction of branch1 give never \nend-of-line diffs.\n\nWhat has changed somewhen are the gitattributes.\n\nWe estimate that this becomes a problem when applying the diffs from \nbranch1 in the direction of\nthe master. Finally then the diffs result in a different state from the \nmaster.\n\nBut when the master is checked out freshly, this difference does not appear.\n\nVery, very annoying.\n\nWe check now every time when these end-of-line diffs appear, if they are \nreally end of line diffs\n\ngit diff --ignore-space-at-eol\n\nand then try the procedure above.\n\nBut to have a dependence from the direction of the checkout is somewhat \nirritating.\n\nIf this is not a bug - then how to avoid it ?\n\nbye for now\n\nThomas\n\n-- \n++++++++++++++++++++++++++++++++++++++++++++++++++++++\niVISION GmbH\t\t|Industrial Inspection Systems\nJülicherstr. 336 B\t|\nD-52070 Aachen\t\t|\nTel: +49-(0)241-961233\t|\nFAX: +49-(0)241-980 2064|\nwww.ivision.de\ntvie@ivision.de\n++++++++++++++++++++++++++++++++++++++++++++++++++++++\n"},{"id":"265235","messageId":"5592AA93.9090002@web.de","threadId":"39755","inReplyTo":"5592A3D9.3080706@ivision.de","subject":"Re: end-of-line diff checkout direction dependence problem","fromName":"Torsten Bögershausen","fromEmail":"tboegi@web.de","sentAt":"2015-06-30T14:41:23Z","receivedAt":"2015-06-30T14:41:23Z","isPatch":false,"sender":{"key":"tboegi@web.de","avatar":"https://avatars.githubusercontent.com/u/7138363?v=4"},"body":"On 2015-06-30 16.12, Thomas Vieten wrote:\n> We face a very inconvenient problem with end-of-line diffs which are not \"real\".\n> We know the end-of-line problem very well as we thought.\n> But now we found a new phenomenon and nobody mentioning it.\n> \n> Consider the following repository structure:\n> \n>           -----------|----|------------->branch1\n>         /\n> master\n>         \\\n> ----------|-------|---------|--------------->branch2\n> \n> The branches are based on master/head.\n> We just consider one branch here, e.g. branch1 .\n> \n> Working with the head of branch1 gives no problems. No end-of-line diffs.\n> Also going back in the direction of master - no problems.\n> Only in the case if we do a checkout from branch1 to master, then\n> all of a sudden end-of-line diffs appear.\n> The files might be changed, but the end-of-line attributes in gitattributes are\n> not changed in the branch.\n> \n> It seems to be the case that since the last change to the files which pop up\n> with eol differences, gittattributes where changed and touch their extensions.\n> \n> With the operation\n> \n> git ls-files -z | xargs -0 rm -f  # delete all the files of this version - here\n> master/head\n> git reset --hard                  # force checkout of master/head and reset index\n> \n> The problem disappears! No eol problems anymore. Something like a brute force\n> checkout.\n> \n> Also checking out versions in the direction of branch1 give never end-of-line\n> diffs.\n> \n> What has changed somewhen are the gitattributes.\n> \n> We estimate that this becomes a problem when applying the diffs from branch1 in\n> the direction of\n> the master. Finally then the diffs result in a different state from the master.\n> \n> But when the master is checked out freshly, this difference does not appear.\n> \n> Very, very annoying.\n> \n> We check now every time when these end-of-line diffs appear, if they are really\n> end of line diffs\n> \n> git diff --ignore-space-at-eol\n> \n> and then try the procedure above.\n> \n> But to have a dependence from the direction of the checkout is somewhat irritating.\n> \n> If this is not a bug - then how to avoid it ?\n> \n> bye for now\n> \n> Thomas\n> \nThe things which are described don't sound unfamilar.\nFirst some questions:\nWhich Git/OS are you running on ?\n\nCYGWIN ?\nGit-for-Windows ?\nLinux ?\nOther ?\n\nWhich versions ?\nHow does your .gitattribute file look like ?\n\nIt may be, that you need to \"nornalize\" your repo:\n\nhttps://www.kernel.org/pub/software/scm/git/docs/gitattributes.html\nThe search for this text\n\"When text=auto normalization\"\nand follow the instructions:\n"},{"id":"265397","messageId":"559543E9.5040706@ivision.de","threadId":"39755","inReplyTo":"5593F73E.60305@ivision.de","subject":"Re: end-of-line diff checkout direction dependence problem","fromName":"Thomas Vieten","fromEmail":"tvie@ivision.de","sentAt":"2015-07-02T14:00:09Z","receivedAt":"2015-07-02T14:00:09Z","isPatch":false,"sender":{"key":"tvie@ivision.de","avatar":null},"body":"\nAnswers and new response below.\n\nThomas Vieten wrote:\n> On 2015-06-30 16.12, Thomas Vieten wrote:\n>> We face a very inconvenient problem with end-of-line diffs which are \n>> not \"real\".\n>> We know the end-of-line problem very well as we thought.\n>> But now we found a new phenomenon and nobody mentioning it.\n>>\n>> Consider the following repository structure:\n>>\n>>           -----------|----|------------->branch1\n>>         /\n>> master\n>>         \\\n>> ----------|-------|---------|--------------->branch2\n>>\n>> The branches are based on master/head.\n>> We just consider one branch here, e.g. branch1 .\n>>\n>> Working with the head of branch1 gives no problems. No end-of-line \n>> diffs.\n>> Also going back in the direction of master - no problems.\n>> Only in the case if we do a checkout from branch1 to master, then\n>> all of a sudden end-of-line diffs appear.\n>> The files might be changed, but the end-of-line attributes in \n>> gitattributes are\n>> not changed in the branch.\n>>\n>> It seems to be the case that since the last change to the files which \n>> pop up\n>> with eol differences, gittattributes where changed and touch their \n>> extensions.\n>>\n>> With the operation\n>>\n>> git ls-files -z | xargs -0 rm -f  # delete all the files of this \n>> version - here\n>> master/head\n>> git reset --hard                  # force checkout of master/head and \n>> reset index\n>>\n>> The problem disappears! No eol problems anymore. Something like a \n>> brute force\n>> checkout.\n>>\n>> Also checking out versions in the direction of branch1 give never \n>> end-of-line\n>> diffs.\n>>\n>> What has changed somewhen are the gitattributes.\n>>\n>> We estimate that this becomes a problem when applying the diffs from \n>> branch1 in\n>> the direction of\n>> the master. Finally then the diffs result in a different state from \n>> the master.\n>>\n>> But when the master is checked out freshly, this difference does not \n>> appear.\n>>\n>> Very, very annoying.\n>>\n>> We check now every time when these end-of-line diffs appear, if they \n>> are really\n>> end of line diffs\n>>\n>> git diff --ignore-space-at-eol\n>>\n>> and then try the procedure above.\n>>\n>> But to have a dependence from the direction of the checkout is \n>> somewhat irritating.\n>>\n>> If this is not a bug - then how to avoid it ?\n>>\n>> bye for now\n>>\n>> Thomas\n>>\n> The things which are described don't sound unfamilar.\n> First some questions:\n> Which Git/OS are you running on ?\n>\n> CYGWIN ?\n> Git-for-Windows ?\ngit version 1.9.2.msysgit.0\n+tortoisegit, for viewing and conflicts\n> Linux ?\n> Other ?\nWindows 7 Pro\n>\n> Which versions ?\n> How does your .gitattribute file look like ?\n>\nsee the file attachend to the end of this message\n> It may be, that you need to \"nornalize\" your repo:\n>\n> https://www.kernel.org/pub/software/scm/git/docs/gitattributes.html\n> The search for this text\n> \"When text=auto normalization\"\n> and follow the instructions:\n>\nin principle we know all this.\nWhat is remarkable that we are able to checkout a version of master \nwhich is not consistent with the repo, and more, dependent from the \ncheckout direction (if the direction is the positive or negative history \nin time). And on the other hand we can checkout a version of master \nwhich is in sync with the master.\n\nNormally such conflicts with not normalised repos appear immediately \nalso in positive history direction. And then it is possible to detect them.\nThe other way around - negative history and diffs - it causes a big \nquestionmark.\n\nOn the other hand this would lead to the mandatory work flow advice: \n\"Always normalize the repository after changes within the gitattributes \nfile\"\nAnd then: Should this then not be automatically be done somehow in the \nbackground by git ?\nReasoning: if the \"git machine\" is causing this behaviour \nsystematically, shouldn't the machine itself have compensation, correction?\n\nDepending on your point of view this could be seen as a bug.\n\nThere is also a big question open: will normalisation really help ? \nBecause there must be one commit with the new gitattributes and then you \nnormalize.\nBut the \"wrong diff\" is in the repo and will cause the problem when \ngoing back to master in the negative direction.\nThis is how understand it up to now.\n\nBut at this point git is complex and we are not really the experts.\n\nbest regards\n\nThomas V.\n\n\n--------------------------------------------------------------------------------------------------------\n.gitattributes\n#Set default behaviour, in case users don't have core.autocrlf set.\n* text=auto\n*.PC                text\n*.in                text\n*.hex               text\n*.inc               text\n*.DAT               binary\n*.lic               binary\n*.chm               binary\n*.dat               binary\n*.cam               binary\n*.cam.new           binary\n*.cam.cam           binary\n*.am                text\n*.cc                text\n*.m4                eol=lf\n*.exe               binary\n*.sys               binary\n*.dll               binary\n*.cxf               eol=lf\n*.vcproj            eol=crlf\n*.vcxproj           text\n*.vcxproj.filters   text\n*.sln               eol=crlf\n*.dsp               eol=crlf\n*.dsw               eol=crlf\n*.sh                text\n*.txt               text\n*.png               binary\n*.jpg               binary\n*.tif               binary\n*.odt               binary\n*.db                binary\n*.vsd               binary\n*.dbf               binary\n*.zip               binary\n*.emf               binary\n*.psp               binary\n*.dot               binary\n*.doc               diff=word\n*.sql               text\n*.vsd               binary\n*.sys               binary\n*.xls               binary\n*.vxd               binary\n*.jpg               binary\n*.jpeg              binary\n*.sch               binary\n*.pdf               binary\n*.s#*               binary\n*.ocx               binary\n*.gif               binary\n*.gz                binary\n*.obj               binary\n*.ncb               binary\n*.bin               binary\n*.bmp               binary\n*.cmd               eol=crlf\n*.bat               eol=crlf\n*.htm               eol=crlf\n*.html              eol=crlf\n*.dsp               eol=crlf\n*.dbf               binary\n*.vtg               binary\n*.mdb               binary\n*.msi               binary\n*.cbproj            binary\n*.groupproj         binary\n*.bdsgroup          binary\n*.tgr               binary\n*.pbd               binary\n*.pbl               binary\n*.tag               binary\nmakefile            text\n*.c                 text\n*.cpp               text\n*.h                 text\n*.hpp               text\n"},{"id":"265544","messageId":"559A25E0.1090908@web.de","threadId":"39755","inReplyTo":"559543E9.5040706@ivision.de","subject":"Re: end-of-line diff checkout direction dependence problem","fromName":"Torsten Bögershausen","fromEmail":"tboegi@web.de","sentAt":"2015-07-06T06:53:20Z","receivedAt":"2015-07-06T06:53:20Z","isPatch":false,"sender":{"key":"tboegi@web.de","avatar":"https://avatars.githubusercontent.com/u/7138363?v=4"},"body":"On 2015-07-02 16.00, Thomas Vieten wrote:\n[]\n> see the file attachend to the end of this message\nThanks for the info\n>\n>> It may be, that you need to \"nornalize\" your repo:\n>>\n> in principle we know all this.\n> What is remarkable that we are able to checkout a version of master which is\n> not consistent with the repo, and more, dependent from the checkout direction\n> (if the direction is the positive or negative history in time). And on the\n> other hand we can checkout a version of master which is in sync with the master.\n>\n> Normally such conflicts with not normalised repos appear immediately also in\n> positive history direction. And then it is possible to detect them.\n> The other way around - negative history and diffs - it causes a big questionmark.\n>\n> On the other hand this would lead to the mandatory work flow advice: \"Always\n> normalize the repository after changes within the gitattributes file\"\nYes, I think this sounds reasonable, and I think the documentations says this.\nBut it is a manual step, which is typically done only once.\nIt there anything which can be improved here ?\n> And then: Should this then not be automatically be done somehow in the\n> background by git ?\nThis could make sense, but the word \"background\" should mean visible to the user ?\nRecently we got the \"untracked cache\" into Git, which keeps track about the\n.gitignore files\nA similar logic can be used to keep track of the .gitattributes file(s) (There\ncan be more than one)\nPatches are welcome.\n>\n> Reasoning: if the \"git machine\" is causing this behaviour systematically,\n> shouldn't the machine itself have compensation, correction?\n>\nQuestion 1: The documentation should be clear enough:\n Whenever someone \"introduces .gitattributes,\" the repo should be normalized.\n\n> Depending on your point of view this could be seen as a bug.\nI have run into this myself, but never in a reproducable way.\nThe day I can reproduce it, I may send a patch.\nOr, somebody else sends a patch before that day.\n>\n> There is also a big question open: will normalisation really help ? Because\n> there must be one commit with the new gitattributes and then you normalize.\nTo my experience it does.\n Normalizing means one single commit (not 2), including the .gitattributes and\nthe normalized files.\nQuestion 2:\nIs there something in the documentation that could be improved.\n> But the \"wrong diff\" is in the repo and will cause the problem when going back\n> to master in the negative direction.\n> This is how understand it up to now.\n>\n> But at this point git is complex and we are not really the experts.\n>\n> best regards\n>\n> Thomas V.\n>\nThanks for the report(s)\n"}]}