{"thread":{"id":"46449","subject":"change the filetype from binary to text after the file is commited to a git repo","startedAt":"2017-07-24T05:11:11Z","lastAt":"2017-07-24T20:34:51Z","messageCount":6,"participants":["tonka tonka","Jeff King","tonka3100@gmail.com"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"324939","messageId":"CAEor4NHDNL4CQP_FpCuJ_3F86-A3P8i=yvp1AZr-NKt0i1=cXw@mail.gmail.com","threadId":"46449","inReplyTo":null,"subject":"change the filetype from binary to text after the file is commited to a git repo","fromName":"tonka tonka","fromEmail":"tonka3100@gmail.com","sentAt":"2017-07-24T05:11:06Z","receivedAt":"2017-07-24T05:11:11Z","isPatch":false,"sender":{"key":"tonka3100@gmail.com","avatar":null},"body":"Hey everybody,\n\nI have a problem with an already committed file into my repo. This git\nrepo was converted from svn to git some years ago. Last week I have\nchange some lines in a file and I saw in the diff that it is marked as\nbinary (it's a simple .cpp file). I think on the first commit it was\ndetected as an utf-16 file (on windows). But no matter what I do I\ncan't get it back to a \"normal text\" text file (git does not detect\nthat), but I is now only utf-8. I also replace the whole content of\nthe file with just 'a' and git say it's binary.\n\n\nIs the only way to get it back to text-mode?:\n* copy a utf-8 version of the original file\n* delete the file\n* make a commit\n* add the old file as a new one\n\nI think that will work but it will also break my history.\n\nIs there a better way to get these behavior without losing history?\n\nBest regards\nTonka\n"},{"id":"324966","messageId":"20170724181835.zo3lcv7pxmkif2jr@sigill.intra.peff.net","threadId":"46449","inReplyTo":"CAEor4NHDNL4CQP_FpCuJ_3F86-A3P8i=yvp1AZr-NKt0i1=cXw@mail.gmail.com","subject":"Re: change the filetype from binary to text after the file is commited to a git repo","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2017-07-24T18:18:36Z","receivedAt":"2017-07-24T18:19:33Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Mon, Jul 24, 2017 at 07:11:06AM +0200, tonka tonka wrote:\n\n> I have a problem with an already committed file into my repo. This git\n> repo was converted from svn to git some years ago. Last week I have\n> change some lines in a file and I saw in the diff that it is marked as\n> binary (it's a simple .cpp file). I think on the first commit it was\n> detected as an utf-16 file (on windows). But no matter what I do I\n> can't get it back to a \"normal text\" text file (git does not detect\n> that), but I is now only utf-8. I also replace the whole content of\n> the file with just 'a' and git say it's binary.\n\nGit doesn't store a flag for \"binary-ness\" on each file (though see\nbelow). As the diffs are generated on the fly when you ask to compare\ntwo versions, so too is the determination of \"is this binary\".\n\nThe default heuristic looks at file size (by default, if the file is\nover 500MB it's considered binary) and whether it has any zero-byte\ncharacters in the first few kilobytes. But note that if _either_ side of\na diff is considered binary, then Git won't show a text diff.\n\nIf you want a particular diff to show all content, even if it doesn't\nlook like text, add \"-a\" to your git invocation (e.g., \"git show -a\").\n\nThat said, you can also use .gitattributes (see \"git help attributes\")\nto mark a file as binary or not-binary, skipping the heuristic check.\nI'm guessing since you converted from svn that you don't have a\n.gitattributes file, but it's possible that somebody later added one\nthat marks the file as binary (and so the solution would be to drop that\nentry).\n\n-Peff\n"},{"id":"324969","messageId":"7DE91669-D603-4C83-9151-78F2D5676E36@gmail.com","threadId":"46449","inReplyTo":"20170724181835.zo3lcv7pxmkif2jr@sigill.intra.peff.net","subject":"Re: change the filetype from binary to text after the file is commited to a git repo","fromName":"tonka3100@gmail.com","fromEmail":"tonka3100@gmail.com","sentAt":"2017-07-24T19:02:12Z","receivedAt":"2017-07-24T19:02:21Z","isPatch":false,"sender":{"key":"tonka3100@gmail.com","avatar":null},"body":"Hey Jeff,\n\nThx for your answer.\n\nThere is no .gitattributes file in the repo. I think that the git heuristic will also detect utf-16 files as binary (in windows), so i think that is the reason why my file is binary (i have to check that tomorrow). If i add a .gitattribute file i have the problem that git diff will treat the old and the new blob as utf-8, which generate garbage.\n\nDo you have another idea?\nCould it be possible to add only a space in code (utf-8) and then add the real content in a second commit, so the old and the new one are both utf-8?\n\n> Am 24.07.2017 um 20:18 schrieb Jeff King <peff@peff.net>:\n> \n>> On Mon, Jul 24, 2017 at 07:11:06AM +0200, tonka tonka wrote:\n>> \n>> I have a problem with an already committed file into my repo. This git\n>> repo was converted from svn to git some years ago. Last week I have\n>> change some lines in a file and I saw in the diff that it is marked as\n>> binary (it's a simple .cpp file). I think on the first commit it was\n>> detected as an utf-16 file (on windows). But no matter what I do I\n>> can't get it back to a \"normal text\" text file (git does not detect\n>> that), but I is now only utf-8. I also replace the whole content of\n>> the file with just 'a' and git say it's binary.\n> \n> Git doesn't store a flag for \"binary-ness\" on each file (though see\n> below). As the diffs are generated on the fly when you ask to compare\n> two versions, so too is the determination of \"is this binary\".\n> \n> The default heuristic looks at file size (by default, if the file is\n> over 500MB it's considered binary) and whether it has any zero-byte\n> characters in the first few kilobytes. But note that if _either_ side of\n> a diff is considered binary, then Git won't show a text diff.\n> \n> If you want a particular diff to show all content, even if it doesn't\n> look like text, add \"-a\" to your git invocation (e.g., \"git show -a\").\n> \n> That said, you can also use .gitattributes (see \"git help attributes\")\n> to mark a file as binary or not-binary, skipping the heuristic check.\n> I'm guessing since you converted from svn that you don't have a\n> .gitattributes file, but it's possible that somebody later added one\n> that marks the file as binary (and so the solution would be to drop that\n> entry).\n> \n> -Peff\n"},{"id":"324975","messageId":"20170724192357.y3oj5urfuie7ffjr@sigill.intra.peff.net","threadId":"46449","inReplyTo":"7DE91669-D603-4C83-9151-78F2D5676E36@gmail.com","subject":"Re: change the filetype from binary to text after the file is commited to a git repo","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2017-07-24T19:23:57Z","receivedAt":"2017-07-24T19:24:22Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Mon, Jul 24, 2017 at 09:02:12PM +0200, tonka3100@gmail.com wrote:\n\n> There is no .gitattributes file in the repo. I think that the git\n> heuristic will also detect utf-16 files as binary (in windows), so i\n> think that is the reason why my file is binary (i have to check that\n> tomorrow).\n\nCorrect. UTF-16 _is_ binary, if you are trying to include it alongside\nASCII content (like the rest of the text diff headers). The two cannot\nmix.\n\n> If i add a .gitattribute file i have the problem that git\n> diff will treat the old and the new blob as utf-8, which generate\n> garbage.\n\nGit's diff doesn't look at encodings at all; it does a diff of the\nactual bytes without respect to any encoding. So yes, if you use \"-a\" or\na gitattribute to ask git to show you the bytes, the UTF-16 is likely to\nlook like garbage (and a commit rewriting from utf-16 to utf-8 will\nbasically be a rewrite of the whole file contents).\n\n> Do you have another idea?  Could it be possible to add only a space in\n> code (utf-8) and then add the real content in a second commit, so the\n> old and the new one are both utf-8?\n\nI'm not sure exactly what you're trying to accomplish. If you're unhappy\nwith the file as utf-16, then you should probably convert to utf-8 as a\nsingle commit (since the diff will otherwise be unreadable) and then\nmake further changes in utf-8.\n\nIf you need the file to remain utf-16 but you want more readable diffs\nfor those versions, you can ask git to convert to utf-8 before\nperforming the diff. Such a diff couldn't be applied, but would be\nuseful for reading. E.g., try:\n\n  echo 'file diff=utf16' >.gitattributes\n  git config diff.utf16.textconv 'iconv -f utf16 -t utf8'\n\nYou can read more about how this works in the \"textconv\" section of \"git\nhelp attributes\".\n\nNote that I'm relying on the external \"iconv\" tool to do the conversion\nthere. It's pretty standard on most Unix systems, but I don't know what\nwould be the best tool on Windows.\n\n-Peff\n"},{"id":"324980","messageId":"20170724203215.nnktj55xyvqfmcvj@sigill.intra.peff.net","threadId":"46449","inReplyTo":"DBBA7352-5276-4972-A437-F27F5F4C2641@gmail.com","subject":"Re: change the filetype from binary to text after the file is commited to a git repo","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2017-07-24T20:32:15Z","receivedAt":"2017-07-24T20:32:31Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Mon, Jul 24, 2017 at 10:26:22PM +0200, tonka3100@gmail.com wrote:\n\n> > I'm not sure exactly what you're trying to accomplish. If you're unhappy\n> > with the file as utf-16, then you should probably convert to utf-8 as a\n> > single commit (since the diff will otherwise be unreadable) and then\n> > make further changes in utf-8.\n\n> That was exactly what i'm searching for. The utf-16 back in the days\n> was by accident (thx to visual studio). So if the last commit and the\n> acutal change are both utf-8 the diff should work again.  Just for my\n> understanding. Git just take the bytes of the whole file on every\n> commit, so there is no general problem with that, the size of the\n> utf-16 is just twice as big as the utf-8 one, is that correct?\n\nRight. The diff switching the encodings will be listed as \"binary\" (and\nyou should write a good commit message explaining what's going on!), but\nthen after that the changes to the utf-8 version will display as normal\ntext.  Git only looks at the actual bytes being diffed, not older\nversions of the file.\n\n-Peff\n"},{"id":"324981","messageId":"1C78174C-36D7-44B6-92A9-8A77D12AC6D2@gmail.com","threadId":"46449","inReplyTo":"20170724203215.nnktj55xyvqfmcvj@sigill.intra.peff.net","subject":"Re: change the filetype from binary to text after the file is commited to a git repo","fromName":"tonka3100@gmail.com","fromEmail":"tonka3100@gmail.com","sentAt":"2017-07-24T20:34:43Z","receivedAt":"2017-07-24T20:34:51Z","isPatch":false,"sender":{"key":"tonka3100@gmail.com","avatar":null},"body":"Thx jeff, i will try it tomorrow.\n\n> Am 24.07.2017 um 22:32 schrieb Jeff King <peff@peff.net>:\n> \n> On Mon, Jul 24, 2017 at 10:26:22PM +0200, tonka3100@gmail.com wrote:\n> \n>>> I'm not sure exactly what you're trying to accomplish. If you're unhappy\n>>> with the file as utf-16, then you should probably convert to utf-8 as a\n>>> single commit (since the diff will otherwise be unreadable) and then\n>>> make further changes in utf-8.\n> \n>> That was exactly what i'm searching for. The utf-16 back in the days\n>> was by accident (thx to visual studio). So if the last commit and the\n>> acutal change are both utf-8 the diff should work again.  Just for my\n>> understanding. Git just take the bytes of the whole file on every\n>> commit, so there is no general problem with that, the size of the\n>> utf-16 is just twice as big as the utf-8 one, is that correct?\n> \n> Right. The diff switching the encodings will be listed as \"binary\" (and\n> you should write a good commit message explaining what's going on!), but\n> then after that the changes to the utf-8 version will display as normal\n> text.  Git only looks at the actual bytes being diffed, not older\n> versions of the file.\n> \n> -Peff\n"}]}