{"thread":{"id":"63292","subject":"Potential bug: Git instantly converts utf-8-bom to utf-8","startedAt":"2025-04-14T15:59:53Z","lastAt":"2025-04-14T19:28:26Z","messageCount":4,"participants":["messester","Lucas Seiki Oshiro","Torsten Bögershausen","Junio C Hamano"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"516108","messageId":"DM6PR12MB386578E1F28B532600ADF575F7B32@DM6PR12MB3865.namprd12.prod.outlook.com","threadId":"63292","inReplyTo":null,"subject":"Potential bug: Git instantly converts utf-8-bom to utf-8","fromName":"messester","fromEmail":"cnotsomark@gmail.com","sentAt":"2025-04-14T15:59:51Z","receivedAt":"2025-04-14T15:59:53Z","isPatch":false,"sender":{"key":"cnotsomark@gmail.com","avatar":null},"body":"How this occurs:\nWhen I clone the git repository https://github.com/NinekoTheCat/infinilore.cs.git into an empty directory and run git status it shows changes where all the files are deleted and then added again.\nThe difference seems to be in the first line as opening it in a text editor aware of encodings shows the change from utf-8-bom to utf-8.\n(Attached to this email is an example patch file).\n\nMy git configuration in the directory is:\n---\ndiff.astextplain.textconv=astextplain\nfilter.lfs.clean=git-lfs clean -- %f\nfilter.lfs.smudge=git-lfs smudge -- %f\nfilter.lfs.process=git-lfs filter-process\nfilter.lfs.required=true\nhttp.sslbackend=openssl\nhttp.sslcainfo=REDACT.crt\ncore.autocrlf=true\ncore.fscache=true\ncore.symlinks=false\npull.rebase=false\ncredential.helper=manager\ncredential.https://dev.azure.com.usehttppath=true\ninit.defaultbranch=main\nfilter.lfs.clean=git-lfs clean -- %f\nfilter.lfs.smudge=git-lfs smudge -- %f\nfilter.lfs.process=git-lfs filter-process\nfilter.lfs.required=true\nuser.name=REDACT\nuser.email=REDACT\nuser.signingkey=4D973217092A925B\ncore.editor=\"C:\\Users\\cnots\\AppData\\Local\\Programs\\Microsoft VS Code\\bin\\code\" --wait\nsafe.directory=REDACT\ncommit.gpgsign=true\ngpg.program=C:\\Program Files (x86)\\GnuPG\\bin\\gpg.exe\ncredential.https://github.com.helper=\ncredential.https://github.com.helper=!REDACT\ncredential.https://gist.github.com.helper=\ncredential.https://gist.github.com.helper=!REDACT\n---\nand the version of git I use is 2.49.0.windows.1.\n\nI'd be very very grateful if anyone could explain what's going on and have a solution."},{"id":"516118","messageId":"A743D2B5-0397-4AF3-899B-FC3D3AD2B5CB@gmail.com","threadId":"63292","inReplyTo":"DM6PR12MB386578E1F28B532600ADF575F7B32@DM6PR12MB3865.namprd12.prod.outlook.com","subject":"Re: Potential bug: Git instantly converts utf-8-bom to utf-8","fromName":"Lucas Seiki Oshiro","fromEmail":"lucasseikioshiro@gmail.com","sentAt":"2025-04-14T17:19:05Z","receivedAt":"2025-04-14T17:19:19Z","isPatch":false,"sender":{"key":"lucasseikioshiro@gmail.com","avatar":"https://avatars.githubusercontent.com/u/12701580?v=4"},"body":"\n> When I clone the git repository https://github.com/NinekoTheCat/infinilore.cs.git into an empty directory and run git status it shows changes where all the files are deleted and then added again.\n\nI tried here. I'm in a Mac, it shows that several file changes.\n\n> The difference seems to be in the first line as opening it in a text editor aware of encodings shows the change from utf-8-bom to utf-8.\n\nGiven the warning messages it looks like it is more related\nto CRLF/LF than to UTF-8.\n\nAt least here, for example, the listed modified '*.cs' files\nare the ones that ends their lines in CRLF. This is, running:\n\n```\nfile $(git ls-files '*.cs') | grep CRLF | cut -d ':' -f1 | sort\n```\n\nshows me the same files as\n\n```\ngit status '*.cs'\n```\n\nYou can also see that are some '*.cs' files that are not\nCRLF-terminated by running:\n\n```\nfile $(git ls-files '*.cs') | grep -v CRLF | sort\n```\n\n> I'd be very very grateful if anyone could explain what's going on and have a solution.<patch.cat>\n\nPerhaps a solution is:\n\n- Define a CRLF or LF as a standard\n- Convert all the files that don't follow the standard that you\n  have chosen to the one that you have chosen\n- Declare it in the .gitattributes file\n\nHope that it helps you!\n\n\n"},{"id":"516119","messageId":"20250414180338.GA2398@tb-raspi4","threadId":"63292","inReplyTo":"A743D2B5-0397-4AF3-899B-FC3D3AD2B5CB@gmail.com","subject":"Re: Potential bug: Git instantly converts utf-8-bom to utf-8","fromName":"Torsten Bögershausen","fromEmail":"tboegi@web.de","sentAt":"2025-04-14T18:03:38Z","receivedAt":"2025-04-14T18:03:46Z","isPatch":false,"sender":{"key":"tboegi@web.de","avatar":"https://avatars.githubusercontent.com/u/7138363?v=4"},"body":"On Mon, Apr 14, 2025 at 02:19:05PM -0300, Lucas Seiki Oshiro wrote:\n>\n> > When I clone the git repository https://github.com/NinekoTheCat/infinilore.cs.git into an empty directory and run git status it shows changes where all the files are deleted and then added again.\n>\n> I tried here. I'm in a Mac, it shows that several file changes.\n>\n> > The difference seems to be in the first line as opening it in a text editor aware of encodings shows the change from utf-8-bom to utf-8.\n>\n> Given the warning messages it looks like it is more related\n> to CRLF/LF than to UTF-8.\n>\n> At least here, for example, the listed modified '*.cs' files\n> are the ones that ends their lines in CRLF. This is, running:\n>\n> ```\n> file $(git ls-files '*.cs') | grep CRLF | cut -d ':' -f1 | sort\n> ```\n>\n> shows me the same files as\n>\n> ```\n> git status '*.cs'\n> ```\n>\n> You can also see that are some '*.cs' files that are not\n> CRLF-terminated by running:\n>\n> ```\n> file $(git ls-files '*.cs') | grep -v CRLF | sort\n> ```\n>\n> > I'd be very very grateful if anyone could explain what's going on and have a solution.<patch.cat>\n>\n> Perhaps a solution is:\n>\n> - Define a CRLF or LF as a standard\n> - Convert all the files that don't follow the standard that you\n>   have chosen to the one that you have chosen\n> - Declare it in the .gitattributes file\n>\n> Hope that it helps you!\n\nI think that this is already done :-)\nHowever, the repo must be normalized:\n\ngit add --renormalize .\n\nwill do that.\n\n"},{"id":"516123","messageId":"xmqq1ptur19k.fsf@gitster.g","threadId":"63292","inReplyTo":"20250414180338.GA2398@tb-raspi4","subject":"Re: Potential bug: Git instantly converts utf-8-bom to utf-8","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-04-14T19:28:23Z","receivedAt":"2025-04-14T19:28:26Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Torsten Bögershausen <tboegi@web.de> writes:\n\n> I think that this is already done :-)\n> However, the repo must be normalized:\n>\n> git add --renormalize .\n>\n> will do that.\n\nThanks ;-).\n\n"}]}