# Potential bug: Git instantly converts utf-8-bom to utf-8 4 messages from 2025-04-14 to 2025-04-14. Participants: messester, Lucas Seiki Oshiro, Torsten Bögershausen, Junio C Hamano. Thread: https://gitlist.dev/t/63292 ## messester, 2025-04-14 15:59 Subject: Potential bug: Git instantly converts utf-8-bom to utf-8 Message-ID: URL: https://gitlist.dev/e/DM6PR12MB386578E1F28B532600ADF575F7B32%40DM6PR12MB3865.namprd12.prod.outlook.com ``` How this occurs: When I clone the git repository https://github.com/NinekoTheCat/infinilore.cs.git into an empty directory and run git status it shows changes where all the files are deleted and then added again. The difference seems to be in the first line as opening it in a text editor aware of encodings shows the change from utf-8-bom to utf-8. (Attached to this email is an example patch file). My git configuration in the directory is: --- diff.astextplain.textconv=astextplain filter.lfs.clean=git-lfs clean -- %f filter.lfs.smudge=git-lfs smudge -- %f filter.lfs.process=git-lfs filter-process filter.lfs.required=true http.sslbackend=openssl http.sslcainfo=REDACT.crt core.autocrlf=true core.fscache=true core.symlinks=false pull.rebase=false credential.helper=manager credential.https://dev.azure.com.usehttppath=true init.defaultbranch=main filter.lfs.clean=git-lfs clean -- %f filter.lfs.smudge=git-lfs smudge -- %f filter.lfs.process=git-lfs filter-process filter.lfs.required=true user.name=REDACT user.email=REDACT user.signingkey=4D973217092A925B core.editor="C:\Users\cnots\AppData\Local\Programs\Microsoft VS Code\bin\code" --wait safe.directory=REDACT commit.gpgsign=true gpg.program=C:\Program Files (x86)\GnuPG\bin\gpg.exe credential.https://github.com.helper= credential.https://github.com.helper=!REDACT credential.https://gist.github.com.helper= credential.https://gist.github.com.helper=!REDACT --- and the version of git I use is 2.49.0.windows.1. I'd be very very grateful if anyone could explain what's going on and have a solution. ``` ## Lucas Seiki Oshiro, 2025-04-14 17:19 Subject: Re: Potential bug: Git instantly converts utf-8-bom to utf-8 Message-ID: URL: https://gitlist.dev/e/A743D2B5-0397-4AF3-899B-FC3D3AD2B5CB%40gmail.com In-Reply-To: ``` > When I clone the git repository https://github.com/NinekoTheCat/infinilore.cs.git into an empty directory and run git status it shows changes where all the files are deleted and then added again. I tried here. I'm in a Mac, it shows that several file changes. > The difference seems to be in the first line as opening it in a text editor aware of encodings shows the change from utf-8-bom to utf-8. Given the warning messages it looks like it is more related to CRLF/LF than to UTF-8. At least here, for example, the listed modified '*.cs' files are the ones that ends their lines in CRLF. This is, running: ˋˋˋ file $(git ls-files '*.cs') | grep CRLF | cut -d ':' -f1 | sort ˋˋˋ shows me the same files as ˋˋˋ git status '*.cs' ˋˋˋ You can also see that are some '*.cs' files that are not CRLF-terminated by running: ˋˋˋ file $(git ls-files '*.cs') | grep -v CRLF | sort ˋˋˋ > I'd be very very grateful if anyone could explain what's going on and have a solution. Perhaps a solution is: - Define a CRLF or LF as a standard - Convert all the files that don't follow the standard that you have chosen to the one that you have chosen - Declare it in the .gitattributes file Hope that it helps you! ``` ## Torsten Bögershausen, 2025-04-14 18:03 Subject: Re: Potential bug: Git instantly converts utf-8-bom to utf-8 Message-ID: <20250414180338.GA2398@tb-raspi4> URL: https://gitlist.dev/e/20250414180338.GA2398%40tb-raspi4 In-Reply-To: ``` On Mon, Apr 14, 2025 at 02:19:05PM -0300, Lucas Seiki Oshiro wrote: > > > When I clone the git repository https://github.com/NinekoTheCat/infinilore.cs.git into an empty directory and run git status it shows changes where all the files are deleted and then added again. > > I tried here. I'm in a Mac, it shows that several file changes. > > > The difference seems to be in the first line as opening it in a text editor aware of encodings shows the change from utf-8-bom to utf-8. > > Given the warning messages it looks like it is more related > to CRLF/LF than to UTF-8. > > At least here, for example, the listed modified '*.cs' files > are the ones that ends their lines in CRLF. This is, running: > > ˋˋˋ > file $(git ls-files '*.cs') | grep CRLF | cut -d ':' -f1 | sort > ˋˋˋ > > shows me the same files as > > ˋˋˋ > git status '*.cs' > ˋˋˋ > > You can also see that are some '*.cs' files that are not > CRLF-terminated by running: > > ˋˋˋ > file $(git ls-files '*.cs') | grep -v CRLF | sort > ˋˋˋ > > > I'd be very very grateful if anyone could explain what's going on and have a solution. > > Perhaps a solution is: > > - Define a CRLF or LF as a standard > - Convert all the files that don't follow the standard that you > have chosen to the one that you have chosen > - Declare it in the .gitattributes file > > Hope that it helps you! I think that this is already done :-) However, the repo must be normalized: git add --renormalize . will do that. ``` ## Junio C Hamano, 2025-04-14 19:28 Subject: Re: Potential bug: Git instantly converts utf-8-bom to utf-8 Message-ID: URL: https://gitlist.dev/e/xmqq1ptur19k.fsf%40gitster.g In-Reply-To: <20250414180338.GA2398@tb-raspi4> ``` Torsten Bögershausen writes: > I think that this is already done :-) > However, the repo must be normalized: > > git add --renormalize . > > will do that. Thanks ;-). ```