Re: Problems with CRLF line endings
- From
brian m. carlson <sandals@crustytoothpaste.net>
- Date
- Apr 20, 2021, 23:34 UTC
- Message-ID
- <YH9lFnJYpejqTOhk@camp.crustytoothpaste.net>
- In-Reply-To
- <DM6PR17MB2106CEAA1CED29E738B81602F9489@DM6PR17MB2106.namprd17.prod.outlook.com>
On 2021-04-20 at 16:33:40, Dennis Worthem wrote:
Show 9 quoted lines
> We as a multi-developer team are using git version 2.31.0.windows.1 on the developer side which we use through Git Bash on Windows 10. On the server side we use Azure Dev Ops. > Approximate 20 developers are working from one repository. We want all files to use CRLF line endings with working with the files. All the developers have a .bashrc file with contents of "git config --global core.autocrlf true". > Additionally, the repository with a .gitattributes file with contents: > *.raml text eol=crlf > *.json text eol=crlf > *.vb text eol=crlf > *.cs text eol=crlf > *.vbproj text eol=crlf > *.csproj text eol=crlf
text eol=crlf tells Git to store LF endings in the repository and use CRLF endings in the working tree. This is the recommended way to configure things, since it means that operations like diffs don't show trailing whitespace.
Show 5 quoted lines
> The problem we are having is that when pulling branches from the repository we still have recurring files with phantom changes. > We have told developer who experience this to either reset the indices cache and do a hard reset or renormalize and check in and push as indicated in a number of web sites. But it still seems to be happening. > Our repository has a release branch and a develop branch and the problems seems to be happening with mainly develop which has more developer traffic. > > It seems inconsistent on changes, The file will download with all CRLF endings but the file is marked as modified and it is not clear what has changed.
This happens when the file that's checked into the repository hasn't been properly checked in and either a filter (e.g., Git LFS), a line ending change, or a working tree encoding causes the file as checked out to differ when checked into the index.
For example, since you've told Git to store LF line endings in the index, if the file is checked into the repository with CRLF endings but then it gets updated, Git will re-read and convert the file into LF endings in the index, and it will appear modified. That can happen because the timestamp changes or for other reasons that make Git think it's dirty.
The proper solution to this is to do "git add --renormalize ." and then commit. You may want to also set up your CI such that CRLF line endings in the repository cause a hard failure. For example, you may wish to do this:
---- #!/bin/sh
if git ls-files --eol | grep i/crlf
then
echo "The above files have CRLF line endings in the index. Aborting." >&2
false
fi
----It's unclear to me why this is happening if you have the above configured, but it could be files which have not been updated and still contain the old CRLF line endings in the repository, which the renormalize should fix. It's also possible someone is using a different Git implementation, possibly one integrated in their editor, and it doesn't honor the .gitattributes file. Or someone could have a .git/info/attributes that's set to override the value.
By adding a hard fail on this in your CI system, you'll figure out where the source of the problem is very quickly. Most of the projects I'm on have hard failures on trailing whitespace, and this is very effective in stopping it from reappearing in the codebase.
> Does anyone have recommendations on how we can get this to work (always CRLF) without the phantom changes occurring in an apparently non-deterministic manner? Have I configured this completely?
It's unclear to me why this happens inconsistently, but it seems to mostly affect Windows users. Users of other OSes don't seem to have files in the working tree re-cleaned unexpectedly as often. That's why this appears to be nondeterministic; I'm not sure what it is about Windows that seems to cause this problem to show up more often there.
Show 5 quoted lines
> The problem may be that one or more developers have not updated their local indices. Also would it help to have everyone do a one-time reclone the repository? > > One file marked as modified has > $ git ls-files --eol | grep WCAPI.Core/WCAPI.Core.csproj > i/crlf w/crlf attr/text eol=crlf WCAPI.Core/WCAPI.Core.csproj
This says that the index has CRLF line endings. That's not correct.
> and a similar one not marked as modified has > > $ git ls-files --eol | grep WCAPI/WCAPI.csproj > i/lf w/crlf attr/text eol=crlf WCAPI/WCAPI.csproj
This is correct.
> Why does the modified one have 'i/crlf' and the unmodified ones have 'i/lf'
Because it was originally checked in that way and hasn't been fixed.
-- brian m. carlson (he/him or they/them) Houston, Texas, US