Re: Changing encoding of a file : What should happen to CRLF in file ?

From: Ashish Negi <ashishnegi33@gmail.com>
Date: Nov 15, 2017, 08:11 UTC
Message-ID: <CAJ_+vJ6SxLOmZfG79Sa-vskBR-XG=C97--PB-vpijjUKym=jYw@mail.gmail.com>
In-Reply-To: <f9dc6482-587d-50a9-d649-aed63be18fad@web.de>

> If you commit the file, it will be stored with LF in the index,

This is what i believe is not happening.

Lets do this with a public repository and steps which are reproducible. I have created a repo : https://github.com/ashishnegi/git_encoding

If you clone this repo in linux and run `git status`, you will find that file is modified.

About repo : Repo have 2 commits, done on windows machine. First one check in a utf-16le encoded file which has crlf. crlf will not be converted to lf in index as git treats it as binary file. 2nd commit changes encoding to utf-8 and commits. This commit does not change crlf to lf in index, even though new format is utf-8 which is text based for git. This is the crux of problem.

I have attached all commands i ran on windows while creating the repo. I tried to capture all information that i could give. Please have a look. It might be useful.

Finally, thank you Torsten for giving your time to the problem. Really appreciate it.

On Tue, Nov 14, 2017 at 10:39 PM, Torsten Bögershausen <tboegi@web.de> wrote:

Show 43 quoted lines

> (Back to the beginning)
>
> You have a file ApplicationManifest.xml
> It is encoded in UTF-16 (and has CRLF)
>
> You convert it into UTF-8
> The file has still CRLF (in the worktree)
>
> Now you add it and make a commit.
> Under both Linux and Windows you have "text=auto".
>
> I assume that you have efficiently core.eol=lf under Linux
> and core.eol=crlf on Windows.
>
> (That is the default, when you don't change anything)
>
> Now, what happens to the CRLF?
> If you commit the file, it will be stored with LF in the index,
> on both systems.
> On checkout, Windows will convert them into CRLF, but Linux will not.
>
> That why you see
>>On linux, during committing i get warning : warning: CRLF will be
>>replaced by LF in …file_name..
>
> All in all there is nothing wrong, at least as I see it.
>
> The question remains:
> Do you need CRLF in Linux ?
> Probably not, but if yes, plase add a line
>
> *.xml text eol=crlf
>
> to your
> .gitattributes
>
> Otherwise your .gitconfig looks good to  me.
>
>
>
>
>
>

git_encoding_repo>git config --global core.safecrlf true

git_encoding_repo>git config core.autocrlf false

git_encoding_repo>git config --get core.autocrlf false

git_encoding_repo>cat .gitattributes
# Set the default behavior, in case people don't have core.autocrlf set.
* text=auto

*.vcxproj eol=crlf *.sh eol=lf

# Denote all files that are truly binary and should not be modified. *.exe binary *.dll binary *.pdb binary *.ico binary *.png binary *.jpg binary

git_encoding_repo>git status On branch master

No commits yet

Untracked files:
  (use "git add <file>..." to include in what will be committed)

        .gitattributes
        file_name.txt

nothing added to commit but untracked files present (use "git add" to track)

git_encoding_repo>git ls-files --eol file_name.txt

git_encoding_repo>git add .

git_encoding_repo>git status On branch master

No commits yet

Changes to be committed:
  (use "git rm --cached <file>..." to unstage)

        new file:   .gitattributes
        new file:   file_name.txt

git_encoding_repo>git ls-files --eol file_name.txt i/-text w/-text attr/text=auto file_name.txt

git_encoding_repo>git commit -m "Commit utf-16le encoded file which has crlf."
[master (root-commit) 91fe3bd] Commit utf-16le encoded file which has crlf.
 2 files changed, 13 insertions(+)
 create mode 100644 .gitattributes
 create mode 100644 file_name.txt

## At this time, i changed file encoding to utf-8.

git_encoding_repo>git status On branch master nothing to commit, working tree clean

git_encoding_repo>git add -p Only binary files changed.

git_encoding_repo>git status
On branch master
Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)

        modified:   file_name.txt

no changes added to commit (use "git add" and/or "git commit -a")

git_encoding_repo>git add file_name.txt

git_encoding_repo>git status
On branch master
Changes to be committed:
  (use "git reset HEAD <file>..." to unstage)

        modified:   file_name.txt

git_encoding_repo>git commit -m "Change encoding of file to utf-8"
[master 179c27b] Change encoding of file to utf-8
 1 file changed, 0 insertions(+), 0 deletions(-)
 rewrite file_name.txt (100%)

git_encoding_repo>git remote add origin https://github.com/ashishnegi/git_encoding.git

git_encoding_repo>git push -u origin master
Counting objects: 7, done.
Delta compression using up to 12 threads.
Compressing objects: 100% (7/7), done.
Writing objects: 100% (7/7), 837 bytes | 837.00 KiB/s, done.
Total 7 (delta 0), reused 0 (delta 0)
To https://github.com/ashishnegi/git_encoding.git
 * [new branch]      master -> master
Branch master set up to track remote branch master from origin.

git_encoding_repo>git ls-files --eol file_name.txt i/crlf w/crlf attr/text=auto file_name.txt

git_encoding_repo>