git blame swallows up lines in case of mixed line endings

Hi Folks,

I encounter unexpected behavior in the following case:

file content:

line1<CR><LF> line2<CR> line3<CR><LF> line4

This is what I get as console output (on Windows):

7db36436 1) line1 line3436 2) line2 7db36436 3) line4

This is the real content:

blame.txt opened in Notepad++:

7db36436 1) line1 <CR><LF> 7db36436 2) line2 <CR> line3 <CR><LF> 7db36436 3) line4 <LF>

Admittedly, very stupid editors, such as Windows Notepad, cannot handle mixed line endings as well. But is this also the way git blame should behave?

Kind regards Konstantin

On 2015-02-19 14.48, Sokolov, Konstantin (ext) wrote:

Git (and other tools) do not handle a single CR as a line ending. A line ending in Git is either CRLF or LF.

A "mixed line ending" in Git sense is a mixture between CRLF and LF, but not a lone CR. (And in this sense it is the expected behavior)

Are you sure you want to use it, or is it a typo ?

It's not that I want to use CR as line ending. We just had to process a repository where it was the case. We use git blame as an automatic processing step in a bigger system. So if a line does not exist for git blame but it does exist for other parts of the system (which read the same file) we run into consistency problems. 

-----Ursprüngliche Nachricht-----
Von: Torsten Bögershausen [mailto:tboegi@web.de] 
Gesendet: Samstag, 21. Februar 2015 14:46
An: Sokolov, Konstantin (ext); 'git@vger.kernel.org'
Betreff: Re: git blame swallows up lines in case of mixed line endings

On 2015-02-19 14.48, Sokolov, Konstantin (ext) wrote:

Git (and other tools) do not handle a single CR as a line ending. A line ending in Git is either CRLF or LF.

A "mixed line ending" in Git sense is a mixture between CRLF and LF, but not a lone CR. (And in this sense it is the expected behavior)

Are you sure you want to use it, or is it a typo ?

Torsten Bögershausen <tboegi@web.de> writes:

You can mark a file as <CRLF> terminated via attributes system and have Git convert them to use <LF> as end-of-line when file contents are stored in Git ("<LF> as end-of-line" is the representation "git blame" uses internally). Konstantin said "on Windows", and I guessed initially that the lines are marked as such, but after looking at the blame.txt output I am not sure.

That means the contents of the lines are:

    First Line:  "line1"
    Second Line: "line2" + CR + "line3"
    Third Line:  "line4"

or if CRLF conversion is not specified in Konstantin's repository:

    First Line:  "line1" + CR
    Second Line: "line2" + CR + CR + "line3" + CR
    Third Line:  "line4" + CR

Either way, that makes the observed behavior totally expected and understandable.

As printing CR moves the cursor to the beginning of line on many terminals, it is understandable to see the above output. After printing the first line, it will show

    7db36436 2) line2

and then go back to the leftmost column and then overwrite 7db3... with "line3".

Even though we have an option to mark <CR> alone as the end of line marker, because the blamed content can go through the textconv filter, it may be possible to define a textconv filter for the path via the attribute system and convert such "mixed line endings" contents to a series of <LF>-terminated lines. That would split the second line in the original input into two lines and may produce desired result.

Thank you for going into the matter. I was not aware of the textconv filter. This is definitely a decent solution. But what exactly do you mean by "Even though we have an option to mark <CR> alone as the end of line marker"? Is there really such an option in Git or did you mean that it's possible in general (by changing the implementation)?

Kind Regards Konstantin

-----Ursprüngliche Nachricht-----
Von: Junio C Hamano [mailto:gitster@pobox.com] 
Gesendet: Montag, 23. Februar 2015 19:45
An: Torsten Bögershausen
Cc: Sokolov, Konstantin (ext); 'git@vger.kernel.org'
Betreff: Re: git blame swallows up lines in case of mixed line endings

Torsten Bögershausen <tboegi@web.de> writes:

You can mark a file as <CRLF> terminated via attributes system and have Git convert them to use <LF> as end-of-line when file contents are stored in Git ("<LF> as end-of-line" is the representation "git blame" uses internally).  Konstantin said "on Windows", and I guessed initially that the lines are marked as such, but after looking at the blame.txt output I am not sure.

That means the contents of the lines are:

    First Line:  "line1"
    Second Line: "line2" + CR + "line3"
    Third Line:  "line4"

or if CRLF conversion is not specified in Konstantin's repository:

    First Line:  "line1" + CR
    Second Line: "line2" + CR + CR + "line3" + CR
    Third Line:  "line4" + CR

Either way, that makes the observed behavior totally expected and understandable.

As printing CR moves the cursor to the beginning of line on many terminals, it is understandable to see the above output.  After printing the first line, it will show 

    7db36436 2) line2

and then go back to the leftmost column and then overwrite 7db3... with "line3".

Even though we have an option to mark <CR> alone as the end of line marker, because the blamed content can go through the textconv filter, it may be possible to define a textconv filter for the path via the attribute system and convert such "mixed line endings" contents to a series of <LF>-terminated lines. That would split the second line in the original input into two lines and may produce desired result.

"Sokolov, Konstantin (ext)" <konstantin.sokolov.ext@siemens.com> writes:

s/have/do not have/; sorry for a false hope ;-)