suggestion for improved docs on autocrlf

I hope this is the right mailing list, hope someone will redirect me if not... The git documentation (git help config) for core.autocrlf doesn't mention that false is a valid option; it only mentions true and input. Further, the docs for "input" are misleading, in that they lead the reader to assume it behaves like "false".

Would probably be nice if this were fixed :) Mark.

---------------------------------------------------------------------- This message, and any attachments, is for the intended recipient(s) only, may contain information that is privileged, confidential and/or proprietary and subject to important terms and conditions available at http://www.bankofamerica.com/emaildisclaimer. If you are not the intended recipient, please delete this message.

On Wed, Aug 07, 2019 at 01:08:22PM +0000, Yagnatinsky, Mark wrote:

Yes, you are at the right place, wellcome to the Git community.

This is somewhat common to all config settings: WHen a new feature is introduced, Git treats "config not defined" as false. This allows later Git versions to behave like older versions, when the config is not set. In that sense "not set" means the same as "false", and is the opposite to "true". (At least for core.autocrlf).

Later the third value "input" was introduced.

We can agree on that.

Yes, do I read this as "I will send a patch" ? That would be nice.

Probably not, but you can read it as "I will cook up better wording and reply to this thread"

---------------------------------------------------------------------- This message, and any attachments, is for the intended recipient(s) only, may contain information that is privileged, confidential and/or proprietary and subject to important terms and conditions available at http://www.bankofamerica.com/emaildisclaimer. If you are not the intended recipient, please delete this message.

Okay, my attempt at better wording for the docs is not going well, because it turns I that I still don't understand the behavior here! I thought that "input" means that CRLF will become LF on "git add" but that seems to be true only sometimes. For instance, consider the following 11-line shell script:

mkdir empty cd empty git init git config core.autocrlf false echo -e '1\r\n2' > test.txt git add . git commit -m test git config core.autocrlf input cp test.txt t2.txt echo 'a\r\nb' > test.txt git add .

The output from the last git add is:

warning: CRLF will be replaced by LF in t2.txt. The file will have its original line endings in your working directory.

Which is a very good warning indeed, but why does it only apply to the new file t2.txt? Why does the existing file test.txt retain its CRLF line endings.

I am so confused.

Any pointers?

---------------------------------------------------------------------- This message, and any attachments, is for the intended recipient(s) only, may contain information that is privileged, confidential and/or proprietary and subject to important terms and conditions available at http://www.bankofamerica.com/emaildisclaimer. If you are not the intended recipient, please delete this message.

On Thu, Aug 08, 2019 at 11:08:14PM +0000, Yagnatinsky, Mark wrote:

Because there are no CRLF ;-)

This line echo 'a\r\nb' > test.txt should probably be written as echo -e 'a\r\nb' > test.txt or better printf 'a\r\nb' > test.txt

Please run

od -c test.txt and may be git ls-files --eol

https://git-scm.com/docs/git-add

What happens if you run the following: mkdir empty cd empty git init echo "* -text" >.gitattributes printf '1\r\n2' > test.txt git add . git commit -m "Add test.txt with CRLF"

echo "* text=auto eol=lf" >.gitattributes cp test.txt t2.txt printf 'a\r\nb' > test.txt git add .

git commit -m "cp text.txt t2.txt; change test.txt; eol=input"

git add --renormailze test.txt git commit -m "Normalize line endings in text.txt"

After correcting spelling of renormalize, the end result of the script you gave is that line endings in working directory are CRLF, and in the repo are LF. Is that expected? Surprising? Not sure what you were trying to test there. I also fixed my script to use printf, new version is: (using copy/paste this time, instead typing from memory like last time; learned my lesson).

mkdir empty cd empty git init git config core.autocrlf false printf '1\r\n2' > test.txt git add . git commit -m test git config core.autocrlf input cp test.txt t2.txt printf 'a\r\nb' > test.txt git add .

Thank you for ls-files, I've been resorting to deleting from working directory and then doing a checkout. And also thank you for the gem of "od -c"! The output of ls-files after running my script is:

i/lf w/crlf attr/ t2.txt i/crlf w/crlf attr/ test.txt

So I'm still confused.  Why didn't test.txt lose its CRs while t2.txt did?

---------------------------------------------------------------------- This message, and any attachments, is for the intended recipient(s) only, may contain information that is privileged, confidential and/or proprietary and subject to important terms and conditions available at http://www.bankofamerica.com/emaildisclaimer. If you are not the intended recipient, please delete this message.

On Fri, Aug 09, 2019 at 03:34:05PM +0000, Yagnatinsky, Mark wrote:

Yes. "git add" does typically not touch the files on disk at all.

May be. Typically the CRLF is done in the .gitattributes file, which is commited and pushed/pulled between the different machines. In opposite, core.autocrlf is just local, and different contributors may use different settings, leading to all kind of CRLF frustrations.

Having said that, the .gitattributes file is established early in the lifetime of a project (=repo). I typically set up the .gitattributes file as the first commit, before adding any other files.

If, however, the .gitattributes file is added to a repo later (better late than never), and the developer messes it up for some reasons, then the original files are still in the working tree on disk.

Correcttions can be made after looking at 'git ls-files --eol' or other diagnostics.

There ? I probably lost the script.

This is because the "auto" feature does not change the line endings, when a file had been commited with CRLF. They may be there on purpose, or not. Git can not make this decision - the user can.

This is why we have 'git add --renormalize".

Wait a second... suppose a file is committed with CRLF line endings. You're saying that even if I have autocrlf set to "input" or "auto", the file will never get "converted" to LF format unless I explicitly renormalize? That sounds like a fairly sensible behavior, but it's not what I've observed in the past at all! There have been plenty of times when I had "autocrlf" set to input which resulted in me changing line endings on commit I had no intention of changing! Indeed, the whole reason I was looking at the git docs recently is that this happened again and I was trying to make it stop happening! Or is that not what you meant?

---------------------------------------------------------------------- This message, and any attachments, is for the intended recipient(s) only, may contain information that is privileged, confidential and/or proprietary and subject to important terms and conditions available at http://www.bankofamerica.com/emaildisclaimer. If you are not the intended recipient, please delete this message.

"Yagnatinsky, Mark" <mark.yagnatinsky@bofa.com> writes:

Isn't that merely an effect of the fact that git add does not bother with paths it knows unmodified, and the decision it makes if a path is modified does not involve if the path's contents predate what you have in the current .gitattributes and .git/config?

If you had committed two paths with CRLF, checked them out, flipped the attribute so that they would be converted upon input, modified one of them but not the other, and say "git add .", because the relative age of .gitattributes and file contents do not get in the picture, "git add ." would say "Ah, you have one modified file, so we need to add its new contents to the index. And you are telling me to convert its line endings to LF while doing so."

But I don't even have a git attributes file! There's no attribute flipping going on, I think. The CRLF'd file was committed by someone else on my team, who probably has git configured differently than I do. Or am I missing the point?

---------------------------------------------------------------------- This message, and any attachments, is for the intended recipient(s) only, may contain information that is privileged, confidential and/or proprietary and subject to important terms and conditions available at http://www.bankofamerica.com/emaildisclaimer. If you are not the intended recipient, please delete this message.

On Mon, Aug 12, 2019 at 01:47:18PM +0000, Yagnatinsky, Mark wrote:

Yes.

Only changing core.autocrlf to input will not change the line endings in the repo. That is intentional and allows to to keep core.autocrlf and jump force-and-back in the history by checking out older versions or later versions.

That is why I would recommend a .gitattributes file, which travels with the commits and through push and pull.

My feeling is that the docemntation for core.autocrlf should be: This setting is deprecated. Use a .gitattributes file instead, and please see the documentation. But that is clearly debatable.

Okay, I feel like I'm on the verge of understanding, but it keeps eluding me, because you keep answering the question I actually asked, rather than the one I should have asked... let me try again, and bear with me if it seems like I'm repeating the same question over and over, because I don't understand which "irrelevant" differences matter and which don't. (You've been amazingly patient so far actually. Thank you for that.)

(Incidentally, you've successfully convinced me that attributes are a far better way to deal with this. I still feel that the behavior should be documented though.)

Setup: Suppose there exists a repo where sample.txt has CRLF line endings.
Meanwhile, my git config has autocrlf as "input".  Now, I clone the repo.
I edit sample.txt, and then commit.  Assuming the repo has no .gitattributes,
is it possible to predict what line endings sample.txt will end up with in my repo?
Or does it depend on more information than what I've just written?

---------------------------------------------------------------------- This message, and any attachments, is for the intended recipient(s) only, may contain information that is privileged, confidential and/or proprietary and subject to important terms and conditions available at http://www.bankofamerica.com/emaildisclaimer. If you are not the intended recipient, please delete this message.

"Yagnatinsky, Mark" <mark.yagnatinsky@bofa.com> writes:

Binary packagers can ship custom attributes and config that applies globally to the installation, which may affect operations in all of your repositories, can't they?

Okay, let me be more explicit :) Suppose "git check-attr -a ." produces no output. Now, is predictable?

-----Original Message-----
From: git-owner@vger.kernel.org [mailto:git-owner@vger.kernel.org] On Behalf Of Junio C Hamano
Sent: Monday, August 12, 2019 2:19 PM
To: Yagnatinsky, Mark <mark.yagnatinsky@bofa.com>
Cc: 'Torsten Bögershausen' <tboegi@web.de>; 'git@vger.kernel.org' <git@vger.kernel.org>
Subject: Re: suggestion for improved docs on autocrlf

"Yagnatinsky, Mark" <mark.yagnatinsky@bofa.com> writes:

Binary packagers can ship custom attributes and config that applies globally to the installation, which may affect operations in all of your repositories, can't they?

---------------------------------------------------------------------- This message, and any attachments, is for the intended recipient(s) only, may contain information that is privileged, confidential and/or proprietary and subject to important terms and conditions available at http://www.bankofamerica.com/emaildisclaimer. If you are not the intended recipient, please delete this message.

On Mon, Aug 12, 2019 at 11:18:35AM -0700, Junio C Hamano wrote:

Let me try to split my answer into 3 parts:
- A Git distribution can ship with a global attributes file, but I am not aware of any
  distribution doing this.
  Especially Git for Windpws does not add any (global) .gitattributes file somewhere,
  as far as I know.
- The default value for the global core.autocrlf is typically unset in all
  Unix-like distributions, which does the same as core.autocrlf=false.
- The installer for Git for Windows allows the user to choose what the global
  core.autocrlf should be (false, true, input).
  The help text of the installer mentions what
  e.g. "false" means (line endings "as is"),
  and I don't rember the text for "true" and "input"  out of my head.

Now back to the original question:

Person A creates a repo with core.autocrlf=false and commite sample.txt with CRLF. Person B clones that repo with CRLF in the repo. When the worktree is checked out, sample.txt will have CRLF. Always.

Even if B uses core.autocrlf=input, sample.txt will keep the CRLF when it is commited. If the line endings need to be changed, there are 2 ways: Either user B runs dos2unix and commits file with LF. Or user B runs `git add --renormalize` sample.txt. That needs a `rm sample.txt && git checkout sample.txt` to update the working tree.

All this is predictable. A fresh clone does not have any "modified files"; all files with CRLF stay with CLRF unless they are normalized.

Our documentation mentions "files that any contributor introduces to the repository"

Is that part understandalble and the distincion between "existing" and "new" files clear enough ?

Thank you once more. Finally, I believe I understood everything you said. I was about to say that this contradicts my own experience. But then I remembered that I normally use my IDE rather than the command line. And I just checked that indeed that the behavior of my IDE is totally different! It renormalizes line endings of existing files whenever autocrlf is set to input. Sigh. Okay, great, life makes sense again. I want to yell at my IDE now.

I now feel brave enough to attempt to come up with better wording for autocrlf docs, if you think that's worth trying.

---------------------------------------------------------------------- This message, and any attachments, is for the intended recipient(s) only, may contain information that is privileged, confidential and/or proprietary and subject to important terms and conditions available at http://www.bankofamerica.com/emaildisclaimer. If you are not the intended recipient, please delete this message.

On Tue, Aug 13, 2019 at 03:31:43PM +0000, Yagnatinsky, Mark wrote:

Would you like to elobarate which IDE that is? The core.autocrlf handling has been improved a couple of times, and it seams that your IDE does things different from git.git (Which is not ideal)

That would be good, I am happy to review patches.

IntelliJ from JetBrains.  I'm planning on emailing them to at least mention this somewhere in their own docs, because I think it's not there.

Will attempt revised wording soon-ish, but not promising today.

---------------------------------------------------------------------- This message, and any attachments, is for the intended recipient(s) only, may contain information that is privileged, confidential and/or proprietary and subject to important terms and conditions available at http://www.bankofamerica.com/emaildisclaimer. If you are not the intended recipient, please delete this message.

Turns out that this is not intentional behavior of IntelliJ but an unfortunate interaction with a bug in git. Had I been using a more recent version of git, it wouldn't be happening. https://youtrack.jetbrains.com/issue/IDEA-205601

Patched docs on the way eventually.  Hopefully it's okay if my syntax is a bit off, I don't know AsciiDoc.

---------------------------------------------------------------------- This message, and any attachments, is for the intended recipient(s) only, may contain information that is privileged, confidential and/or proprietary and subject to important terms and conditions available at http://www.bankofamerica.com/emaildisclaimer. If you are not the intended recipient, please delete this message.

Okay, first attempt at better phrasing. This may need more paragraph breaks, or something. Right now it's very wall-of-texty. And probably in a style way too different from the rest of the git docs. Also, the syntax is probably closer to markdown than AsciiDoc; sorry. Anyway, enough disclaimers, here's the first draft:

This variable has three valid settings: true, input, and false.
(Leaving it unset is equivalent to setting it to "false".)
1. Set it to "true" if you want to have CRLF line endings in your 
working directory and the repository has LF line endings. 
Setting it to true is the same as setting the `text` attribute to
"auto" on all files and core.eol to "crlf".  In other words: any file
that has LF line endings in the repository will have CRLF line
endings in your working directory.  If you  commit a new file to
the repository, then git will commit it with LF line endings, even
if it has CRLF endings in your working  directory.  However, if you
edit an existing file that has CRLF line endings in the repository,
then git will not convert it to LF line endings when you commit it.

2. If you set it to "input" then git will not do any line ending conversions
when checking files out of the repository into your working directory.
That is, immediately after a checkout, the line endings in your working
directory will match those in the repository.  When committing a new
file to the repository, git will commit it with LF line endings, even if it has
CRLF line endings in your working directory.  If you edit an existing file in
the repository, then:
        * If the file had LF line endings in the repository, it will still have them,
            no matter what line endings are in the working directory.
        * If the file has LF line endings in the working directory, then it will be
            committed with LF line endings , no matter what line endings it used
            to have in the repository.
If neither of the above two cases apply, (in other words, if the file has CRLF
endings in the repository and in also in the working directory), then it will be
committed with CRLF line endings.

3. The simplest setting to explain is "false".  In this setting, git will not
perform any line ending conversion; all files will be checked out into
the working directory exactly as they are in the repo, and will be
committed to the repo exactly as they are in the working directory.
This setting is recommended; if you are tempted to use "input" or "true"
instead of "false", then consider looking into committing a .gitattributes file
into your repository instead.  Settings in that file override this configuration
variable, and since the same attributes file is being used by everyone who
works on the repo, the results end up being more predictable.

Or something like that.

---------------------------------------------------------------------- This message, and any attachments, is for the intended recipient(s) only, may contain information that is privileged, confidential and/or proprietary and subject to important terms and conditions available at http://www.bankofamerica.com/emaildisclaimer. If you are not the intended recipient, please delete this message.

On Thu, Aug 15, 2019 at 04:56:05AM +0000, Yagnatinsky, Mark wrote:

Thanks so much for the suggestion.

Feedback and Input like this from Git users are really appreciated. I will re-read it a couple of times, and probably find some time to make a patch out of it within the next days (or weeks).

Yay!  Feel free to tweak and/or butcher the wording, I won't be offended :)

-----Original Message-----
From: Torsten Bögershausen [mailto:tboegi@web.de] 
Sent: Friday, August 16, 2019 12:20 AM
To: Yagnatinsky, Mark <mark.yagnatinsky@bofa.com>
Cc: 'Junio C Hamano' <gitster@pobox.com>; 'git@vger.kernel.org' <git@vger.kernel.org>
Subject: Re: suggestion for improved docs on autocrlf

On Thu, Aug 15, 2019 at 04:56:05AM +0000, Yagnatinsky, Mark wrote:

Thanks so much for the suggestion.

Feedback and Input like this from Git users are really appreciated. I will re-read it a couple of times, and probably find some time to make a patch out of it within the next days (or weeks).

---------------------------------------------------------------------- This message, and any attachments, is for the intended recipient(s) only, may contain information that is privileged, confidential and/or proprietary and subject to important terms and conditions available at http://www.bankofamerica.com/emaildisclaimer. If you are not the intended recipient, please delete this message.

Torsten Bögershausen <tboegi@web.de> writes:

Good.  Thanks for a good discussion.