[PATCH v3 0/6] Abide by our own rules regarding line endings

Over the past decade, there have been a couple of attempts to remedy the situation regarding line endings (Windows/DOS line endings are traditionally CR/LF even if many current applications can handle LF, too, Linux/MacOSX/*BSD/Unix uses LF-only line handlings, and old MacOS software used to use CR-only line endings).

The current idea seems to be that the default line endings differ depending on the platform, and for files that should be exempt from that default, we strongly recommend using .gitattributes to define what the source code requires.

It is time to heed our own recommendation and mark the files that require LF-only line endings in our very own .gitattributes.

For starters, those files include shell scripts: the most prevalent shell interpreter in use (and certainly used in Git for Windows) is Bash, and Bash does not handle CR/LF line endings gracefully.

There are even two shell scripts that are used in the test suite even if they are not technically supposed to be part of core Git, as indicated by their habitat inside contrib/: git-new-workdir and git-completion.bash.

Related to shell scripts: when generating common-cmds.h, we use tools that generally operate on the assumption that input and output deliminate their lines using LF-only line endings. Consequently, they would happily copy the CR byte verbatim into the strings in common-cmds.h, which in turn makes the C preprocessor barf (that interprets them as MacOS-style line endings).

Further, the most obvious required fixes concern tests' support files committed verbatim, to be compared to Git's output, which is always LF-only. This includes the two files in t/diff-lib/ (which does not, in fact contain library functions as suggested by its name, but a copy of the README and the COPYING files, specifically for use in the tests).

There are a few SVN dump files, too, supporting the Subversion-related tests, requiring LF-only line endings.

Without these fixes, Git will fail to build and pass the test suite, as can be verified even on Linux using this cadence:

	git config core.autocrlf true
	rm .git/index && git stash
	make DEVELOPER=1 -j15 test

Note: I separated out the change marking t/t4051/* as LF-only into an
individual commit for one reason: it would appear that some grep builds
on Windows automagically convert CR/LF input into LF-only output. I went
the easy route to work around this issue, as I do not want to bother
changing MSYS2 grep's behavior.

Changes since v2:

- marked t/diff-lib/* as LF-only, dropping the ugly `tr -d "\015"` patch.

Johannes Schindelin (6):
  Fix build with core.autocrlf=true
  git-new-workdir: mark script as LF-only
  completion: mark bash script as LF-only
  t3901: move supporting files into t/t3901/
  Fix the remaining tests that failed with core.autocrlf=true
  t4051: mark supporting files as requiring LF-only line endings

 .gitattributes                           |  8 ++++++-
 contrib/completion/.gitattributes        |  1 +
 contrib/workdir/.gitattributes           |  1 +
 git-gui/.gitattributes                   |  1 +
 t/.gitattributes                         | 22 +++++++++++++++++-
 t/t0203-gettext-setlocale-sanity.sh      |  4 ++--
 t/t3901-i18n-patch.sh                    | 38 ++++++++++++++++----------------
 t/{t3901-8859-1.txt => t3901/8859-1.txt} |  0
 t/{t3901-utf8.txt => t3901/utf8.txt}     |  0
 t/t9350-fast-export.sh                   |  2 +-
 t/t9500-gitweb-standalone-no-errors.sh   |  4 ++--
 11 files changed, 55 insertions(+), 26 deletions(-)
 create mode 100644 contrib/completion/.gitattributes
 create mode 100644 contrib/workdir/.gitattributes
 rename t/{t3901-8859-1.txt => t3901/8859-1.txt} (100%)
 rename t/{t3901-utf8.txt => t3901/utf8.txt} (100%)

base-commit: 9b669787fc6ebc527df9ad058c4bcaf46bacc267
Published-As: https://github.com/dscho/git/releases/tag/lf-attrs-v3
Fetch-It-Via: git fetch https://github.com/dscho/git lf-attrs-v3

Interdiff vs v2:

 diff --git a/t/.gitattributes b/t/.gitattributes
 index bdd82cf31f7..3bd959ae523 100644
 --- a/t/.gitattributes
 +++ b/t/.gitattributes
 @@ -1,4 +1,5 @@
  t[0-9][0-9][0-9][0-9]/* -whitespace
 +/diff-lib/* eol=lf
  /t0110/url-* binary
  /t3900/*.txt eol=lf
  /t3901/*.txt eol=lf
 diff --git a/t/t4003-diff-rename-1.sh b/t/t4003-diff-rename-1.sh
 index c3e0a3c3fc9..df2accb6555 100755
 --- a/t/t4003-diff-rename-1.sh
 +++ b/t/t4003-diff-rename-1.sh
 @@ -11,7 +11,7 @@ test_description='More rename detection
  
  test_expect_success \
      'prepare reference tree' \
 -    'tr -d "\015" <"$TEST_DIRECTORY"/diff-lib/COPYING >COPYING &&
 +    'cat "$TEST_DIRECTORY"/diff-lib/COPYING >COPYING &&
       echo frotz >rezrov &&
      git update-index --add COPYING rezrov &&
      tree=$(git write-tree) &&
 @@ -99,7 +99,7 @@ test_expect_success \
  
  test_expect_success \
      'prepare work tree once again' \
 -    'tr -d "\015" <"$TEST_DIRECTORY"/diff-lib/COPYING >COPYING &&
 +    'cat "$TEST_DIRECTORY"/diff-lib/COPYING >COPYING &&
       git update-index --add --remove COPYING COPYING.1'
  
  # tree has COPYING and rezrov.  work tree has COPYING and COPYING.1,
 diff --git a/t/t4005-diff-rename-2.sh b/t/t4005-diff-rename-2.sh
 index f1641c35ee2..135addbfbda 100755
 --- a/t/t4005-diff-rename-2.sh
 +++ b/t/t4005-diff-rename-2.sh
 @@ -11,7 +11,7 @@ test_description='Same rename detection as t4003 but testing diff-raw.
  
  test_expect_success \
      'prepare reference tree' \
 -    'tr -d "\015" <"$TEST_DIRECTORY"/diff-lib/COPYING >COPYING &&
 +    'cat "$TEST_DIRECTORY"/diff-lib/COPYING >COPYING &&
       echo frotz >rezrov &&
      git update-index --add COPYING rezrov &&
      tree=$(git write-tree) &&
 @@ -71,7 +71,7 @@ test_expect_success \
  
  test_expect_success \
      'prepare work tree once again' \
 -    'tr -d "\015" <"$TEST_DIRECTORY"/diff-lib/COPYING >COPYING &&
 +    'cat "$TEST_DIRECTORY"/diff-lib/COPYING >COPYING &&
       git update-index --add --remove COPYING COPYING.1'
  
  git diff-index -C --find-copies-harder $tree >current
 diff --git a/t/t4007-rename-3.sh b/t/t4007-rename-3.sh
 index 0157fde5503..dae327fabbf 100755
 --- a/t/t4007-rename-3.sh
 +++ b/t/t4007-rename-3.sh
 @@ -11,7 +11,7 @@ test_description='Rename interaction with pathspec.
  
  test_expect_success 'prepare reference tree' '
  	mkdir path0 path1 &&
 -	tr -d "\015" <"$TEST_DIRECTORY"/diff-lib/COPYING >path0/COPYING &&
 +	cp "$TEST_DIRECTORY"/diff-lib/COPYING path0/COPYING &&
  	git update-index --add path0/COPYING &&
  	tree=$(git write-tree) &&
  	echo $tree
 diff --git a/t/t4008-diff-break-rewrite.sh b/t/t4008-diff-break-rewrite.sh
 index 5af4fa6aadb..9dd1bc5e162 100755
 --- a/t/t4008-diff-break-rewrite.sh
 +++ b/t/t4008-diff-break-rewrite.sh
 @@ -25,8 +25,8 @@ Further, with -B and -M together, these should turn into two renames.
  . "$TEST_DIRECTORY"/diff-lib.sh ;# test-lib chdir's into trash
  
  test_expect_success setup '
 -	tr -d "\015" <"$TEST_DIRECTORY"/diff-lib/README >file0 &&
 -	tr -d "\015" <"$TEST_DIRECTORY"/diff-lib/COPYING >file1 &&
 +	cat "$TEST_DIRECTORY"/diff-lib/README >file0 &&
 +	cat "$TEST_DIRECTORY"/diff-lib/COPYING >file1 &&
  	git update-index --add file0 file1 &&
  	git tag reference $(git write-tree)
  '