{"thread":{"id":"37022","subject":"[PATCH 2/2] t0027: combinations of core.autocrlf, core.eol and text","startedAt":"2014-06-29T06:34:54Z","lastAt":"2014-06-30T18:27:42Z","messageCount":2,"participants":["Torsten Bögershausen","Junio C Hamano"],"isPatch":true,"patchVersion":1,"patchTotal":2},"messages":[{"id":"245150","messageId":"53AFB38E.8000401@web.de","threadId":"37022","inReplyTo":null,"subject":"[PATCH 2/2] t0027: combinations of core.autocrlf, core.eol and text","fromName":"Torsten Bögershausen","fromEmail":"tboegi@web.de","sentAt":"2014-06-29T06:34:54Z","receivedAt":"2014-06-29T06:34:54Z","isPatch":true,"sender":{"key":"tboegi@web.de","avatar":"https://avatars.githubusercontent.com/u/7138363?v=4"},"body":"Historically there are 3 different parameters controlling how line endings\nare handled by Git:\n- core.autocrlf\n- core.eol\n- the \"text\" attribute in .gitattributes\n\nThere are different types of content:\n- (1) Files with only LF\n- (2) Files with only CRLF\n- (3) Files with mixed LF and CRLF\n- (4) Files with LF and/or CRLF with CR not followed by LF\n- (5) Files which are binary (e.g. have NUL bytes)\n\nRecently the question came up, how files with mixed EOLs are handled by Git\n(and libgit2) when they are checked out and core.autocrlf=true.\n\nSee\nhttp://git.661346.n2.nabble.com/The-different-EOL-behavior-between-libgit2-based-software-and-official-Git-td7613670.html#a7613801\n\nAdd the EXPENSIVE t0027-auto-crlf.sh to test all combination of files\nand parameters for both \"git add/commit\" and \"git checkout\".\n\nSigned-off-by: Torsten Bögershausen <tboegi@web.de>\n---\n t/t0027-auto-crlf.sh | 265 +++++++++++++++++++++++++++++++++++++++++++++++++++\n 1 file changed, 265 insertions(+)\n create mode 100755 t/t0027-auto-crlf.sh\n\ndiff --git a/t/t0027-auto-crlf.sh b/t/t0027-auto-crlf.sh\nnew file mode 100755\nindex 0000000..72dd3e8\n--- /dev/null\n+++ b/t/t0027-auto-crlf.sh\n@@ -0,0 +1,265 @@\n+#!/bin/sh\n+\n+test_description='CRLF conversion all combinations'\n+\n+. ./test-lib.sh\n+\n+if ! test_have_prereq EXPENSIVE\n+then\n+\tskip_all=\"EXPENSIVE not set\"\n+\ttest_done\n+fi\n+\n+\n+compare_files()\n+{\n+\tod -c <\"$1\" >\"$1\".expect &&\n+\tod -c <\"$2\" >\"$2\".actual &&\n+\ttest_cmp \"$1\".expect \"$2\".actual &&\n+\trm \"$1\".expect \"$2\".actual\n+}\n+\n+compare_ws_file()\n+{\n+\tpfx=$1\n+\texp=$2.expect\n+\tact=$pfx.actual.$3\n+\tod -c <\"$2\" >\"$exp\" &&\n+\tod -c <\"$3\" >\"$act\" &&\n+\ttest_cmp $exp $act &&\n+\trm $exp $act\n+}\n+\n+create_gitattributes()\n+{\n+\ttxtbin=$1\n+\tcase \"$txtbin\" in\n+\t\tauto)\n+\t\techo \"*.txt text=auto\" >.gitattributes\n+\t\t;;\n+\t\ttext)\n+\t\techo \"*.txt text\" >.gitattributes\n+\t\t;;\n+\t\t-text)\n+\t\techo \"*.txt -text\" >.gitattributes\n+\t\t;;\n+\t\t*)\n+\t\techo >.gitattributes\n+\t\t;;\n+\tesac\n+}\n+\n+create_file_in_repo()\n+{\n+\tcrlf=$1\n+\ttxtbin=$2\n+\tcreate_gitattributes \"$txtbin\" &&\n+\tfor f in LF CRLF LF_mix_CR CRLF_mix_LF CRLF_nul\n+\tdo\n+\t\tpfx=crlf_${crlf}_attr_${txtbin}_$f.txt &&\n+\t\tcp $f $pfx && git -c core.autocrlf=$crlf add $pfx\n+\tdone &&\n+\tgit commit -m \"core.autocrlf $crlf\"\n+}\n+\n+check_files_in_repo()\n+{\n+\tcrlf=$1\n+\ttxtbin=$2\n+\tlfname=$3\n+\tcrlfname=$4\n+\tlfmixcrlf=$5\n+\tlfmixcr=$6\n+\tcrlfnul=$7\n+\tpfx=crlf_${crlf}_attr_${txtbin}_ &&\n+\tcompare_files $lfname ${pfx}LF.txt &&\n+\tcompare_files $crlfname ${pfx}CRLF.txt &&\n+\tcompare_files $lfmixcrlf ${pfx}CRLF_mix_LF.txt &&\n+\tcompare_files $lfmixcr ${pfx}LF_mix_CR.txt &&\n+\tcompare_files $crlfnul ${pfx}CRLF_nul.txt\n+}\n+\n+\n+check_files_in_ws()\n+{\n+\teol=$1\n+\tcrlf=$2\n+\ttxtbin=$3\n+\tlfname=$4\n+\tcrlfname=$5\n+\tlfmixcrlf=$6\n+\tlfmixcr=$7\n+\tcrlfnul=$8\n+\tcreate_gitattributes $txtbin &&\n+\tgit config core.autocrlf $crlf &&\n+\tpfx=eol_${eol}_crlf_${crlf}_attr_${txtbin}_ &&\n+\tsrc=crlf_false_attr__ &&\n+\tfor f in LF CRLF LF_mix_CR CRLF_mix_LF CRLF_nul\n+\tdo\n+\t\trm $src$f.txt &&\n+\t\tif test -z \"$eol\"; then\n+\t\t\tgit checkout $src$f.txt\n+\t\telse\n+\t\t\tgit -c core.eol=$eol checkout $src$f.txt\n+\t\tfi\n+\tdone\n+\n+\n+\ttest_expect_success \"checkout core.eol=$eol core.autocrlf=$crlf gitattributes=$txtbin file=LF\" \"\n+\t\tcompare_ws_file $pfx $lfname    ${src}LF.txt\n+\t\"\n+\ttest_expect_success \"checkout core.eol=$eol core.autocrlf=$crlf gitattributes=$txtbin file=CRLF\" \"\n+\t\tcompare_ws_file $pfx $crlfname  ${src}CRLF.txt\n+\t\"\n+\ttest_expect_success \"checkout core.eol=$eol core.autocrlf=$crlf gitattributes=$txtbin file=CRLF_mix_LF\" \"\n+\t\tcompare_ws_file $pfx $lfmixcrlf ${src}CRLF_mix_LF.txt\n+\t\"\n+\ttest_expect_success \"checkout core.eol=$eol core.autocrlf=$crlf gitattributes=$txtbin file=LF_mix_CR\" \"\n+\t\tcompare_ws_file $pfx $lfmixcr   ${src}LF_mix_CR.txt\n+\t\"\n+\ttest_expect_success \"checkout core.eol=$eol core.autocrlf=$crlf gitattributes=$txtbin file=CRLF_nul\" \"\n+\t\tcompare_ws_file $pfx $crlfnul   ${src}CRLF_nul.txt\n+\t\"\n+}\n+\n+#######\n+(\n+\ttype od >/dev/null &&\n+\tprintf \"line1Q\\r\\nline2\\r\\nline3\" | q_to_nul >CRLF_nul &&\n+\tcat >expect <<-EOF &&\n+\t0000000 l i n e 1 \\0 \\r \\n l i n e 2 \\r \\n l\n+\t0000020 i n e 3\n+\t0000024\n+EOF\n+\tod -c CRLF_nul | sed -e \"s/[ \t][\t ]*/ /g\" -e \"s/ *$//\" >actual\n+\ttest_cmp expect actual &&\n+\trm expect actual\n+) || {\n+\t\tskip_all=\"od not found or od -c not usable\"\n+\t\texit 0\n+\t\ttest_done\n+}\n+\n+test_expect_success 'setup master' '\n+\techo >.gitattributes &&\n+\tgit checkout -b master &&\n+\tgit add .gitattributes &&\n+\tgit commit -m \"add .gitattributes\" \"\" &&\n+\tprintf \"line1\\nline2\\nline3\"     >LF &&\n+\tprintf \"line1\\r\\nline2\\r\\nline3\" >CRLF &&\n+\tprintf \"line1\\r\\nline2\\nline3\"   >CRLF_mix_LF &&\n+\tprintf \"line1\\nline2\\rline3\"     >LF_mix_CR &&\n+\tprintf \"line1\\r\\nline2\\rline3\"   >CRLF_mix_CR &&\n+\tprintf \"line1Q\\nline2\\nline3\" | q_to_nul >LF_nul\n+'\n+#  CRLF_nul had been created above\n+\n+test_expect_success 'create files' '\n+\tcreate_file_in_repo false \"\" &&\n+\tcreate_file_in_repo true  \"\" &&\n+\tcreate_file_in_repo input \"\" &&\n+\n+\tcreate_file_in_repo false \"auto\" &&\n+\tcreate_file_in_repo true  \"auto\" &&\n+\tcreate_file_in_repo input \"auto\" &&\n+\n+\tcreate_file_in_repo false \"text\" &&\n+\tcreate_file_in_repo true  \"text\" &&\n+\tcreate_file_in_repo input \"text\" &&\n+\n+\tcreate_file_in_repo false \"-text\" &&\n+\tcreate_file_in_repo true  \"-text\" &&\n+\tcreate_file_in_repo input \"-text\" &&\n+\trm -f *.txt &&\n+\tgit reset --hard\n+'\n+\n+test_expect_success 'commit empty gitattribues' '\n+\tcheck_files_in_repo false \"\"      LF CRLF CRLF_mix_LF LF_mix_CR CRLF_nul &&\n+\tcheck_files_in_repo true  \"\"      LF LF   LF          LF_mix_CR CRLF_nul &&\n+\tcheck_files_in_repo input \"\"      LF LF   LF          LF_mix_CR CRLF_nul\n+'\n+\n+test_expect_success 'commit text=auto' '\n+\tcheck_files_in_repo false \"auto\"  LF LF   LF          LF_mix_CR CRLF_nul &&\n+\tcheck_files_in_repo true  \"auto\"  LF LF   LF          LF_mix_CR CRLF_nul &&\n+\tcheck_files_in_repo input \"auto\"  LF LF   LF          LF_mix_CR CRLF_nul\n+'\n+\n+test_expect_success 'commit text' '\n+\tcheck_files_in_repo false \"text\"  LF LF   LF          LF_mix_CR LF_nul &&\n+\tcheck_files_in_repo true  \"text\"  LF LF   LF          LF_mix_CR LF_nul &&\n+\tcheck_files_in_repo input \"text\"  LF LF   LF          LF_mix_CR LF_nul\n+'\n+\n+test_expect_success 'commit -text' '\n+\tcheck_files_in_repo false \"-text\" LF CRLF CRLF_mix_LF LF_mix_CR CRLF_nul &&\n+\tcheck_files_in_repo true  \"-text\" LF CRLF CRLF_mix_LF LF_mix_CR CRLF_nul &&\n+\tcheck_files_in_repo input \"-text\" LF CRLF CRLF_mix_LF LF_mix_CR CRLF_nul\n+'\n+\n+################################################################################\n+# Check how files in the repo are changed when they are checked out\n+# How to read the table below:\n+# - check_files_in_ws will check multiple files, see below\n+# - parameter $1 : core.eol               lf | crlf\n+# - parameter $2 : core.autocrlf          false | true | input\n+# - parameter $3 : text in .gitattributs  \"\" (empty) | auto | text | -text\n+# - parameter $4 : reference for a file with only LF in the repo\n+# - parameter $5 : reference for a file with only CRLF in the repo\n+# - parameter $6 : reference for a file with mixed LF and CRLF in the repo\n+# - parameter $7 : reference for a file with LF and CR in the repo (does somebody uses this ?)\n+# - parameter $8 : reference for a file with CRLF and a NUL (should be handled as binary when auto)\n+\n+check_files_in_ws lf      false  \"\"       LF    CRLF  CRLF_mix_LF  LF_mix_CR    CRLF_nul\n+check_files_in_ws lf      true   \"\"       CRLF  CRLF  CRLF_mix_LF  LF_mix_CR    CRLF_nul\n+check_files_in_ws lf      input  \"\"       LF    CRLF  CRLF_mix_LF  LF_mix_CR    CRLF_nul\n+\n+check_files_in_ws lf      false \"auto\"    LF    CRLF  CRLF_mix_LF  LF_mix_CR    CRLF_nul\n+check_files_in_ws lf      true  \"auto\"    CRLF  CRLF  CRLF         LF_mix_CR    CRLF_nul\n+check_files_in_ws lf      input \"auto\"    LF    CRLF  CRLF_mix_LF  LF_mix_CR    CRLF_nul\n+\n+check_files_in_ws lf      false \"text\"    LF    CRLF  CRLF_mix_LF  LF_mix_CR    CRLF_nul\n+check_files_in_ws lf      true  \"text\"    CRLF  CRLF  CRLF         CRLF_mix_CR  CRLF_nul\n+check_files_in_ws lf      input \"text\"    LF    CRLF  CRLF_mix_LF  LF_mix_CR    CRLF_nul\n+\n+check_files_in_ws lf      false \"-text\"   LF    CRLF  CRLF_mix_LF  LF_mix_CR    CRLF_nul\n+check_files_in_ws lf      true  \"-text\"   LF    CRLF  CRLF_mix_LF  LF_mix_CR    CRLF_nul\n+check_files_in_ws lf      input \"-text\"   LF    CRLF  CRLF_mix_LF  LF_mix_CR    CRLF_nul\n+\n+###########\n+#core.autocrlf=input is forbidden with core.eol=crlf\n+check_files_in_ws crlf    false \"\"        LF    CRLF  CRLF_mix_LF  LF_mix_CR    CRLF_nul\n+check_files_in_ws crlf    true  \"\"        CRLF  CRLF  CRLF_mix_LF  LF_mix_CR    CRLF_nul\n+\n+check_files_in_ws crlf    false \"auto\"    CRLF  CRLF  CRLF         LF_mix_CR    CRLF_nul\n+check_files_in_ws crlf    true  \"auto\"    CRLF  CRLF  CRLF         LF_mix_CR    CRLF_nul\n+\n+check_files_in_ws crlf    false \"text\"    CRLF  CRLF  CRLF         CRLF_mix_CR  CRLF_nul\n+check_files_in_ws crlf    true  \"text\"    CRLF  CRLF  CRLF         CRLF_mix_CR  CRLF_nul\n+\n+check_files_in_ws crlf    false \"-text\"   LF    CRLF  CRLF_mix_LF  LF_mix_CR    CRLF_nul\n+check_files_in_ws crlf    true  \"-text\"   LF    CRLF  CRLF_mix_LF  LF_mix_CR    CRLF_nul\n+\n+if test_have_prereq MINGW\n+then\n+check_files_in_ws \"\"      false \"\"        LF    CRLF  CRLF_mix_LF  LF_mix_CR    CRLF_nul\n+check_files_in_ws \"\"      true  \"\"        CRLF  CRLF  CRLF_mix_LF  LF_mix_CR    CRLF_nul\n+check_files_in_ws \"\"      false \"auto\"    LF    CRLF  CRLF_mix_LF  LF_mix_CR    CRLF_nul\n+check_files_in_ws \"\"      true  \"auto\"    CRLF  CRLF  CRLF         LF_mix_CR    CRLF_nul\n+check_files_in_ws \"\"      false \"text\"    LF    CRLF  CRLF_mix_LF  LF_mix_CR    CRLF_nul\n+check_files_in_ws \"\"      true  \"text\"    CRLF  CRLF  CRLF         CRLF_mix_CR  CRLF_nul\n+check_files_in_ws \"\"      false \"-text\"   LF    CRLF  CRLF_mix_LF  LF_mix_CR    CRLF_nul\n+check_files_in_ws \"\"      true  \"-text\"   LF    CRLF  CRLF_mix_LF  LF_mix_CR    CRLF_nul\n+\n+check_files_in_ws native  false \"\"        LF    CRLF  CRLF_mix_LF  LF_mix_CR    CRLF_nul\n+check_files_in_ws native  true  \"\"        CRLF  CRLF  CRLF_mix_LF  LF_mix_CR    CRLF_nul\n+check_files_in_ws native  false \"auto\"    LF    CRLF  CRLF_mix_LF  LF_mix_CR    CRLF_nul\n+check_files_in_ws native  true  \"auto\"    CRLF  CRLF  CRLF         LF_mix_CR    CRLF_nul\n+check_files_in_ws native  false \"text\"    LF    CRLF  CRLF_mix_LF  LF_mix_CR    CRLF_nul\n+check_files_in_ws native  true  \"text\"    CRLF  CRLF  CRLF         CRLF_mix_CR  CRLF_nul\n+check_files_in_ws native  false \"-text\"   LF    CRLF  CRLF_mix_LF  LF_mix_CR    CRLF_nul\n+check_files_in_ws native  true  \"-text\"   LF    CRLF  CRLF_mix_LF  LF_mix_CR    CRLF_nul\n+fi\n+\n+test_done\n-- \n2.0.0.9631.g7e872d2\n"},{"id":"245213","messageId":"xmqq61jiz201.fsf@gitster.dls.corp.google.com","threadId":"37022","inReplyTo":"53AFB38E.8000401@web.de","subject":"Re: [PATCH 2/2] t0027: combinations of core.autocrlf, core.eol and text","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2014-06-30T18:27:42Z","receivedAt":"2014-06-30T18:27:42Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Torsten Bögershausen <tboegi@web.de> writes:\n\n> Historically there are 3 different parameters controlling how line endings\n> are handled by Git:\n> - core.autocrlf\n> - core.eol\n> - the \"text\" attribute in .gitattributes\n>\n> There are different types of content:\n> - (1) Files with only LF\n> - (2) Files with only CRLF\n> - (3) Files with mixed LF and CRLF\n> - (4) Files with LF and/or CRLF with CR not followed by LF\n> - (5) Files which are binary (e.g. have NUL bytes)\n>\n> Recently the question came up, how files with mixed EOLs are handled by Git\n> (and libgit2) when they are checked out and core.autocrlf=true.\n\nI have a slight suspicion that it may be better to leave behaviour\non some nonsensical combinations (e.g. 3 and 4) as \"unspecified\",\nwhich would mean we shouldn't be able to pick between\ntest_expect_success and test_expect_failure for some of them.\n\n> +test_description='CRLF conversion all combinations'\n> +\n> +check_files_in_ws()\n> +{\n\n(minor) Please have SP between \"ws\" and \"()\".\n\n\n> +(\n> +\ttype od >/dev/null &&\n> +\tprintf \"line1Q\\r\\nline2\\r\\nline3\" | q_to_nul >CRLF_nul &&\n> +\tcat >expect <<-EOF &&\n> +\t0000000 l i n e 1 \\0 \\r \\n l i n e 2 \\r \\n l\n> +\t0000020 i n e 3\n> +\t0000024\n> +EOF\n> +\tod -c CRLF_nul | sed -e \"s/[ \t][\t ]*/ /g\" -e \"s/ *$//\" >actual\n> +\ttest_cmp expect actual &&\n> +\trm expect actual\n> +) || {\n> +\t\tskip_all=\"od not found or od -c not usable\"\n> +\t\texit 0\n> +\t\ttest_done\n> +}\n\nI am not sure how cast-in-stone portable \"od -c\" output is across\nplatforms in practice.  You can use tr to convert NUL, CR and LF to\nsome distinctive and otherwise unused character, just like you use\nq_to_nul to map NUL to \"Q\" already.  And that would allow you to\nforget about such portability issues.\n\n\n> +test_expect_success 'setup master' '\n> +\techo >.gitattributes &&\n> +\tgit checkout -b master &&\n> +\tgit add .gitattributes &&\n> +\tgit commit -m \"add .gitattributes\" \"\" &&\n> +\tprintf \"line1\\nline2\\nline3\"     >LF &&\n> +\tprintf \"line1\\r\\nline2\\r\\nline3\" >CRLF &&\n> +\tprintf \"line1\\r\\nline2\\nline3\"   >CRLF_mix_LF &&\n> +\tprintf \"line1\\nline2\\rline3\"     >LF_mix_CR &&\n> +\tprintf \"line1\\r\\nline2\\rline3\"   >CRLF_mix_CR &&\n> +\tprintf \"line1Q\\nline2\\nline3\" | q_to_nul >LF_nul\n\nI do not see why you would want files that end with incomplete lines\nfor these.  Please have the line-end for the last line in them.\n"}]}