{"thread":{"id":"42304","subject":"Bug report: Duplicate CRLF rewrite warnings on commit","startedAt":"2016-05-13T13:49:53Z","lastAt":"2016-06-07T15:20:51Z","messageCount":53,"participants":["Adam Dinwoodie","Junio C Hamano","Jeff King","Torsten Bögershausen","tboegi@web.de","Eric Sunshine"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"286459","messageId":"20160513134953.GE2345@dinwoodie.org","threadId":"42304","inReplyTo":null,"subject":"Bug report: Duplicate CRLF rewrite warnings on commit","fromName":"Adam Dinwoodie","fromEmail":"adam@dinwoodie.org","sentAt":"2016-05-13T13:49:53Z","receivedAt":"2016-05-13T13:49:53Z","isPatch":false,"sender":{"key":"adam@dinwoodie.org","avatar":"https://avatars.githubusercontent.com/u/1397507?v=4"},"body":"If you use .gitattributes to enable CRLF->LF rewriting, then commit a\nfile that would have its line endings rewritten, the \"CRLF will be\nreplaced by LF\" warning is printed several times over; I'd expect it to\nbe printed only once.\n\nThere's a test case in t0020 -- \"safecrlf: print warning only once\" --\nthat checks the warning is only printed once when using `git add`, but\nnothing that seems to check the same thing on `git commit`. The\nunnecessary multiple warnings seem to have existed since at least v1.6.0\n(the warnings appear to have been added in v1.5.5 in 21e5ad5, but I\ncouldn't get that to build on my current box), and I'm seeing them on my\nCygwin box's build of the current next branch (d10caa2) and on my CentOS\nbox's v2.8.1 release.\n\nExample:\n\n $ git init\n Initialized empty Git repository in /home/Adam/test/.git/\n\n $ echo '* text' >.gitattributes\n\n $ echo 'lf-terminated line' >text\n\n $ git add .gitattributes text && git commit -m 'Initial commit'\n [master (root-commit) 9a18d39] Initial commit\n 2 files changed, 2 insertions(+)\n create mode 100644 .gitattributes\n create mode 100644 text\n\n $ echo 'crlf-terminated line' | unix2dos >text\n\n $ git add text # Single CRLF warning as expected\n warning: CRLF will be replaced by LF in text.\n The file will have its original line endings in your working directory.\n\n $ git commit -m 'CRLF' # Should have one CRLF warning, actually get two\n warning: CRLF will be replaced by LF in text.\n The file will have its original line endings in your working directory.\n [master 4a8b1cb] CRLF\n warning: CRLF will be replaced by LF in text.\n The file will have its original line endings in your working directory.\n 1 file changed, 1 insertion(+), 1 deletion(-)\n\n(Tangentially: what's the accepted practice for submitting failing test\nscripts? I've written a short test case to add to t0020 that shows this\nbugged behaviour, but I've got the vague impression from past emails\nthat leading with the patch email adding the failing test case is not\nthe expected way to do things on this list...)\n\nCheers,\n\nAdam\n"},{"id":"286465","messageId":"xmqqinyhaoxn.fsf@gitster.mtv.corp.google.com","threadId":"42304","inReplyTo":"20160513134953.GE2345@dinwoodie.org","subject":"Re: Bug report: Duplicate CRLF rewrite warnings on commit","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-05-13T16:43:48Z","receivedAt":"2016-05-13T16:43:48Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Adam Dinwoodie writes:\n\n> If you use .gitattributes to enable CRLF->LF rewriting, then commit a\n> file that would have its line endings rewritten, the \"CRLF will be\n> replaced by LF\" warning is printed several times over; I'd expect it to\n> be printed only once.\n>\n> There's a test case in t0020 -- \"safecrlf: print warning only once\" --\n> that checks the warning is only printed once when using `git add`, but\n> nothing that seems to check the same thing on `git commit`. The\n> unnecessary multiple warnings seem to have existed since at least v1.6.0\n> (the warnings appear to have been added in v1.5.5 in 21e5ad5, but I\n> couldn't get that to build on my current box), and I'm seeing them on my\n> Cygwin box's build of the current next branch (d10caa2) and on my CentOS\n> box's v2.8.1 release.\n\nTorsten, I know you are not heavily invested in \"commit\" codepath,\nbut since you've been actively touching the CRLF area, I wonder\nperhaps you might be interested in taking a look?\n\n>\n> Example:\n>\n> $ git init\n> Initialized empty Git repository in /home/Adam/test/.git/\n>\n> $ echo '* text' >.gitattributes\n>\n> $ echo 'lf-terminated line' >text\n>\n> $ git add .gitattributes text && git commit -m 'Initial commit'\n> [master (root-commit) 9a18d39] Initial commit\n> 2 files changed, 2 insertions(+)\n> create mode 100644 .gitattributes\n> create mode 100644 text\n>\n> $ echo 'crlf-terminated line' | unix2dos >text\n>\n> $ git add text # Single CRLF warning as expected\n> warning: CRLF will be replaced by LF in text.\n> The file will have its original line endings in your working directory.\n>\n> $ git commit -m 'CRLF' # Should have one CRLF warning, actually get two\n> warning: CRLF will be replaced by LF in text.\n> The file will have its original line endings in your working directory.\n> [master 4a8b1cb] CRLF\n> warning: CRLF will be replaced by LF in text.\n> The file will have its original line endings in your working directory.\n> 1 file changed, 1 insertion(+), 1 deletion(-)\n>\n> (Tangentially: what's the accepted practice for submitting failing test\n> scripts? I've written a short test case to add to t0020 that shows this\n> bugged behaviour, but I've got the vague impression from past emails\n> that leading with the patch email adding the failing test case is not\n> the expected way to do things on this list...)\n>\n> Cheers,\n>\n> Adam\n"},{"id":"286474","messageId":"20160513181255.GA30700@sigill.intra.peff.net","threadId":"42304","inReplyTo":"20160513134953.GE2345@dinwoodie.org","subject":"Re: Bug report: Duplicate CRLF rewrite warnings on commit","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2016-05-13T18:12:55Z","receivedAt":"2016-05-13T18:12:55Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Fri, May 13, 2016 at 02:49:53PM +0100, Adam Dinwoodie wrote:\n\n> (Tangentially: what's the accepted practice for submitting failing test\n> scripts? I've written a short test case to add to t0020 that shows this\n> bugged behaviour, but I've got the vague impression from past emails\n> that leading with the patch email adding the failing test case is not\n> the expected way to do things on this list...)\n\nWe don't want commits that fail the test suite, since it makes bisection\nmore difficult. But you can mark known bugs like:\n\n test_expect_failure 'git-foo should output bar' '\n\t...\n '\n\nI think it's OK to submit a patch like that. Hopefully somebody picks\nthat up and combines it with their fix patch, but if not, then it at\nleast documents the failure for later generations.\n\n-Peff\n"},{"id":"286483","messageId":"xmqq1t55agge.fsf@gitster.mtv.corp.google.com","threadId":"42304","inReplyTo":"20160513181255.GA30700@sigill.intra.peff.net","subject":"Re: Bug report: Duplicate CRLF rewrite warnings on commit","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-05-13T19:46:57Z","receivedAt":"2016-05-13T19:46:57Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jeff King writes:\n\n> On Fri, May 13, 2016 at 02:49:53PM +0100, Adam Dinwoodie wrote:\n>\n>> (Tangentially: what's the accepted practice for submitting failing test\n>> scripts? I've written a short test case to add to t0020 that shows this\n>> bugged behaviour, but I've got the vague impression from past emails\n>> that leading with the patch email adding the failing test case is not\n>> the expected way to do things on this list...)\n>\n> We don't want commits that fail the test suite, since it makes bisection\n> more difficult. But you can mark known bugs like:\n>\n> test_expect_failure 'git-foo should output bar' '\n> \t...\n> '\n>\n> I think it's OK to submit a patch like that. Hopefully somebody picks\n> that up and combines it with their fix patch, but if not, then it at\n> least documents the failure for later generations.\n\n... but we do not want to overdo this.\n\nAn expect_failure helps when somebody is actually starting to work\non a breakage by setting a goal, but otherwise nobody would look at\nthem. I do not think we know if 120+ instances of expect_failure in\nt/ we already have are still all valid; some of them even might be\nrequesting a behaviour that we decided is a wrong expectation later.\n"},{"id":"286489","messageId":"20160513195347.GD9890@sigill.intra.peff.net","threadId":"42304","inReplyTo":"xmqq1t55agge.fsf@gitster.mtv.corp.google.com","subject":"Re: Bug report: Duplicate CRLF rewrite warnings on commit","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2016-05-13T19:53:48Z","receivedAt":"2016-05-13T19:53:48Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Fri, May 13, 2016 at 12:46:57PM -0700, Junio C Hamano wrote:\n\n> > We don't want commits that fail the test suite, since it makes bisection\n> > more difficult. But you can mark known bugs like:\n> >\n> > test_expect_failure 'git-foo should output bar' '\n> > \t...\n> > '\n> >\n> > I think it's OK to submit a patch like that. Hopefully somebody picks\n> > that up and combines it with their fix patch, but if not, then it at\n> > least documents the failure for later generations.\n> \n> ... but we do not want to overdo this.\n> \n> An expect_failure helps when somebody is actually starting to work\n> on a breakage by setting a goal, but otherwise nobody would look at\n> them. I do not think we know if 120+ instances of expect_failure in\n> t/ we already have are still all valid; some of them even might be\n> requesting a behaviour that we decided is a wrong expectation later.\n\nTrue. I almost wrote more on this, but held back. But since you seem\ninterested... :)\n\nI think that it's a really nice thing to write up your test as\ntest_expect_failure and post it to the list as a patch that is ready to\nbe applied, _even if we don't apply it_. Because that format makes it\nsuper-easy for somebody who does want to work on it to just \"git am\"\nyour patch and focus on fixing it.\n\nAnd your patch can either go at the front of their series, or they\ncan squash it in and acknowledge you in the commit message. But either\nway it reduces the amount of work they have to do (whether they work on\nthe bug soon, or much later).\n\n-Peff\n"},{"id":"286523","messageId":"998a4fcd-ecd8-7e0c-860e-7f12ec02b1f9@web.de","threadId":"42304","inReplyTo":"xmqqinyhaoxn.fsf@gitster.mtv.corp.google.com","subject":"Re: Bug report: Duplicate CRLF rewrite warnings on commit","fromName":"Torsten Bögershausen","fromEmail":"tboegi@web.de","sentAt":"2016-05-14T05:40:11Z","receivedAt":"2016-05-14T05:40:11Z","isPatch":false,"sender":{"key":"tboegi@web.de","avatar":"https://avatars.githubusercontent.com/u/7138363?v=4"},"body":"On 13.05.16 18:43, Junio C Hamano wrote:\n> Adam Dinwoodie writes:\n> \n>> If you use .gitattributes to enable CRLF->LF rewriting, then commit a\n>> file that would have its line endings rewritten, the \"CRLF will be\n>> replaced by LF\" warning is printed several times over; I'd expect it to\n>> be printed only once.\n>>\n>> There's a test case in t0020 -- \"safecrlf: print warning only once\" --\n>> that checks the warning is only printed once when using `git add`, but\n>> nothing that seems to check the same thing on `git commit`. The\n>> unnecessary multiple warnings seem to have existed since at least v1.6.0\n>> (the warnings appear to have been added in v1.5.5 in 21e5ad5, but I\n>> couldn't get that to build on my current box), and I'm seeing them on my\n>> Cygwin box's build of the current next branch (d10caa2) and on my CentOS\n>> box's v2.8.1 release.\n> \n> Torsten, I know you are not heavily invested in \"commit\" codepath,\n> but since you've been actively touching the CRLF area, I wonder\n> perhaps you might be interested in taking a look?\n> \nI have seen this double warning as well, but never digged further,\nas it is most probably in the code outside convert.c\n\nI can put it on the TODO list, and I think it would make sense to add\na TC in t0020, which is marked as test_expect_failure.\n \n"},{"id":"286529","messageId":"20160514111732.GF2345@dinwoodie.org","threadId":"42304","inReplyTo":"998a4fcd-ecd8-7e0c-860e-7f12ec02b1f9@web.de","subject":"Re: Bug report: Duplicate CRLF rewrite warnings on commit","fromName":"Adam Dinwoodie","fromEmail":"adam@dinwoodie.org","sentAt":"2016-05-14T11:17:32Z","receivedAt":"2016-05-14T11:17:32Z","isPatch":false,"sender":{"key":"adam@dinwoodie.org","avatar":"https://avatars.githubusercontent.com/u/1397507?v=4"},"body":"On Sat, May 14, 2016 at 07:40:11AM +0200, Torsten Bögershausen wrote:\n> On 13.05.16 18:43, Junio C Hamano wrote:\n> > Adam Dinwoodie writes:\n> > \n> >> If you use .gitattributes to enable CRLF->LF rewriting, then commit a\n> >> file that would have its line endings rewritten, the \"CRLF will be\n> >> replaced by LF\" warning is printed several times over; I'd expect it to\n> >> be printed only once.\n> >>\n> >> There's a test case in t0020 -- \"safecrlf: print warning only once\" --\n> >> that checks the warning is only printed once when using `git add`, but\n> >> nothing that seems to check the same thing on `git commit`. The\n> >> unnecessary multiple warnings seem to have existed since at least v1.6.0\n> >> (the warnings appear to have been added in v1.5.5 in 21e5ad5, but I\n> >> couldn't get that to build on my current box), and I'm seeing them on my\n> >> Cygwin box's build of the current next branch (d10caa2) and on my CentOS\n> >> box's v2.8.1 release.\n> > \n> > Torsten, I know you are not heavily invested in \"commit\" codepath,\n> > but since you've been actively touching the CRLF area, I wonder\n> > perhaps you might be interested in taking a look?\n> > \n> I have seen this double warning as well, but never digged further,\n> as it is most probably in the code outside convert.c\n> \n> I can put it on the TODO list, and I think it would make sense to add\n> a TC in t0020, which is marked as test_expect_failure.\n\nGrand. I'll submit the test patch now.\n"},{"id":"286557","messageId":"1463292500-12521-1-git-send-email-tboegi@web.de","threadId":"42304","inReplyTo":"20160513134953.GE2345@dinwoodie.org","subject":"[PATCH/RFC v1 0/1] Quickfix ?No duplicate CRLF rewrite warnings on commit","fromName":"","fromEmail":"tboegi@web.de","sentAt":"2016-05-15T06:08:20Z","receivedAt":"2016-05-15T06:08:20Z","isPatch":true,"sender":{"key":"tboegi@web.de","avatar":"https://avatars.githubusercontent.com/u/7138363?v=4"},"body":"From: Torsten Bögershausen \n\nIt may be that this patch only covers over a sympton, rather\nthan fixing the root cause.\n\nTorsten Bögershausen (1):\n No duplicate CRLF rewrite warnings on commit\n\n diff.c | 2 ++\n diffcore-break.c | 6 ++++--\n diffcore.h | 1 +\n t/t0020-crlf.sh | 14 ++++++++++++++\n 4 files changed, 21 insertions(+), 2 deletions(-)\n\n-- \n2.0.0.rc1.6318.g0c2c796\n"},{"id":"286556","messageId":"1463292503-12559-1-git-send-email-tboegi@web.de","threadId":"42304","inReplyTo":"20160513134953.GE2345@dinwoodie.org","subject":"[PATCH/RFC v1 1/1] No duplicate CRLF rewrite warnings on commit","fromName":"","fromEmail":"tboegi@web.de","sentAt":"2016-05-15T06:08:23Z","receivedAt":"2016-05-15T06:08:23Z","isPatch":true,"sender":{"key":"tboegi@web.de","avatar":"https://avatars.githubusercontent.com/u/7138363?v=4"},"body":"From: Torsten Bögershausen \n\nIf .gitattributes are used to enable CRLF->LF rewriting,\nthen commiting a file that would have its line endings rewritten,\nthe \"CRLF will be replaced by LF\" warning is printed 2 times.\nA user expects it to be printed only once.\nThe automatic rename detection by Git runs the conversion twice,\nsuppress the warning in the second run.\n\nReported-By: Adam Dinwoodie \nSigned-off-by: Torsten Bögershausen \n---\n diff.c | 2 ++\n diffcore-break.c | 6 ++++--\n diffcore.h | 1 +\n t/t0020-crlf.sh | 14 ++++++++++++++\n 4 files changed, 21 insertions(+), 2 deletions(-)\n\ndiff --git a/diff.c b/diff.c\nindex d3734d3..c965afc 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -2749,6 +2749,8 @@ int diff_populate_filespec(struct diff_filespec *s, unsigned int flags)\n \tenum safe_crlf crlf_warn = (safe_crlf == SAFE_CRLF_FAIL\n \t\t\t\t ? SAFE_CRLF_WARN\n \t\t\t\t : safe_crlf);\n+\tif (flags & CHECK_IGNORE_CRLF_WARNING)\n+\t\tcrlf_warn = SAFE_CRLF_FALSE;\n \n \tif (!DIFF_FILE_VALID(s))\n \t\tdie(\"internal error: asking to populate invalid file.\");\ndiff --git a/diffcore-break.c b/diffcore-break.c\nindex 5473493..4a75681 100644\n--- a/diffcore-break.c\n+++ b/diffcore-break.c\n@@ -47,6 +47,7 @@ static int should_break(struct diff_filespec *src,\n \t */\n \tunsigned long delta_size, max_size;\n \tunsigned long src_copied, literal_added, src_removed;\n+\tunsigned flags2 = 0;\n \n \t*merge_score_p = 0; /* assume no deletion --- \"do not break\"\n \t\t\t * is the default.\n@@ -61,9 +62,10 @@ static int should_break(struct diff_filespec *src,\n \t !hashcmp(src->sha1, dst->sha1))\n \t\treturn 0; /* they are the same */\n \n-\tif (diff_populate_filespec(src, 0) || diff_populate_filespec(dst, 0))\n+\tif (!strcmp(src->path, dst->path))\n+\t\tflags2 = CHECK_IGNORE_CRLF_WARNING;\n+\tif (diff_populate_filespec(src, 0) || diff_populate_filespec(dst, flags2))\n \t\treturn 0; /* error but caught downstream */\n-\n \tmax_size = ((src->size > dst->size) ? src->size : dst->size);\n \tif (max_size < MINIMUM_BREAK_SIZE)\n \t\treturn 0; /* we do not break too small filepair */\ndiff --git a/diffcore.h b/diffcore.h\nindex 33ea2de..91464aa 100644\n--- a/diffcore.h\n+++ b/diffcore.h\n@@ -57,6 +57,7 @@ extern void fill_filespec(struct diff_filespec *, const unsigned char *,\n \n #define CHECK_SIZE_ONLY 1\n #define CHECK_BINARY 2\n+#define CHECK_IGNORE_CRLF_WARNING 4\n extern int diff_populate_filespec(struct diff_filespec *, unsigned int);\n extern void diff_free_filespec_data(struct diff_filespec *);\n extern void diff_free_filespec_blob(struct diff_filespec *);\ndiff --git a/t/t0020-crlf.sh b/t/t0020-crlf.sh\nindex f94120a..f7d2f74 100755\n--- a/t/t0020-crlf.sh\n+++ b/t/t0020-crlf.sh\n@@ -86,6 +86,20 @@ test_expect_success 'safecrlf: print warning only once' '\n \ttest $(git add doublewarn 2>&1 | grep \"CRLF will be replaced by LF\" | wc -l) = 1\n '\n \n+test_expect_success 'safecrlf: print warning only once on commit' '\n+\n+\tgit config core.autocrlf input &&\n+\tgit config core.safecrlf warn &&\n+\n+\tfor w in I am all LF; do echo $w; done >doublewarn2 &&\n+\tgit add doublewarn2 &&\n+\tgit commit -m \"nowarn\" &&\n+\tfor w in Oh here is CRLFQ in text; do echo $w; done | q_to_cr >doublewarn2 &&\n+\tgit add doublewarn2 2>&1 &&\n+\tgit commit -m Message 2>&1 | grep \"CRLF will be replaced by LF\" >actual &&\n+\techo \"warning: CRLF will be replaced by LF in doublewarn2.\" >expected &&\n+\ttest_cmp expected actual\n+'\n \n test_expect_success 'safecrlf: git diff demotes safecrlf=true to warn' '\n \tgit config core.autocrlf input &&\n-- \n2.0.0.rc1.6318.g0c2c796\n"},{"id":"286558","messageId":"CAPig+cTrijGb5C1DRYTYtas0Rp5Fb34KmkxW5Fqb-T5-bdyD9Q@mail.gmail.com","threadId":"42304","inReplyTo":"1463292503-12559-1-git-send-email-tboegi@web.de","subject":"Re: [PATCH/RFC v1 1/1] No duplicate CRLF rewrite warnings on commit","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2016-05-15T06:15:15Z","receivedAt":"2016-05-15T06:15:15Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Sun, May 15, 2016 at 2:08 AM, wrote:\n> If .gitattributes are used to enable CRLF->LF rewriting,\n> then commiting a file that would have its line endings rewritten,\n> the \"CRLF will be replaced by LF\" warning is printed 2 times.\n> A user expects it to be printed only once.\n> The automatic rename detection by Git runs the conversion twice,\n> suppress the warning in the second run.\n>\n> Reported-By: Adam Dinwoodie \n> Signed-off-by: Torsten Bögershausen \n> ---\n> diff --git a/t/t0020-crlf.sh b/t/t0020-crlf.sh\n> @@ -86,6 +86,20 @@ test_expect_success 'safecrlf: print warning only once' '\n> +test_expect_success 'safecrlf: print warning only once on commit' '\n> +\n> + git config core.autocrlf input &&\n> + git config core.safecrlf warn &&\n> +\n> + for w in I am all LF; do echo $w; done >doublewarn2 &&\n\nI would typically say something about how you could instead use:\n\n test_write_lines I am all LF >doublewarn2 &&\n\nbut since you're just mimicking existing style in this script, I won't\nmention it.\n\n> + git add doublewarn2 &&\n> + git commit -m \"nowarn\" &&\n> + for w in Oh here is CRLFQ in text; do echo $w; done | q_to_cr >doublewarn2 &&\n\nLikewise; note my silence.\n\n> + git add doublewarn2 2>&1 &&\n> + git commit -m Message 2>&1 | grep \"CRLF will be replaced by LF\" >actual &&\n> + echo \"warning: CRLF will be replaced by LF in doublewarn2.\" >expected &&\n> + test_cmp expected actual\n> +'\n"},{"id":"286559","messageId":"1463294263-20135-1-git-send-email-tboegi@web.de","threadId":"42304","inReplyTo":"20160513134953.GE2345@dinwoodie.org","subject":"[PATCH v1 0/3] CRLF-Handling: bug fix around ce_compare_data()","fromName":"","fromEmail":"tboegi@web.de","sentAt":"2016-05-15T06:37:43Z","receivedAt":"2016-05-15T06:37:43Z","isPatch":true,"sender":{"key":"tboegi@web.de","avatar":"https://avatars.githubusercontent.com/u/7138363?v=4"},"body":"From: Torsten Bögershausen \n\nBreak up the old 10/10 series about CLRF handling into smaller\nseries.\n\n1/10..4/10 are now found in tb/core-eol-fix\n\nThis series includes 3 from the 10/10 series:\n09/10 t6038; use crlf on all platforms #now 1/3\n05/10 read-cache: factor out get_sha1_from_index() helper #now 2/3\n10/10 Fix for ce_compare_data(), done right. #now 3/3\n\nThe most important patch is 3/3\n\nThe reminding 3 patches,\nstream-and-early-out,\nconvert-unify-the-auto-handling-of-CRLF\nmore-safer-crlf-handling-with-text-attr\nNeed to be re-done, re-sorted and re-written, thanks for all the comments.\n\nTorsten Bögershausen (3):\n t6038; use crlf on all platforms\n read-cache: factor out get_sha1_from_index() helper\n convert: ce_compare_data() checks for a sha1 of a path\n\n cache.h | 4 ++++\n convert.c | 33 ++++++++++++++++++++++-----------\n convert.h | 23 +++++++++++++++++++----\n read-cache.c | 33 +++++++++++++++++++++------------\n sha1_file.c | 17 +++++++++++++----\n t/t6038-merge-text-auto.sh | 37 +++++++++++--------------------------\n 6 files changed, 90 insertions(+), 57 deletions(-)\n\n-- \n2.0.0.rc1.6318.g0c2c796\n"},{"id":"286562","messageId":"1463294280-20176-1-git-send-email-tboegi@web.de","threadId":"42304","inReplyTo":"20160513134953.GE2345@dinwoodie.org","subject":"[PATCH v1 1/3] t6038; use crlf on all platforms","fromName":"","fromEmail":"tboegi@web.de","sentAt":"2016-05-15T06:38:00Z","receivedAt":"2016-05-15T06:38:00Z","isPatch":true,"sender":{"key":"tboegi@web.de","avatar":"https://avatars.githubusercontent.com/u/7138363?v=4"},"body":"From: Torsten Bögershausen \n\nt6038 uses different code, dependig if NATIVE_CRLF is set ot not.\nWhen the native line endings are LF, merge.renormalize is not tested very well.\nChange the test to always use CRLF by setting core.eol=crlf.\nAfter doing so, the test fails:\n\nrm '.gitattributes'\nrm 'control_file'\nrm 'file'\nrm 'inert_file'\nHEAD is now at 0d9ffb6 add line from b\nerror: addinfo_cache failed for path 'file'\nfile: unmerged (cbd69ec7cd12dd0989e853923867d94c8519aa52)\nfile: unmerged (ad55e240aeb42e0d9a0e18d6d8b02dd82ee3e527)\nfile: unmerged (99b633103c15c20cebebf821133ab526b0ff90b2)\nfatal: git write-tree failed to write a tree\nMerging:\n0d9ffb6 add line from b\nvirtual a\nfound 1 common ancestor:\n1c56df1 Initial\nAuto-merging file\nnot ok 4 - Merge addition of text=auto\n\nThis will be addressed in the next commit.\n---\n t/t6038-merge-text-auto.sh | 37 +++++++++++--------------------------\n 1 file changed, 11 insertions(+), 26 deletions(-)\n\ndiff --git a/t/t6038-merge-text-auto.sh b/t/t6038-merge-text-auto.sh\nindex 85c10b0..4dc8c1a 100755\n--- a/t/t6038-merge-text-auto.sh\n+++ b/t/t6038-merge-text-auto.sh\n@@ -18,6 +18,7 @@ test_have_prereq SED_STRIPS_CR && SED_OPTIONS=-b\n \n test_expect_success setup '\n \tgit config core.autocrlf false &&\n+\tgit config core.eol crlf &&\n \n \techo first line | append_cr >file &&\n \techo first line >control_file &&\n@@ -72,10 +73,8 @@ test_expect_success 'Merge after setting text=auto' '\n \tsame line\n \tEOF\n \n-\tif test_have_prereq NATIVE_CRLF; then\n-\t\tappend_cr expected.temp &&\n-\t\tmv expected.temp expected\n-\tfi &&\n+\tappend_cr expected.temp &&\n+\tmv expected.temp expected &&\n \tgit config merge.renormalize true &&\n \tgit rm -fr . &&\n \trm -f .gitattributes &&\n@@ -90,10 +89,8 @@ test_expect_success 'Merge addition of text=auto' '\n \tsame line\n \tEOF\n \n-\tif test_have_prereq NATIVE_CRLF; then\n-\t\tappend_cr expected.temp &&\n-\t\tmv expected.temp expected\n-\tfi &&\n+\tappend_cr expected.temp &&\n+\tmv expected.temp expected &&\n \tgit config merge.renormalize true &&\n \tgit rm -fr . &&\n \trm -f .gitattributes &&\n@@ -104,15 +101,9 @@ test_expect_success 'Merge addition of text=auto' '\n \n test_expect_success 'Detect CRLF/LF conflict after setting text=auto' '\n \techo \"<<<<<<<\" >expected &&\n-\tif test_have_prereq NATIVE_CRLF; then\n-\t\techo first line | append_cr >>expected &&\n-\t\techo same line | append_cr >>expected &&\n-\t\techo ======= | append_cr >>expected\n-\telse\n-\t\techo first line >>expected &&\n-\t\techo same line >>expected &&\n-\t\techo ======= >>expected\n-\tfi &&\n+\techo first line | append_cr >>expected &&\n+\techo same line | append_cr >>expected &&\n+\techo ======= | append_cr >>expected &&\n \techo first line | append_cr >>expected &&\n \techo same line | append_cr >>expected &&\n \techo \">>>>>>>\" >>expected &&\n@@ -128,15 +119,9 @@ test_expect_success 'Detect LF/CRLF conflict from addition of text=auto' '\n \techo \"<<<<<<<\" >expected &&\n \techo first line | append_cr >>expected &&\n \techo same line | append_cr >>expected &&\n-\tif test_have_prereq NATIVE_CRLF; then\n-\t\techo ======= | append_cr >>expected &&\n-\t\techo first line | append_cr >>expected &&\n-\t\techo same line | append_cr >>expected\n-\telse\n-\t\techo ======= >>expected &&\n-\t\techo first line >>expected &&\n-\t\techo same line >>expected\n-\tfi &&\n+\techo ======= | append_cr >>expected &&\n+\techo first line | append_cr >>expected &&\n+\techo same line | append_cr >>expected &&\n \techo \">>>>>>>\" >>expected &&\n \tgit config merge.renormalize false &&\n \trm -f .gitattributes &&\n-- \n2.0.0.rc1.6318.g0c2c796\n"},{"id":"286560","messageId":"1463294297-20215-1-git-send-email-tboegi@web.de","threadId":"42304","inReplyTo":"20160513134953.GE2345@dinwoodie.org","subject":"[PATCH v1 2/3] read-cache: factor out get_sha1_from_index() helper","fromName":"","fromEmail":"tboegi@web.de","sentAt":"2016-05-15T06:38:17Z","receivedAt":"2016-05-15T06:38:17Z","isPatch":true,"sender":{"key":"tboegi@web.de","avatar":"https://avatars.githubusercontent.com/u/7138363?v=4"},"body":"From: Torsten Bögershausen \n\nFactor out the retrieval of the sha1 for a given path in\nread_blob_data_from_index() into the function get_sha1_from_index().\n\nThis will be used in the next commit, when convert.c can do the\nanalyze for \"text=auto\" without slurping the whole blob into memory\nat once.\n\nAdd a wrapper definition get_sha1_from_cache().\n\nSigned-off-by: Torsten Bögershausen \n---\n cache.h | 3 +++\n read-cache.c | 29 ++++++++++++++++++-----------\n 2 files changed, 21 insertions(+), 11 deletions(-)\n\ndiff --git a/cache.h b/cache.h\nindex 160f8e3..15a2a10 100644\n--- a/cache.h\n+++ b/cache.h\n@@ -379,6 +379,7 @@ extern void free_name_hash(struct index_state *istate);\n #define unmerge_cache_entry_at(at) unmerge_index_entry_at(&the_index, at)\n #define unmerge_cache(pathspec) unmerge_index(&the_index, pathspec)\n #define read_blob_data_from_cache(path, sz) read_blob_data_from_index(&the_index, (path), (sz))\n+#define get_sha1_from_cache(path) get_sha1_from_index (&the_index, (path))\n #endif\n \n enum object_type {\n@@ -1038,6 +1039,8 @@ static inline void *read_sha1_file(const unsigned char *sha1, enum object_type *\n \treturn read_sha1_file_extended(sha1, type, size, LOOKUP_REPLACE_OBJECT);\n }\n \n+const unsigned char *get_sha1_from_index(struct index_state *istate, const char *path);\n+\n /*\n * This internal function is only declared here for the benefit of\n * lookup_replace_object(). Please do not call it directly.\ndiff --git a/read-cache.c b/read-cache.c\nindex d9fb78b..a3ef967 100644\n--- a/read-cache.c\n+++ b/read-cache.c\n@@ -2263,13 +2263,27 @@ int index_name_is_other(const struct index_state *istate, const char *name,\n \n void *read_blob_data_from_index(struct index_state *istate, const char *path, unsigned long *size)\n {\n-\tint pos, len;\n+\tconst unsigned char *sha1;\n \tunsigned long sz;\n \tenum object_type type;\n \tvoid *data;\n \n-\tlen = strlen(path);\n-\tpos = index_name_pos(istate, path, len);\n+\tsha1 = get_sha1_from_index(istate, path);\n+\tif (!sha1)\n+\t\treturn NULL;\n+\tdata = read_sha1_file(sha1, &type, &sz);\n+\tif (!data || type != OBJ_BLOB) {\n+\t\tfree(data);\n+\t\treturn NULL;\n+\t}\n+\tif (size)\n+\t\t*size = sz;\n+\treturn data;\n+}\n+\n+const unsigned char *get_sha1_from_index(struct index_state *istate, const char *path)\n+{\n+\tint pos = index_name_pos(istate, path, strlen(path));\n \tif (pos < 0) {\n \t\t/*\n \t\t * We might be in the middle of a merge, in which\n@@ -2285,14 +2299,7 @@ void *read_blob_data_from_index(struct index_state *istate, const char *path, un\n \t}\n \tif (pos < 0)\n \t\treturn NULL;\n-\tdata = read_sha1_file(istate->cache[pos]->sha1, &type, &sz);\n-\tif (!data || type != OBJ_BLOB) {\n-\t\tfree(data);\n-\t\treturn NULL;\n-\t}\n-\tif (size)\n-\t\t*size = sz;\n-\treturn data;\n+\treturn (istate->cache[pos]->sha1);\n }\n \n void stat_validity_clear(struct stat_validity *sv)\n-- \n2.0.0.rc1.6318.g0c2c796\n"},{"id":"286561","messageId":"1463294299-20256-1-git-send-email-tboegi@web.de","threadId":"42304","inReplyTo":"20160513134953.GE2345@dinwoodie.org","subject":"[PATCH v1 3/3] convert: ce_compare_data() checks for a sha1 of a path","fromName":"","fromEmail":"tboegi@web.de","sentAt":"2016-05-15T06:38:19Z","receivedAt":"2016-05-15T06:38:19Z","isPatch":true,"sender":{"key":"tboegi@web.de","avatar":"https://avatars.githubusercontent.com/u/7138363?v=4"},"body":"From: Torsten Bögershausen \n\nTo compare a file in working tree with the index, convert_to_git() is used,\nthe the result is hashed and the hash value compared with ce->sha1.\n\nDeep down would_convert_crlf_at_commit() is invoked, to check if CRLF\nare converted or not: When a CRLF had been in the index before, CRLF in\nthe working tree are not converted.\n\nWhile in a merge, a file name in the working tree has different blobs\nin the index with different hash values.\nForwarding ce->sha1 from ce_compare_data() into crlf_to_git() makes sure\nthe would_convert_crlf_at_commit() looks at the appropriate blob.\n\nSigned-off-by: Torsten Bögershausen \n---\n cache.h | 1 +\n convert.c | 33 ++++++++++++++++++++++-----------\n convert.h | 23 +++++++++++++++++++----\n read-cache.c | 4 +++-\n sha1_file.c | 17 +++++++++++++----\n 5 files changed, 58 insertions(+), 20 deletions(-)\n\ndiff --git a/cache.h b/cache.h\nindex 15a2a10..a5136c0 100644\n--- a/cache.h\n+++ b/cache.h\n@@ -605,6 +605,7 @@ extern int ie_modified(const struct index_state *, const struct cache_entry *, s\n \n #define HASH_WRITE_OBJECT 1\n #define HASH_FORMAT_CHECK 2\n+#define HASH_CE_HAS_SHA1 4\n extern int index_fd(unsigned char *sha1, int fd, struct stat *st, enum object_type type, const char *path, unsigned flags);\n extern int index_path(unsigned char *sha1, const char *path, struct stat *st, unsigned flags);\n \ndiff --git a/convert.c b/convert.c\nindex f524b8d..5a0cd03 100644\n--- a/convert.c\n+++ b/convert.c\n@@ -217,21 +217,29 @@ static void check_safe_crlf(const char *path, enum crlf_action crlf_action,\n \t}\n }\n \n-static int has_cr_in_index(const char *path)\n+static int has_cr_in_index(const char *path, const unsigned char *sha1)\n {\n \tunsigned long sz;\n \tvoid *data;\n \tint has_cr;\n-\n-\tdata = read_blob_data_from_cache(path, &sz);\n-\tif (!data)\n+\tenum object_type type;\n+\tif (!sha1)\n+\t\tsha1 = get_sha1_from_cache(path);\n+\tif (!sha1)\n+\t\treturn 0;\n+\tdata = read_sha1_file(sha1, &type, &sz);\n+\tif (!data || type != OBJ_BLOB) {\n+\t\tfree(data);\n \t\treturn 0;\n+\t}\n+\n \thas_cr = memchr(data, '\\r', sz) != NULL;\n \tfree(data);\n \treturn has_cr;\n }\n \n-static int crlf_to_git(const char *path, const char *src, size_t len,\n+static int crlf_to_git(const char *path, const unsigned char *sha1,\n+\t\t const char *src, size_t len,\n \t\t struct strbuf *buf,\n \t\t enum crlf_action crlf_action, enum safe_crlf checksafe)\n {\n@@ -260,7 +268,7 @@ static int crlf_to_git(const char *path, const char *src, size_t len,\n \t\t\t * If the file in the index has any CR in it, do not convert.\n \t\t\t * This is the new safer autocrlf handling.\n \t\t\t */\n-\t\t\tif (has_cr_in_index(path))\n+\t\t\tif (has_cr_in_index(path, sha1))\n \t\t\t\treturn 0;\n \t\t}\n \t}\n@@ -852,8 +860,9 @@ const char *get_convert_attr_ascii(const char *path)\n \treturn \"\";\n }\n \n-int convert_to_git(const char *path, const char *src, size_t len,\n- struct strbuf *dst, enum safe_crlf checksafe)\n+int convert_to_git_ce_sha1(const char *path, const unsigned char *sha1,\n+\t\t\t const char *src, size_t len,\n+\t\t\t struct strbuf *dst, enum safe_crlf checksafe)\n {\n \tint ret = 0;\n \tconst char *filter = NULL;\n@@ -874,7 +883,7 @@ int convert_to_git(const char *path, const char *src, size_t len,\n \t\tsrc = dst->buf;\n \t\tlen = dst->len;\n \t}\n-\tret |= crlf_to_git(path, src, len, dst, ca.crlf_action, checksafe);\n+\tret |= crlf_to_git(path, sha1, src, len, dst, ca.crlf_action, checksafe);\n \tif (ret && dst) {\n \t\tsrc = dst->buf;\n \t\tlen = dst->len;\n@@ -882,7 +891,9 @@ int convert_to_git(const char *path, const char *src, size_t len,\n \treturn ret | ident_to_git(path, src, len, dst, ca.ident);\n }\n \n-void convert_to_git_filter_fd(const char *path, int fd, struct strbuf *dst,\n+void convert_to_git_filter_fd(const char *path,\n+\t\t\t const unsigned char *sha1,\n+\t\t\t int fd, struct strbuf *dst,\n \t\t\t enum safe_crlf checksafe)\n {\n \tstruct conv_attrs ca;\n@@ -894,7 +905,7 @@ void convert_to_git_filter_fd(const char *path, int fd, struct strbuf *dst,\n \tif (!apply_filter(path, NULL, 0, fd, dst, ca.drv->clean))\n \t\tdie(\"%s: clean filter '%s' failed\", path, ca.drv->name);\n \n-\tcrlf_to_git(path, dst->buf, dst->len, dst, ca.crlf_action, checksafe);\n+\tcrlf_to_git(path, sha1, dst->buf, dst->len, dst, ca.crlf_action, checksafe);\n \tident_to_git(path, dst->buf, dst->len, dst, ca.ident);\n }\n \ndiff --git a/convert.h b/convert.h\nindex ccf436b..1634d37 100644\n--- a/convert.h\n+++ b/convert.h\n@@ -37,8 +37,16 @@ extern const char *get_wt_convert_stats_ascii(const char *path);\n extern const char *get_convert_attr_ascii(const char *path);\n \n /* returns 1 if *dst was used */\n-extern int convert_to_git(const char *path, const char *src, size_t len,\n-\t\t\t struct strbuf *dst, enum safe_crlf checksafe);\n+extern int convert_to_git_ce_sha1(const char *path, const unsigned char *sha1,\n+\t\t\t\t const char *src, size_t len,\n+\t\t\t\t struct strbuf *dst, enum safe_crlf checksafe);\n+\n+static inline int convert_to_git(const char *path, const char *src, size_t len,\n+\t\t\t\t struct strbuf *dst, enum safe_crlf checksafe)\n+{\n+\treturn convert_to_git_ce_sha1(path, NULL, src, len, dst, checksafe);\n+}\n+\n extern int convert_to_working_tree(const char *path, const char *src,\n \t\t\t\t size_t len, struct strbuf *dst);\n extern int renormalize_buffer(const char *path, const char *src, size_t len,\n@@ -47,9 +55,16 @@ static inline int would_convert_to_git(const char *path)\n {\n \treturn convert_to_git(path, NULL, 0, NULL, 0);\n }\n+static inline int would_convert_to_git_ce_sha1(const char *path,\n+\t\t\t\t\t const unsigned char *sha1)\n+{\n+\treturn convert_to_git_ce_sha1(path, sha1, NULL, 0, NULL, 0);\n+}\n+\n /* Precondition: would_convert_to_git_filter_fd(path) == true */\n-extern void convert_to_git_filter_fd(const char *path, int fd,\n-\t\t\t\t struct strbuf *dst,\n+extern void convert_to_git_filter_fd(const char *path,\n+\t\t\t\t const unsigned char *sha1,\n+\t\t\t\t int fd, struct strbuf *dst,\n \t\t\t\t enum safe_crlf checksafe);\n extern int would_convert_to_git_filter_fd(const char *path);\n \ndiff --git a/read-cache.c b/read-cache.c\nindex a3ef967..0ebc237 100644\n--- a/read-cache.c\n+++ b/read-cache.c\n@@ -163,7 +163,9 @@ static int ce_compare_data(const struct cache_entry *ce, struct stat *st)\n \n \tif (fd >= 0) {\n \t\tunsigned char sha1[20];\n-\t\tif (!index_fd(sha1, fd, st, OBJ_BLOB, ce->name, 0))\n+\t\tunsigned flags = HASH_CE_HAS_SHA1;\n+\t\tmemcpy(sha1, ce->sha1, sizeof(sha1));\n+\t\tif (!index_fd(sha1, fd, st, OBJ_BLOB, ce->name, flags))\n \t\t\tmatch = hashcmp(sha1, ce->sha1);\n \t\t/* index_fd() closed the file descriptor already */\n \t}\ndiff --git a/sha1_file.c b/sha1_file.c\nindex d0f2aa0..dd013d5 100644\n--- a/sha1_file.c\n+++ b/sha1_file.c\n@@ -3275,6 +3275,7 @@ static int index_mem(unsigned char *sha1, void *buf, size_t size,\n {\n \tint ret, re_allocated = 0;\n \tint write_object = flags & HASH_WRITE_OBJECT;\n+\tconst int valid_sha1 = flags & HASH_CE_HAS_SHA1;\n \n \tif (!type)\n \t\ttype = OBJ_BLOB;\n@@ -3284,8 +3285,11 @@ static int index_mem(unsigned char *sha1, void *buf, size_t size,\n \t */\n \tif ((type == OBJ_BLOB) && path) {\n \t\tstruct strbuf nbuf = STRBUF_INIT;\n-\t\tif (convert_to_git(path, buf, size, &nbuf,\n-\t\t\t\t write_object ? safe_crlf : SAFE_CRLF_FALSE)) {\n+\t\tif (convert_to_git_ce_sha1(path,\n+\t\t\t\t\t valid_sha1 ? sha1 : NULL,\n+\t\t\t\t\t buf, size, &nbuf,\n+\t\t\t\t\t write_object ? safe_crlf : SAFE_CRLF_FALSE)){\n+\n \t\t\tbuf = strbuf_detach(&nbuf, &size);\n \t\t\tre_allocated = 1;\n \t\t}\n@@ -3313,12 +3317,15 @@ static int index_stream_convert_blob(unsigned char *sha1, int fd,\n {\n \tint ret;\n \tconst int write_object = flags & HASH_WRITE_OBJECT;\n+\tconst int valid_sha1 = flags & HASH_CE_HAS_SHA1;\n \tstruct strbuf sbuf = STRBUF_INIT;\n \n \tassert(path);\n \tassert(would_convert_to_git_filter_fd(path));\n \n-\tconvert_to_git_filter_fd(path, fd, &sbuf,\n+\tconvert_to_git_filter_fd(path,\n+\t\t\t\t valid_sha1 ? sha1 : NULL,\n+\t\t\t\t fd, &sbuf,\n \t\t\t\t write_object ? safe_crlf : SAFE_CRLF_FALSE);\n \n \tif (write_object)\n@@ -3396,6 +3403,8 @@ int index_fd(unsigned char *sha1, int fd, struct stat *st,\n \t enum object_type type, const char *path, unsigned flags)\n {\n \tint ret;\n+\tconst unsigned char *sha1_ce;\n+\tsha1_ce = flags & HASH_CE_HAS_SHA1 ? sha1 : NULL;\n \n \t/*\n \t * Call xsize_t() only when needed to avoid potentially unnecessary\n@@ -3406,7 +3415,7 @@ int index_fd(unsigned char *sha1, int fd, struct stat *st,\n \telse if (!S_ISREG(st->st_mode))\n \t\tret = index_pipe(sha1, fd, type, path, flags);\n \telse if (st->st_size <= big_file_threshold || type != OBJ_BLOB ||\n-\t\t (path && would_convert_to_git(path)))\n+\t\t (path && would_convert_to_git_ce_sha1(path,sha1_ce)))\n \t\tret = index_core(sha1, fd, xsize_t(st->st_size), type, path,\n \t\t\t\t flags);\n \telse\n-- \n2.0.0.rc1.6318.g0c2c796\n"},{"id":"286564","messageId":"CAPig+cRj_=+hBw_BpBHfJLwBmMH1k8w1S1WMFes2avSOOkc77g@mail.gmail.com","threadId":"42304","inReplyTo":"1463294280-20176-1-git-send-email-tboegi@web.de","subject":"Re: [PATCH v1 1/3] t6038; use crlf on all platforms","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2016-05-15T06:42:38Z","receivedAt":"2016-05-15T06:42:38Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Sun, May 15, 2016 at 2:38 AM, wrote:\n> t6038 uses different code, dependig if NATIVE_CRLF is set ot not.\n\ns/dependig/depending/\ns/ot/or/\n\n> When the native line endings are LF, merge.renormalize is not tested very well.\n> Change the test to always use CRLF by setting core.eol=crlf.\n> After doing so, the test fails:\n> [...snip...]\n> This will be addressed in the next commit.\n\nDoes this mean that the below tests now fail? If so, they should be\nswitched to use test_expect_failure here, and then swapped back to\ntest_expect_success in the patch which fixes the problem.\n\n> ---\n> diff --git a/t/t6038-merge-text-auto.sh b/t/t6038-merge-text-auto.sh\n> index 85c10b0..4dc8c1a 100755\n> --- a/t/t6038-merge-text-auto.sh\n> +++ b/t/t6038-merge-text-auto.sh\n> @@ -18,6 +18,7 @@ test_have_prereq SED_STRIPS_CR && SED_OPTIONS=-b\n>\n> test_expect_success setup '\n> git config core.autocrlf false &&\n> + git config core.eol crlf &&\n>\n> echo first line | append_cr >file &&\n> echo first line >control_file &&\n> @@ -72,10 +73,8 @@ test_expect_success 'Merge after setting text=auto' '\n> same line\n> EOF\n>\n> - if test_have_prereq NATIVE_CRLF; then\n> - append_cr expected.temp &&\n> - mv expected.temp expected\n> - fi &&\n> + append_cr expected.temp &&\n> + mv expected.temp expected &&\n> git config merge.renormalize true &&\n> git rm -fr . &&\n> rm -f .gitattributes &&\n> @@ -90,10 +89,8 @@ test_expect_success 'Merge addition of text=auto' '\n> same line\n> EOF\n>\n> - if test_have_prereq NATIVE_CRLF; then\n> - append_cr expected.temp &&\n> - mv expected.temp expected\n> - fi &&\n> + append_cr expected.temp &&\n> + mv expected.temp expected &&\n> git config merge.renormalize true &&\n> git rm -fr . &&\n> rm -f .gitattributes &&\n> @@ -104,15 +101,9 @@ test_expect_success 'Merge addition of text=auto' '\n>\n> test_expect_success 'Detect CRLF/LF conflict after setting text=auto' '\n> echo \"<<<<<<<\" >expected &&\n> - if test_have_prereq NATIVE_CRLF; then\n> - echo first line | append_cr >>expected &&\n> - echo same line | append_cr >>expected &&\n> - echo ======= | append_cr >>expected\n> - else\n> - echo first line >>expected &&\n> - echo same line >>expected &&\n> - echo ======= >>expected\n> - fi &&\n> + echo first line | append_cr >>expected &&\n> + echo same line | append_cr >>expected &&\n> + echo ======= | append_cr >>expected &&\n> echo first line | append_cr >>expected &&\n> echo same line | append_cr >>expected &&\n> echo \">>>>>>>\" >>expected &&\n> @@ -128,15 +119,9 @@ test_expect_success 'Detect LF/CRLF conflict from addition of text=auto' '\n> echo \"<<<<<<<\" >expected &&\n> echo first line | append_cr >>expected &&\n> echo same line | append_cr >>expected &&\n> - if test_have_prereq NATIVE_CRLF; then\n> - echo ======= | append_cr >>expected &&\n> - echo first line | append_cr >>expected &&\n> - echo same line | append_cr >>expected\n> - else\n> - echo ======= >>expected &&\n> - echo first line >>expected &&\n> - echo same line >>expected\n> - fi &&\n> + echo ======= | append_cr >>expected &&\n> + echo first line | append_cr >>expected &&\n> + echo same line | append_cr >>expected &&\n> echo \">>>>>>>\" >>expected &&\n> git config merge.renormalize false &&\n> rm -f .gitattributes &&\n> --\n> 2.0.0.rc1.6318.g0c2c796\n"},{"id":"286565","messageId":"CAPig+cRiu_oJZhPj+QPY6JpcEn=d8VWdHU90XThN9_88JryZ_g@mail.gmail.com","threadId":"42304","inReplyTo":"1463294297-20215-1-git-send-email-tboegi@web.de","subject":"Re: [PATCH v1 2/3] read-cache: factor out get_sha1_from_index() helper","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2016-05-15T06:45:49Z","receivedAt":"2016-05-15T06:45:49Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Sun, May 15, 2016 at 2:38 AM, wrote:\n> Factor out the retrieval of the sha1 for a given path in\n> read_blob_data_from_index() into the function get_sha1_from_index().\n>\n> This will be used in the next commit, when convert.c can do the\n> analyze for \"text=auto\" without slurping the whole blob into memory\n> at once.\n>\n> Add a wrapper definition get_sha1_from_cache().\n>\n> Signed-off-by: Torsten Bögershausen \n> ---\n> diff --git a/cache.h b/cache.h\n> @@ -379,6 +379,7 @@ extern void free_name_hash(struct index_state *istate);\n> #define read_blob_data_from_cache(path, sz) read_blob_data_from_index(&the_index, (path), (sz))\n> +#define get_sha1_from_cache(path) get_sha1_from_index (&the_index, (path))\n\ns/\\s+/ /\n"},{"id":"286567","messageId":"CAPig+cSYU-+G5p=SaYs=PvMkLiB745Nf7sY3sKe3aozfVvq+iQ@mail.gmail.com","threadId":"42304","inReplyTo":"1463294299-20256-1-git-send-email-tboegi@web.de","subject":"Re: [PATCH v1 3/3] convert: ce_compare_data() checks for a sha1 of a path","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2016-05-15T06:52:50Z","receivedAt":"2016-05-15T06:52:50Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Sun, May 15, 2016 at 2:38 AM, wrote:\n> To compare a file in working tree with the index, convert_to_git() is used,\n> the the result is hashed and the hash value compared with ce->sha1.\n>\n> Deep down would_convert_crlf_at_commit() is invoked, to check if CRLF\n> are converted or not: When a CRLF had been in the index before, CRLF in\n> the working tree are not converted.\n>\n> While in a merge, a file name in the working tree has different blobs\n> in the index with different hash values.\n> Forwarding ce->sha1 from ce_compare_data() into crlf_to_git() makes sure\n> the would_convert_crlf_at_commit() looks at the appropriate blob.\n>\n> Signed-off-by: Torsten Bögershausen \n> ---\n> diff --git a/convert.c b/convert.c\n> @@ -217,21 +217,29 @@ static void check_safe_crlf(const char *path, enum crlf_action crlf_action,\n> -static int has_cr_in_index(const char *path)\n> +static int has_cr_in_index(const char *path, const unsigned char *sha1)\n> {\n> unsigned long sz;\n> void *data;\n> int has_cr;\n> -\n> - data = read_blob_data_from_cache(path, &sz);\n> - if (!data)\n> + enum object_type type;\n> + if (!sha1)\n> + sha1 = get_sha1_from_cache(path);\n> + if (!sha1)\n> + return 0;\n> + data = read_sha1_file(sha1, &type, &sz);\n> + if (!data || type != OBJ_BLOB) {\n> + free(data);\n> return 0;\n> + }\n> +\n> has_cr = memchr(data, '\\r', sz) != NULL;\n> free(data);\n> return has_cr;\n> }\n\nPossible rewrite which would make it harder to forget to free 'data':\n\n int has_cr = 0;\n ...\n data = read_sha1_file(...);\n if (data && type == OBJ_BLOB)\n has_cr = memchr(...);\n free(data);\n return has_cr;\n"},{"id":"286589","messageId":"1463317328-2632-1-git-send-email-tboegi@web.de","threadId":"42304","inReplyTo":"20160513134953.GE2345@dinwoodie.org","subject":"[PATCH v2 0/3] CRLF-Handling: bug fix around ce_compare_data()","fromName":"","fromEmail":"tboegi@web.de","sentAt":"2016-05-15T13:02:08Z","receivedAt":"2016-05-15T13:02:08Z","isPatch":true,"sender":{"key":"tboegi@web.de","avatar":"https://avatars.githubusercontent.com/u/7138363?v=4"},"body":"From: Torsten Bögershausen \n\nChanges since v1:\n - Re-order the patches\n - t6038; use crlf on all platforms\n This did actually not break anything (without old 7/10)\n Adapt the commit message\n - ce_compare_data():\n Simplify the logic around free() in has_cr_in_index(),\n Thanks Eric Sunshine\n\nBreak up the old 10/10 series about CLRF handling into smaller\nseries.\n\n1/10..4/10 are now found in tb/core-eol-fix\n\nThis series includes 3 from the 10/10 series:\n05/10 read-cache: factor out get_sha1_from_index() helper #now 1/3\n10/10 Fix for ce_compare_data(), done right. #now 2/3\n09/10 t6038; use crlf on all platforms #now 3/3\n\nThe most important patch is 2/3\n\nThe reminding 3 patches,\nstream-and-early-out,\nconvert-unify-the-auto-handling-of-CRLF\nmore-safer-crlf-handling-with-text-attr\nNeed to be re-done, re-sorted and re-written, thanks for all the comments.\n\nTorsten Bögershausen (3):\n read-cache: factor out get_sha1_from_index() helper\n convert: ce_compare_data() checks for a sha1 of a path\n t6038; use crlf on all platforms\n\n cache.h | 4 ++++\n convert.c | 34 ++++++++++++++++++++++------------\n convert.h | 23 +++++++++++++++++++----\n read-cache.c | 33 +++++++++++++++++++++------------\n sha1_file.c | 17 +++++++++++++----\n t/t6038-merge-text-auto.sh | 37 +++++++++++--------------------------\n 6 files changed, 90 insertions(+), 58 deletions(-)\n\n-- \n2.0.0.rc1.6318.g0c2c796\n"},{"id":"286590","messageId":"1463317330-2671-1-git-send-email-tboegi@web.de","threadId":"42304","inReplyTo":"20160513134953.GE2345@dinwoodie.org","subject":"[PATCH v2 1/3] read-cache: factor out get_sha1_from_index() helper","fromName":"","fromEmail":"tboegi@web.de","sentAt":"2016-05-15T13:02:10Z","receivedAt":"2016-05-15T13:02:10Z","isPatch":true,"sender":{"key":"tboegi@web.de","avatar":"https://avatars.githubusercontent.com/u/7138363?v=4"},"body":"From: Torsten Bögershausen \n\nFactor out the retrieval of the sha1 for a given path in\nread_blob_data_from_index() into the function get_sha1_from_index().\n\nThis will be used in the next commit, when convert.c can do the\nanalyze for \"text=auto\" without slurping the whole blob into memory\nat once.\n\nAdd a wrapper definition get_sha1_from_cache().\n\nSigned-off-by: Torsten Bögershausen \n---\n cache.h | 3 +++\n read-cache.c | 29 ++++++++++++++++++-----------\n 2 files changed, 21 insertions(+), 11 deletions(-)\n\ndiff --git a/cache.h b/cache.h\nindex 160f8e3..15a2a10 100644\n--- a/cache.h\n+++ b/cache.h\n@@ -379,6 +379,7 @@ extern void free_name_hash(struct index_state *istate);\n #define unmerge_cache_entry_at(at) unmerge_index_entry_at(&the_index, at)\n #define unmerge_cache(pathspec) unmerge_index(&the_index, pathspec)\n #define read_blob_data_from_cache(path, sz) read_blob_data_from_index(&the_index, (path), (sz))\n+#define get_sha1_from_cache(path) get_sha1_from_index (&the_index, (path))\n #endif\n \n enum object_type {\n@@ -1038,6 +1039,8 @@ static inline void *read_sha1_file(const unsigned char *sha1, enum object_type *\n \treturn read_sha1_file_extended(sha1, type, size, LOOKUP_REPLACE_OBJECT);\n }\n \n+const unsigned char *get_sha1_from_index(struct index_state *istate, const char *path);\n+\n /*\n * This internal function is only declared here for the benefit of\n * lookup_replace_object(). Please do not call it directly.\ndiff --git a/read-cache.c b/read-cache.c\nindex d9fb78b..a3ef967 100644\n--- a/read-cache.c\n+++ b/read-cache.c\n@@ -2263,13 +2263,27 @@ int index_name_is_other(const struct index_state *istate, const char *name,\n \n void *read_blob_data_from_index(struct index_state *istate, const char *path, unsigned long *size)\n {\n-\tint pos, len;\n+\tconst unsigned char *sha1;\n \tunsigned long sz;\n \tenum object_type type;\n \tvoid *data;\n \n-\tlen = strlen(path);\n-\tpos = index_name_pos(istate, path, len);\n+\tsha1 = get_sha1_from_index(istate, path);\n+\tif (!sha1)\n+\t\treturn NULL;\n+\tdata = read_sha1_file(sha1, &type, &sz);\n+\tif (!data || type != OBJ_BLOB) {\n+\t\tfree(data);\n+\t\treturn NULL;\n+\t}\n+\tif (size)\n+\t\t*size = sz;\n+\treturn data;\n+}\n+\n+const unsigned char *get_sha1_from_index(struct index_state *istate, const char *path)\n+{\n+\tint pos = index_name_pos(istate, path, strlen(path));\n \tif (pos < 0) {\n \t\t/*\n \t\t * We might be in the middle of a merge, in which\n@@ -2285,14 +2299,7 @@ void *read_blob_data_from_index(struct index_state *istate, const char *path, un\n \t}\n \tif (pos < 0)\n \t\treturn NULL;\n-\tdata = read_sha1_file(istate->cache[pos]->sha1, &type, &sz);\n-\tif (!data || type != OBJ_BLOB) {\n-\t\tfree(data);\n-\t\treturn NULL;\n-\t}\n-\tif (size)\n-\t\t*size = sz;\n-\treturn data;\n+\treturn (istate->cache[pos]->sha1);\n }\n \n void stat_validity_clear(struct stat_validity *sv)\n-- \n2.0.0.rc1.6318.g0c2c796\n"},{"id":"286591","messageId":"1463317349-2710-1-git-send-email-tboegi@web.de","threadId":"42304","inReplyTo":"20160513134953.GE2345@dinwoodie.org","subject":"[PATCH v2 2/3] convert: ce_compare_data() checks for a sha1 of a path","fromName":"","fromEmail":"tboegi@web.de","sentAt":"2016-05-15T13:02:29Z","receivedAt":"2016-05-15T13:02:29Z","isPatch":true,"sender":{"key":"tboegi@web.de","avatar":"https://avatars.githubusercontent.com/u/7138363?v=4"},"body":"From: Torsten Bögershausen \n\nTo compare a file in working tree with the index, convert_to_git() is used,\nthe the result is hashed and the hash value compared with ce->sha1.\n\nDeep down would_convert_crlf_at_commit() is invoked, to check if CRLF\nare converted or not: When a CRLF had been in the index before, CRLF in\nthe working tree are not converted.\n\nWhile in a merge, a file name in the working tree has different blobs\nin the index with different hash values.\nForwarding ce->sha1 from ce_compare_data() into crlf_to_git() makes sure\nthe would_convert_crlf_at_commit() looks at the appropriate blob.\n\nSigned-off-by: Torsten Bögershausen \n---\n cache.h | 1 +\n convert.c | 34 ++++++++++++++++++++++------------\n convert.h | 23 +++++++++++++++++++----\n read-cache.c | 4 +++-\n sha1_file.c | 17 +++++++++++++----\n 5 files changed, 58 insertions(+), 21 deletions(-)\n\ndiff --git a/cache.h b/cache.h\nindex 15a2a10..a5136c0 100644\n--- a/cache.h\n+++ b/cache.h\n@@ -605,6 +605,7 @@ extern int ie_modified(const struct index_state *, const struct cache_entry *, s\n \n #define HASH_WRITE_OBJECT 1\n #define HASH_FORMAT_CHECK 2\n+#define HASH_CE_HAS_SHA1 4\n extern int index_fd(unsigned char *sha1, int fd, struct stat *st, enum object_type type, const char *path, unsigned flags);\n extern int index_path(unsigned char *sha1, const char *path, struct stat *st, unsigned flags);\n \ndiff --git a/convert.c b/convert.c\nindex f524b8d..c972d14 100644\n--- a/convert.c\n+++ b/convert.c\n@@ -217,21 +217,28 @@ static void check_safe_crlf(const char *path, enum crlf_action crlf_action,\n \t}\n }\n \n-static int has_cr_in_index(const char *path)\n+static int has_cr_in_index(const char *path, const unsigned char *sha1)\n {\n \tunsigned long sz;\n \tvoid *data;\n-\tint has_cr;\n-\n-\tdata = read_blob_data_from_cache(path, &sz);\n+\tint has_cr = 0;\n+\tenum object_type type;\n+\tif (!sha1)\n+\t\tsha1 = get_sha1_from_cache(path);\n+\tif (!sha1)\n+\t\treturn 0;\n+\tdata = read_sha1_file(sha1, &type, &sz);\n \tif (!data)\n \t\treturn 0;\n-\thas_cr = memchr(data, '\\r', sz) != NULL;\n+\tif (type == OBJ_BLOB)\n+\t\thas_cr = memchr(data, '\\r', sz) != NULL;\n+\n \tfree(data);\n \treturn has_cr;\n }\n \n-static int crlf_to_git(const char *path, const char *src, size_t len,\n+static int crlf_to_git(const char *path, const unsigned char *sha1,\n+\t\t const char *src, size_t len,\n \t\t struct strbuf *buf,\n \t\t enum crlf_action crlf_action, enum safe_crlf checksafe)\n {\n@@ -260,7 +267,7 @@ static int crlf_to_git(const char *path, const char *src, size_t len,\n \t\t\t * If the file in the index has any CR in it, do not convert.\n \t\t\t * This is the new safer autocrlf handling.\n \t\t\t */\n-\t\t\tif (has_cr_in_index(path))\n+\t\t\tif (has_cr_in_index(path, sha1))\n \t\t\t\treturn 0;\n \t\t}\n \t}\n@@ -852,8 +859,9 @@ const char *get_convert_attr_ascii(const char *path)\n \treturn \"\";\n }\n \n-int convert_to_git(const char *path, const char *src, size_t len,\n- struct strbuf *dst, enum safe_crlf checksafe)\n+int convert_to_git_ce_sha1(const char *path, const unsigned char *sha1,\n+\t\t\t const char *src, size_t len,\n+\t\t\t struct strbuf *dst, enum safe_crlf checksafe)\n {\n \tint ret = 0;\n \tconst char *filter = NULL;\n@@ -874,7 +882,7 @@ int convert_to_git(const char *path, const char *src, size_t len,\n \t\tsrc = dst->buf;\n \t\tlen = dst->len;\n \t}\n-\tret |= crlf_to_git(path, src, len, dst, ca.crlf_action, checksafe);\n+\tret |= crlf_to_git(path, sha1, src, len, dst, ca.crlf_action, checksafe);\n \tif (ret && dst) {\n \t\tsrc = dst->buf;\n \t\tlen = dst->len;\n@@ -882,7 +890,9 @@ int convert_to_git(const char *path, const char *src, size_t len,\n \treturn ret | ident_to_git(path, src, len, dst, ca.ident);\n }\n \n-void convert_to_git_filter_fd(const char *path, int fd, struct strbuf *dst,\n+void convert_to_git_filter_fd(const char *path,\n+\t\t\t const unsigned char *sha1,\n+\t\t\t int fd, struct strbuf *dst,\n \t\t\t enum safe_crlf checksafe)\n {\n \tstruct conv_attrs ca;\n@@ -894,7 +904,7 @@ void convert_to_git_filter_fd(const char *path, int fd, struct strbuf *dst,\n \tif (!apply_filter(path, NULL, 0, fd, dst, ca.drv->clean))\n \t\tdie(\"%s: clean filter '%s' failed\", path, ca.drv->name);\n \n-\tcrlf_to_git(path, dst->buf, dst->len, dst, ca.crlf_action, checksafe);\n+\tcrlf_to_git(path, sha1, dst->buf, dst->len, dst, ca.crlf_action, checksafe);\n \tident_to_git(path, dst->buf, dst->len, dst, ca.ident);\n }\n \ndiff --git a/convert.h b/convert.h\nindex ccf436b..1634d37 100644\n--- a/convert.h\n+++ b/convert.h\n@@ -37,8 +37,16 @@ extern const char *get_wt_convert_stats_ascii(const char *path);\n extern const char *get_convert_attr_ascii(const char *path);\n \n /* returns 1 if *dst was used */\n-extern int convert_to_git(const char *path, const char *src, size_t len,\n-\t\t\t struct strbuf *dst, enum safe_crlf checksafe);\n+extern int convert_to_git_ce_sha1(const char *path, const unsigned char *sha1,\n+\t\t\t\t const char *src, size_t len,\n+\t\t\t\t struct strbuf *dst, enum safe_crlf checksafe);\n+\n+static inline int convert_to_git(const char *path, const char *src, size_t len,\n+\t\t\t\t struct strbuf *dst, enum safe_crlf checksafe)\n+{\n+\treturn convert_to_git_ce_sha1(path, NULL, src, len, dst, checksafe);\n+}\n+\n extern int convert_to_working_tree(const char *path, const char *src,\n \t\t\t\t size_t len, struct strbuf *dst);\n extern int renormalize_buffer(const char *path, const char *src, size_t len,\n@@ -47,9 +55,16 @@ static inline int would_convert_to_git(const char *path)\n {\n \treturn convert_to_git(path, NULL, 0, NULL, 0);\n }\n+static inline int would_convert_to_git_ce_sha1(const char *path,\n+\t\t\t\t\t const unsigned char *sha1)\n+{\n+\treturn convert_to_git_ce_sha1(path, sha1, NULL, 0, NULL, 0);\n+}\n+\n /* Precondition: would_convert_to_git_filter_fd(path) == true */\n-extern void convert_to_git_filter_fd(const char *path, int fd,\n-\t\t\t\t struct strbuf *dst,\n+extern void convert_to_git_filter_fd(const char *path,\n+\t\t\t\t const unsigned char *sha1,\n+\t\t\t\t int fd, struct strbuf *dst,\n \t\t\t\t enum safe_crlf checksafe);\n extern int would_convert_to_git_filter_fd(const char *path);\n \ndiff --git a/read-cache.c b/read-cache.c\nindex a3ef967..0ebc237 100644\n--- a/read-cache.c\n+++ b/read-cache.c\n@@ -163,7 +163,9 @@ static int ce_compare_data(const struct cache_entry *ce, struct stat *st)\n \n \tif (fd >= 0) {\n \t\tunsigned char sha1[20];\n-\t\tif (!index_fd(sha1, fd, st, OBJ_BLOB, ce->name, 0))\n+\t\tunsigned flags = HASH_CE_HAS_SHA1;\n+\t\tmemcpy(sha1, ce->sha1, sizeof(sha1));\n+\t\tif (!index_fd(sha1, fd, st, OBJ_BLOB, ce->name, flags))\n \t\t\tmatch = hashcmp(sha1, ce->sha1);\n \t\t/* index_fd() closed the file descriptor already */\n \t}\ndiff --git a/sha1_file.c b/sha1_file.c\nindex d0f2aa0..dd013d5 100644\n--- a/sha1_file.c\n+++ b/sha1_file.c\n@@ -3275,6 +3275,7 @@ static int index_mem(unsigned char *sha1, void *buf, size_t size,\n {\n \tint ret, re_allocated = 0;\n \tint write_object = flags & HASH_WRITE_OBJECT;\n+\tconst int valid_sha1 = flags & HASH_CE_HAS_SHA1;\n \n \tif (!type)\n \t\ttype = OBJ_BLOB;\n@@ -3284,8 +3285,11 @@ static int index_mem(unsigned char *sha1, void *buf, size_t size,\n \t */\n \tif ((type == OBJ_BLOB) && path) {\n \t\tstruct strbuf nbuf = STRBUF_INIT;\n-\t\tif (convert_to_git(path, buf, size, &nbuf,\n-\t\t\t\t write_object ? safe_crlf : SAFE_CRLF_FALSE)) {\n+\t\tif (convert_to_git_ce_sha1(path,\n+\t\t\t\t\t valid_sha1 ? sha1 : NULL,\n+\t\t\t\t\t buf, size, &nbuf,\n+\t\t\t\t\t write_object ? safe_crlf : SAFE_CRLF_FALSE)){\n+\n \t\t\tbuf = strbuf_detach(&nbuf, &size);\n \t\t\tre_allocated = 1;\n \t\t}\n@@ -3313,12 +3317,15 @@ static int index_stream_convert_blob(unsigned char *sha1, int fd,\n {\n \tint ret;\n \tconst int write_object = flags & HASH_WRITE_OBJECT;\n+\tconst int valid_sha1 = flags & HASH_CE_HAS_SHA1;\n \tstruct strbuf sbuf = STRBUF_INIT;\n \n \tassert(path);\n \tassert(would_convert_to_git_filter_fd(path));\n \n-\tconvert_to_git_filter_fd(path, fd, &sbuf,\n+\tconvert_to_git_filter_fd(path,\n+\t\t\t\t valid_sha1 ? sha1 : NULL,\n+\t\t\t\t fd, &sbuf,\n \t\t\t\t write_object ? safe_crlf : SAFE_CRLF_FALSE);\n \n \tif (write_object)\n@@ -3396,6 +3403,8 @@ int index_fd(unsigned char *sha1, int fd, struct stat *st,\n \t enum object_type type, const char *path, unsigned flags)\n {\n \tint ret;\n+\tconst unsigned char *sha1_ce;\n+\tsha1_ce = flags & HASH_CE_HAS_SHA1 ? sha1 : NULL;\n \n \t/*\n \t * Call xsize_t() only when needed to avoid potentially unnecessary\n@@ -3406,7 +3415,7 @@ int index_fd(unsigned char *sha1, int fd, struct stat *st,\n \telse if (!S_ISREG(st->st_mode))\n \t\tret = index_pipe(sha1, fd, type, path, flags);\n \telse if (st->st_size <= big_file_threshold || type != OBJ_BLOB ||\n-\t\t (path && would_convert_to_git(path)))\n+\t\t (path && would_convert_to_git_ce_sha1(path,sha1_ce)))\n \t\tret = index_core(sha1, fd, xsize_t(st->st_size), type, path,\n \t\t\t\t flags);\n \telse\n-- \n2.0.0.rc1.6318.g0c2c796\n"},{"id":"286592","messageId":"1463317350-2749-1-git-send-email-tboegi@web.de","threadId":"42304","inReplyTo":"20160513134953.GE2345@dinwoodie.org","subject":"[PATCH v2 3/3] t6038; use crlf on all platforms","fromName":"","fromEmail":"tboegi@web.de","sentAt":"2016-05-15T13:02:30Z","receivedAt":"2016-05-15T13:02:30Z","isPatch":true,"sender":{"key":"tboegi@web.de","avatar":"https://avatars.githubusercontent.com/u/7138363?v=4"},"body":"From: Torsten Bögershausen \n\nt6038 uses different code, dependig if NATIVE_CRLF is set ot not.\nWhen the native line endings are LF, merge.renormalize is not tested very well.\nChange the test to always use CRLF by setting core.eol=crlf.\n---\n t/t6038-merge-text-auto.sh | 37 +++++++++++--------------------------\n 1 file changed, 11 insertions(+), 26 deletions(-)\n\ndiff --git a/t/t6038-merge-text-auto.sh b/t/t6038-merge-text-auto.sh\nindex 85c10b0..4dc8c1a 100755\n--- a/t/t6038-merge-text-auto.sh\n+++ b/t/t6038-merge-text-auto.sh\n@@ -18,6 +18,7 @@ test_have_prereq SED_STRIPS_CR && SED_OPTIONS=-b\n \n test_expect_success setup '\n \tgit config core.autocrlf false &&\n+\tgit config core.eol crlf &&\n \n \techo first line | append_cr >file &&\n \techo first line >control_file &&\n@@ -72,10 +73,8 @@ test_expect_success 'Merge after setting text=auto' '\n \tsame line\n \tEOF\n \n-\tif test_have_prereq NATIVE_CRLF; then\n-\t\tappend_cr expected.temp &&\n-\t\tmv expected.temp expected\n-\tfi &&\n+\tappend_cr expected.temp &&\n+\tmv expected.temp expected &&\n \tgit config merge.renormalize true &&\n \tgit rm -fr . &&\n \trm -f .gitattributes &&\n@@ -90,10 +89,8 @@ test_expect_success 'Merge addition of text=auto' '\n \tsame line\n \tEOF\n \n-\tif test_have_prereq NATIVE_CRLF; then\n-\t\tappend_cr expected.temp &&\n-\t\tmv expected.temp expected\n-\tfi &&\n+\tappend_cr expected.temp &&\n+\tmv expected.temp expected &&\n \tgit config merge.renormalize true &&\n \tgit rm -fr . &&\n \trm -f .gitattributes &&\n@@ -104,15 +101,9 @@ test_expect_success 'Merge addition of text=auto' '\n \n test_expect_success 'Detect CRLF/LF conflict after setting text=auto' '\n \techo \"<<<<<<<\" >expected &&\n-\tif test_have_prereq NATIVE_CRLF; then\n-\t\techo first line | append_cr >>expected &&\n-\t\techo same line | append_cr >>expected &&\n-\t\techo ======= | append_cr >>expected\n-\telse\n-\t\techo first line >>expected &&\n-\t\techo same line >>expected &&\n-\t\techo ======= >>expected\n-\tfi &&\n+\techo first line | append_cr >>expected &&\n+\techo same line | append_cr >>expected &&\n+\techo ======= | append_cr >>expected &&\n \techo first line | append_cr >>expected &&\n \techo same line | append_cr >>expected &&\n \techo \">>>>>>>\" >>expected &&\n@@ -128,15 +119,9 @@ test_expect_success 'Detect LF/CRLF conflict from addition of text=auto' '\n \techo \"<<<<<<<\" >expected &&\n \techo first line | append_cr >>expected &&\n \techo same line | append_cr >>expected &&\n-\tif test_have_prereq NATIVE_CRLF; then\n-\t\techo ======= | append_cr >>expected &&\n-\t\techo first line | append_cr >>expected &&\n-\t\techo same line | append_cr >>expected\n-\telse\n-\t\techo ======= >>expected &&\n-\t\techo first line >>expected &&\n-\t\techo same line >>expected\n-\tfi &&\n+\techo ======= | append_cr >>expected &&\n+\techo first line | append_cr >>expected &&\n+\techo same line | append_cr >>expected &&\n \techo \">>>>>>>\" >>expected &&\n \tgit config merge.renormalize false &&\n \trm -f .gitattributes &&\n-- \n2.0.0.rc1.6318.g0c2c796\n"},{"id":"286610","messageId":"xmqq7fev55qk.fsf@gitster.mtv.corp.google.com","threadId":"42304","inReplyTo":"1463294299-20256-1-git-send-email-tboegi@web.de","subject":"Re: [PATCH v1 3/3] convert: ce_compare_data() checks for a sha1 of a path","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-05-15T22:14:11Z","receivedAt":"2016-05-15T22:14:11Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"tboegi@web.de writes:\n\n> -static int has_cr_in_index(const char *path)\n> +static int has_cr_in_index(const char *path, const unsigned char *sha1)\n> {\n> \tunsigned long sz;\n> \tvoid *data;\n> \tint has_cr;\n> -\n> -\tdata = read_blob_data_from_cache(path, &sz);\n> -\tif (!data)\n> +\tenum object_type type;\n> +\tif (!sha1)\n> +\t\tsha1 = get_sha1_from_cache(path);\n> +\tif (!sha1)\n> +\t\treturn 0;\n> +\tdata = read_sha1_file(sha1, &type, &sz);\n> +\tif (!data || type != OBJ_BLOB) {\n> +\t\tfree(data);\n> \t\treturn 0;\n> +\t}\n> +\n> \thas_cr = memchr(data, '\\r', sz) != NULL;\n> \tfree(data);\n> \treturn has_cr;\n> }\n\nDoes this really need 2/3? Wouldn't this be equivalent to\n\n\tif (!sha1) {\n \tdata = read_blob_data_from_cache(path, &sz);\n\t} else {\n \tdata = read_sha1_file(sha1, &type, &sz);\n\t}\n\tif (!data || type != OBJ_BLOB) {\n \tfree(data);\n return 0;\n\t}\n\n has_cr = ...\n"},{"id":"286667","messageId":"1463413889-12490-1-git-send-email-tboegi@web.de","threadId":"42304","inReplyTo":"xmqq7fev55qk.fsf@gitster.mtv.corp.google.com","subject":"[PATCH v3 0/1] CRLF-Handling: bug fix around ce_compare_data()","fromName":"","fromEmail":"tboegi@web.de","sentAt":"2016-05-16T15:51:29Z","receivedAt":"2016-05-16T15:51:29Z","isPatch":true,"sender":{"key":"tboegi@web.de","avatar":"https://avatars.githubusercontent.com/u/7138363?v=4"},"body":"From: Torsten Bögershausen \n\nChanges since v2:\n\n- Only 1 patch, the t6038 needs to go as a separate \"series\"\n\nhas_cr_in_index() uses either\n\tdata = read_sha1_file(sha1, &type, &sz);\nor\n\tdata = read_blob_data_from_cache(path, &sz);\n\n(I like v2 better, since there is a single code to get a sha object\ninto memory, which later will be replaced by a streaming object)\nBut in any case, here is v3\n\nTorsten Bögershausen (1):\n convert: ce_compare_data() checks for a sha1 of a path\n\n cache.h | 1 +\n convert.c | 35 +++++++++++++++++++++--------------\n convert.h | 23 +++++++++++++++++++----\n read-cache.c | 4 +++-\n sha1_file.c | 17 +++++++++++++----\n\n-- \n2.0.0.rc1.6318.g0c2c796\n"},{"id":"286666","messageId":"1463413891-12529-1-git-send-email-tboegi@web.de","threadId":"42304","inReplyTo":"xmqq7fev55qk.fsf@gitster.mtv.corp.google.com","subject":"[PATCH v3 1/1] convert: ce_compare_data() checks for a sha1 of a path","fromName":"","fromEmail":"tboegi@web.de","sentAt":"2016-05-16T15:51:31Z","receivedAt":"2016-05-16T15:51:31Z","isPatch":true,"sender":{"key":"tboegi@web.de","avatar":"https://avatars.githubusercontent.com/u/7138363?v=4"},"body":"From: Torsten Bögershausen \n\nTo compare a file in working tree with the index, convert_to_git() is used,\nthe the result is hashed and the hash value compared with ce->sha1.\n\nDeep down would_convert_crlf_at_commit() is invoked, to check if CRLF\nare converted or not: When a CRLF had been in the index before, CRLF in\nthe working tree are not converted.\n\nWhile in a merge, a file name in the working tree has different blobs\nin the index with different hash values.\nForwarding ce->sha1 from ce_compare_data() into crlf_to_git() makes sure\nthe would_convert_crlf_at_commit() looks at the appropriate blob.\n\nSigned-off-by: Torsten Bögershausen \n---\n cache.h | 1 +\n convert.c | 35 +++++++++++++++++++++--------------\n convert.h | 23 +++++++++++++++++++----\n read-cache.c | 4 +++-\n sha1_file.c | 17 +++++++++++++----\n 5 files changed, 57 insertions(+), 23 deletions(-)\n\ndiff --git a/cache.h b/cache.h\nindex 160f8e3..5b25462 100644\n--- a/cache.h\n+++ b/cache.h\n@@ -604,6 +604,7 @@ extern int ie_modified(const struct index_state *, const struct cache_entry *, s\n \n #define HASH_WRITE_OBJECT 1\n #define HASH_FORMAT_CHECK 2\n+#define HASH_CE_HAS_SHA1 4\n extern int index_fd(unsigned char *sha1, int fd, struct stat *st, enum object_type type, const char *path, unsigned flags);\n extern int index_path(unsigned char *sha1, const char *path, struct stat *st, unsigned flags);\n \ndiff --git a/convert.c b/convert.c\nindex f524b8d..dc86899 100644\n--- a/convert.c\n+++ b/convert.c\n@@ -217,21 +217,25 @@ static void check_safe_crlf(const char *path, enum crlf_action crlf_action,\n \t}\n }\n \n-static int has_cr_in_index(const char *path)\n+static int has_cr_in_index(const char *path, const unsigned char *sha1)\n {\n \tunsigned long sz;\n \tvoid *data;\n-\tint has_cr;\n-\n-\tdata = read_blob_data_from_cache(path, &sz);\n-\tif (!data)\n-\t\treturn 0;\n-\thas_cr = memchr(data, '\\r', sz) != NULL;\n+\tint has_cr = 0;\n+\tenum object_type type = OBJ_BLOB;\n+\tif (sha1)\n+\t\tdata = read_sha1_file(sha1, &type, &sz);\n+\telse\n+\t\tdata = read_blob_data_from_cache(path, &sz);\n+\n+\tif (data)\n+\t\thas_cr = memchr(data, '\\r', sz) != NULL;\n \tfree(data);\n \treturn has_cr;\n }\n \n-static int crlf_to_git(const char *path, const char *src, size_t len,\n+static int crlf_to_git(const char *path, const unsigned char *sha1,\n+\t\t const char *src, size_t len,\n \t\t struct strbuf *buf,\n \t\t enum crlf_action crlf_action, enum safe_crlf checksafe)\n {\n@@ -260,7 +264,7 @@ static int crlf_to_git(const char *path, const char *src, size_t len,\n \t\t\t * If the file in the index has any CR in it, do not convert.\n \t\t\t * This is the new safer autocrlf handling.\n \t\t\t */\n-\t\t\tif (has_cr_in_index(path))\n+\t\t\tif (has_cr_in_index(path, sha1))\n \t\t\t\treturn 0;\n \t\t}\n \t}\n@@ -852,8 +856,9 @@ const char *get_convert_attr_ascii(const char *path)\n \treturn \"\";\n }\n \n-int convert_to_git(const char *path, const char *src, size_t len,\n- struct strbuf *dst, enum safe_crlf checksafe)\n+int convert_to_git_ce_sha1(const char *path, const unsigned char *sha1,\n+\t\t\t const char *src, size_t len,\n+\t\t\t struct strbuf *dst, enum safe_crlf checksafe)\n {\n \tint ret = 0;\n \tconst char *filter = NULL;\n@@ -874,7 +879,7 @@ int convert_to_git(const char *path, const char *src, size_t len,\n \t\tsrc = dst->buf;\n \t\tlen = dst->len;\n \t}\n-\tret |= crlf_to_git(path, src, len, dst, ca.crlf_action, checksafe);\n+\tret |= crlf_to_git(path, sha1, src, len, dst, ca.crlf_action, checksafe);\n \tif (ret && dst) {\n \t\tsrc = dst->buf;\n \t\tlen = dst->len;\n@@ -882,7 +887,9 @@ int convert_to_git(const char *path, const char *src, size_t len,\n \treturn ret | ident_to_git(path, src, len, dst, ca.ident);\n }\n \n-void convert_to_git_filter_fd(const char *path, int fd, struct strbuf *dst,\n+void convert_to_git_filter_fd(const char *path,\n+\t\t\t const unsigned char *sha1,\n+\t\t\t int fd, struct strbuf *dst,\n \t\t\t enum safe_crlf checksafe)\n {\n \tstruct conv_attrs ca;\n@@ -894,7 +901,7 @@ void convert_to_git_filter_fd(const char *path, int fd, struct strbuf *dst,\n \tif (!apply_filter(path, NULL, 0, fd, dst, ca.drv->clean))\n \t\tdie(\"%s: clean filter '%s' failed\", path, ca.drv->name);\n \n-\tcrlf_to_git(path, dst->buf, dst->len, dst, ca.crlf_action, checksafe);\n+\tcrlf_to_git(path, sha1, dst->buf, dst->len, dst, ca.crlf_action, checksafe);\n \tident_to_git(path, dst->buf, dst->len, dst, ca.ident);\n }\n \ndiff --git a/convert.h b/convert.h\nindex ccf436b..1634d37 100644\n--- a/convert.h\n+++ b/convert.h\n@@ -37,8 +37,16 @@ extern const char *get_wt_convert_stats_ascii(const char *path);\n extern const char *get_convert_attr_ascii(const char *path);\n \n /* returns 1 if *dst was used */\n-extern int convert_to_git(const char *path, const char *src, size_t len,\n-\t\t\t struct strbuf *dst, enum safe_crlf checksafe);\n+extern int convert_to_git_ce_sha1(const char *path, const unsigned char *sha1,\n+\t\t\t\t const char *src, size_t len,\n+\t\t\t\t struct strbuf *dst, enum safe_crlf checksafe);\n+\n+static inline int convert_to_git(const char *path, const char *src, size_t len,\n+\t\t\t\t struct strbuf *dst, enum safe_crlf checksafe)\n+{\n+\treturn convert_to_git_ce_sha1(path, NULL, src, len, dst, checksafe);\n+}\n+\n extern int convert_to_working_tree(const char *path, const char *src,\n \t\t\t\t size_t len, struct strbuf *dst);\n extern int renormalize_buffer(const char *path, const char *src, size_t len,\n@@ -47,9 +55,16 @@ static inline int would_convert_to_git(const char *path)\n {\n \treturn convert_to_git(path, NULL, 0, NULL, 0);\n }\n+static inline int would_convert_to_git_ce_sha1(const char *path,\n+\t\t\t\t\t const unsigned char *sha1)\n+{\n+\treturn convert_to_git_ce_sha1(path, sha1, NULL, 0, NULL, 0);\n+}\n+\n /* Precondition: would_convert_to_git_filter_fd(path) == true */\n-extern void convert_to_git_filter_fd(const char *path, int fd,\n-\t\t\t\t struct strbuf *dst,\n+extern void convert_to_git_filter_fd(const char *path,\n+\t\t\t\t const unsigned char *sha1,\n+\t\t\t\t int fd, struct strbuf *dst,\n \t\t\t\t enum safe_crlf checksafe);\n extern int would_convert_to_git_filter_fd(const char *path);\n \ndiff --git a/read-cache.c b/read-cache.c\nindex d9fb78b..4b0e3c3 100644\n--- a/read-cache.c\n+++ b/read-cache.c\n@@ -163,7 +163,9 @@ static int ce_compare_data(const struct cache_entry *ce, struct stat *st)\n \n \tif (fd >= 0) {\n \t\tunsigned char sha1[20];\n-\t\tif (!index_fd(sha1, fd, st, OBJ_BLOB, ce->name, 0))\n+\t\tunsigned flags = HASH_CE_HAS_SHA1;\n+\t\tmemcpy(sha1, ce->sha1, sizeof(sha1));\n+\t\tif (!index_fd(sha1, fd, st, OBJ_BLOB, ce->name, flags))\n \t\t\tmatch = hashcmp(sha1, ce->sha1);\n \t\t/* index_fd() closed the file descriptor already */\n \t}\ndiff --git a/sha1_file.c b/sha1_file.c\nindex d0f2aa0..dd013d5 100644\n--- a/sha1_file.c\n+++ b/sha1_file.c\n@@ -3275,6 +3275,7 @@ static int index_mem(unsigned char *sha1, void *buf, size_t size,\n {\n \tint ret, re_allocated = 0;\n \tint write_object = flags & HASH_WRITE_OBJECT;\n+\tconst int valid_sha1 = flags & HASH_CE_HAS_SHA1;\n \n \tif (!type)\n \t\ttype = OBJ_BLOB;\n@@ -3284,8 +3285,11 @@ static int index_mem(unsigned char *sha1, void *buf, size_t size,\n \t */\n \tif ((type == OBJ_BLOB) && path) {\n \t\tstruct strbuf nbuf = STRBUF_INIT;\n-\t\tif (convert_to_git(path, buf, size, &nbuf,\n-\t\t\t\t write_object ? safe_crlf : SAFE_CRLF_FALSE)) {\n+\t\tif (convert_to_git_ce_sha1(path,\n+\t\t\t\t\t valid_sha1 ? sha1 : NULL,\n+\t\t\t\t\t buf, size, &nbuf,\n+\t\t\t\t\t write_object ? safe_crlf : SAFE_CRLF_FALSE)){\n+\n \t\t\tbuf = strbuf_detach(&nbuf, &size);\n \t\t\tre_allocated = 1;\n \t\t}\n@@ -3313,12 +3317,15 @@ static int index_stream_convert_blob(unsigned char *sha1, int fd,\n {\n \tint ret;\n \tconst int write_object = flags & HASH_WRITE_OBJECT;\n+\tconst int valid_sha1 = flags & HASH_CE_HAS_SHA1;\n \tstruct strbuf sbuf = STRBUF_INIT;\n \n \tassert(path);\n \tassert(would_convert_to_git_filter_fd(path));\n \n-\tconvert_to_git_filter_fd(path, fd, &sbuf,\n+\tconvert_to_git_filter_fd(path,\n+\t\t\t\t valid_sha1 ? sha1 : NULL,\n+\t\t\t\t fd, &sbuf,\n \t\t\t\t write_object ? safe_crlf : SAFE_CRLF_FALSE);\n \n \tif (write_object)\n@@ -3396,6 +3403,8 @@ int index_fd(unsigned char *sha1, int fd, struct stat *st,\n \t enum object_type type, const char *path, unsigned flags)\n {\n \tint ret;\n+\tconst unsigned char *sha1_ce;\n+\tsha1_ce = flags & HASH_CE_HAS_SHA1 ? sha1 : NULL;\n \n \t/*\n \t * Call xsize_t() only when needed to avoid potentially unnecessary\n@@ -3406,7 +3415,7 @@ int index_fd(unsigned char *sha1, int fd, struct stat *st,\n \telse if (!S_ISREG(st->st_mode))\n \t\tret = index_pipe(sha1, fd, type, path, flags);\n \telse if (st->st_size <= big_file_threshold || type != OBJ_BLOB ||\n-\t\t (path && would_convert_to_git(path)))\n+\t\t (path && would_convert_to_git_ce_sha1(path,sha1_ce)))\n \t\tret = index_core(sha1, fd, xsize_t(st->st_size), type, path,\n \t\t\t\t flags);\n \telse\n-- \n2.0.0.rc1.6318.g0c2c796\n"},{"id":"286672","messageId":"xmqqshxi2d80.fsf@gitster.mtv.corp.google.com","threadId":"42304","inReplyTo":"1463413889-12490-1-git-send-email-tboegi@web.de","subject":"Re: [PATCH v3 0/1] CRLF-Handling: bug fix around ce_compare_data()","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-05-16T16:13:03Z","receivedAt":"2016-05-16T16:13:03Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"tboegi@web.de writes:\n\n> From: Torsten Bögershausen \n>\n> Changes since v2:\n>\n> - Only 1 patch, the t6038 needs to go as a separate \"series\"\n>\n> has_cr_in_index() uses either\n> \tdata = read_sha1_file(sha1, &type, &sz);\n> or\n> \tdata = read_blob_data_from_cache(path, &sz);\n>\n> (I like v2 better, since there is a single code to get a sha object\n> into memory, which later will be replaced by a streaming object)\n\nWait a minute, please. I only asked the reason why you did it that\nway and mentioned that the end result seemed equivalent. \"The end\nresult seems equivalent\" does not automatically mean \"therefore you\nmust avoid changing the code.\"\n\nIf you still prefer the original code, and your preference is backed\nby a solid reasoning, don't change the code to a less preferrable\nversion. You may have to explain what you wrote in () above clearly\nin an updated log message to save other readers from asking the same\nquestion as I asked, though.\n\nThanks.\n"},{"id":"286772","messageId":"573A993F.8020205@web.de","threadId":"42304","inReplyTo":"xmqqshxi2d80.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH v3 0/1] CRLF-Handling: bug fix around ce_compare_data()","fromName":"Torsten Bögershausen","fromEmail":"tboegi@web.de","sentAt":"2016-05-17T04:08:31Z","receivedAt":"2016-05-17T04:08:31Z","isPatch":true,"sender":{"key":"tboegi@web.de","avatar":"https://avatars.githubusercontent.com/u/7138363?v=4"},"body":"On 05/16/2016 06:13 PM, Junio C Hamano wrote:\n> Wait a minute, please. I only asked the reason why you did it that\n> way and mentioned that the end result seemed equivalent. \"The end\n> result seems equivalent\" does not automatically mean \"therefore you\n> must avoid changing the code.\"\n>\n> If you still prefer the original code, and your preference is backed\n> by a solid reasoning, don't change the code to a less preferrable\n> version. You may have to explain what you wrote in () above clearly\n> in an updated log message to save other readers from asking the same\n> question as I asked, though.\n>\n> Thanks.\n> --\n>\nNo problem.\nI will re-send v4 in some time and pull out the update of t6038 into an \nown path\n"},{"id":"286790","messageId":"1463501398-8608-1-git-send-email-tboegi@web.de","threadId":"42304","inReplyTo":"573A993F.8020205@web.de","subject":"[PATCH v1 1/1] t6038; use crlf on all platforms","fromName":"","fromEmail":"tboegi@web.de","sentAt":"2016-05-17T16:09:58Z","receivedAt":"2016-05-17T16:09:58Z","isPatch":true,"sender":{"key":"tboegi@web.de","avatar":"https://avatars.githubusercontent.com/u/7138363?v=4"},"body":"From: Torsten Bögershausen \n\nt6038 uses different code, depending if NATIVE_CRLF is set ot not.\nWhen the native line endings are LF, merge.renormalize is not tested very well.\nChange the test to always use CRLF by setting core.eol=crlf.\n---\nBroke the 10/10 series into smaller pieces, this is the update of t6038\n t/t6038-merge-text-auto.sh | 37 +++++++++++--------------------------\n 1 file changed, 11 insertions(+), 26 deletions(-)\n\ndiff --git a/t/t6038-merge-text-auto.sh b/t/t6038-merge-text-auto.sh\nindex 85c10b0..4dc8c1a 100755\n--- a/t/t6038-merge-text-auto.sh\n+++ b/t/t6038-merge-text-auto.sh\n@@ -18,6 +18,7 @@ test_have_prereq SED_STRIPS_CR && SED_OPTIONS=-b\n \n test_expect_success setup '\n \tgit config core.autocrlf false &&\n+\tgit config core.eol crlf &&\n \n \techo first line | append_cr >file &&\n \techo first line >control_file &&\n@@ -72,10 +73,8 @@ test_expect_success 'Merge after setting text=auto' '\n \tsame line\n \tEOF\n \n-\tif test_have_prereq NATIVE_CRLF; then\n-\t\tappend_cr expected.temp &&\n-\t\tmv expected.temp expected\n-\tfi &&\n+\tappend_cr expected.temp &&\n+\tmv expected.temp expected &&\n \tgit config merge.renormalize true &&\n \tgit rm -fr . &&\n \trm -f .gitattributes &&\n@@ -90,10 +89,8 @@ test_expect_success 'Merge addition of text=auto' '\n \tsame line\n \tEOF\n \n-\tif test_have_prereq NATIVE_CRLF; then\n-\t\tappend_cr expected.temp &&\n-\t\tmv expected.temp expected\n-\tfi &&\n+\tappend_cr expected.temp &&\n+\tmv expected.temp expected &&\n \tgit config merge.renormalize true &&\n \tgit rm -fr . &&\n \trm -f .gitattributes &&\n@@ -104,15 +101,9 @@ test_expect_success 'Merge addition of text=auto' '\n \n test_expect_success 'Detect CRLF/LF conflict after setting text=auto' '\n \techo \"<<<<<<<\" >expected &&\n-\tif test_have_prereq NATIVE_CRLF; then\n-\t\techo first line | append_cr >>expected &&\n-\t\techo same line | append_cr >>expected &&\n-\t\techo ======= | append_cr >>expected\n-\telse\n-\t\techo first line >>expected &&\n-\t\techo same line >>expected &&\n-\t\techo ======= >>expected\n-\tfi &&\n+\techo first line | append_cr >>expected &&\n+\techo same line | append_cr >>expected &&\n+\techo ======= | append_cr >>expected &&\n \techo first line | append_cr >>expected &&\n \techo same line | append_cr >>expected &&\n \techo \">>>>>>>\" >>expected &&\n@@ -128,15 +119,9 @@ test_expect_success 'Detect LF/CRLF conflict from addition of text=auto' '\n \techo \"<<<<<<<\" >expected &&\n \techo first line | append_cr >>expected &&\n \techo same line | append_cr >>expected &&\n-\tif test_have_prereq NATIVE_CRLF; then\n-\t\techo ======= | append_cr >>expected &&\n-\t\techo first line | append_cr >>expected &&\n-\t\techo same line | append_cr >>expected\n-\telse\n-\t\techo ======= >>expected &&\n-\t\techo first line >>expected &&\n-\t\techo same line >>expected\n-\tfi &&\n+\techo ======= | append_cr >>expected &&\n+\techo first line | append_cr >>expected &&\n+\techo same line | append_cr >>expected &&\n \techo \">>>>>>>\" >>expected &&\n \tgit config merge.renormalize false &&\n \trm -f .gitattributes &&\n-- \n2.0.0.rc1.6318.g0c2c796\n"},{"id":"286792","messageId":"1463503266-3552-1-git-send-email-tboegi@web.de","threadId":"42304","inReplyTo":"573A993F.8020205@web.de","subject":"[PATCH v4 0/2] CRLF: ce_compare_data() checks for a sha1 of a path","fromName":"","fromEmail":"tboegi@web.de","sentAt":"2016-05-17T16:41:06Z","receivedAt":"2016-05-17T16:41:06Z","isPatch":true,"sender":{"key":"tboegi@web.de","avatar":"https://avatars.githubusercontent.com/u/7138363?v=4"},"body":"From: Torsten Bögershausen \n\nBreak up the old 10/10 series about CLRF handling into smaller\nseries. This is a small bugfix, when merge.renormalize is used\nwith core.autocrlf (and no attributes are set).\nPrepare the refactoring to use the streaming interface.\n\nTorsten Bögershausen (2):\n read-cache: factor out get_sha1_from_index() helper\n convert: ce_compare_data() checks for a sha1 of a path\n\n cache.h | 4 ++++\n convert.c | 34 ++++++++++++++++++++++------------\n convert.h | 23 +++++++++++++++++++----\n read-cache.c | 33 +++++++++++++++++++++------------\n sha1_file.c | 17 +++++++++++++----\n 5 files changed, 79 insertions(+), 32 deletions(-)\n\n-- \n2.0.0.rc1.6318.g0c2c796\n"},{"id":"286793","messageId":"1463503284-3592-1-git-send-email-tboegi@web.de","threadId":"42304","inReplyTo":"573A993F.8020205@web.de","subject":"[PATCH v4 1/2] read-cache: factor out get_sha1_from_index() helper","fromName":"","fromEmail":"tboegi@web.de","sentAt":"2016-05-17T16:41:24Z","receivedAt":"2016-05-17T16:41:24Z","isPatch":true,"sender":{"key":"tboegi@web.de","avatar":"https://avatars.githubusercontent.com/u/7138363?v=4"},"body":"From: Torsten Bögershausen \n\nFactor out the retrieval of the sha1 for a given path in\nread_blob_data_from_index() into the function get_sha1_from_index().\n\nThis will be used in the next commit, when convert.c can do the\nanalyze for \"text=auto\" without slurping the whole blob into memory\nat once.\n\nAdd a wrapper definition get_sha1_from_cache().\n\nSigned-off-by: Torsten Bögershausen \n---\n cache.h | 3 +++\n read-cache.c | 29 ++++++++++++++++++-----------\n 2 files changed, 21 insertions(+), 11 deletions(-)\n\ndiff --git a/cache.h b/cache.h\nindex 160f8e3..15a2a10 100644\n--- a/cache.h\n+++ b/cache.h\n@@ -379,6 +379,7 @@ extern void free_name_hash(struct index_state *istate);\n #define unmerge_cache_entry_at(at) unmerge_index_entry_at(&the_index, at)\n #define unmerge_cache(pathspec) unmerge_index(&the_index, pathspec)\n #define read_blob_data_from_cache(path, sz) read_blob_data_from_index(&the_index, (path), (sz))\n+#define get_sha1_from_cache(path) get_sha1_from_index (&the_index, (path))\n #endif\n \n enum object_type {\n@@ -1038,6 +1039,8 @@ static inline void *read_sha1_file(const unsigned char *sha1, enum object_type *\n \treturn read_sha1_file_extended(sha1, type, size, LOOKUP_REPLACE_OBJECT);\n }\n \n+const unsigned char *get_sha1_from_index(struct index_state *istate, const char *path);\n+\n /*\n * This internal function is only declared here for the benefit of\n * lookup_replace_object(). Please do not call it directly.\ndiff --git a/read-cache.c b/read-cache.c\nindex d9fb78b..a3ef967 100644\n--- a/read-cache.c\n+++ b/read-cache.c\n@@ -2263,13 +2263,27 @@ int index_name_is_other(const struct index_state *istate, const char *name,\n \n void *read_blob_data_from_index(struct index_state *istate, const char *path, unsigned long *size)\n {\n-\tint pos, len;\n+\tconst unsigned char *sha1;\n \tunsigned long sz;\n \tenum object_type type;\n \tvoid *data;\n \n-\tlen = strlen(path);\n-\tpos = index_name_pos(istate, path, len);\n+\tsha1 = get_sha1_from_index(istate, path);\n+\tif (!sha1)\n+\t\treturn NULL;\n+\tdata = read_sha1_file(sha1, &type, &sz);\n+\tif (!data || type != OBJ_BLOB) {\n+\t\tfree(data);\n+\t\treturn NULL;\n+\t}\n+\tif (size)\n+\t\t*size = sz;\n+\treturn data;\n+}\n+\n+const unsigned char *get_sha1_from_index(struct index_state *istate, const char *path)\n+{\n+\tint pos = index_name_pos(istate, path, strlen(path));\n \tif (pos < 0) {\n \t\t/*\n \t\t * We might be in the middle of a merge, in which\n@@ -2285,14 +2299,7 @@ void *read_blob_data_from_index(struct index_state *istate, const char *path, un\n \t}\n \tif (pos < 0)\n \t\treturn NULL;\n-\tdata = read_sha1_file(istate->cache[pos]->sha1, &type, &sz);\n-\tif (!data || type != OBJ_BLOB) {\n-\t\tfree(data);\n-\t\treturn NULL;\n-\t}\n-\tif (size)\n-\t\t*size = sz;\n-\treturn data;\n+\treturn (istate->cache[pos]->sha1);\n }\n \n void stat_validity_clear(struct stat_validity *sv)\n-- \n2.0.0.rc1.6318.g0c2c796\n"},{"id":"286794","messageId":"1463503301-3634-1-git-send-email-tboegi@web.de","threadId":"42304","inReplyTo":"573A993F.8020205@web.de","subject":"[PATCH v4 2/2] convert: ce_compare_data() checks for a sha1 of a path","fromName":"","fromEmail":"tboegi@web.de","sentAt":"2016-05-17T16:41:41Z","receivedAt":"2016-05-17T16:41:41Z","isPatch":true,"sender":{"key":"tboegi@web.de","avatar":"https://avatars.githubusercontent.com/u/7138363?v=4"},"body":"From: Torsten Bögershausen \n\nTo compare a file in working tree with the index, convert_to_git() is used,\nthe the result is hashed and the hash value compared with ce->sha1.\n\nDeep down would_convert_crlf_at_commit() is invoked, to check if CRLF\nare converted or not: When a CRLF had been in the index before, CRLF in\nthe working tree are not converted.\n\nWhile in a merge, a file name in the working tree has different blobs\nin the index with different hash values.\nForwarding ce->sha1 from ce_compare_data() into crlf_to_git() makes sure\nthe would_convert_crlf_at_commit() looks at the appropriate blob.\n\nhas_cr_in_index() does not use read_blob_data_from_cache() any more.\nA single function to read data for a give sha value makes it easier to\nrefactor has_cr_in_index() to use the streaming interface.\n\nSigned-off-by: Torsten Bögershausen \n---\n cache.h | 1 +\n convert.c | 34 ++++++++++++++++++++++------------\n convert.h | 23 +++++++++++++++++++----\n read-cache.c | 4 +++-\n sha1_file.c | 17 +++++++++++++----\n 5 files changed, 58 insertions(+), 21 deletions(-)\n\ndiff --git a/cache.h b/cache.h\nindex 15a2a10..a5136c0 100644\n--- a/cache.h\n+++ b/cache.h\n@@ -605,6 +605,7 @@ extern int ie_modified(const struct index_state *, const struct cache_entry *, s\n \n #define HASH_WRITE_OBJECT 1\n #define HASH_FORMAT_CHECK 2\n+#define HASH_CE_HAS_SHA1 4\n extern int index_fd(unsigned char *sha1, int fd, struct stat *st, enum object_type type, const char *path, unsigned flags);\n extern int index_path(unsigned char *sha1, const char *path, struct stat *st, unsigned flags);\n \ndiff --git a/convert.c b/convert.c\nindex f524b8d..c972d14 100644\n--- a/convert.c\n+++ b/convert.c\n@@ -217,21 +217,28 @@ static void check_safe_crlf(const char *path, enum crlf_action crlf_action,\n \t}\n }\n \n-static int has_cr_in_index(const char *path)\n+static int has_cr_in_index(const char *path, const unsigned char *sha1)\n {\n \tunsigned long sz;\n \tvoid *data;\n-\tint has_cr;\n-\n-\tdata = read_blob_data_from_cache(path, &sz);\n+\tint has_cr = 0;\n+\tenum object_type type;\n+\tif (!sha1)\n+\t\tsha1 = get_sha1_from_cache(path);\n+\tif (!sha1)\n+\t\treturn 0;\n+\tdata = read_sha1_file(sha1, &type, &sz);\n \tif (!data)\n \t\treturn 0;\n-\thas_cr = memchr(data, '\\r', sz) != NULL;\n+\tif (type == OBJ_BLOB)\n+\t\thas_cr = memchr(data, '\\r', sz) != NULL;\n+\n \tfree(data);\n \treturn has_cr;\n }\n \n-static int crlf_to_git(const char *path, const char *src, size_t len,\n+static int crlf_to_git(const char *path, const unsigned char *sha1,\n+\t\t const char *src, size_t len,\n \t\t struct strbuf *buf,\n \t\t enum crlf_action crlf_action, enum safe_crlf checksafe)\n {\n@@ -260,7 +267,7 @@ static int crlf_to_git(const char *path, const char *src, size_t len,\n \t\t\t * If the file in the index has any CR in it, do not convert.\n \t\t\t * This is the new safer autocrlf handling.\n \t\t\t */\n-\t\t\tif (has_cr_in_index(path))\n+\t\t\tif (has_cr_in_index(path, sha1))\n \t\t\t\treturn 0;\n \t\t}\n \t}\n@@ -852,8 +859,9 @@ const char *get_convert_attr_ascii(const char *path)\n \treturn \"\";\n }\n \n-int convert_to_git(const char *path, const char *src, size_t len,\n- struct strbuf *dst, enum safe_crlf checksafe)\n+int convert_to_git_ce_sha1(const char *path, const unsigned char *sha1,\n+\t\t\t const char *src, size_t len,\n+\t\t\t struct strbuf *dst, enum safe_crlf checksafe)\n {\n \tint ret = 0;\n \tconst char *filter = NULL;\n@@ -874,7 +882,7 @@ int convert_to_git(const char *path, const char *src, size_t len,\n \t\tsrc = dst->buf;\n \t\tlen = dst->len;\n \t}\n-\tret |= crlf_to_git(path, src, len, dst, ca.crlf_action, checksafe);\n+\tret |= crlf_to_git(path, sha1, src, len, dst, ca.crlf_action, checksafe);\n \tif (ret && dst) {\n \t\tsrc = dst->buf;\n \t\tlen = dst->len;\n@@ -882,7 +890,9 @@ int convert_to_git(const char *path, const char *src, size_t len,\n \treturn ret | ident_to_git(path, src, len, dst, ca.ident);\n }\n \n-void convert_to_git_filter_fd(const char *path, int fd, struct strbuf *dst,\n+void convert_to_git_filter_fd(const char *path,\n+\t\t\t const unsigned char *sha1,\n+\t\t\t int fd, struct strbuf *dst,\n \t\t\t enum safe_crlf checksafe)\n {\n \tstruct conv_attrs ca;\n@@ -894,7 +904,7 @@ void convert_to_git_filter_fd(const char *path, int fd, struct strbuf *dst,\n \tif (!apply_filter(path, NULL, 0, fd, dst, ca.drv->clean))\n \t\tdie(\"%s: clean filter '%s' failed\", path, ca.drv->name);\n \n-\tcrlf_to_git(path, dst->buf, dst->len, dst, ca.crlf_action, checksafe);\n+\tcrlf_to_git(path, sha1, dst->buf, dst->len, dst, ca.crlf_action, checksafe);\n \tident_to_git(path, dst->buf, dst->len, dst, ca.ident);\n }\n \ndiff --git a/convert.h b/convert.h\nindex ccf436b..1634d37 100644\n--- a/convert.h\n+++ b/convert.h\n@@ -37,8 +37,16 @@ extern const char *get_wt_convert_stats_ascii(const char *path);\n extern const char *get_convert_attr_ascii(const char *path);\n \n /* returns 1 if *dst was used */\n-extern int convert_to_git(const char *path, const char *src, size_t len,\n-\t\t\t struct strbuf *dst, enum safe_crlf checksafe);\n+extern int convert_to_git_ce_sha1(const char *path, const unsigned char *sha1,\n+\t\t\t\t const char *src, size_t len,\n+\t\t\t\t struct strbuf *dst, enum safe_crlf checksafe);\n+\n+static inline int convert_to_git(const char *path, const char *src, size_t len,\n+\t\t\t\t struct strbuf *dst, enum safe_crlf checksafe)\n+{\n+\treturn convert_to_git_ce_sha1(path, NULL, src, len, dst, checksafe);\n+}\n+\n extern int convert_to_working_tree(const char *path, const char *src,\n \t\t\t\t size_t len, struct strbuf *dst);\n extern int renormalize_buffer(const char *path, const char *src, size_t len,\n@@ -47,9 +55,16 @@ static inline int would_convert_to_git(const char *path)\n {\n \treturn convert_to_git(path, NULL, 0, NULL, 0);\n }\n+static inline int would_convert_to_git_ce_sha1(const char *path,\n+\t\t\t\t\t const unsigned char *sha1)\n+{\n+\treturn convert_to_git_ce_sha1(path, sha1, NULL, 0, NULL, 0);\n+}\n+\n /* Precondition: would_convert_to_git_filter_fd(path) == true */\n-extern void convert_to_git_filter_fd(const char *path, int fd,\n-\t\t\t\t struct strbuf *dst,\n+extern void convert_to_git_filter_fd(const char *path,\n+\t\t\t\t const unsigned char *sha1,\n+\t\t\t\t int fd, struct strbuf *dst,\n \t\t\t\t enum safe_crlf checksafe);\n extern int would_convert_to_git_filter_fd(const char *path);\n \ndiff --git a/read-cache.c b/read-cache.c\nindex a3ef967..0ebc237 100644\n--- a/read-cache.c\n+++ b/read-cache.c\n@@ -163,7 +163,9 @@ static int ce_compare_data(const struct cache_entry *ce, struct stat *st)\n \n \tif (fd >= 0) {\n \t\tunsigned char sha1[20];\n-\t\tif (!index_fd(sha1, fd, st, OBJ_BLOB, ce->name, 0))\n+\t\tunsigned flags = HASH_CE_HAS_SHA1;\n+\t\tmemcpy(sha1, ce->sha1, sizeof(sha1));\n+\t\tif (!index_fd(sha1, fd, st, OBJ_BLOB, ce->name, flags))\n \t\t\tmatch = hashcmp(sha1, ce->sha1);\n \t\t/* index_fd() closed the file descriptor already */\n \t}\ndiff --git a/sha1_file.c b/sha1_file.c\nindex d0f2aa0..dd013d5 100644\n--- a/sha1_file.c\n+++ b/sha1_file.c\n@@ -3275,6 +3275,7 @@ static int index_mem(unsigned char *sha1, void *buf, size_t size,\n {\n \tint ret, re_allocated = 0;\n \tint write_object = flags & HASH_WRITE_OBJECT;\n+\tconst int valid_sha1 = flags & HASH_CE_HAS_SHA1;\n \n \tif (!type)\n \t\ttype = OBJ_BLOB;\n@@ -3284,8 +3285,11 @@ static int index_mem(unsigned char *sha1, void *buf, size_t size,\n \t */\n \tif ((type == OBJ_BLOB) && path) {\n \t\tstruct strbuf nbuf = STRBUF_INIT;\n-\t\tif (convert_to_git(path, buf, size, &nbuf,\n-\t\t\t\t write_object ? safe_crlf : SAFE_CRLF_FALSE)) {\n+\t\tif (convert_to_git_ce_sha1(path,\n+\t\t\t\t\t valid_sha1 ? sha1 : NULL,\n+\t\t\t\t\t buf, size, &nbuf,\n+\t\t\t\t\t write_object ? safe_crlf : SAFE_CRLF_FALSE)){\n+\n \t\t\tbuf = strbuf_detach(&nbuf, &size);\n \t\t\tre_allocated = 1;\n \t\t}\n@@ -3313,12 +3317,15 @@ static int index_stream_convert_blob(unsigned char *sha1, int fd,\n {\n \tint ret;\n \tconst int write_object = flags & HASH_WRITE_OBJECT;\n+\tconst int valid_sha1 = flags & HASH_CE_HAS_SHA1;\n \tstruct strbuf sbuf = STRBUF_INIT;\n \n \tassert(path);\n \tassert(would_convert_to_git_filter_fd(path));\n \n-\tconvert_to_git_filter_fd(path, fd, &sbuf,\n+\tconvert_to_git_filter_fd(path,\n+\t\t\t\t valid_sha1 ? sha1 : NULL,\n+\t\t\t\t fd, &sbuf,\n \t\t\t\t write_object ? safe_crlf : SAFE_CRLF_FALSE);\n \n \tif (write_object)\n@@ -3396,6 +3403,8 @@ int index_fd(unsigned char *sha1, int fd, struct stat *st,\n \t enum object_type type, const char *path, unsigned flags)\n {\n \tint ret;\n+\tconst unsigned char *sha1_ce;\n+\tsha1_ce = flags & HASH_CE_HAS_SHA1 ? sha1 : NULL;\n \n \t/*\n \t * Call xsize_t() only when needed to avoid potentially unnecessary\n@@ -3406,7 +3415,7 @@ int index_fd(unsigned char *sha1, int fd, struct stat *st,\n \telse if (!S_ISREG(st->st_mode))\n \t\tret = index_pipe(sha1, fd, type, path, flags);\n \telse if (st->st_size <= big_file_threshold || type != OBJ_BLOB ||\n-\t\t (path && would_convert_to_git(path)))\n+\t\t (path && would_convert_to_git_ce_sha1(path,sha1_ce)))\n \t\tret = index_core(sha1, fd, xsize_t(st->st_size), type, path,\n \t\t\t\t flags);\n \telse\n-- \n2.0.0.rc1.6318.g0c2c796\n"},{"id":"286816","messageId":"xmqqa8jowmu0.fsf@gitster.mtv.corp.google.com","threadId":"42304","inReplyTo":"1463501398-8608-1-git-send-email-tboegi@web.de","subject":"Re: [PATCH v1 1/1] t6038; use crlf on all platforms","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-05-17T18:39:35Z","receivedAt":"2016-05-17T18:39:35Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"tboegi@web.de writes:\n\n> From: Torsten Bögershausen \n>\n> t6038 uses different code, depending if NATIVE_CRLF is set ot not.\n> When the native line endings are LF, merge.renormalize is not tested very well.\n> Change the test to always use CRLF by setting core.eol=crlf.\n> ---\n> Broke the 10/10 series into smaller pieces, this is the update of t6038\n\nThanks.\n\n - Missing sign-off.\n\n - \"... is not tested very well\", which implies \"with this change,\n it will be tested\", is a secondary benefit. The reader need to\n agree that, whether the platform native line ending is CRLF or\n LF, 'git merge' should behave identically on any platform, as\n long as the line ending convention used in the repository is\n explicitly set in the same way, before agreeing that this is a\n good thing to do in general. And that is a bigger benefit, no?\n\n - But doesn't the same principle apply in the other direction?\n When forced to do core.eol=lf, a platform with NATIVE_CRLF should\n behave identically to how a platform without NATIVE_CRLF would?\n\n With this patch, we lose test coverage for core.eol=lf case,\n which used to be tested on non-NATIVE_CRLF platforms. Isn't that\n a concern to us?\n\n> t/t6038-merge-text-auto.sh | 37 +++++++++++--------------------------\n> 1 file changed, 11 insertions(+), 26 deletions(-)\n>\n> diff --git a/t/t6038-merge-text-auto.sh b/t/t6038-merge-text-auto.sh\n> index 85c10b0..4dc8c1a 100755\n> --- a/t/t6038-merge-text-auto.sh\n> +++ b/t/t6038-merge-text-auto.sh\n> @@ -18,6 +18,7 @@ test_have_prereq SED_STRIPS_CR && SED_OPTIONS=-b\n> \n> test_expect_success setup '\n> \tgit config core.autocrlf false &&\n> +\tgit config core.eol crlf &&\n> \n> \techo first line | append_cr >file &&\n> \techo first line >control_file &&\n> @@ -72,10 +73,8 @@ test_expect_success 'Merge after setting text=auto' '\n> \tsame line\n> \tEOF\n> \n> -\tif test_have_prereq NATIVE_CRLF; then\n> -\t\tappend_cr expected.temp &&\n> -\t\tmv expected.temp expected\n> -\tfi &&\n> +\tappend_cr expected.temp &&\n> +\tmv expected.temp expected &&\n> \tgit config merge.renormalize true &&\n> \tgit rm -fr . &&\n> \trm -f .gitattributes &&\n> @@ -90,10 +89,8 @@ test_expect_success 'Merge addition of text=auto' '\n> \tsame line\n> \tEOF\n> \n> -\tif test_have_prereq NATIVE_CRLF; then\n> -\t\tappend_cr expected.temp &&\n> -\t\tmv expected.temp expected\n> -\tfi &&\n> +\tappend_cr expected.temp &&\n> +\tmv expected.temp expected &&\n> \tgit config merge.renormalize true &&\n> \tgit rm -fr . &&\n> \trm -f .gitattributes &&\n> @@ -104,15 +101,9 @@ test_expect_success 'Merge addition of text=auto' '\n> \n> test_expect_success 'Detect CRLF/LF conflict after setting text=auto' '\n> \techo \"<<<<<<<\" >expected &&\n> -\tif test_have_prereq NATIVE_CRLF; then\n> -\t\techo first line | append_cr >>expected &&\n> -\t\techo same line | append_cr >>expected &&\n> -\t\techo ======= | append_cr >>expected\n> -\telse\n> -\t\techo first line >>expected &&\n> -\t\techo same line >>expected &&\n> -\t\techo ======= >>expected\n> -\tfi &&\n> +\techo first line | append_cr >>expected &&\n> +\techo same line | append_cr >>expected &&\n> +\techo ======= | append_cr >>expected &&\n> \techo first line | append_cr >>expected &&\n> \techo same line | append_cr >>expected &&\n> \techo \">>>>>>>\" >>expected &&\n> @@ -128,15 +119,9 @@ test_expect_success 'Detect LF/CRLF conflict from addition of text=auto' '\n> \techo \"<<<<<<<\" >expected &&\n> \techo first line | append_cr >>expected &&\n> \techo same line | append_cr >>expected &&\n> -\tif test_have_prereq NATIVE_CRLF; then\n> -\t\techo ======= | append_cr >>expected &&\n> -\t\techo first line | append_cr >>expected &&\n> -\t\techo same line | append_cr >>expected\n> -\telse\n> -\t\techo ======= >>expected &&\n> -\t\techo first line >>expected &&\n> -\t\techo same line >>expected\n> -\tfi &&\n> +\techo ======= | append_cr >>expected &&\n> +\techo first line | append_cr >>expected &&\n> +\techo same line | append_cr >>expected &&\n> \techo \">>>>>>>\" >>expected &&\n> \tgit config merge.renormalize false &&\n> \trm -f .gitattributes &&\n"},{"id":"286817","messageId":"xmqq60ucwlz8.fsf@gitster.mtv.corp.google.com","threadId":"42304","inReplyTo":"1463503301-3634-1-git-send-email-tboegi@web.de","subject":"Re: [PATCH v4 2/2] convert: ce_compare_data() checks for a sha1 of a path","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-05-17T18:58:03Z","receivedAt":"2016-05-17T18:58:03Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"tboegi@web.de writes:\n\n> #define HASH_WRITE_OBJECT 1\n> #define HASH_FORMAT_CHECK 2\n> +#define HASH_CE_HAS_SHA1 4\n\nHow does one pronounce the words in this constant? Does it make a\nlistener understand what this constant means?\n\n\n/*\n * We need a comment around here to say what these two\n * parameters mean. I am guessing that (1) if sha1 is not NULL,\n * path is ignored and the function inspects if it has CR; (2)\n * otherwise it checks the index entry at path and inspects if\n * it has CR.\n */\n> \n> -static int has_cr_in_index(const char *path)\n> +static int has_cr_in_index(const char *path, const unsigned char *sha1)\n> {\n\nThis makes me seriously wonder if it is a good idea to modify this\nfunction like this. (1) means this function is not about IN INDEX\nat all!\n\nPerhaps add a \"static int blob_has_cr(const unsigned char *sha1)\"\nand call it from the real caller you added that wants to call this\nbutchered two-param version that has sha1 is a better solution?\n\n> -static int crlf_to_git(const char *path, const char *src, size_t len,\n> +static int crlf_to_git(const char *path, const unsigned char *sha1,\n> +\t\t const char *src, size_t len,\n> \t\t struct strbuf *buf,\n> \t\t enum crlf_action crlf_action, enum safe_crlf checksafe)\n> {\n> @@ -260,7 +267,7 @@ static int crlf_to_git(const char *path, const char *src, size_t len,\n> \t\t\t * If the file in the index has any CR in it, do not convert.\n> \t\t\t * This is the new safer autocrlf handling.\n> \t\t\t */\n> -\t\t\tif (has_cr_in_index(path))\n> +\t\t\tif (has_cr_in_index(path, sha1))\n\nI think this change is too ugly. The new \"sha1\" parameter is\ntelling us that \"in order to see if the indexed version has CR, do\nnot look at the index, but look at the contents of this blob\".\n\nBut wouldn't the result become more understandable if instead we\npassed a new parameter \"cr-state-for-conversion\" that takes UNKNOWN,\nHAS_CR, or NO_CR to this function?\n\nIn other words, what if, when the caller knows it does not care\nwhat's in the index but wants to instead see if a different blob has\nCR, we make it the caller's responsibility to figure it out by\ncalling blob_has_cr() before calling into this codepath and pass the\nresult of that check down? When cr-state-for-conversion is UNKNOWN,\nthis code knows that it needs to call has_cr_in_index(path) to\nfigure it out itself. Otherwise, it can and should use the\ncaller-supplied value without asking has_cr_in_index(path).\n"},{"id":"286854","messageId":"573BEEEF.8040305@web.de","threadId":"42304","inReplyTo":"xmqq60ucwlz8.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH v4 2/2] convert: ce_compare_data() checks for a sha1 of a path","fromName":"Torsten Bögershausen","fromEmail":"tboegi@web.de","sentAt":"2016-05-18T04:26:23Z","receivedAt":"2016-05-18T04:26:23Z","isPatch":true,"sender":{"key":"tboegi@web.de","avatar":"https://avatars.githubusercontent.com/u/7138363?v=4"},"body":"On 05/17/2016 08:58 PM, Junio C Hamano wrote:\n> tboegi@web.de writes:\n>\n>> #define HASH_WRITE_OBJECT 1\n>> #define HASH_FORMAT_CHECK 2\n>> +#define HASH_CE_HAS_SHA1 4\n>\n> How does one pronounce the words in this constant? Does it make a\n> listener understand what this constant means?\nHow about\nHASH_USE_SHA1_FROM_CE\nor\nHASH_CE_HAS_VALID_SHA1\n\n\n>\n>\n> /*\n> * We need a comment around here to say what these two\n> * parameters mean. I am guessing that (1) if sha1 is not NULL,\n> * path is ignored and the function inspects if it has CR; (2)\n> * otherwise it checks the index entry at path and inspects if\n> * it has CR.\n> */\n>>\n>> -static int has_cr_in_index(const char *path)\n>> +static int has_cr_in_index(const char *path, const unsigned char *sha1)\n>> {\n>\n> This makes me seriously wonder if it is a good idea to modify this\n> function like this. (1) means this function is not about IN INDEX\n> at all!\n>\n> Perhaps add a \"static int blob_has_cr(const unsigned char *sha1)\"\n> and call it from the real caller you added that wants to call this\n> butchered two-param version that has sha1 is a better solution?\n>\n>> -static int crlf_to_git(const char *path, const char *src, size_t len,\n>> +static int crlf_to_git(const char *path, const unsigned char *sha1,\n>> +\t\t const char *src, size_t len,\n>> \t\t struct strbuf *buf,\n>> \t\t enum crlf_action crlf_action, enum safe_crlf checksafe)\n>> {\n>> @@ -260,7 +267,7 @@ static int crlf_to_git(const char *path, const char *src, size_t len,\n>> \t\t\t * If the file in the index has any CR in it, do not convert.\n>> \t\t\t * This is the new safer autocrlf handling.\n>> \t\t\t */\n>> -\t\t\tif (has_cr_in_index(path))\n>> +\t\t\tif (has_cr_in_index(path, sha1))\n\n>\n> I think this change is too ugly. The new \"sha1\" parameter is\n> telling us that \"in order to see if the indexed version has CR, do\n> not look at the index, but look at the contents of this blob\".\nThanks for some fresh eyes, I guess that\nblob_has_cr(sha1)\nwould make most sense.\n"},{"id":"286864","messageId":"04175530-b0b8-572d-1fea-06a2bd029029@web.de","threadId":"42304","inReplyTo":"573BEEEF.8040305@web.de","subject":"Re: [PATCH v4 2/2] convert: ce_compare_data() checks for a sha1 of a path","fromName":"Torsten Bögershausen","fromEmail":"tboegi@web.de","sentAt":"2016-05-18T15:10:06Z","receivedAt":"2016-05-18T15:10:06Z","isPatch":true,"sender":{"key":"tboegi@web.de","avatar":"https://avatars.githubusercontent.com/u/7138363?v=4"},"body":"On 18.05.16 06:26, Torsten Bögershausen wrote:\n> On 05/17/2016 08:58 PM, Junio C Hamano wrote:\n>> tboegi@web.de writes:\n>>\n>>> #define HASH_WRITE_OBJECT 1\n>>> #define HASH_FORMAT_CHECK 2\n>>> +#define HASH_CE_HAS_SHA1 4\n>>\n>> How does one pronounce the words in this constant? Does it make a\n>> listener understand what this constant means?\n> How about\n> HASH_USE_SHA1_FROM_CE\n> or\n> HASH_CE_HAS_VALID_SHA1\nor, before I send a new patch,\nHASH_USE_SHA_NOT_PATH \n"},{"id":"286982","messageId":"1463667675-25930-1-git-send-email-tboegi@web.de","threadId":"42304","inReplyTo":"573A993F.8020205@web.de","subject":"[PATCH v5 0/2] CRLF: ce_compare_data() checks for a sha1 of a path","fromName":"","fromEmail":"tboegi@web.de","sentAt":"2016-05-19T14:21:15Z","receivedAt":"2016-05-19T14:21:15Z","isPatch":true,"sender":{"key":"tboegi@web.de","avatar":"https://avatars.githubusercontent.com/u/7138363?v=4"},"body":"From: Torsten Bögershausen \n\nBreak up the old 10/10 series about CLRF handling into smaller\nseries. This is a small bugfix, when merge.renormalize is used\nwith core.autocrlf (and no attributes are set).\nPrepare the refactoring to use the streaming interface.\nChanges since v4:\n - Rename #define in cache.h into HASH_USE_SHA_NOT_PATH\n - convert.c: Rename has_cr_in_index into blob_has_cr()\n Better logic when sha1 != NULL,\n Adjusted the commit message\n\nTorsten Bögershausen (2):\n read-cache: factor out get_sha1_from_index() helper\n convert: ce_compare_data() checks for a sha1 of a path\n\n cache.h | 4 ++++\n convert.c | 34 ++++++++++++++++++++++------------\n convert.h | 23 +++++++++++++++++++----\n read-cache.c | 33 +++++++++++++++++++++------------\n sha1_file.c | 17 +++++++++++++----\n 5 files changed, 79 insertions(+), 32 deletions(-)\n\n-- \n2.0.0.rc1.6318.g0c2c796\n"},{"id":"286983","messageId":"1463667678-25969-1-git-send-email-tboegi@web.de","threadId":"42304","inReplyTo":"573A993F.8020205@web.de","subject":"[PATCH v5 1/2] read-cache: factor out get_sha1_from_index() helper","fromName":"","fromEmail":"tboegi@web.de","sentAt":"2016-05-19T14:21:18Z","receivedAt":"2016-05-19T14:21:18Z","isPatch":true,"sender":{"key":"tboegi@web.de","avatar":"https://avatars.githubusercontent.com/u/7138363?v=4"},"body":"From: Torsten Bögershausen \n\nFactor out the retrieval of the sha1 for a given path in\nread_blob_data_from_index() into the function get_sha1_from_index().\n\nThis will be used in the next commit, when convert.c can do the\nanalyze for \"text=auto\" without slurping the whole blob into memory\nat once.\n\nAdd a wrapper definition get_sha1_from_cache().\n\nSigned-off-by: Torsten Bögershausen \n---\n cache.h | 3 +++\n read-cache.c | 29 ++++++++++++++++++-----------\n 2 files changed, 21 insertions(+), 11 deletions(-)\n\ndiff --git a/cache.h b/cache.h\nindex 160f8e3..15a2a10 100644\n--- a/cache.h\n+++ b/cache.h\n@@ -379,6 +379,7 @@ extern void free_name_hash(struct index_state *istate);\n #define unmerge_cache_entry_at(at) unmerge_index_entry_at(&the_index, at)\n #define unmerge_cache(pathspec) unmerge_index(&the_index, pathspec)\n #define read_blob_data_from_cache(path, sz) read_blob_data_from_index(&the_index, (path), (sz))\n+#define get_sha1_from_cache(path) get_sha1_from_index (&the_index, (path))\n #endif\n \n enum object_type {\n@@ -1038,6 +1039,8 @@ static inline void *read_sha1_file(const unsigned char *sha1, enum object_type *\n \treturn read_sha1_file_extended(sha1, type, size, LOOKUP_REPLACE_OBJECT);\n }\n \n+const unsigned char *get_sha1_from_index(struct index_state *istate, const char *path);\n+\n /*\n * This internal function is only declared here for the benefit of\n * lookup_replace_object(). Please do not call it directly.\ndiff --git a/read-cache.c b/read-cache.c\nindex d9fb78b..a3ef967 100644\n--- a/read-cache.c\n+++ b/read-cache.c\n@@ -2263,13 +2263,27 @@ int index_name_is_other(const struct index_state *istate, const char *name,\n \n void *read_blob_data_from_index(struct index_state *istate, const char *path, unsigned long *size)\n {\n-\tint pos, len;\n+\tconst unsigned char *sha1;\n \tunsigned long sz;\n \tenum object_type type;\n \tvoid *data;\n \n-\tlen = strlen(path);\n-\tpos = index_name_pos(istate, path, len);\n+\tsha1 = get_sha1_from_index(istate, path);\n+\tif (!sha1)\n+\t\treturn NULL;\n+\tdata = read_sha1_file(sha1, &type, &sz);\n+\tif (!data || type != OBJ_BLOB) {\n+\t\tfree(data);\n+\t\treturn NULL;\n+\t}\n+\tif (size)\n+\t\t*size = sz;\n+\treturn data;\n+}\n+\n+const unsigned char *get_sha1_from_index(struct index_state *istate, const char *path)\n+{\n+\tint pos = index_name_pos(istate, path, strlen(path));\n \tif (pos < 0) {\n \t\t/*\n \t\t * We might be in the middle of a merge, in which\n@@ -2285,14 +2299,7 @@ void *read_blob_data_from_index(struct index_state *istate, const char *path, un\n \t}\n \tif (pos < 0)\n \t\treturn NULL;\n-\tdata = read_sha1_file(istate->cache[pos]->sha1, &type, &sz);\n-\tif (!data || type != OBJ_BLOB) {\n-\t\tfree(data);\n-\t\treturn NULL;\n-\t}\n-\tif (size)\n-\t\t*size = sz;\n-\treturn data;\n+\treturn (istate->cache[pos]->sha1);\n }\n \n void stat_validity_clear(struct stat_validity *sv)\n-- \n2.0.0.rc1.6318.g0c2c796\n"},{"id":"286981","messageId":"1463667680-26008-1-git-send-email-tboegi@web.de","threadId":"42304","inReplyTo":"573A993F.8020205@web.de","subject":"[PATCH v5 2/2] convert: ce_compare_data() checks for a sha1 of a path","fromName":"","fromEmail":"tboegi@web.de","sentAt":"2016-05-19T14:21:20Z","receivedAt":"2016-05-19T14:21:20Z","isPatch":true,"sender":{"key":"tboegi@web.de","avatar":"https://avatars.githubusercontent.com/u/7138363?v=4"},"body":"From: Torsten Bögershausen \n\nTo compare a file in working tree with the index, convert_to_git() is used,\nthe the result is hashed and the hash value compared with ce->sha1.\n\nDeep down would_convert_crlf_at_commit() is invoked, to check if CRLF\nare converted or not: When a CRLF had been in the index before, CRLF in\nthe working tree are not converted.\n\nWhile in a merge, a file name in the working tree has different blobs\nin the index with different hash values.\nForwarding ce->sha1 from ce_compare_data() into crlf_to_git() makes sure\nthe would_convert_crlf_at_commit() looks at the appropriate blob.\n\nWhile at it, rename has_cr_in_index() into blob_has_cr()\n\nSigned-off-by: Torsten Bögershausen \n---\n cache.h | 1 +\n convert.c | 34 ++++++++++++++++++++++------------\n convert.h | 23 +++++++++++++++++++----\n read-cache.c | 4 +++-\n sha1_file.c | 17 +++++++++++++----\n 5 files changed, 58 insertions(+), 21 deletions(-)\n\ndiff --git a/cache.h b/cache.h\nindex 15a2a10..868599e 100644\n--- a/cache.h\n+++ b/cache.h\n@@ -605,6 +605,7 @@ extern int ie_modified(const struct index_state *, const struct cache_entry *, s\n \n #define HASH_WRITE_OBJECT 1\n #define HASH_FORMAT_CHECK 2\n+#define HASH_USE_SHA_NOT_PATH 4\n extern int index_fd(unsigned char *sha1, int fd, struct stat *st, enum object_type type, const char *path, unsigned flags);\n extern int index_path(unsigned char *sha1, const char *path, struct stat *st, unsigned flags);\n \ndiff --git a/convert.c b/convert.c\nindex f524b8d..92ddfb1 100644\n--- a/convert.c\n+++ b/convert.c\n@@ -217,21 +217,26 @@ static void check_safe_crlf(const char *path, enum crlf_action crlf_action,\n \t}\n }\n \n-static int has_cr_in_index(const char *path)\n+static int blob_has_cr(const unsigned char *sha1)\n {\n \tunsigned long sz;\n \tvoid *data;\n-\tint has_cr;\n-\n-\tdata = read_blob_data_from_cache(path, &sz);\n+\tint has_cr = 0;\n+\tenum object_type type;\n+\tif (!sha1)\n+\t\treturn 0;\n+\tdata = read_sha1_file(sha1, &type, &sz);\n \tif (!data)\n \t\treturn 0;\n-\thas_cr = memchr(data, '\\r', sz) != NULL;\n+\tif (type == OBJ_BLOB)\n+\t\thas_cr = memchr(data, '\\r', sz) != NULL;\n+\n \tfree(data);\n \treturn has_cr;\n }\n \n-static int crlf_to_git(const char *path, const char *src, size_t len,\n+static int crlf_to_git(const char *path, const unsigned char *sha1,\n+\t\t const char *src, size_t len,\n \t\t struct strbuf *buf,\n \t\t enum crlf_action crlf_action, enum safe_crlf checksafe)\n {\n@@ -260,7 +265,9 @@ static int crlf_to_git(const char *path, const char *src, size_t len,\n \t\t\t * If the file in the index has any CR in it, do not convert.\n \t\t\t * This is the new safer autocrlf handling.\n \t\t\t */\n-\t\t\tif (has_cr_in_index(path))\n+\t\t\tif (!sha1)\n+\t\t\t\tsha1 = get_sha1_from_cache(path);\n+\t\t\tif (blob_has_cr(sha1))\n \t\t\t\treturn 0;\n \t\t}\n \t}\n@@ -852,8 +859,9 @@ const char *get_convert_attr_ascii(const char *path)\n \treturn \"\";\n }\n \n-int convert_to_git(const char *path, const char *src, size_t len,\n- struct strbuf *dst, enum safe_crlf checksafe)\n+int convert_to_git_ce_sha1(const char *path, const unsigned char *sha1,\n+\t\t\t const char *src, size_t len,\n+\t\t\t struct strbuf *dst, enum safe_crlf checksafe)\n {\n \tint ret = 0;\n \tconst char *filter = NULL;\n@@ -874,7 +882,7 @@ int convert_to_git(const char *path, const char *src, size_t len,\n \t\tsrc = dst->buf;\n \t\tlen = dst->len;\n \t}\n-\tret |= crlf_to_git(path, src, len, dst, ca.crlf_action, checksafe);\n+\tret |= crlf_to_git(path, sha1, src, len, dst, ca.crlf_action, checksafe);\n \tif (ret && dst) {\n \t\tsrc = dst->buf;\n \t\tlen = dst->len;\n@@ -882,7 +890,9 @@ int convert_to_git(const char *path, const char *src, size_t len,\n \treturn ret | ident_to_git(path, src, len, dst, ca.ident);\n }\n \n-void convert_to_git_filter_fd(const char *path, int fd, struct strbuf *dst,\n+void convert_to_git_filter_fd(const char *path,\n+\t\t\t const unsigned char *sha1,\n+\t\t\t int fd, struct strbuf *dst,\n \t\t\t enum safe_crlf checksafe)\n {\n \tstruct conv_attrs ca;\n@@ -894,7 +904,7 @@ void convert_to_git_filter_fd(const char *path, int fd, struct strbuf *dst,\n \tif (!apply_filter(path, NULL, 0, fd, dst, ca.drv->clean))\n \t\tdie(\"%s: clean filter '%s' failed\", path, ca.drv->name);\n \n-\tcrlf_to_git(path, dst->buf, dst->len, dst, ca.crlf_action, checksafe);\n+\tcrlf_to_git(path, sha1, dst->buf, dst->len, dst, ca.crlf_action, checksafe);\n \tident_to_git(path, dst->buf, dst->len, dst, ca.ident);\n }\n \ndiff --git a/convert.h b/convert.h\nindex ccf436b..1634d37 100644\n--- a/convert.h\n+++ b/convert.h\n@@ -37,8 +37,16 @@ extern const char *get_wt_convert_stats_ascii(const char *path);\n extern const char *get_convert_attr_ascii(const char *path);\n \n /* returns 1 if *dst was used */\n-extern int convert_to_git(const char *path, const char *src, size_t len,\n-\t\t\t struct strbuf *dst, enum safe_crlf checksafe);\n+extern int convert_to_git_ce_sha1(const char *path, const unsigned char *sha1,\n+\t\t\t\t const char *src, size_t len,\n+\t\t\t\t struct strbuf *dst, enum safe_crlf checksafe);\n+\n+static inline int convert_to_git(const char *path, const char *src, size_t len,\n+\t\t\t\t struct strbuf *dst, enum safe_crlf checksafe)\n+{\n+\treturn convert_to_git_ce_sha1(path, NULL, src, len, dst, checksafe);\n+}\n+\n extern int convert_to_working_tree(const char *path, const char *src,\n \t\t\t\t size_t len, struct strbuf *dst);\n extern int renormalize_buffer(const char *path, const char *src, size_t len,\n@@ -47,9 +55,16 @@ static inline int would_convert_to_git(const char *path)\n {\n \treturn convert_to_git(path, NULL, 0, NULL, 0);\n }\n+static inline int would_convert_to_git_ce_sha1(const char *path,\n+\t\t\t\t\t const unsigned char *sha1)\n+{\n+\treturn convert_to_git_ce_sha1(path, sha1, NULL, 0, NULL, 0);\n+}\n+\n /* Precondition: would_convert_to_git_filter_fd(path) == true */\n-extern void convert_to_git_filter_fd(const char *path, int fd,\n-\t\t\t\t struct strbuf *dst,\n+extern void convert_to_git_filter_fd(const char *path,\n+\t\t\t\t const unsigned char *sha1,\n+\t\t\t\t int fd, struct strbuf *dst,\n \t\t\t\t enum safe_crlf checksafe);\n extern int would_convert_to_git_filter_fd(const char *path);\n \ndiff --git a/read-cache.c b/read-cache.c\nindex a3ef967..c109b6d 100644\n--- a/read-cache.c\n+++ b/read-cache.c\n@@ -163,7 +163,9 @@ static int ce_compare_data(const struct cache_entry *ce, struct stat *st)\n \n \tif (fd >= 0) {\n \t\tunsigned char sha1[20];\n-\t\tif (!index_fd(sha1, fd, st, OBJ_BLOB, ce->name, 0))\n+\t\tunsigned flags = HASH_USE_SHA_NOT_PATH;\n+\t\tmemcpy(sha1, ce->sha1, sizeof(sha1));\n+\t\tif (!index_fd(sha1, fd, st, OBJ_BLOB, ce->name, flags))\n \t\t\tmatch = hashcmp(sha1, ce->sha1);\n \t\t/* index_fd() closed the file descriptor already */\n \t}\ndiff --git a/sha1_file.c b/sha1_file.c\nindex d0f2aa0..1fdbfd3 100644\n--- a/sha1_file.c\n+++ b/sha1_file.c\n@@ -3275,6 +3275,7 @@ static int index_mem(unsigned char *sha1, void *buf, size_t size,\n {\n \tint ret, re_allocated = 0;\n \tint write_object = flags & HASH_WRITE_OBJECT;\n+\tconst int valid_sha1 = flags & HASH_USE_SHA_NOT_PATH;\n \n \tif (!type)\n \t\ttype = OBJ_BLOB;\n@@ -3284,8 +3285,11 @@ static int index_mem(unsigned char *sha1, void *buf, size_t size,\n \t */\n \tif ((type == OBJ_BLOB) && path) {\n \t\tstruct strbuf nbuf = STRBUF_INIT;\n-\t\tif (convert_to_git(path, buf, size, &nbuf,\n-\t\t\t\t write_object ? safe_crlf : SAFE_CRLF_FALSE)) {\n+\t\tif (convert_to_git_ce_sha1(path,\n+\t\t\t\t\t valid_sha1 ? sha1 : NULL,\n+\t\t\t\t\t buf, size, &nbuf,\n+\t\t\t\t\t write_object ? safe_crlf : SAFE_CRLF_FALSE)){\n+\n \t\t\tbuf = strbuf_detach(&nbuf, &size);\n \t\t\tre_allocated = 1;\n \t\t}\n@@ -3313,12 +3317,15 @@ static int index_stream_convert_blob(unsigned char *sha1, int fd,\n {\n \tint ret;\n \tconst int write_object = flags & HASH_WRITE_OBJECT;\n+\tconst int valid_sha1 = flags & HASH_USE_SHA_NOT_PATH;\n \tstruct strbuf sbuf = STRBUF_INIT;\n \n \tassert(path);\n \tassert(would_convert_to_git_filter_fd(path));\n \n-\tconvert_to_git_filter_fd(path, fd, &sbuf,\n+\tconvert_to_git_filter_fd(path,\n+\t\t\t\t valid_sha1 ? sha1 : NULL,\n+\t\t\t\t fd, &sbuf,\n \t\t\t\t write_object ? safe_crlf : SAFE_CRLF_FALSE);\n \n \tif (write_object)\n@@ -3396,6 +3403,8 @@ int index_fd(unsigned char *sha1, int fd, struct stat *st,\n \t enum object_type type, const char *path, unsigned flags)\n {\n \tint ret;\n+\tconst unsigned char *sha1_ce;\n+\tsha1_ce = flags & HASH_USE_SHA_NOT_PATH ? sha1 : NULL;\n \n \t/*\n \t * Call xsize_t() only when needed to avoid potentially unnecessary\n@@ -3406,7 +3415,7 @@ int index_fd(unsigned char *sha1, int fd, struct stat *st,\n \telse if (!S_ISREG(st->st_mode))\n \t\tret = index_pipe(sha1, fd, type, path, flags);\n \telse if (st->st_size <= big_file_threshold || type != OBJ_BLOB ||\n-\t\t (path && would_convert_to_git(path)))\n+\t\t (path && would_convert_to_git_ce_sha1(path,sha1_ce)))\n \t\tret = index_core(sha1, fd, xsize_t(st->st_size), type, path,\n \t\t\t\t flags);\n \telse\n-- \n2.0.0.rc1.6318.g0c2c796\n"},{"id":"287047","messageId":"xmqqk2iphcqe.fsf@gitster.mtv.corp.google.com","threadId":"42304","inReplyTo":"1463667680-26008-1-git-send-email-tboegi@web.de","subject":"Re: [PATCH v5 2/2] convert: ce_compare_data() checks for a sha1 of a path","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-05-19T23:03:37Z","receivedAt":"2016-05-19T23:03:37Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"tboegi@web.de writes:\n\n> +int convert_to_git_ce_sha1(const char *path, const unsigned char *sha1,\n> +\t\t\t const char *src, size_t len,\n> +\t\t\t struct strbuf *dst, enum safe_crlf checksafe)\n\nThat's a strange name for the function, as \"ce\" and \"sha1\" gives no\nhints what they are about.\n\nIf I understand correctly:\n\n - convert_to_git() is about converting into , and\n \"path\" is not for reading the contents to be converted. It is\n used to tell crlf_to_git() the index entry to pay attention to\n when defeating the usual conversion rules due to strange contents\n in the index (it also is used to report errors for safe-crlf\n check).\n\n - This one adds \"sha1\", and that is not about the contents to be\n converted, either. Like \"path\", \"sha1\" is used to tell what blob\n to check when disabling the usual conversion rules.\n\nDoes the above description help in coming up with a better\ndescription for the ce/sha1 thing? The comment near the code that\nuses them reads like so:\n\n\t/*\n\t * If the file in the index has any CR in it, do not convert.\n\t * This is the new safer autocrlf handling.\n\t */\n\nWhat is a good name to call the input to that logic? \"This\nfunction, in addition to convert_to_git(), takes an extra parameter,\nthat tells it an extra piece of information 'X'\"; what is X?\n\nAt the same time, the parameter \"sha1\" needs to be renamed to\nclarify what object it is and what purpose it serves. \"sha1\" alone\nis an overly generic name, and it does not hint that it may not even\nbe given to the function, and that it doesn't have anything to do\nwith the contents points at.\n\n\tNote. Perhaps you wanted _ce_sha1 suffix to tell the readers\n\tthat it takes \"an object name of the cache entry\" that\n\tfurther affects the conversion? If so the sha1 parameter\n\tshould be renamed to match (and make it clear to readers\n\tthat is what you meant).\n\nThe \"sha1\" is pretending to be the more important input for the\nprimary function of this function by being in early part of the\nparameter list. This may need to be rethought; we probably should\nhave done so as part of your previous refactoring of this file.\n\nconvert_to_git() takes the data for in and gives\nresult in , so these four should be its first parameters. The\ndetail of the way the conversion works may be affected by additional\nparameters, e.g. controls if extra warnings are given.\n\nThe is to influence the conversion logic further to disable\nthe crlf-to-git conversion by inspecting a blob, and it tells the\nfunction that the blob comes from an unusual place (as opposed to\nthe index entry at ). So it should sit next to checksafe as\nan auxiliary input parameter.\n\nThe logic implemented by the patch looks correct, but I'd have to\nsay that the result is an unmaintainable mess. Right now, I can see\nwhat is going on in the new code. I am sure that in 6 months, with\npoorly named helper functions and parameters, I will have hard time\nrecalling how the new code works.\n"},{"id":"287048","messageId":"xmqqfutdhcej.fsf@gitster.mtv.corp.google.com","threadId":"42304","inReplyTo":"1463667675-25930-1-git-send-email-tboegi@web.de","subject":"Re: [PATCH v5 0/2] CRLF: ce_compare_data() checks for a sha1 of a path","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-05-19T23:10:44Z","receivedAt":"2016-05-19T23:10:44Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"tboegi@web.de writes:\n\n> From: Torsten Bögershausen \n>\n> Break up the old 10/10 series about CLRF handling into smaller\n> series. This is a small bugfix, when merge.renormalize is used\n> with core.autocrlf (and no attributes are set).\n\nIs it worth protecting the fix with a new test? Or does this flip\nan existing \"expect-failure\" to \"expect-success\"?\n\n> Prepare the refactoring to use the streaming interface.\n\n> Changes since v4:\n> - Rename #define in cache.h into HASH_USE_SHA_NOT_PATH\n> - convert.c: Rename has_cr_in_index into blob_has_cr()\n> Better logic when sha1 != NULL,\n> Adjusted the commit message\n>\n> Torsten Bögershausen (2):\n> read-cache: factor out get_sha1_from_index() helper\n> convert: ce_compare_data() checks for a sha1 of a path\n>\n> cache.h | 4 ++++\n> convert.c | 34 ++++++++++++++++++++++------------\n> convert.h | 23 +++++++++++++++++++----\n> read-cache.c | 33 +++++++++++++++++++++------------\n> sha1_file.c | 17 +++++++++++++----\n> 5 files changed, 79 insertions(+), 32 deletions(-)\n"},{"id":"287110","messageId":"1463764344-21643-1-git-send-email-tboegi@web.de","threadId":"42304","inReplyTo":"xmqqk2iphcqe.fsf@gitster.mtv.corp.google.com","subject":"[PATCH v6 1/2] read-cache: factor out get_sha1_from_index() helper","fromName":"","fromEmail":"tboegi@web.de","sentAt":"2016-05-20T17:12:24Z","receivedAt":"2016-05-20T17:12:24Z","isPatch":true,"sender":{"key":"tboegi@web.de","avatar":"https://avatars.githubusercontent.com/u/7138363?v=4"},"body":"From: Torsten Bögershausen \n\nFactor out the retrieval of the sha1 for a given path in\nread_blob_data_from_index() into the function get_sha1_from_index().\n\nThis will be used in the next commit, when convert.c can do the\nanalyze for \"text=auto\" without slurping the whole blob into memory\nat once.\n\nAdd a wrapper definition get_sha1_from_cache().\n\nSigned-off-by: Torsten Bögershausen \n---\n cache.h | 3 +++\n read-cache.c | 29 ++++++++++++++++++-----------\n 2 files changed, 21 insertions(+), 11 deletions(-)\n\ndiff --git a/cache.h b/cache.h\nindex 160f8e3..15a2a10 100644\n--- a/cache.h\n+++ b/cache.h\n@@ -379,6 +379,7 @@ extern void free_name_hash(struct index_state *istate);\n #define unmerge_cache_entry_at(at) unmerge_index_entry_at(&the_index, at)\n #define unmerge_cache(pathspec) unmerge_index(&the_index, pathspec)\n #define read_blob_data_from_cache(path, sz) read_blob_data_from_index(&the_index, (path), (sz))\n+#define get_sha1_from_cache(path) get_sha1_from_index (&the_index, (path))\n #endif\n \n enum object_type {\n@@ -1038,6 +1039,8 @@ static inline void *read_sha1_file(const unsigned char *sha1, enum object_type *\n \treturn read_sha1_file_extended(sha1, type, size, LOOKUP_REPLACE_OBJECT);\n }\n \n+const unsigned char *get_sha1_from_index(struct index_state *istate, const char *path);\n+\n /*\n * This internal function is only declared here for the benefit of\n * lookup_replace_object(). Please do not call it directly.\ndiff --git a/read-cache.c b/read-cache.c\nindex d9fb78b..a3ef967 100644\n--- a/read-cache.c\n+++ b/read-cache.c\n@@ -2263,13 +2263,27 @@ int index_name_is_other(const struct index_state *istate, const char *name,\n \n void *read_blob_data_from_index(struct index_state *istate, const char *path, unsigned long *size)\n {\n-\tint pos, len;\n+\tconst unsigned char *sha1;\n \tunsigned long sz;\n \tenum object_type type;\n \tvoid *data;\n \n-\tlen = strlen(path);\n-\tpos = index_name_pos(istate, path, len);\n+\tsha1 = get_sha1_from_index(istate, path);\n+\tif (!sha1)\n+\t\treturn NULL;\n+\tdata = read_sha1_file(sha1, &type, &sz);\n+\tif (!data || type != OBJ_BLOB) {\n+\t\tfree(data);\n+\t\treturn NULL;\n+\t}\n+\tif (size)\n+\t\t*size = sz;\n+\treturn data;\n+}\n+\n+const unsigned char *get_sha1_from_index(struct index_state *istate, const char *path)\n+{\n+\tint pos = index_name_pos(istate, path, strlen(path));\n \tif (pos < 0) {\n \t\t/*\n \t\t * We might be in the middle of a merge, in which\n@@ -2285,14 +2299,7 @@ void *read_blob_data_from_index(struct index_state *istate, const char *path, un\n \t}\n \tif (pos < 0)\n \t\treturn NULL;\n-\tdata = read_sha1_file(istate->cache[pos]->sha1, &type, &sz);\n-\tif (!data || type != OBJ_BLOB) {\n-\t\tfree(data);\n-\t\treturn NULL;\n-\t}\n-\tif (size)\n-\t\t*size = sz;\n-\treturn data;\n+\treturn (istate->cache[pos]->sha1);\n }\n \n void stat_validity_clear(struct stat_validity *sv)\n-- \n2.0.0.rc1.6318.g0c2c796\n"},{"id":"287109","messageId":"1463764366-21683-1-git-send-email-tboegi@web.de","threadId":"42304","inReplyTo":"xmqqk2iphcqe.fsf@gitster.mtv.corp.google.com","subject":"[PATCH v6 2/2] convert: ce_compare_data() checks for a sha1 of a path","fromName":"","fromEmail":"tboegi@web.de","sentAt":"2016-05-20T17:12:46Z","receivedAt":"2016-05-20T17:12:46Z","isPatch":true,"sender":{"key":"tboegi@web.de","avatar":"https://avatars.githubusercontent.com/u/7138363?v=4"},"body":"From: Torsten Bögershausen \n\nTo compare a file in working tree with the index, convert_to_git() is used,\nthe result is hashed and the hash value compared with ce->sha1.\n\nDeep down would_convert_crlf_at_commit() is invoked, to check if CRLF\nare converted or not: When a CRLF had been in the index before, CRLF in\nthe working tree are not converted.\n\nWhile in a merge, a file name in the working tree has different blobs\nin the index with different hash values.\nForwarding ce->sha1 from ce_compare_data() into crlf_to_git() makes sure\nthe would_convert_crlf_at_commit() looks at the appropriate blob.\n\nForward sha1 from ce_compare_data() into convert_to_git().\nAll other callers use NULL, and the sha1 it is determined from path using\nget_sha1_from_cache(path), this is the same handling as before.\n\nRe-order the arguments for convert_to_git() according to their importance:\n `src`, `len` and `dst` are the place in memory, where the conversion is done\n `path` is the file name to look up the attributes.\n `sha1` is needed by the \"new safer autocrlf handling\".\n `checksafe` determines, if a warning is printed or an error is raised.\n\nIn the same spirit, forward the sha1 into would_convert_to_git().\n\nWhile at it, rename has_cr_in_index() into blob_has_cr()\n\nSigned-off-by: Torsten Bögershausen \nChanges sinve v6:\n decrease the messiness with 12 %\n convert_to_git() has a re-ordered parameter list.\n Describe whats going on better in the commit msg.\n Cleanup: 0 -> SAFE_CRLF_FALSE at some places\n---\n builtin/apply.c | 3 ++-\n builtin/blame.c | 2 +-\n cache.h | 1 +\n combine-diff.c | 4 +++-\n convert.c | 38 +++++++++++++++++++++++++-------------\n convert.h | 20 ++++++++++++++------\n diff.c | 3 ++-\n dir.c | 2 +-\n read-cache.c | 4 +++-\n sha1_file.c | 17 +++++++++++++----\n 10 files changed, 65 insertions(+), 29 deletions(-)\n\ndiff --git a/builtin/apply.c b/builtin/apply.c\nindex 8e4da2e..c01654a 100644\n--- a/builtin/apply.c\n+++ b/builtin/apply.c\n@@ -2140,7 +2140,8 @@ static int read_old_data(struct stat *st, const char *path, struct strbuf *buf)\n \tcase S_IFREG:\n \t\tif (strbuf_read_file(buf, path, st->st_size) != st->st_size)\n \t\t\treturn error(_(\"unable to open or read %s\"), path);\n-\t\tconvert_to_git(path, buf->buf, buf->len, buf, 0);\n+\t\tconvert_to_git(buf->buf, buf->len, buf,\n+\t\t\t path, NULL, SAFE_CRLF_FALSE);\n \t\treturn 0;\n \tdefault:\n \t\treturn -1;\ndiff --git a/builtin/blame.c b/builtin/blame.c\nindex 21f42b0..4a01e20 100644\n--- a/builtin/blame.c\n+++ b/builtin/blame.c\n@@ -2377,7 +2377,7 @@ static struct commit *fake_working_tree_commit(struct diff_options *opt,\n \t\tif (strbuf_read(&buf, 0, 0) < 0)\n \t\t\tdie_errno(\"failed to read from stdin\");\n \t}\n-\tconvert_to_git(path, buf.buf, buf.len, &buf, 0);\n+\tconvert_to_git(buf.buf, buf.len, &buf, path, NULL, SAFE_CRLF_FALSE);\n \torigin->file.ptr = buf.buf;\n \torigin->file.size = buf.len;\n \tpretend_sha1_file(buf.buf, buf.len, OBJ_BLOB, origin->blob_sha1);\ndiff --git a/cache.h b/cache.h\nindex 15a2a10..868599e 100644\n--- a/cache.h\n+++ b/cache.h\n@@ -605,6 +605,7 @@ extern int ie_modified(const struct index_state *, const struct cache_entry *, s\n \n #define HASH_WRITE_OBJECT 1\n #define HASH_FORMAT_CHECK 2\n+#define HASH_USE_SHA_NOT_PATH 4\n extern int index_fd(unsigned char *sha1, int fd, struct stat *st, enum object_type type, const char *path, unsigned flags);\n extern int index_path(unsigned char *sha1, const char *path, struct stat *st, unsigned flags);\n \ndiff --git a/combine-diff.c b/combine-diff.c\nindex 0e1d4b0..cac4c81 100644\n--- a/combine-diff.c\n+++ b/combine-diff.c\n@@ -1053,7 +1053,9 @@ static void show_patch_diff(struct combine_diff_path *elem, int num_parent,\n \t\t\tif (is_file) {\n \t\t\t\tstruct strbuf buf = STRBUF_INIT;\n \n-\t\t\t\tif (convert_to_git(elem->path, result, len, &buf, safe_crlf)) {\n+\t\t\t\tif (convert_to_git(result, len, &buf,\n+\t\t\t\t\t\t elem->path, NULL,\n+\t\t\t\t\t\t safe_crlf)) {\n \t\t\t\t\tfree(result);\n \t\t\t\t\tresult = strbuf_detach(&buf, &len);\n \t\t\t\t\tresult_size = len;\ndiff --git a/convert.c b/convert.c\nindex f524b8d..a58bb26 100644\n--- a/convert.c\n+++ b/convert.c\n@@ -217,21 +217,26 @@ static void check_safe_crlf(const char *path, enum crlf_action crlf_action,\n \t}\n }\n \n-static int has_cr_in_index(const char *path)\n+static int blob_has_cr(const unsigned char *sha1)\n {\n \tunsigned long sz;\n \tvoid *data;\n-\tint has_cr;\n-\n-\tdata = read_blob_data_from_cache(path, &sz);\n+\tint has_cr = 0;\n+\tenum object_type type;\n+\tif (!sha1)\n+\t\treturn 0;\n+\tdata = read_sha1_file(sha1, &type, &sz);\n \tif (!data)\n \t\treturn 0;\n-\thas_cr = memchr(data, '\\r', sz) != NULL;\n+\tif (type == OBJ_BLOB)\n+\t\thas_cr = memchr(data, '\\r', sz) != NULL;\n+\n \tfree(data);\n \treturn has_cr;\n }\n \n-static int crlf_to_git(const char *path, const char *src, size_t len,\n+static int crlf_to_git(const char *path, const unsigned char *sha1,\n+\t\t const char *src, size_t len,\n \t\t struct strbuf *buf,\n \t\t enum crlf_action crlf_action, enum safe_crlf checksafe)\n {\n@@ -260,7 +265,9 @@ static int crlf_to_git(const char *path, const char *src, size_t len,\n \t\t\t * If the file in the index has any CR in it, do not convert.\n \t\t\t * This is the new safer autocrlf handling.\n \t\t\t */\n-\t\t\tif (has_cr_in_index(path))\n+\t\t\tif (!sha1)\n+\t\t\t\tsha1 = get_sha1_from_cache(path);\n+\t\t\tif (blob_has_cr(sha1))\n \t\t\t\treturn 0;\n \t\t}\n \t}\n@@ -852,8 +859,10 @@ const char *get_convert_attr_ascii(const char *path)\n \treturn \"\";\n }\n \n-int convert_to_git(const char *path, const char *src, size_t len,\n- struct strbuf *dst, enum safe_crlf checksafe)\n+int convert_to_git(const char *src, size_t len,\n+\t\t struct strbuf *dst,\n+\t\t const char *path, const unsigned char *sha1,\n+\t\t enum safe_crlf checksafe)\n {\n \tint ret = 0;\n \tconst char *filter = NULL;\n@@ -874,7 +883,7 @@ int convert_to_git(const char *path, const char *src, size_t len,\n \t\tsrc = dst->buf;\n \t\tlen = dst->len;\n \t}\n-\tret |= crlf_to_git(path, src, len, dst, ca.crlf_action, checksafe);\n+\tret |= crlf_to_git(path, sha1, src, len, dst, ca.crlf_action, checksafe);\n \tif (ret && dst) {\n \t\tsrc = dst->buf;\n \t\tlen = dst->len;\n@@ -882,7 +891,9 @@ int convert_to_git(const char *path, const char *src, size_t len,\n \treturn ret | ident_to_git(path, src, len, dst, ca.ident);\n }\n \n-void convert_to_git_filter_fd(const char *path, int fd, struct strbuf *dst,\n+void convert_to_git_filter_fd(const char *path,\n+\t\t\t const unsigned char *sha1,\n+\t\t\t int fd, struct strbuf *dst,\n \t\t\t enum safe_crlf checksafe)\n {\n \tstruct conv_attrs ca;\n@@ -894,7 +905,7 @@ void convert_to_git_filter_fd(const char *path, int fd, struct strbuf *dst,\n \tif (!apply_filter(path, NULL, 0, fd, dst, ca.drv->clean))\n \t\tdie(\"%s: clean filter '%s' failed\", path, ca.drv->name);\n \n-\tcrlf_to_git(path, dst->buf, dst->len, dst, ca.crlf_action, checksafe);\n+\tcrlf_to_git(path, sha1, dst->buf, dst->len, dst, ca.crlf_action, checksafe);\n \tident_to_git(path, dst->buf, dst->len, dst, ca.ident);\n }\n \n@@ -949,7 +960,8 @@ int renormalize_buffer(const char *path, const char *src, size_t len, struct str\n \t\tsrc = dst->buf;\n \t\tlen = dst->len;\n \t}\n-\treturn ret | convert_to_git(path, src, len, dst, SAFE_CRLF_FALSE);\n+\tret |= convert_to_git(src, len, dst, path, NULL, SAFE_CRLF_FALSE);\n+\treturn ret;\n }\n \n /*****************************************************************\ndiff --git a/convert.h b/convert.h\nindex ccf436b..12fe767 100644\n--- a/convert.h\n+++ b/convert.h\n@@ -37,19 +37,27 @@ extern const char *get_wt_convert_stats_ascii(const char *path);\n extern const char *get_convert_attr_ascii(const char *path);\n \n /* returns 1 if *dst was used */\n-extern int convert_to_git(const char *path, const char *src, size_t len,\n-\t\t\t struct strbuf *dst, enum safe_crlf checksafe);\n+extern int convert_to_git(const char *src, size_t len,\n+\t\t\t struct strbuf *dst,\n+\t\t\t const char *path, const unsigned char *sha1,\n+\t\t\t enum safe_crlf checksafe);\n+\n extern int convert_to_working_tree(const char *path, const char *src,\n \t\t\t\t size_t len, struct strbuf *dst);\n extern int renormalize_buffer(const char *path, const char *src, size_t len,\n \t\t\t struct strbuf *dst);\n-static inline int would_convert_to_git(const char *path)\n+\n+static inline int would_convert_to_git(const char *path,\n+\t\t\t\t const unsigned char *sha1)\n {\n-\treturn convert_to_git(path, NULL, 0, NULL, 0);\n+\treturn convert_to_git(NULL, 0, NULL, path, sha1, SAFE_CRLF_FALSE);\n }\n+\n+\n /* Precondition: would_convert_to_git_filter_fd(path) == true */\n-extern void convert_to_git_filter_fd(const char *path, int fd,\n-\t\t\t\t struct strbuf *dst,\n+extern void convert_to_git_filter_fd(const char *path,\n+\t\t\t\t const unsigned char *sha1,\n+\t\t\t\t int fd, struct strbuf *dst,\n \t\t\t\t enum safe_crlf checksafe);\n extern int would_convert_to_git_filter_fd(const char *path);\n \ndiff --git a/diff.c b/diff.c\nindex d3734d3..9c00973 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -2810,7 +2810,8 @@ int diff_populate_filespec(struct diff_filespec *s, unsigned int flags)\n \t\t/*\n \t\t * Convert from working tree format to canonical git format\n \t\t */\n-\t\tif (convert_to_git(s->path, s->data, s->size, &buf, crlf_warn)) {\n+\t\tif (convert_to_git(s->data, s->size, &buf, s->path, NULL,\n+\t\t\t\t crlf_warn)) {\n \t\t\tsize_t size = 0;\n \t\t\tmunmap(s->data, s->size);\n \t\t\ts->should_munmap = 0;\ndiff --git a/dir.c b/dir.c\nindex 656f272..5ac379d 100644\n--- a/dir.c\n+++ b/dir.c\n@@ -713,7 +713,7 @@ static int add_excludes(const char *fname, const char *base, int baselen,\n \t\t\t\t (pos = cache_name_pos(fname, strlen(fname))) >= 0 &&\n \t\t\t\t !ce_stage(active_cache[pos]) &&\n \t\t\t\t ce_uptodate(active_cache[pos]) &&\n-\t\t\t\t !would_convert_to_git(fname))\n+\t\t\t\t !would_convert_to_git(fname, NULL))\n \t\t\t\thashcpy(sha1_stat->sha1, active_cache[pos]->sha1);\n \t\t\telse\n \t\t\t\thash_sha1_file(buf, size, \"blob\", sha1_stat->sha1);\ndiff --git a/read-cache.c b/read-cache.c\nindex a3ef967..c109b6d 100644\n--- a/read-cache.c\n+++ b/read-cache.c\n@@ -163,7 +163,9 @@ static int ce_compare_data(const struct cache_entry *ce, struct stat *st)\n \n \tif (fd >= 0) {\n \t\tunsigned char sha1[20];\n-\t\tif (!index_fd(sha1, fd, st, OBJ_BLOB, ce->name, 0))\n+\t\tunsigned flags = HASH_USE_SHA_NOT_PATH;\n+\t\tmemcpy(sha1, ce->sha1, sizeof(sha1));\n+\t\tif (!index_fd(sha1, fd, st, OBJ_BLOB, ce->name, flags))\n \t\t\tmatch = hashcmp(sha1, ce->sha1);\n \t\t/* index_fd() closed the file descriptor already */\n \t}\ndiff --git a/sha1_file.c b/sha1_file.c\nindex d0f2aa0..48906b0 100644\n--- a/sha1_file.c\n+++ b/sha1_file.c\n@@ -3275,6 +3275,7 @@ static int index_mem(unsigned char *sha1, void *buf, size_t size,\n {\n \tint ret, re_allocated = 0;\n \tint write_object = flags & HASH_WRITE_OBJECT;\n+\tconst int valid_sha1 = flags & HASH_USE_SHA_NOT_PATH;\n \n \tif (!type)\n \t\ttype = OBJ_BLOB;\n@@ -3284,8 +3285,11 @@ static int index_mem(unsigned char *sha1, void *buf, size_t size,\n \t */\n \tif ((type == OBJ_BLOB) && path) {\n \t\tstruct strbuf nbuf = STRBUF_INIT;\n-\t\tif (convert_to_git(path, buf, size, &nbuf,\n-\t\t\t\t write_object ? safe_crlf : SAFE_CRLF_FALSE)) {\n+\t\tif (convert_to_git(\n+\t\t\t buf, size, &nbuf,path,\n+\t\t\t valid_sha1 ? sha1 : NULL,\n+\t\t\t write_object ? safe_crlf : SAFE_CRLF_FALSE)){\n+\n \t\t\tbuf = strbuf_detach(&nbuf, &size);\n \t\t\tre_allocated = 1;\n \t\t}\n@@ -3313,12 +3317,15 @@ static int index_stream_convert_blob(unsigned char *sha1, int fd,\n {\n \tint ret;\n \tconst int write_object = flags & HASH_WRITE_OBJECT;\n+\tconst int valid_sha1 = flags & HASH_USE_SHA_NOT_PATH;\n \tstruct strbuf sbuf = STRBUF_INIT;\n \n \tassert(path);\n \tassert(would_convert_to_git_filter_fd(path));\n \n-\tconvert_to_git_filter_fd(path, fd, &sbuf,\n+\tconvert_to_git_filter_fd(path,\n+\t\t\t\t valid_sha1 ? sha1 : NULL,\n+\t\t\t\t fd, &sbuf,\n \t\t\t\t write_object ? safe_crlf : SAFE_CRLF_FALSE);\n \n \tif (write_object)\n@@ -3396,6 +3403,8 @@ int index_fd(unsigned char *sha1, int fd, struct stat *st,\n \t enum object_type type, const char *path, unsigned flags)\n {\n \tint ret;\n+\tconst unsigned char *sha1_ce;\n+\tsha1_ce = flags & HASH_USE_SHA_NOT_PATH ? sha1 : NULL;\n \n \t/*\n \t * Call xsize_t() only when needed to avoid potentially unnecessary\n@@ -3406,7 +3415,7 @@ int index_fd(unsigned char *sha1, int fd, struct stat *st,\n \telse if (!S_ISREG(st->st_mode))\n \t\tret = index_pipe(sha1, fd, type, path, flags);\n \telse if (st->st_size <= big_file_threshold || type != OBJ_BLOB ||\n-\t\t (path && would_convert_to_git(path)))\n+\t\t (path && would_convert_to_git(path,sha1_ce)))\n \t\tret = index_core(sha1, fd, xsize_t(st->st_size), type, path,\n \t\t\t\t flags);\n \telse\n-- \n2.0.0.rc1.6318.g0c2c796\n"},{"id":"287113","messageId":"xmqqshxcei66.fsf@gitster.mtv.corp.google.com","threadId":"42304","inReplyTo":"1463764366-21683-1-git-send-email-tboegi@web.de","subject":"Re: [PATCH v6 2/2] convert: ce_compare_data() checks for a sha1 of a path","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-05-20T17:46:41Z","receivedAt":"2016-05-20T17:46:41Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"tboegi@web.de writes:\n\n> From: Torsten Bögershausen \n>\n> To compare a file in working tree with the index, convert_to_git() is used,\n> the result is hashed and the hash value compared with ce->sha1.\n>\n> Deep down would_convert_crlf_at_commit() is invoked, to check if CRLF\n> are converted or not: When a CRLF had been in the index before, CRLF in\n> the working tree are not converted.\n>\n> While in a merge, a file name in the working tree has different blobs\n> in the index with different hash values.\n> Forwarding ce->sha1 from ce_compare_data() into crlf_to_git() makes sure\n> the would_convert_crlf_at_commit() looks at the appropriate blob.\n>\n> Forward sha1 from ce_compare_data() into convert_to_git().\n> All other callers use NULL, and the sha1 it is determined from path using\n> get_sha1_from_cache(path), this is the same handling as before.\n>\n> Re-order the arguments for convert_to_git() according to their importance:\n> `src`, `len` and `dst` are the place in memory, where the conversion is done\n> `path` is the file name to look up the attributes.\n> `sha1` is needed by the \"new safer autocrlf handling\".\n> `checksafe` determines, if a warning is printed or an error is raised.\n>\n> In the same spirit, forward the sha1 into would_convert_to_git().\n>\n> While at it, rename has_cr_in_index() into blob_has_cr()\n>\n> Signed-off-by: Torsten Bögershausen \n\n\n> Changes sinve v6:\n> decrease the messiness with 12 %\n\nWhat does that mean????\n\n> convert_to_git() has a re-ordered parameter list.\n\nThat is not what I suggested, though. being\nthe first three would be fine, and that would be in line with\nhelpers ${frotz}_to_git() that are used from there.\n\nThe primary thing that made me worried was a new parameter with a\nbland name \"sha1\" whose purpose is unclear was added near the\nbeginning, leading readers to confusion.\n\nWhether you keep at the beginning of move it to later\ntogether with , helpers like crlf_to_git() need to be updated\nto match. E.g. I would say that this\n\n> -\tret |= crlf_to_git(path, src, len, dst, ca.crlf_action, checksafe);\n> +\tret |= crlf_to_git(path, sha1, src, len, dst, ca.crlf_action, checksafe);\n\nwould want to be ordered more like this:\n\n ret |= crlf_to_git(path, src, len, dst,\n ca.crlf_action, checksafe, index_blob_sha1);\n\nif you choose to keep the first four intact for convert_to_git(),\nthat is.\n\n> Describe whats going on better in the commit msg.\n\nThe suggestion to rename the parameter was to allow readers of the\ncode to immediately know what kind of SHA1 it is. They cannot\nafford to run \"git blame\" every time to find the commit to read the\ncommit log message; for a public function like convert_to_git(),\nin-code comment is also necessary.\n"},{"id":"287146","messageId":"1463824906-10190-1-git-send-email-tboegi@web.de","threadId":"42304","inReplyTo":"xmqqk2iphcqe.fsf@gitster.mtv.corp.google.com","subject":"[PATCH v7 1/2] read-cache: factor out get_sha1_from_index() helper","fromName":"","fromEmail":"tboegi@web.de","sentAt":"2016-05-21T10:01:46Z","receivedAt":"2016-05-21T10:01:46Z","isPatch":true,"sender":{"key":"tboegi@web.de","avatar":"https://avatars.githubusercontent.com/u/7138363?v=4"},"body":"From: Torsten Bögershausen \n\nFactor out the retrieval of the sha1 for a given path in\nread_blob_data_from_index() into the function get_sha1_from_index().\n\nThis will be used in the next commit, when convert.c can do the\nanalyze for \"text=auto\" without slurping the whole blob into memory\nat once.\n\nAdd a wrapper definition get_sha1_from_cache().\n\nSigned-off-by: Torsten Bögershausen \n---\n cache.h | 3 +++\n read-cache.c | 29 ++++++++++++++++++-----------\n 2 files changed, 21 insertions(+), 11 deletions(-)\n\ndiff --git a/cache.h b/cache.h\nindex 160f8e3..15a2a10 100644\n--- a/cache.h\n+++ b/cache.h\n@@ -379,6 +379,7 @@ extern void free_name_hash(struct index_state *istate);\n #define unmerge_cache_entry_at(at) unmerge_index_entry_at(&the_index, at)\n #define unmerge_cache(pathspec) unmerge_index(&the_index, pathspec)\n #define read_blob_data_from_cache(path, sz) read_blob_data_from_index(&the_index, (path), (sz))\n+#define get_sha1_from_cache(path) get_sha1_from_index (&the_index, (path))\n #endif\n \n enum object_type {\n@@ -1038,6 +1039,8 @@ static inline void *read_sha1_file(const unsigned char *sha1, enum object_type *\n \treturn read_sha1_file_extended(sha1, type, size, LOOKUP_REPLACE_OBJECT);\n }\n \n+const unsigned char *get_sha1_from_index(struct index_state *istate, const char *path);\n+\n /*\n * This internal function is only declared here for the benefit of\n * lookup_replace_object(). Please do not call it directly.\ndiff --git a/read-cache.c b/read-cache.c\nindex d9fb78b..a3ef967 100644\n--- a/read-cache.c\n+++ b/read-cache.c\n@@ -2263,13 +2263,27 @@ int index_name_is_other(const struct index_state *istate, const char *name,\n \n void *read_blob_data_from_index(struct index_state *istate, const char *path, unsigned long *size)\n {\n-\tint pos, len;\n+\tconst unsigned char *sha1;\n \tunsigned long sz;\n \tenum object_type type;\n \tvoid *data;\n \n-\tlen = strlen(path);\n-\tpos = index_name_pos(istate, path, len);\n+\tsha1 = get_sha1_from_index(istate, path);\n+\tif (!sha1)\n+\t\treturn NULL;\n+\tdata = read_sha1_file(sha1, &type, &sz);\n+\tif (!data || type != OBJ_BLOB) {\n+\t\tfree(data);\n+\t\treturn NULL;\n+\t}\n+\tif (size)\n+\t\t*size = sz;\n+\treturn data;\n+}\n+\n+const unsigned char *get_sha1_from_index(struct index_state *istate, const char *path)\n+{\n+\tint pos = index_name_pos(istate, path, strlen(path));\n \tif (pos < 0) {\n \t\t/*\n \t\t * We might be in the middle of a merge, in which\n@@ -2285,14 +2299,7 @@ void *read_blob_data_from_index(struct index_state *istate, const char *path, un\n \t}\n \tif (pos < 0)\n \t\treturn NULL;\n-\tdata = read_sha1_file(istate->cache[pos]->sha1, &type, &sz);\n-\tif (!data || type != OBJ_BLOB) {\n-\t\tfree(data);\n-\t\treturn NULL;\n-\t}\n-\tif (size)\n-\t\t*size = sz;\n-\treturn data;\n+\treturn (istate->cache[pos]->sha1);\n }\n \n void stat_validity_clear(struct stat_validity *sv)\n-- \n2.0.0.rc1.6318.g0c2c796\n"},{"id":"287147","messageId":"1463824909-10229-1-git-send-email-tboegi@web.de","threadId":"42304","inReplyTo":"xmqqk2iphcqe.fsf@gitster.mtv.corp.google.com","subject":"[PATCH v7 2/2] convert: ce_compare_data() checks for a sha1 of a path","fromName":"","fromEmail":"tboegi@web.de","sentAt":"2016-05-21T10:01:49Z","receivedAt":"2016-05-21T10:01:49Z","isPatch":true,"sender":{"key":"tboegi@web.de","avatar":"https://avatars.githubusercontent.com/u/7138363?v=4"},"body":"From: Torsten Bögershausen \n\nTo compare a file in working tree with the index, convert_to_git() is used,\nthe result is hashed and the hash value compared with ce->sha1.\n\nDeep down would_convert_crlf_at_commit() is invoked, to check if CRLF\nare converted or not.\nThe \"new safer autocrlf handling\" checks if CRLF had been in the index before,\nand if, the CRLF in the working tree are not converted.\n\nWhile in a merge, a file name in the working tree has different blobs\nin the index with different hash values.\nForwarding ce->sha1 from ce_compare_data() into crlf_to_git() makes sure\nthe would_convert_crlf_at_commit() looks at the appropriate blob.\n\nAdd a new parameter index_blob_sha1 to convert_to_git(), and forward the\nsha1 from ce_compare_data() into convert_to_git(). Other callers use NULL\nfor index_blob_sha1, and the sha1 is determined from path\nusing get_sha1_from_cache(path). This is the same handling as before.\n\nIn the same spirit, forward the sha1 into would_convert_to_git().\n\nWhile at it, rename has_cr_in_index() into blob_has_cr() and replace\n0 with SAFE_CRLF_FALSE.\n\nSigned-off-by: Torsten Bögershausen \n---\n V6 went into the wrong direction.\n V5 -> V7: Adds parameter index_blob_sha1 to (convert_to_git() and would_convert_to_git().\n\n builtin/apply.c | 3 ++-\n builtin/blame.c | 2 +-\n cache.h | 1 +\n combine-diff.c | 3 ++-\n convert.c | 34 ++++++++++++++++++++++------------\n convert.h | 15 +++++++++++----\n diff.c | 3 ++-\n dir.c | 2 +-\n read-cache.c | 4 +++-\n sha1_file.c | 12 +++++++++---\n 10 files changed, 54 insertions(+), 25 deletions(-)\n\ndiff --git a/builtin/apply.c b/builtin/apply.c\nindex 8e4da2e..0cf9a0a 100644\n--- a/builtin/apply.c\n+++ b/builtin/apply.c\n@@ -2140,7 +2140,8 @@ static int read_old_data(struct stat *st, const char *path, struct strbuf *buf)\n \tcase S_IFREG:\n \t\tif (strbuf_read_file(buf, path, st->st_size) != st->st_size)\n \t\t\treturn error(_(\"unable to open or read %s\"), path);\n-\t\tconvert_to_git(path, buf->buf, buf->len, buf, 0);\n+\t\tconvert_to_git(path, buf->buf, buf->len, buf,\n+\t\t\t SAFE_CRLF_FALSE, NULL);\n \t\treturn 0;\n \tdefault:\n \t\treturn -1;\ndiff --git a/builtin/blame.c b/builtin/blame.c\nindex 21f42b0..1c523b6 100644\n--- a/builtin/blame.c\n+++ b/builtin/blame.c\n@@ -2377,7 +2377,7 @@ static struct commit *fake_working_tree_commit(struct diff_options *opt,\n \t\tif (strbuf_read(&buf, 0, 0) < 0)\n \t\t\tdie_errno(\"failed to read from stdin\");\n \t}\n-\tconvert_to_git(path, buf.buf, buf.len, &buf, 0);\n+\tconvert_to_git(path, buf.buf, buf.len, &buf, SAFE_CRLF_FALSE, NULL);\n \torigin->file.ptr = buf.buf;\n \torigin->file.size = buf.len;\n \tpretend_sha1_file(buf.buf, buf.len, OBJ_BLOB, origin->blob_sha1);\ndiff --git a/cache.h b/cache.h\nindex 15a2a10..868599e 100644\n--- a/cache.h\n+++ b/cache.h\n@@ -605,6 +605,7 @@ extern int ie_modified(const struct index_state *, const struct cache_entry *, s\n \n #define HASH_WRITE_OBJECT 1\n #define HASH_FORMAT_CHECK 2\n+#define HASH_USE_SHA_NOT_PATH 4\n extern int index_fd(unsigned char *sha1, int fd, struct stat *st, enum object_type type, const char *path, unsigned flags);\n extern int index_path(unsigned char *sha1, const char *path, struct stat *st, unsigned flags);\n \ndiff --git a/combine-diff.c b/combine-diff.c\nindex 0e1d4b0..c4fa884 100644\n--- a/combine-diff.c\n+++ b/combine-diff.c\n@@ -1053,7 +1053,8 @@ static void show_patch_diff(struct combine_diff_path *elem, int num_parent,\n \t\t\tif (is_file) {\n \t\t\t\tstruct strbuf buf = STRBUF_INIT;\n \n-\t\t\t\tif (convert_to_git(elem->path, result, len, &buf, safe_crlf)) {\n+\t\t\t\tif (convert_to_git(elem->path, result, len,\n+\t\t\t\t\t\t &buf, safe_crlf, NULL)) {\n \t\t\t\t\tfree(result);\n \t\t\t\t\tresult = strbuf_detach(&buf, &len);\n \t\t\t\t\tresult_size = len;\ndiff --git a/convert.c b/convert.c\nindex f524b8d..f0eb4ed 100644\n--- a/convert.c\n+++ b/convert.c\n@@ -217,23 +217,28 @@ static void check_safe_crlf(const char *path, enum crlf_action crlf_action,\n \t}\n }\n \n-static int has_cr_in_index(const char *path)\n+static int blob_has_cr(const unsigned char *index_blob_sha1)\n {\n \tunsigned long sz;\n \tvoid *data;\n-\tint has_cr;\n-\n-\tdata = read_blob_data_from_cache(path, &sz);\n+\tint has_cr = 0;\n+\tenum object_type type;\n+\tif (!index_blob_sha1)\n+\t\treturn 0;\n+\tdata = read_sha1_file(index_blob_sha1, &type, &sz);\n \tif (!data)\n \t\treturn 0;\n-\thas_cr = memchr(data, '\\r', sz) != NULL;\n+\tif (type == OBJ_BLOB)\n+\t\thas_cr = memchr(data, '\\r', sz) != NULL;\n+\n \tfree(data);\n \treturn has_cr;\n }\n \n static int crlf_to_git(const char *path, const char *src, size_t len,\n \t\t struct strbuf *buf,\n-\t\t enum crlf_action crlf_action, enum safe_crlf checksafe)\n+\t\t enum crlf_action crlf_action, enum safe_crlf checksafe,\n+\t\t const unsigned char *index_blob_sha1)\n {\n \tstruct text_stat stats;\n \tchar *dst;\n@@ -260,7 +265,9 @@ static int crlf_to_git(const char *path, const char *src, size_t len,\n \t\t\t * If the file in the index has any CR in it, do not convert.\n \t\t\t * This is the new safer autocrlf handling.\n \t\t\t */\n-\t\t\tif (has_cr_in_index(path))\n+\t\t\tif (!index_blob_sha1)\n+\t\t\t\tindex_blob_sha1 = get_sha1_from_cache(path);\n+\t\t\tif (blob_has_cr(index_blob_sha1))\n \t\t\t\treturn 0;\n \t\t}\n \t}\n@@ -853,7 +860,8 @@ const char *get_convert_attr_ascii(const char *path)\n }\n \n int convert_to_git(const char *path, const char *src, size_t len,\n- struct strbuf *dst, enum safe_crlf checksafe)\n+\t\t struct strbuf *dst, enum safe_crlf checksafe,\n+\t\t const unsigned char *index_blob_sha1)\n {\n \tint ret = 0;\n \tconst char *filter = NULL;\n@@ -874,7 +882,7 @@ int convert_to_git(const char *path, const char *src, size_t len,\n \t\tsrc = dst->buf;\n \t\tlen = dst->len;\n \t}\n-\tret |= crlf_to_git(path, src, len, dst, ca.crlf_action, checksafe);\n+\tret |= crlf_to_git(path, src, len, dst, ca.crlf_action, checksafe, index_blob_sha1);\n \tif (ret && dst) {\n \t\tsrc = dst->buf;\n \t\tlen = dst->len;\n@@ -883,7 +891,8 @@ int convert_to_git(const char *path, const char *src, size_t len,\n }\n \n void convert_to_git_filter_fd(const char *path, int fd, struct strbuf *dst,\n-\t\t\t enum safe_crlf checksafe)\n+\t\t\t enum safe_crlf checksafe,\n+\t\t\t const unsigned char *index_blob_sha1)\n {\n \tstruct conv_attrs ca;\n \tconvert_attrs(&ca, path);\n@@ -894,7 +903,8 @@ void convert_to_git_filter_fd(const char *path, int fd, struct strbuf *dst,\n \tif (!apply_filter(path, NULL, 0, fd, dst, ca.drv->clean))\n \t\tdie(\"%s: clean filter '%s' failed\", path, ca.drv->name);\n \n-\tcrlf_to_git(path, dst->buf, dst->len, dst, ca.crlf_action, checksafe);\n+\tcrlf_to_git(path, dst->buf, dst->len, dst, ca.crlf_action,\n+\t\t checksafe, index_blob_sha1);\n \tident_to_git(path, dst->buf, dst->len, dst, ca.ident);\n }\n \n@@ -949,7 +959,7 @@ int renormalize_buffer(const char *path, const char *src, size_t len, struct str\n \t\tsrc = dst->buf;\n \t\tlen = dst->len;\n \t}\n-\treturn ret | convert_to_git(path, src, len, dst, SAFE_CRLF_FALSE);\n+\treturn ret | convert_to_git(path, src, len, dst, SAFE_CRLF_FALSE, NULL);\n }\n \n /*****************************************************************\ndiff --git a/convert.h b/convert.h\nindex ccf436b..60c46b8 100644\n--- a/convert.h\n+++ b/convert.h\n@@ -38,19 +38,26 @@ extern const char *get_convert_attr_ascii(const char *path);\n \n /* returns 1 if *dst was used */\n extern int convert_to_git(const char *path, const char *src, size_t len,\n-\t\t\t struct strbuf *dst, enum safe_crlf checksafe);\n+\t\t\t struct strbuf *dst, enum safe_crlf checksafe,\n+\t\t\t const unsigned char *index_blob_sha1);\n+\n extern int convert_to_working_tree(const char *path, const char *src,\n \t\t\t\t size_t len, struct strbuf *dst);\n extern int renormalize_buffer(const char *path, const char *src, size_t len,\n \t\t\t struct strbuf *dst);\n-static inline int would_convert_to_git(const char *path)\n+static inline int would_convert_to_git(const char *path,\n+\t\t\t\t const unsigned char *index_blob_sha1)\n {\n-\treturn convert_to_git(path, NULL, 0, NULL, 0);\n+\treturn convert_to_git(path, NULL, 0, NULL, SAFE_CRLF_FALSE,\n+\t\t\t index_blob_sha1);\n }\n+\n /* Precondition: would_convert_to_git_filter_fd(path) == true */\n extern void convert_to_git_filter_fd(const char *path, int fd,\n \t\t\t\t struct strbuf *dst,\n-\t\t\t\t enum safe_crlf checksafe);\n+\t\t\t\t enum safe_crlf checksafe,\n+\t\t\t\t const unsigned char *index_blob_sha1);\n+\n extern int would_convert_to_git_filter_fd(const char *path);\n \n /*****************************************************************\ndiff --git a/diff.c b/diff.c\nindex d3734d3..a8308e0 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -2810,7 +2810,8 @@ int diff_populate_filespec(struct diff_filespec *s, unsigned int flags)\n \t\t/*\n \t\t * Convert from working tree format to canonical git format\n \t\t */\n-\t\tif (convert_to_git(s->path, s->data, s->size, &buf, crlf_warn)) {\n+\t\tif (convert_to_git(s->path, s->data, s->size, &buf,\n+\t\t\t\t crlf_warn, NULL)) {\n \t\t\tsize_t size = 0;\n \t\t\tmunmap(s->data, s->size);\n \t\t\ts->should_munmap = 0;\ndiff --git a/dir.c b/dir.c\nindex 656f272..5ac379d 100644\n--- a/dir.c\n+++ b/dir.c\n@@ -713,7 +713,7 @@ static int add_excludes(const char *fname, const char *base, int baselen,\n \t\t\t\t (pos = cache_name_pos(fname, strlen(fname))) >= 0 &&\n \t\t\t\t !ce_stage(active_cache[pos]) &&\n \t\t\t\t ce_uptodate(active_cache[pos]) &&\n-\t\t\t\t !would_convert_to_git(fname))\n+\t\t\t\t !would_convert_to_git(fname, NULL))\n \t\t\t\thashcpy(sha1_stat->sha1, active_cache[pos]->sha1);\n \t\t\telse\n \t\t\t\thash_sha1_file(buf, size, \"blob\", sha1_stat->sha1);\ndiff --git a/read-cache.c b/read-cache.c\nindex a3ef967..c109b6d 100644\n--- a/read-cache.c\n+++ b/read-cache.c\n@@ -163,7 +163,9 @@ static int ce_compare_data(const struct cache_entry *ce, struct stat *st)\n \n \tif (fd >= 0) {\n \t\tunsigned char sha1[20];\n-\t\tif (!index_fd(sha1, fd, st, OBJ_BLOB, ce->name, 0))\n+\t\tunsigned flags = HASH_USE_SHA_NOT_PATH;\n+\t\tmemcpy(sha1, ce->sha1, sizeof(sha1));\n+\t\tif (!index_fd(sha1, fd, st, OBJ_BLOB, ce->name, flags))\n \t\t\tmatch = hashcmp(sha1, ce->sha1);\n \t\t/* index_fd() closed the file descriptor already */\n \t}\ndiff --git a/sha1_file.c b/sha1_file.c\nindex d0f2aa0..52e5c6f 100644\n--- a/sha1_file.c\n+++ b/sha1_file.c\n@@ -3275,6 +3275,7 @@ static int index_mem(unsigned char *sha1, void *buf, size_t size,\n {\n \tint ret, re_allocated = 0;\n \tint write_object = flags & HASH_WRITE_OBJECT;\n+\tconst int valid_sha1 = flags & HASH_USE_SHA_NOT_PATH;\n \n \tif (!type)\n \t\ttype = OBJ_BLOB;\n@@ -3285,7 +3286,8 @@ static int index_mem(unsigned char *sha1, void *buf, size_t size,\n \tif ((type == OBJ_BLOB) && path) {\n \t\tstruct strbuf nbuf = STRBUF_INIT;\n \t\tif (convert_to_git(path, buf, size, &nbuf,\n-\t\t\t\t write_object ? safe_crlf : SAFE_CRLF_FALSE)) {\n+\t\t\t\t write_object ? safe_crlf : SAFE_CRLF_FALSE,\n+\t\t\t\t valid_sha1 ? sha1 : NULL)) {\n \t\t\tbuf = strbuf_detach(&nbuf, &size);\n \t\t\tre_allocated = 1;\n \t\t}\n@@ -3313,13 +3315,15 @@ static int index_stream_convert_blob(unsigned char *sha1, int fd,\n {\n \tint ret;\n \tconst int write_object = flags & HASH_WRITE_OBJECT;\n+\tconst int valid_sha1 = flags & HASH_USE_SHA_NOT_PATH;\n \tstruct strbuf sbuf = STRBUF_INIT;\n \n \tassert(path);\n \tassert(would_convert_to_git_filter_fd(path));\n \n \tconvert_to_git_filter_fd(path, fd, &sbuf,\n-\t\t\t\t write_object ? safe_crlf : SAFE_CRLF_FALSE);\n+\t\t\t\t write_object ? safe_crlf : SAFE_CRLF_FALSE,\n+\t\t\t\t valid_sha1 ? sha1 : NULL);\n \n \tif (write_object)\n \t\tret = write_sha1_file(sbuf.buf, sbuf.len, typename(OBJ_BLOB),\n@@ -3396,6 +3400,8 @@ int index_fd(unsigned char *sha1, int fd, struct stat *st,\n \t enum object_type type, const char *path, unsigned flags)\n {\n \tint ret;\n+\tconst unsigned char *sha1_ce;\n+\tsha1_ce = flags & HASH_USE_SHA_NOT_PATH ? sha1 : NULL;\n \n \t/*\n \t * Call xsize_t() only when needed to avoid potentially unnecessary\n@@ -3406,7 +3412,7 @@ int index_fd(unsigned char *sha1, int fd, struct stat *st,\n \telse if (!S_ISREG(st->st_mode))\n \t\tret = index_pipe(sha1, fd, type, path, flags);\n \telse if (st->st_size <= big_file_threshold || type != OBJ_BLOB ||\n-\t\t (path && would_convert_to_git(path)))\n+\t\t (path && would_convert_to_git(path,sha1_ce)))\n \t\tret = index_core(sha1, fd, xsize_t(st->st_size), type, path,\n \t\t\t\t flags);\n \telse\n-- \n2.0.0.rc1.6318.g0c2c796\n"},{"id":"287436","messageId":"xmqqy46z482d.fsf@gitster.mtv.corp.google.com","threadId":"42304","inReplyTo":"1463824909-10229-1-git-send-email-tboegi@web.de","subject":"Re: [PATCH v7 2/2] convert: ce_compare_data() checks for a sha1 of a path","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-05-24T18:36:26Z","receivedAt":"2016-05-24T18:36:26Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"tboegi@web.de writes:\n\n> From: Torsten Bögershausen \n>\n> To compare a file in working tree with the index, convert_to_git() is used,\n> the result is hashed and the hash value compared with ce->sha1.\n>\n> Deep down would_convert_crlf_at_commit() is invoked, to check if CRLF\n> are converted or not.\n> The \"new safer autocrlf handling\" checks if CRLF had been in the index before,\n> and if, the CRLF in the working tree are not converted.\n>\n> While in a merge, a file name in the working tree has different blobs\n> in the index with different hash values.\n> Forwarding ce->sha1 from ce_compare_data() into crlf_to_git() makes sure\n> the would_convert_crlf_at_commit() looks at the appropriate blob.\n>\n> Add a new parameter index_blob_sha1 to convert_to_git(), and forward the\n> sha1 from ce_compare_data() into convert_to_git(). Other callers use NULL\n> for index_blob_sha1, and the sha1 is determined from path\n> using get_sha1_from_cache(path). This is the same handling as before.\n>\n> In the same spirit, forward the sha1 into would_convert_to_git().\n>\n> While at it, rename has_cr_in_index() into blob_has_cr() and replace\n> 0 with SAFE_CRLF_FALSE.\n>\n> Signed-off-by: Torsten Bögershausen \n> ---\n\nAssuming that it is sensible to pass an extra \"no, no, do not look\nat the index entry at path, but look at this blob to see if the CRLF\nconversion must be disabled\" parameter all the down the callchain,\nthis round looks good.\n\nI really hate to say this this late in the reroll cycle, but this\npart of the description makes me wonder:\n\n > While in a merge, a file name in the working tree has\n > different blobs in the index with different hash values.\n > Forwarding ce->sha1 from ce_compare_data() into crlf_to_git()\n > makes sure the would_convert_crlf_at_commit() looks at the\n > appropriate blob.\n\nWhen we need content-level merges with help from the end user, we\nwould need to \"convert-to-git\" the result of conflict resolution by\nthe user left in the working tree, and that convert-to-git needs to\ntake our original contents into account, i.e. \"did we have CR in the\nblob already? if so, disable the CRLF thing\".\n\nBut we would always have \"our\" original contents at stage #2 in the\nindex in such a case, and would_convert_to_git() eventually calls\ninto read_blob_data_from_cache(), which knows to read from stage #2\n\nEven if we were in a renaming merge conflict, where they renamed\nfile F to G while we kept file F as file F, the conflicted working\ntree file will be made in path G, and stage #2 of the index for path\nG would have our original contents we had at path F.\n\nSo it is not clear why this \"no, no, look at this blob instead\" is\nnecessary in the first place. What problem does this solve? Is\nthis a fix for something that can be easily demonstrated with a new\ntest case?\n"},{"id":"287833","messageId":"1464627642-23994-1-git-send-email-tboegi@web.de","threadId":"42304","inReplyTo":"xmqq7fev55qk.fsf@gitster.mtv.corp.google.com","subject":"[PATCH v1 0/1] t6038-merge-text-auto.sh","fromName":"","fromEmail":"tboegi@web.de","sentAt":"2016-05-30T17:00:42Z","receivedAt":"2016-05-30T17:00:42Z","isPatch":true,"sender":{"key":"tboegi@web.de","avatar":"https://avatars.githubusercontent.com/u/7138363?v=4"},"body":"From: Torsten Bögershausen \n\nSplit of the old 10/10 series.\nThis is the update of t6038, which is needed to\nmotivate the patch\n `convert: ce_compare_data() checks for a sha1 of a path`\non top of\n `convert: unify the \"auto\" handling of CRLF`\n\nWhen files with different eols are merged with\nmerge.renormalize = true,\nit is important to look at the right blob to determine if\nthe crlf came from the blob or are a result of a coversion.\nThis is a little bit of a hen-and-egg problem:\nThe problem comes up after the \"unified auto handling\".\nIn theory, it should have been since before:\nget_sha1_from_index() says:\n\n * We might be in the middle of a merge, in which\n * case we would read stage #2 (ours).\n\nThis seams wrong, as in the merge we sometimes need to\nlook at \"theirs\".\n(But I haven't managed to construct a TC)\n\nt6038-merge-text-auto.sh\n\nTorsten Bögershausen (1):\n t6038: different eol for \"Merge addition of text=auto\"\n\n t/t6038-merge-text-auto.sh | 28 +++++++++++++++++++++++-----\n 1 file changed, 23 insertions(+), 5 deletions(-)\n\n-- \n2.0.0.rc1.6318.g0c2c796\n"},{"id":"287831","messageId":"1464627646-24041-1-git-send-email-tboegi@web.de","threadId":"42304","inReplyTo":"xmqq7fev55qk.fsf@gitster.mtv.corp.google.com","subject":"[PATCH v1 1/1] t6038: different eol for \"Merge addition of text=auto\"","fromName":"","fromEmail":"tboegi@web.de","sentAt":"2016-05-30T17:00:46Z","receivedAt":"2016-05-30T17:00:46Z","isPatch":true,"sender":{"key":"tboegi@web.de","avatar":"https://avatars.githubusercontent.com/u/7138363?v=4"},"body":"From: Torsten Bögershausen \n\nt6038 uses different code, depending if NATIVE_CRLF is set ot not.\n\nEnhance the test coverage for cross-platform testing and run\n\"Merge addition of text=auto\" with both lf and crlf as core.eol.\n\nIt is important to be run this test with crlf under Linux,\nthe day when the \"unified auto handling will be introduced.\n\nSigned-off-by: Torsten Bögershausen \n---\n t/t6038-merge-text-auto.sh | 28 +++++++++++++++++++++++-----\n 1 file changed, 23 insertions(+), 5 deletions(-)\n\ndiff --git a/t/t6038-merge-text-auto.sh b/t/t6038-merge-text-auto.sh\nindex 33b77ee..8846f5d 100755\n--- a/t/t6038-merge-text-auto.sh\n+++ b/t/t6038-merge-text-auto.sh\n@@ -91,16 +91,13 @@ test_expect_success 'Merge after setting text=auto' '\n \tcompare_files expected file\n '\n \n-test_expect_success 'Merge addition of text=auto' '\n+test_expect_success 'Merge addition of text=auto eol=LF' '\n+\tgit config core.eol lf &&\n \tcat <<-\\EOF >expected &&\n \tfirst line\n \tsame line\n \tEOF\n \n-\tif test_have_prereq NATIVE_CRLF; then\n-\t\tappend_cr expected.temp &&\n-\t\tmv expected.temp expected\n-\tfi &&\n \tgit config merge.renormalize true &&\n \tgit rm -fr . &&\n \trm -f .gitattributes &&\n@@ -109,7 +106,28 @@ test_expect_success 'Merge addition of text=auto' '\n \tcompare_files expected file\n '\n \n+test_expect_success 'Merge addition of text=auto eol=CRLF' '\n+\tgit config core.eol crlf &&\n+\tcat <<-\\EOF >expected &&\n+\tfirst line\n+\tsame line\n+\tEOF\n+\n+\tappend_cr expected.temp &&\n+\tmv expected.temp expected &&\n+\tgit config merge.renormalize true &&\n+\tgit rm -fr . &&\n+\trm -f .gitattributes &&\n+\tgit reset --hard b &&\n+\techo >&2 \"After git reset --hard b\" &&\n+\tgit ls-files -s --eol >&2 &&\n+\tgit merge a &&\n+\tcompare_files expected file\n+'\n+\n+\n test_expect_success 'Detect CRLF/LF conflict after setting text=auto' '\n+\tgit config core.eol native &&\n \techo \"<<<<<<<\" >expected &&\n \tif test_have_prereq NATIVE_CRLF; then\n \t\techo first line | append_cr >>expected &&\n-- \n2.0.0.rc1.6318.g0c2c796\n"},{"id":"287835","messageId":"xmqqzir7o286.fsf@gitster.mtv.corp.google.com","threadId":"42304","inReplyTo":"1464627642-23994-1-git-send-email-tboegi@web.de","subject":"Re: [PATCH v1 0/1] t6038-merge-text-auto.sh","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-05-30T18:00:25Z","receivedAt":"2016-05-30T18:00:25Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"tboegi@web.de writes:\n\n> This is a little bit of a hen-and-egg problem:\n> The problem comes up after the \"unified auto handling\".\n> In theory, it should have been since before:\n> get_sha1_from_index() says:\n>\n> * We might be in the middle of a merge, in which\n> * case we would read stage #2 (ours).\n>\n> This seams wrong, as in the merge we sometimes need to\n> look at \"theirs\".\n\nThe two comment you quoted is absolutely the right thing to do.\n\"In a merge, we sometimes need to look at 'theirs'\" is like saying\n\"When we are dong 'git add', we need to look at what is in the\nworking tree\". It is total red herring.\n\nStep back and think why we even look at what in the index in the\nfirst place; it is to decide if we want or do not want to disable\nthe automatic CRLF -> LF conversion. And think again the reason why\ndo we look at the index.\n\nThere may be a line with CRLF line endings in the new contents,\nwhether it came from a merge, cherry-pick, patch application, or\nplain-simple \"git add\" from the working tree. Auto-CRLF usually\nsays \"We want CRLF turned into LF\". But the user misconfigured and\nfor this path the user might not want the conversion take place, in\nwhich case you would disable the conversion. Where do you take that\nhint \"the user might have misconfigured?\" from? By looking at what\nthe user _started_ her update from. If the state before this \"we\nneed to replace the blob in the index with a new contents, so we\nneed to hash the new contents to come up with the updated blob\"\nstarted contains CRLF already, that may be a hint--if we apply the\nCRLF->LF conversion on the original, even if the \"new contents\" were\nidentical to what she already had, we would end up changing the blob\nwith her current configuration. Hence we disable.\n\nIsn't that the reasoning behind that \"safe auto-crlf\" thing?\n\nThe new contents getting integrated into her current state may have\nCRLF, and if a merge or a cherry-pick leaves conflicts, they may be\nstored in stage #3. But paying attention to that to decide if we\nwant to disable Auto-CRLF conversion is simply wrong; you should\nlook at the CRLF in stage #3 as purely something that might need to\nbe converted, not something that affects the decision if it needs to\nbe converted, just like you view CRLF in a working tree file when\nyou do \"git add\"..\n\nImagine that you started from a history where somebody recorded a\ntext file with CRLF in the blob, unconverted. Later the project\ndecided to express their text with LF to support cross-platform\ndevelopment better, and sets up the Auto-CRLF. Your user is working\nnear the tip of that history after the eol correction happened. Now\nshe gets a pull-request of a branch that forked from an old point in\nthe history, before the eol correction and full of CRLF. Yes, to\nintegrate the change being proposed, she needs to look at \"theirs\";\nthat's the whole point of a \"merge\". Why should she revert the eol\ncorrection her history has by getting fooled by the fact that the\nupdate was based on a part of the history before the eol correction?\n"},{"id":"287844","messageId":"xmqqy46r8jre.fsf@gitster.mtv.corp.google.com","threadId":"42304","inReplyTo":"xmqqzir7o286.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH v1 0/1] t6038-merge-text-auto.sh","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-05-30T18:48:21Z","receivedAt":"2016-05-30T18:48:21Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Junio C Hamano writes:\n\n> Imagine that you started from a history where somebody recorded a\n> text file with CRLF in the blob, unconverted. Later the project\n> decided to express their text with LF to support cross-platform\n> development better, and sets up the Auto-CRLF. Your user is working\n> near the tip of that history after the eol correction happened. Now\n> she gets a pull-request of a branch that forked from an old point in\n> the history, before the eol correction and full of CRLF. Yes, to\n> integrate the change being proposed, she needs to look at \"theirs\";\n> that's the whole point of a \"merge\". Why should she revert the eol\n> correction her history has by getting fooled by the fact that the\n> update was based on a part of the history before the eol correction?\n\nThinking aloud along this line a bit further, if you really cared\n(and I don't feel very strongly, as I think \"safe crlf\" is an ugly\nworkaround that lets users keep their misconfiguration by penalizing\ntheir day-to-day operation), you may want to think about doing a\n3-way merge of \"eol preference\" beween all stages.\n\nThat is, if the common ancestor in stage #1 and the current version\nin stage #2 both have its text in LF, and the data being merged in\nstage #3 is in CRLF, you sayr \"do not convert; the change being\nbrought in wants to have CRLF endings, while our history did not\ncare\". Similarly, if the common ancestor in stage #1 and the data\nbeing merged in stage #3 both have CRLF, and your version in stage\n#2 has LF, you say \"We wanted to fix eol since the side branch\nforked, and the side branch predates that fix, so we keep the eol\nfix we did since we diverged\", i.e. \"Do convert\".\n\nFor doing this, you may want to refactor the codepath that decides\n\"Auto-CRLF usually wants to turn CRLF in text to LF, but should we\ndisable that logic now, because the user already has CRLF in the\ncurrent one?\" into a function that takes a single parameter, `path`,\nand returns \"Yes, do convert CRLF to LF / No, do not convert\"\nboolean. Having a low level function that says \"What's the blob at\nthis path in the index\" and have the caller run that logic feels\nunwieldy if we want to go that route.\n"},{"id":"288610","messageId":"1465312846-4791-1-git-send-email-tboegi@web.de","threadId":"42304","inReplyTo":"xmqq7fev55qk.fsf@gitster.mtv.corp.google.com","subject":"[PATCH v2 0/3] unified auto CRLF handling, V2","fromName":"","fromEmail":"tboegi@web.de","sentAt":"2016-06-07T15:20:46Z","receivedAt":"2016-06-07T15:20:46Z","isPatch":true,"sender":{"key":"tboegi@web.de","avatar":"https://avatars.githubusercontent.com/u/7138363?v=4"},"body":"From: Torsten Bögershausen \n\nunified auto CRLF handling, V2\n 1/3 is 7/10 of the old 10/10 series\n 2/3 and \n 3/3 is a replacement for tb/convert-peek-in-index:\n Better commit message, added test case\n\nAll in all we are getting closer.\nMost of the patches had been send & reviewed earlier,\nbut a critical review won't hurt.\n\nTorsten Bögershausen (3):\n convert: unify the \"auto\" handling of CRLF\n read-cache: factor out get_sha1_from_index() helper\n Correct ce_compare_data() in a middle of a merge\n\n Documentation/config.txt | 12 +++----\n Documentation/gitattributes.txt | 15 +++++----\n builtin/apply.c | 3 +-\n builtin/blame.c | 2 +-\n cache.h | 4 +++\n combine-diff.c | 3 +-\n convert.c | 65 +++++++++++++++++++++++-------------\n convert.h | 18 +++++++---\n diff.c | 3 +-\n dir.c | 2 +-\n read-cache.c | 33 +++++++++++-------\n sha1_file.c | 12 +++++--\n t/t0025-crlf-auto.sh | 4 +--\n t/t0027-auto-crlf.sh | 32 +++++++++---------\n t/t6038-merge-text-auto.sh | 74 ++++++++++++++++++++++++-----------------\n 15 files changed, 172 insertions(+), 110 deletions(-)\n\n-- \n2.0.0.rc1.6318.g0c2c796\n"},{"id":"288613","messageId":"1465312848-4830-1-git-send-email-tboegi@web.de","threadId":"42304","inReplyTo":"xmqq7fev55qk.fsf@gitster.mtv.corp.google.com","subject":"[PATCH v2 1/3] convert: unify the \"auto\" handling of CRLF","fromName":"","fromEmail":"tboegi@web.de","sentAt":"2016-06-07T15:20:48Z","receivedAt":"2016-06-07T15:20:48Z","isPatch":true,"sender":{"key":"tboegi@web.de","avatar":"https://avatars.githubusercontent.com/u/7138363?v=4"},"body":"From: Torsten Bögershausen \n\nBefore this change,\n$ echo \"* text=auto\" >.gitattributes\n$ echo \"* eol=crlf\" >>.gitattributes\n\nwould have the same effect as\n$ echo \"* text\" >.gitattributes\n$ git config core.eol crlf\n\nSince the 'eol' attribute had higher priority than 'text=auto', this may\ncorrupt binary files and is not what most users expect to happen.\n\nMake the 'eol' attribute to obey 'text=auto', and now\n$ echo \"* text=auto\" >.gitattributes\n$ echo \"* eol=crlf\" >>.gitattributes\nbehaves the same as\n$ echo \"* text=auto\" >.gitattributes\n$ git config core.eol crlf\n\nIn other words,\n$ echo \"* text=auto eol=crlf\" >.gitattributes\nhas the same effect as\n$ git config core.autocrlf true\n\nand\n$ echo \"* text=auto eol=lf\" >.gitattributes\nhas the same effect as\n$ git config core.autocrlf input\n\nSigned-off-by: Torsten Bögershausen \n---\n Documentation/config.txt | 12 +++++-------\n Documentation/gitattributes.txt | 15 +++++++++------\n convert.c | 42 +++++++++++++++++++++--------------------\n convert.h | 3 ++-\n t/t0025-crlf-auto.sh | 4 ++--\n t/t0027-auto-crlf.sh | 32 +++++++++++++++----------------\n t/t6038-merge-text-auto.sh | 23 ++++++++++++++--------\n 7 files changed, 71 insertions(+), 60 deletions(-)\n\ndiff --git a/Documentation/config.txt b/Documentation/config.txt\nindex 53f00db..208b5de 100644\n--- a/Documentation/config.txt\n+++ b/Documentation/config.txt\n@@ -405,13 +405,11 @@ file with mixed line endings would be reported by the `core.safecrlf`\n mechanism.\n \n core.autocrlf::\n-\tSetting this variable to \"true\" is almost the same as setting\n-\tthe `text` attribute to \"auto\" on all files except that text\n-\tfiles are not guaranteed to be normalized: files that contain\n-\t`CRLF` in the repository will not be touched. Use this\n-\tsetting if you want to have `CRLF` line endings in your\n-\tworking directory even though the repository does not have\n-\tnormalized line endings. This variable can be set to 'input',\n+\tSetting this variable to \"true\" is the same as setting\n+\tthe `text` attribute to \"auto\" on all files and core.eol to \"crlf\".\n+\tSet to true if you want to have `CRLF` line endings in your\n+\tworking directory and the repository has LF line endings.\n+\tThis variable can be set to 'input',\n \tin which case no output conversion is performed.\n \n core.symlinks::\ndiff --git a/Documentation/gitattributes.txt b/Documentation/gitattributes.txt\nindex e3b1de8..d7a124b 100644\n--- a/Documentation/gitattributes.txt\n+++ b/Documentation/gitattributes.txt\n@@ -115,6 +115,7 @@ text file is normalized, its line endings are converted to LF in the\n repository. To control what line ending style is used in the working\n directory, use the `eol` attribute for a single file and the\n `core.eol` configuration variable for all text files.\n+Note that `core.autocrlf` overrides `core.eol`\n \n Set::\n \n@@ -130,8 +131,9 @@ Unset::\n Set to string value \"auto\"::\n \n \tWhen `text` is set to \"auto\", the path is marked for automatic\n-\tend-of-line normalization. If Git decides that the content is\n-\ttext, its line endings are normalized to LF on checkin.\n+\tend-of-line conversion. If Git decides that the content is\n+\ttext, its line endings are converted to LF on checkin.\n+\tWhen the file has been commited with CRLF, no conversion is done.\n \n Unspecified::\n \n@@ -146,7 +148,7 @@ unspecified.\n ^^^^^\n \n This attribute sets a specific line-ending style to be used in the\n-working directory. It enables end-of-line normalization without any\n+working directory. It enables end-of-line conversion without any\n content checks, effectively setting the `text` attribute.\n \n Set to string value \"crlf\"::\n@@ -186,9 +188,10 @@ the working directory, and prevent .jpg files from being normalized\n regardless of their content.\n \n ------------------------\n+* text=auto\n *.txt\t\ttext\n-*.vcproj\teol=crlf\n-*.sh\t\teol=lf\n+*.vcproj\ttext eol=crlf\n+*.sh\t\ttext eol=lf\n *.jpg\t\t-text\n ------------------------\n \n@@ -198,7 +201,7 @@ normalization in Git.\n \n If you simply want to have CRLF line endings in your working directory\n regardless of the repository you are working with, you can set the\n-config variable \"core.autocrlf\" without changing any attributes.\n+config variable \"core.autocrlf\" without using any attributes.\n \n ------------------------\n [core]\ndiff --git a/convert.c b/convert.c\nindex b1614bf..67d69b5 100644\n--- a/convert.c\n+++ b/convert.c\n@@ -176,7 +176,9 @@ static enum eol output_eol(enum crlf_action crlf_action)\n \t\treturn EOL_LF;\n \tcase CRLF_UNDEFINED:\n \tcase CRLF_AUTO_CRLF:\n+\t\treturn EOL_CRLF;\n \tcase CRLF_AUTO_INPUT:\n+\t\treturn EOL_LF;\n \tcase CRLF_TEXT:\n \tcase CRLF_AUTO:\n \t\t/* fall through */\n@@ -254,17 +256,15 @@ static int crlf_to_git(const char *path, const char *src, size_t len,\n \tif (crlf_action == CRLF_AUTO || crlf_action == CRLF_AUTO_INPUT || crlf_action == CRLF_AUTO_CRLF) {\n \t\tif (convert_is_binary(len, &stats))\n \t\t\treturn 0;\n-\n-\t\tif (crlf_action == CRLF_AUTO_INPUT || crlf_action == CRLF_AUTO_CRLF) {\n-\t\t\t/*\n-\t\t\t * If the file in the index has any CR in it, do not convert.\n-\t\t\t * This is the new safer autocrlf handling.\n-\t\t\t */\n-\t\t\tif (has_cr_in_index(path))\n-\t\t\t\treturn 0;\n-\t\t}\n+\t\t/*\n+\t\t * If the file in the index has any CR in it, do not convert.\n+\t\t * This is the new safer autocrlf handling.\n+\t\t */\n+\t\tif (checksafe == SAFE_CRLF_RENORMALIZE)\n+\t\t\tchecksafe = SAFE_CRLF_FALSE;\n+\t\telse if (has_cr_in_index(path))\n+\t\t\treturn 0;\n \t}\n-\n \tcheck_safe_crlf(path, crlf_action, &stats, checksafe);\n \n \t/* Optimization: No CRLF? Nothing to convert, regardless. */\n@@ -320,12 +320,10 @@ static int crlf_to_worktree(const char *path, const char *src, size_t len,\n \t\treturn 0;\n \n \tif (crlf_action == CRLF_AUTO || crlf_action == CRLF_AUTO_INPUT || crlf_action == CRLF_AUTO_CRLF) {\n-\t\tif (crlf_action == CRLF_AUTO_INPUT || crlf_action == CRLF_AUTO_CRLF) {\n-\t\t\t/* If we have any CR or CRLF line endings, we do not touch it */\n-\t\t\t/* This is the new safer autocrlf-handling */\n-\t\t\tif (stats.lonecr || stats.crlf )\n-\t\t\t\treturn 0;\n-\t\t}\n+\t\t/* If we have any CR or CRLF line endings, we do not touch it */\n+\t\t/* This is the new safer autocrlf-handling */\n+\t\tif (stats.lonecr || stats.crlf )\n+\t\t\treturn 0;\n \n \t\tif (convert_is_binary(len, &stats))\n \t\t\treturn 0;\n@@ -786,7 +784,11 @@ static void convert_attrs(struct conv_attrs *ca, const char *path)\n \t\tca->drv = git_path_check_convert(ccheck + 2);\n \t\tif (ca->crlf_action != CRLF_BINARY) {\n \t\t\tenum eol eol_attr = git_path_check_eol(ccheck + 3);\n-\t\t\tif (eol_attr == EOL_LF)\n+\t\t\tif (ca->crlf_action == CRLF_AUTO && eol_attr == EOL_LF)\n+\t\t\t\tca->crlf_action = CRLF_AUTO_INPUT;\n+\t\t\telse if (ca->crlf_action == CRLF_AUTO && eol_attr == EOL_CRLF)\n+\t\t\t\tca->crlf_action = CRLF_AUTO_CRLF;\n+\t\t\telse if (eol_attr == EOL_LF)\n \t\t\t\tca->crlf_action = CRLF_TEXT_INPUT;\n \t\t\telse if (eol_attr == EOL_CRLF)\n \t\t\t\tca->crlf_action = CRLF_TEXT_CRLF;\n@@ -845,9 +847,9 @@ const char *get_convert_attr_ascii(const char *path)\n \tcase CRLF_AUTO:\n \t\treturn \"text=auto\";\n \tcase CRLF_AUTO_CRLF:\n-\t\treturn \"text=auto eol=crlf\"; /* This is not supported yet */\n+\t\treturn \"text=auto eol=crlf\";\n \tcase CRLF_AUTO_INPUT:\n-\t\treturn \"text=auto eol=lf\"; /* This is not supported yet */\n+\t\treturn \"text=auto eol=lf\";\n \t}\n \treturn \"\";\n }\n@@ -949,7 +951,7 @@ int renormalize_buffer(const char *path, const char *src, size_t len, struct str\n \t\tsrc = dst->buf;\n \t\tlen = dst->len;\n \t}\n-\treturn ret | convert_to_git(path, src, len, dst, SAFE_CRLF_FALSE);\n+\treturn ret | convert_to_git(path, src, len, dst, SAFE_CRLF_RENORMALIZE);\n }\n \n /*****************************************************************\ndiff --git a/convert.h b/convert.h\nindex ccf436b..81b6cdf 100644\n--- a/convert.h\n+++ b/convert.h\n@@ -7,7 +7,8 @@\n enum safe_crlf {\n \tSAFE_CRLF_FALSE = 0,\n \tSAFE_CRLF_FAIL = 1,\n-\tSAFE_CRLF_WARN = 2\n+\tSAFE_CRLF_WARN = 2,\n+\tSAFE_CRLF_RENORMALIZE = 4\n };\n \n extern enum safe_crlf safe_crlf;\ndiff --git a/t/t0025-crlf-auto.sh b/t/t0025-crlf-auto.sh\nindex c164b46..d0bee08 100755\n--- a/t/t0025-crlf-auto.sh\n+++ b/t/t0025-crlf-auto.sh\n@@ -114,7 +114,7 @@ test_expect_success 'autocrlf=true does not normalize CRLF files' '\n \ttest -z \"$LFonlydiff\" -a -z \"$CRLFonlydiff\" -a -z \"$LFwithNULdiff\"\n '\n \n-test_expect_success 'text=auto, autocrlf=true _does_ normalize CRLF files' '\n+test_expect_success 'text=auto, autocrlf=true does not normalize CRLF files' '\n \n \trm -f .gitattributes tmp LFonly CRLFonly LFwithNUL &&\n \tgit config core.autocrlf true &&\n@@ -126,7 +126,7 @@ test_expect_success 'text=auto, autocrlf=true _does_ normalize CRLF files' '\n \tLFonlydiff=$(git diff LFonly) &&\n \tCRLFonlydiff=$(git diff CRLFonly) &&\n \tLFwithNULdiff=$(git diff LFwithNUL) &&\n-\ttest -z \"$LFonlydiff\" -a -n \"$CRLFonlydiff\" -a -z \"$LFwithNULdiff\"\n+\ttest -z \"$LFonlydiff\" -a -z \"$CRLFonlydiff\" -a -z \"$LFwithNULdiff\"\n '\n \n test_expect_success 'text=auto, autocrlf=true does not normalize binary files' '\ndiff --git a/t/t0027-auto-crlf.sh b/t/t0027-auto-crlf.sh\nindex 9372589..8367d0b 100755\n--- a/t/t0027-auto-crlf.sh\n+++ b/t/t0027-auto-crlf.sh\n@@ -175,8 +175,8 @@ attr_ascii () {\n \ttext,lf) echo \"text eol=lf\" ;;\n \ttext,crlf) echo \"text eol=crlf\" ;;\n \tauto,) echo \"text=auto\" ;;\n-\tauto,lf) echo \"text eol=lf\" ;;\n-\tauto,crlf) echo \"text eol=crlf\" ;;\n+\tauto,lf) echo \"text=auto eol=lf\" ;;\n+\tauto,crlf) echo \"text=auto eol=crlf\" ;;\n \tlf,) echo \"text eol=lf\" ;;\n \tcrlf,) echo \"text eol=crlf\" ;;\n \t,) echo \"\" ;;\n@@ -397,10 +397,9 @@ commit_chk_wrnNNO \"\" \"\" false \"\" \"\" \"\" \"\"\n commit_chk_wrnNNO \"\" \"\" true LF_CRLF \"\" \"\" \"\" \"\"\n commit_chk_wrnNNO \"\" \"\" input \"\" \"\" \"\" \"\" \"\"\n \n-commit_chk_wrnNNO \"auto\" \"\" false \"$WILC\" \"$WICL\" \"$WAMIX\" \"\" \"\"\n-commit_chk_wrnNNO \"auto\" \"\" true LF_CRLF \"\" LF_CRLF \"\" \"\"\n-commit_chk_wrnNNO \"auto\" \"\" input \"\" CRLF_LF CRLF_LF \"\" \"\"\n-\n+commit_chk_wrnNNO \"auto\" \"\" false \"$WILC\" \"\" \"\" \"\" \"\"\n+commit_chk_wrnNNO \"auto\" \"\" true LF_CRLF \"\" \"\" \"\" \"\"\n+commit_chk_wrnNNO \"auto\" \"\" input \"\" \"\" \"\" \"\" \"\"\n for crlf in true false input\n do\n \tcommit_chk_wrnNNO -text \"\" $crlf \"\" \"\" \"\" \"\" \"\"\n@@ -408,8 +407,8 @@ do\n \tcommit_chk_wrnNNO -text crlf $crlf \"\" \"\" \"\" \"\" \"\"\n \tcommit_chk_wrnNNO \"\" lf $crlf \"\" CRLF_LF CRLF_LF \"\" CRLF_LF\n \tcommit_chk_wrnNNO \"\" crlf $crlf LF_CRLF \"\" LF_CRLF LF_CRLF \"\"\n-\tcommit_chk_wrnNNO auto lf \t$crlf \"\" CRLF_LF CRLF_LF \"\" CRLF_LF\n-\tcommit_chk_wrnNNO auto crlf \t$crlf LF_CRLF \"\" LF_CRLF LF_CRLF \"\"\n+\tcommit_chk_wrnNNO auto lf \t$crlf \"\" \"\" \"\" \"\" \"\"\n+\tcommit_chk_wrnNNO auto crlf \t$crlf LF_CRLF \"\" \"\" \"\" \"\"\n \tcommit_chk_wrnNNO text lf \t$crlf \"\" CRLF_LF CRLF_LF \"\" CRLF_LF\n \tcommit_chk_wrnNNO text crlf \t$crlf LF_CRLF \"\" LF_CRLF LF_CRLF \"\"\n done\n@@ -454,9 +453,9 @@ do\n \tcheck_in_repo_NNO -text \"\" $crlf LF CRLF CRLF_mix_LF LF_mix_CR CRLF_nul\n \tcheck_in_repo_NNO -text lf $crlf LF CRLF CRLF_mix_LF LF_mix_CR CRLF_nul\n \tcheck_in_repo_NNO -text crlf $crlf LF CRLF CRLF_mix_LF LF_mix_CR CRLF_nul\n-\tcheck_in_repo_NNO auto \"\" $crlf LF LF LF LF_mix_CR CRLF_nul\n-\tcheck_in_repo_NNO auto lf $crlf LF LF LF LF_mix_CR LF_nul\n-\tcheck_in_repo_NNO auto crlf $crlf LF LF LF LF_mix_CR LF_nul\n+\tcheck_in_repo_NNO auto \"\" $crlf LF CRLF CRLF_mix_LF LF_mix_CR CRLF_nul\n+\tcheck_in_repo_NNO auto lf $crlf LF CRLF CRLF_mix_LF LF_mix_CR CRLF_nul\n+\tcheck_in_repo_NNO auto crlf $crlf LF CRLF CRLF_mix_LF LF_mix_CR CRLF_nul\n \tcheck_in_repo_NNO text \"\" $crlf LF LF LF LF_mix_CR LF_nul\n \tcheck_in_repo_NNO text lf $crlf LF LF LF LF_mix_CR LF_nul\n \tcheck_in_repo_NNO text crlf $crlf LF LF LF LF_mix_CR LF_nul\n@@ -493,7 +492,8 @@ fi\n export CRLF_MIX_LF_CR MIX NL\n \n # Same handling with and without ident\n-for id in \"\" ident\n+#for id in \"\" ident\n+for id in \"\"\n do\n \tfor ceol in lf crlf native\n \tdo\n@@ -509,7 +509,7 @@ do\n \t\t\tcheckout_files text \"$id\" \"crlf\" \"$crlf\" \"$ceol\" CRLF CRLF CRLF CRLF_mix_CR CRLF_nul\n \t\t\t# currently the same as text, eol=XXX\n \t\t\tcheckout_files auto \"$id\" \"lf\" \"$crlf\" \"$ceol\" LF CRLF CRLF_mix_LF LF_mix_CR LF_nul\n-\t\t\tcheckout_files auto \"$id\" \"crlf\" \"$crlf\" \"$ceol\" CRLF CRLF CRLF CRLF_mix_CR CRLF_nul\n+\t\t\tcheckout_files auto \"$id\" \"crlf\" \"$crlf\" \"$ceol\" CRLF CRLF CRLF_mix_LF LF_mix_CR LF_nul\n \t\tdone\n \n \t\t# core.autocrlf false, different core.eol\n@@ -517,7 +517,7 @@ do\n \t\t# core.autocrlf true\n \t\tcheckout_files \"\" \"$id\" \"\" true \"$ceol\" CRLF CRLF CRLF_mix_LF LF_mix_CR LF_nul\n \t\t# text: core.autocrlf = true overrides core.eol\n-\t\tcheckout_files auto \"$id\" \"\" true \"$ceol\" CRLF CRLF CRLF LF_mix_CR LF_nul\n+\t\tcheckout_files auto \"$id\" \"\" true \"$ceol\" CRLF CRLF CRLF_mix_LF LF_mix_CR LF_nul\n \t\tcheckout_files text \"$id\" \"\" true \"$ceol\" CRLF CRLF CRLF CRLF_mix_CR CRLF_nul\n \t\t# text: core.autocrlf = input overrides core.eol\n \t\tcheckout_files text \"$id\" \"\" input \"$ceol\" LF CRLF CRLF_mix_LF LF_mix_CR LF_nul\n@@ -531,8 +531,8 @@ do\n \tcheckout_files text \"$id\" \"\" false \"\" $NL CRLF $MIX_CRLF_LF $MIX_LF_CR $LFNUL\n \tcheckout_files text \"$id\" \"\" false native $NL CRLF $MIX_CRLF_LF $MIX_LF_CR $LFNUL\n \t# auto: core.autocrlf=false and core.eol unset(or native) uses native eol\n-\tcheckout_files auto \"$id\" \"\" false \"\" $NL CRLF $MIX_CRLF_LF LF_mix_CR LF_nul\n-\tcheckout_files auto \"$id\" \"\" false native $NL CRLF $MIX_CRLF_LF LF_mix_CR LF_nul\n+\tcheckout_files auto \"$id\" \"\" false \"\" $NL CRLF CRLF_mix_LF LF_mix_CR LF_nul\n+\tcheckout_files auto \"$id\" \"\" false native $NL CRLF CRLF_mix_LF LF_mix_CR LF_nul\n done\n \n # Should be the last test case: remove some files from the worktree\ndiff --git a/t/t6038-merge-text-auto.sh b/t/t6038-merge-text-auto.sh\nindex 85c10b0..33b77ee 100755\n--- a/t/t6038-merge-text-auto.sh\n+++ b/t/t6038-merge-text-auto.sh\n@@ -16,6 +16,13 @@ test_description='CRLF merge conflict across text=auto change\n \n test_have_prereq SED_STRIPS_CR && SED_OPTIONS=-b\n \n+compare_files () {\n+\ttr '\\015\\000' QN <\"$1\" >\"$1\".expect &&\n+\ttr '\\015\\000' QN <\"$2\" >\"$2\".actual &&\n+\ttest_cmp \"$1\".expect \"$2\".actual &&\n+\trm \"$1\".expect \"$2\".actual\n+}\n+\n test_expect_success setup '\n \tgit config core.autocrlf false &&\n \n@@ -30,7 +37,7 @@ test_expect_success setup '\n \tgit branch side &&\n \n \techo \"* text=auto\" >.gitattributes &&\n-\ttouch file &&\n+\techo first line >file &&\n \tgit add .gitattributes file &&\n \ttest_tick &&\n \tgit commit -m \"normalize file\" &&\n@@ -81,7 +88,7 @@ test_expect_success 'Merge after setting text=auto' '\n \trm -f .gitattributes &&\n \tgit reset --hard a &&\n \tgit merge b &&\n-\ttest_cmp expected file\n+\tcompare_files expected file\n '\n \n test_expect_success 'Merge addition of text=auto' '\n@@ -99,7 +106,7 @@ test_expect_success 'Merge addition of text=auto' '\n \trm -f .gitattributes &&\n \tgit reset --hard b &&\n \tgit merge a &&\n-\ttest_cmp expected file\n+\tcompare_files expected file\n '\n \n test_expect_success 'Detect CRLF/LF conflict after setting text=auto' '\n@@ -121,7 +128,7 @@ test_expect_success 'Detect CRLF/LF conflict after setting text=auto' '\n \tgit reset --hard a &&\n \ttest_must_fail git merge b &&\n \tfuzz_conflict file >file.fuzzy &&\n-\ttest_cmp expected file.fuzzy\n+\tcompare_files expected file.fuzzy\n '\n \n test_expect_success 'Detect LF/CRLF conflict from addition of text=auto' '\n@@ -143,7 +150,7 @@ test_expect_success 'Detect LF/CRLF conflict from addition of text=auto' '\n \tgit reset --hard b &&\n \ttest_must_fail git merge a &&\n \tfuzz_conflict file >file.fuzzy &&\n-\ttest_cmp expected file.fuzzy\n+\tcompare_files expected file.fuzzy\n '\n \n test_expect_failure 'checkout -m after setting text=auto' '\n@@ -158,7 +165,7 @@ test_expect_failure 'checkout -m after setting text=auto' '\n \tgit reset --hard initial &&\n \tgit checkout a -- . &&\n \tgit checkout -m b &&\n-\ttest_cmp expected file\n+\tcompare_files expected file\n '\n \n test_expect_failure 'checkout -m addition of text=auto' '\n@@ -173,7 +180,7 @@ test_expect_failure 'checkout -m addition of text=auto' '\n \tgit reset --hard initial &&\n \tgit checkout b -- . &&\n \tgit checkout -m a &&\n-\ttest_cmp expected file\n+\tcompare_files expected file\n '\n \n test_expect_failure 'cherry-pick patch from after text=auto was added' '\n@@ -187,7 +194,7 @@ test_expect_failure 'cherry-pick patch from after text=auto was added' '\n \tgit reset --hard b &&\n \ttest_must_fail git cherry-pick a >err 2>&1 &&\n \tgrep \"[Nn]othing added\" err &&\n-\ttest_cmp expected file\n+\tcompare_files expected file\n '\n \n test_expect_success 'Test delete/normalize conflict' '\n-- \n2.0.0.rc1.6318.g0c2c796\n"},{"id":"288611","messageId":"1465312850-4869-1-git-send-email-tboegi@web.de","threadId":"42304","inReplyTo":"xmqq7fev55qk.fsf@gitster.mtv.corp.google.com","subject":"[PATCH v2 2/3] read-cache: factor out get_sha1_from_index() helper","fromName":"","fromEmail":"tboegi@web.de","sentAt":"2016-06-07T15:20:50Z","receivedAt":"2016-06-07T15:20:50Z","isPatch":true,"sender":{"key":"tboegi@web.de","avatar":"https://avatars.githubusercontent.com/u/7138363?v=4"},"body":"From: Torsten Bögershausen \n\nFactor out the retrieval of the sha1 for a given path in\nread_blob_data_from_index() into the function get_sha1_from_index().\n\nThis will be used in the next commit, when convert.c can do the\nanalyze for \"text=auto\" without slurping the whole blob into memory\nat once.\n\nAdd a wrapper definition get_sha1_from_cache().\n\nSigned-off-by: Torsten Bögershausen \n---\n cache.h | 3 +++\n read-cache.c | 29 ++++++++++++++++++-----------\n 2 files changed, 21 insertions(+), 11 deletions(-)\n\ndiff --git a/cache.h b/cache.h\nindex 6049f86..322ee40 100644\n--- a/cache.h\n+++ b/cache.h\n@@ -379,6 +379,7 @@ extern void free_name_hash(struct index_state *istate);\n #define unmerge_cache_entry_at(at) unmerge_index_entry_at(&the_index, at)\n #define unmerge_cache(pathspec) unmerge_index(&the_index, pathspec)\n #define read_blob_data_from_cache(path, sz) read_blob_data_from_index(&the_index, (path), (sz))\n+#define get_sha1_from_cache(path) get_sha1_from_index (&the_index, (path))\n #endif\n \n enum object_type {\n@@ -1050,6 +1051,8 @@ static inline void *read_sha1_file(const unsigned char *sha1, enum object_type *\n \treturn read_sha1_file_extended(sha1, type, size, LOOKUP_REPLACE_OBJECT);\n }\n \n+const unsigned char *get_sha1_from_index(struct index_state *istate, const char *path);\n+\n /*\n * This internal function is only declared here for the benefit of\n * lookup_replace_object(). Please do not call it directly.\ndiff --git a/read-cache.c b/read-cache.c\nindex d9fb78b..a3ef967 100644\n--- a/read-cache.c\n+++ b/read-cache.c\n@@ -2263,13 +2263,27 @@ int index_name_is_other(const struct index_state *istate, const char *name,\n \n void *read_blob_data_from_index(struct index_state *istate, const char *path, unsigned long *size)\n {\n-\tint pos, len;\n+\tconst unsigned char *sha1;\n \tunsigned long sz;\n \tenum object_type type;\n \tvoid *data;\n \n-\tlen = strlen(path);\n-\tpos = index_name_pos(istate, path, len);\n+\tsha1 = get_sha1_from_index(istate, path);\n+\tif (!sha1)\n+\t\treturn NULL;\n+\tdata = read_sha1_file(sha1, &type, &sz);\n+\tif (!data || type != OBJ_BLOB) {\n+\t\tfree(data);\n+\t\treturn NULL;\n+\t}\n+\tif (size)\n+\t\t*size = sz;\n+\treturn data;\n+}\n+\n+const unsigned char *get_sha1_from_index(struct index_state *istate, const char *path)\n+{\n+\tint pos = index_name_pos(istate, path, strlen(path));\n \tif (pos < 0) {\n \t\t/*\n \t\t * We might be in the middle of a merge, in which\n@@ -2285,14 +2299,7 @@ void *read_blob_data_from_index(struct index_state *istate, const char *path, un\n \t}\n \tif (pos < 0)\n \t\treturn NULL;\n-\tdata = read_sha1_file(istate->cache[pos]->sha1, &type, &sz);\n-\tif (!data || type != OBJ_BLOB) {\n-\t\tfree(data);\n-\t\treturn NULL;\n-\t}\n-\tif (size)\n-\t\t*size = sz;\n-\treturn data;\n+\treturn (istate->cache[pos]->sha1);\n }\n \n void stat_validity_clear(struct stat_validity *sv)\n-- \n2.0.0.rc1.6318.g0c2c796\n"},{"id":"288612","messageId":"1465312851-4908-1-git-send-email-tboegi@web.de","threadId":"42304","inReplyTo":"xmqq7fev55qk.fsf@gitster.mtv.corp.google.com","subject":"[PATCH v2 3/3] Correct ce_compare_data() in a middle of a merge","fromName":"","fromEmail":"tboegi@web.de","sentAt":"2016-06-07T15:20:51Z","receivedAt":"2016-06-07T15:20:51Z","isPatch":true,"sender":{"key":"tboegi@web.de","avatar":"https://avatars.githubusercontent.com/u/7138363?v=4"},"body":"From: Torsten Bögershausen \n\nThe following didn't work as expected:\n- In a middle of a merge\n- merge.renormalize is true,\n- .gitattributes = \"* text=auto\"\n- core.eol = crlf\n\nMerge a blob with CRLF \"first line\\r\\nsame line\\r\\n\" and a blob\nwith LF \"first line\\nsame line\\n\".\n\nThe expected result of the merge is \"first line\\nsame line\\n\".\n\nThe content in the working tree is \"first line\\r\\nsame line\\r\\n\",\nand ce_compare_data() should find that the content is clean and return 0.\n\nDeep down crlf_to_git() is invoked, to check if CRLF are converted or not.\n\nThe \"new safer autocrlf handling\" calls blob_has_cr().\n\nInstead of using the sha1 of the blob, (CRLF in this example),\nthe function get_sha1_from_index() is invoked.\nget_sha1_from_index() decides to return \"ours\" when in the middle of\nthe merge, which is LF.\n\nAs a result, the CRLF in the worktree are converted into LF before\nthe comparison.\nThe contents of LF and CRLF don't match any more.\n\nThe problem is that ce_compare_data() has ce->sha1, but the sha1 is lost\non it's way into blob_has_cr().\n\nForwarding ce->sha1 from ce_compare_data() into crlf_to_git() makes sure\nthat blob_has_cr() looks at the appropriate blob.\n\nAdd a new parameter index_blob_sha1 to convert_to_git(), and forward the\nsha1 from ce_compare_data() into convert_to_git(). Other callers use NULL\nfor index_blob_sha1, and the sha1 is determined from path\nusing get_sha1_from_cache(path). This is the same handling as before.\n\nIn the same spirit, forward the sha1 into would_convert_to_git().\n\nWhile at it, rename has_cr_in_index() into blob_has_cr()\nand replace 0 with SAFE_CRLF_FALSE.\n\nAdd a TC in t6038 to have a test coverage under Linux.\n\nSigned-off-by: Torsten Bögershausen \n---\n builtin/apply.c | 3 ++-\n builtin/blame.c | 2 +-\n cache.h | 1 +\n combine-diff.c | 3 ++-\n convert.c | 43 ++++++++++++++++++++++++++------------\n convert.h | 15 ++++++++++----\n diff.c | 3 ++-\n dir.c | 2 +-\n read-cache.c | 4 +++-\n sha1_file.c | 12 ++++++++---\n t/t6038-merge-text-auto.sh | 51 +++++++++++++++++++++++++---------------------\n 11 files changed, 90 insertions(+), 49 deletions(-)\n\ndiff --git a/builtin/apply.c b/builtin/apply.c\nindex 8e4da2e..0cf9a0a 100644\n--- a/builtin/apply.c\n+++ b/builtin/apply.c\n@@ -2140,7 +2140,8 @@ static int read_old_data(struct stat *st, const char *path, struct strbuf *buf)\n \tcase S_IFREG:\n \t\tif (strbuf_read_file(buf, path, st->st_size) != st->st_size)\n \t\t\treturn error(_(\"unable to open or read %s\"), path);\n-\t\tconvert_to_git(path, buf->buf, buf->len, buf, 0);\n+\t\tconvert_to_git(path, buf->buf, buf->len, buf,\n+\t\t\t SAFE_CRLF_FALSE, NULL);\n \t\treturn 0;\n \tdefault:\n \t\treturn -1;\ndiff --git a/builtin/blame.c b/builtin/blame.c\nindex 21f42b0..1c523b6 100644\n--- a/builtin/blame.c\n+++ b/builtin/blame.c\n@@ -2377,7 +2377,7 @@ static struct commit *fake_working_tree_commit(struct diff_options *opt,\n \t\tif (strbuf_read(&buf, 0, 0) < 0)\n \t\t\tdie_errno(\"failed to read from stdin\");\n \t}\n-\tconvert_to_git(path, buf.buf, buf.len, &buf, 0);\n+\tconvert_to_git(path, buf.buf, buf.len, &buf, SAFE_CRLF_FALSE, NULL);\n \torigin->file.ptr = buf.buf;\n \torigin->file.size = buf.len;\n \tpretend_sha1_file(buf.buf, buf.len, OBJ_BLOB, origin->blob_sha1);\ndiff --git a/cache.h b/cache.h\nindex 322ee40..86f48f9 100644\n--- a/cache.h\n+++ b/cache.h\n@@ -605,6 +605,7 @@ extern int ie_modified(const struct index_state *, const struct cache_entry *, s\n \n #define HASH_WRITE_OBJECT 1\n #define HASH_FORMAT_CHECK 2\n+#define HASH_USE_SHA_NOT_PATH 4\n extern int index_fd(unsigned char *sha1, int fd, struct stat *st, enum object_type type, const char *path, unsigned flags);\n extern int index_path(unsigned char *sha1, const char *path, struct stat *st, unsigned flags);\n \ndiff --git a/combine-diff.c b/combine-diff.c\nindex 8f2313d..8b535a7 100644\n--- a/combine-diff.c\n+++ b/combine-diff.c\n@@ -1052,7 +1052,8 @@ static void show_patch_diff(struct combine_diff_path *elem, int num_parent,\n \t\t\tif (is_file) {\n \t\t\t\tstruct strbuf buf = STRBUF_INIT;\n \n-\t\t\t\tif (convert_to_git(elem->path, result, len, &buf, safe_crlf)) {\n+\t\t\t\tif (convert_to_git(elem->path, result, len,\n+\t\t\t\t\t\t &buf, safe_crlf, NULL)) {\n \t\t\t\t\tfree(result);\n \t\t\t\t\tresult = strbuf_detach(&buf, &len);\n \t\t\t\t\tresult_size = len;\ndiff --git a/convert.c b/convert.c\nindex 67d69b5..802ee7c 100644\n--- a/convert.c\n+++ b/convert.c\n@@ -219,23 +219,28 @@ static void check_safe_crlf(const char *path, enum crlf_action crlf_action,\n \t}\n }\n \n-static int has_cr_in_index(const char *path)\n+static int blob_has_cr(const unsigned char *index_blob_sha1)\n {\n \tunsigned long sz;\n \tvoid *data;\n-\tint has_cr;\n-\n-\tdata = read_blob_data_from_cache(path, &sz);\n+\tint has_cr = 0;\n+\tenum object_type type;\n+\tif (!index_blob_sha1)\n+\t\treturn 0;\n+\tdata = read_sha1_file(index_blob_sha1, &type, &sz);\n \tif (!data)\n \t\treturn 0;\n-\thas_cr = memchr(data, '\\r', sz) != NULL;\n+\tif (type == OBJ_BLOB)\n+\t\thas_cr = memchr(data, '\\r', sz) != NULL;\n+\n \tfree(data);\n \treturn has_cr;\n }\n \n static int crlf_to_git(const char *path, const char *src, size_t len,\n \t\t struct strbuf *buf,\n-\t\t enum crlf_action crlf_action, enum safe_crlf checksafe)\n+\t\t enum crlf_action crlf_action, enum safe_crlf checksafe,\n+\t\t const unsigned char *index_blob_sha1)\n {\n \tstruct text_stat stats;\n \tchar *dst;\n@@ -256,14 +261,23 @@ static int crlf_to_git(const char *path, const char *src, size_t len,\n \tif (crlf_action == CRLF_AUTO || crlf_action == CRLF_AUTO_INPUT || crlf_action == CRLF_AUTO_CRLF) {\n \t\tif (convert_is_binary(len, &stats))\n \t\t\treturn 0;\n+\n \t\t/*\n \t\t * If the file in the index has any CR in it, do not convert.\n \t\t * This is the new safer autocrlf handling.\n \t\t */\n \t\tif (checksafe == SAFE_CRLF_RENORMALIZE)\n \t\t\tchecksafe = SAFE_CRLF_FALSE;\n-\t\telse if (has_cr_in_index(path))\n-\t\t\treturn 0;\n+\t\telse {\n+\t\t\t/*\n+\t\t\t * If the file in the index has any CR in it, do not convert.\n+\t\t\t * This is the new safer autocrlf handling.\n+\t\t\t */\n+\t\t\tif (!index_blob_sha1)\n+\t\t\t\tindex_blob_sha1 = get_sha1_from_cache(path);\n+\t\t\tif (blob_has_cr(index_blob_sha1))\n+\t\t\t\treturn 0;\n+\t\t}\n \t}\n \tcheck_safe_crlf(path, crlf_action, &stats, checksafe);\n \n@@ -855,7 +869,8 @@ const char *get_convert_attr_ascii(const char *path)\n }\n \n int convert_to_git(const char *path, const char *src, size_t len,\n- struct strbuf *dst, enum safe_crlf checksafe)\n+\t\t struct strbuf *dst, enum safe_crlf checksafe,\n+\t\t const unsigned char *index_blob_sha1)\n {\n \tint ret = 0;\n \tconst char *filter = NULL;\n@@ -876,7 +891,7 @@ int convert_to_git(const char *path, const char *src, size_t len,\n \t\tsrc = dst->buf;\n \t\tlen = dst->len;\n \t}\n-\tret |= crlf_to_git(path, src, len, dst, ca.crlf_action, checksafe);\n+\tret |= crlf_to_git(path, src, len, dst, ca.crlf_action, checksafe, index_blob_sha1);\n \tif (ret && dst) {\n \t\tsrc = dst->buf;\n \t\tlen = dst->len;\n@@ -885,7 +900,8 @@ int convert_to_git(const char *path, const char *src, size_t len,\n }\n \n void convert_to_git_filter_fd(const char *path, int fd, struct strbuf *dst,\n-\t\t\t enum safe_crlf checksafe)\n+\t\t\t enum safe_crlf checksafe,\n+\t\t\t const unsigned char *index_blob_sha1)\n {\n \tstruct conv_attrs ca;\n \tconvert_attrs(&ca, path);\n@@ -896,7 +912,8 @@ void convert_to_git_filter_fd(const char *path, int fd, struct strbuf *dst,\n \tif (!apply_filter(path, NULL, 0, fd, dst, ca.drv->clean))\n \t\tdie(\"%s: clean filter '%s' failed\", path, ca.drv->name);\n \n-\tcrlf_to_git(path, dst->buf, dst->len, dst, ca.crlf_action, checksafe);\n+\tcrlf_to_git(path, dst->buf, dst->len, dst, ca.crlf_action,\n+\t\t checksafe, index_blob_sha1);\n \tident_to_git(path, dst->buf, dst->len, dst, ca.ident);\n }\n \n@@ -951,7 +968,7 @@ int renormalize_buffer(const char *path, const char *src, size_t len, struct str\n \t\tsrc = dst->buf;\n \t\tlen = dst->len;\n \t}\n-\treturn ret | convert_to_git(path, src, len, dst, SAFE_CRLF_RENORMALIZE);\n+\treturn ret | convert_to_git(path, src, len, dst, SAFE_CRLF_RENORMALIZE, NULL);\n }\n \n /*****************************************************************\ndiff --git a/convert.h b/convert.h\nindex 81b6cdf..8c34dd5 100644\n--- a/convert.h\n+++ b/convert.h\n@@ -39,19 +39,26 @@ extern const char *get_convert_attr_ascii(const char *path);\n \n /* returns 1 if *dst was used */\n extern int convert_to_git(const char *path, const char *src, size_t len,\n-\t\t\t struct strbuf *dst, enum safe_crlf checksafe);\n+\t\t\t struct strbuf *dst, enum safe_crlf checksafe,\n+\t\t\t const unsigned char *index_blob_sha1);\n+\n extern int convert_to_working_tree(const char *path, const char *src,\n \t\t\t\t size_t len, struct strbuf *dst);\n extern int renormalize_buffer(const char *path, const char *src, size_t len,\n \t\t\t struct strbuf *dst);\n-static inline int would_convert_to_git(const char *path)\n+static inline int would_convert_to_git(const char *path,\n+\t\t\t\t const unsigned char *index_blob_sha1)\n {\n-\treturn convert_to_git(path, NULL, 0, NULL, 0);\n+\treturn convert_to_git(path, NULL, 0, NULL, SAFE_CRLF_FALSE,\n+\t\t\t index_blob_sha1);\n }\n+\n /* Precondition: would_convert_to_git_filter_fd(path) == true */\n extern void convert_to_git_filter_fd(const char *path, int fd,\n \t\t\t\t struct strbuf *dst,\n-\t\t\t\t enum safe_crlf checksafe);\n+\t\t\t\t enum safe_crlf checksafe,\n+\t\t\t\t const unsigned char *index_blob_sha1);\n+\n extern int would_convert_to_git_filter_fd(const char *path);\n \n /*****************************************************************\ndiff --git a/diff.c b/diff.c\nindex d3734d3..a8308e0 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -2810,7 +2810,8 @@ int diff_populate_filespec(struct diff_filespec *s, unsigned int flags)\n \t\t/*\n \t\t * Convert from working tree format to canonical git format\n \t\t */\n-\t\tif (convert_to_git(s->path, s->data, s->size, &buf, crlf_warn)) {\n+\t\tif (convert_to_git(s->path, s->data, s->size, &buf,\n+\t\t\t\t crlf_warn, NULL)) {\n \t\t\tsize_t size = 0;\n \t\t\tmunmap(s->data, s->size);\n \t\t\ts->should_munmap = 0;\ndiff --git a/dir.c b/dir.c\nindex 6172b34..66f1ffb 100644\n--- a/dir.c\n+++ b/dir.c\n@@ -712,7 +712,7 @@ static int add_excludes(const char *fname, const char *base, int baselen,\n \t\t\t\t (pos = cache_name_pos(fname, strlen(fname))) >= 0 &&\n \t\t\t\t !ce_stage(active_cache[pos]) &&\n \t\t\t\t ce_uptodate(active_cache[pos]) &&\n-\t\t\t\t !would_convert_to_git(fname))\n+\t\t\t\t !would_convert_to_git(fname, NULL))\n \t\t\t\thashcpy(sha1_stat->sha1, active_cache[pos]->sha1);\n \t\t\telse\n \t\t\t\thash_sha1_file(buf, size, \"blob\", sha1_stat->sha1);\ndiff --git a/read-cache.c b/read-cache.c\nindex a3ef967..c109b6d 100644\n--- a/read-cache.c\n+++ b/read-cache.c\n@@ -163,7 +163,9 @@ static int ce_compare_data(const struct cache_entry *ce, struct stat *st)\n \n \tif (fd >= 0) {\n \t\tunsigned char sha1[20];\n-\t\tif (!index_fd(sha1, fd, st, OBJ_BLOB, ce->name, 0))\n+\t\tunsigned flags = HASH_USE_SHA_NOT_PATH;\n+\t\tmemcpy(sha1, ce->sha1, sizeof(sha1));\n+\t\tif (!index_fd(sha1, fd, st, OBJ_BLOB, ce->name, flags))\n \t\t\tmatch = hashcmp(sha1, ce->sha1);\n \t\t/* index_fd() closed the file descriptor already */\n \t}\ndiff --git a/sha1_file.c b/sha1_file.c\nindex d5e1121..10630f0 100644\n--- a/sha1_file.c\n+++ b/sha1_file.c\n@@ -3273,6 +3273,7 @@ static int index_mem(unsigned char *sha1, void *buf, size_t size,\n {\n \tint ret, re_allocated = 0;\n \tint write_object = flags & HASH_WRITE_OBJECT;\n+\tconst int valid_sha1 = flags & HASH_USE_SHA_NOT_PATH;\n \n \tif (!type)\n \t\ttype = OBJ_BLOB;\n@@ -3283,7 +3284,8 @@ static int index_mem(unsigned char *sha1, void *buf, size_t size,\n \tif ((type == OBJ_BLOB) && path) {\n \t\tstruct strbuf nbuf = STRBUF_INIT;\n \t\tif (convert_to_git(path, buf, size, &nbuf,\n-\t\t\t\t write_object ? safe_crlf : SAFE_CRLF_FALSE)) {\n+\t\t\t\t write_object ? safe_crlf : SAFE_CRLF_FALSE,\n+\t\t\t\t valid_sha1 ? sha1 : NULL)) {\n \t\t\tbuf = strbuf_detach(&nbuf, &size);\n \t\t\tre_allocated = 1;\n \t\t}\n@@ -3311,13 +3313,15 @@ static int index_stream_convert_blob(unsigned char *sha1, int fd,\n {\n \tint ret;\n \tconst int write_object = flags & HASH_WRITE_OBJECT;\n+\tconst int valid_sha1 = flags & HASH_USE_SHA_NOT_PATH;\n \tstruct strbuf sbuf = STRBUF_INIT;\n \n \tassert(path);\n \tassert(would_convert_to_git_filter_fd(path));\n \n \tconvert_to_git_filter_fd(path, fd, &sbuf,\n-\t\t\t\t write_object ? safe_crlf : SAFE_CRLF_FALSE);\n+\t\t\t\t write_object ? safe_crlf : SAFE_CRLF_FALSE,\n+\t\t\t\t valid_sha1 ? sha1 : NULL);\n \n \tif (write_object)\n \t\tret = write_sha1_file(sbuf.buf, sbuf.len, typename(OBJ_BLOB),\n@@ -3394,6 +3398,8 @@ int index_fd(unsigned char *sha1, int fd, struct stat *st,\n \t enum object_type type, const char *path, unsigned flags)\n {\n \tint ret;\n+\tconst unsigned char *sha1_ce;\n+\tsha1_ce = flags & HASH_USE_SHA_NOT_PATH ? sha1 : NULL;\n \n \t/*\n \t * Call xsize_t() only when needed to avoid potentially unnecessary\n@@ -3404,7 +3410,7 @@ int index_fd(unsigned char *sha1, int fd, struct stat *st,\n \telse if (!S_ISREG(st->st_mode))\n \t\tret = index_pipe(sha1, fd, type, path, flags);\n \telse if (st->st_size <= big_file_threshold || type != OBJ_BLOB ||\n-\t\t (path && would_convert_to_git(path)))\n+\t\t (path && would_convert_to_git(path,sha1_ce)))\n \t\tret = index_core(sha1, fd, xsize_t(st->st_size), type, path,\n \t\t\t\t flags);\n \telse\ndiff --git a/t/t6038-merge-text-auto.sh b/t/t6038-merge-text-auto.sh\nindex 33b77ee..5e8d5fa 100755\n--- a/t/t6038-merge-text-auto.sh\n+++ b/t/t6038-merge-text-auto.sh\n@@ -91,16 +91,13 @@ test_expect_success 'Merge after setting text=auto' '\n \tcompare_files expected file\n '\n \n-test_expect_success 'Merge addition of text=auto' '\n+test_expect_success 'Merge addition of text=auto eol=LF' '\n+\tgit config core.eol lf &&\n \tcat <<-\\EOF >expected &&\n \tfirst line\n \tsame line\n \tEOF\n \n-\tif test_have_prereq NATIVE_CRLF; then\n-\t\tappend_cr expected.temp &&\n-\t\tmv expected.temp expected\n-\tfi &&\n \tgit config merge.renormalize true &&\n \tgit rm -fr . &&\n \trm -f .gitattributes &&\n@@ -109,17 +106,31 @@ test_expect_success 'Merge addition of text=auto' '\n \tcompare_files expected file\n '\n \n+test_expect_success 'Merge addition of text=auto eol=CRLF' '\n+\tgit config core.eol crlf &&\n+\tcat <<-\\EOF >expected &&\n+\tfirst line\n+\tsame line\n+\tEOF\n+\n+\tappend_cr expected.temp &&\n+\tmv expected.temp expected &&\n+\tgit config merge.renormalize true &&\n+\tgit rm -fr . &&\n+\trm -f .gitattributes &&\n+\tgit reset --hard b &&\n+\techo >&2 \"After git reset --hard b\" &&\n+\tgit ls-files -s --eol >&2 &&\n+\tgit merge a &&\n+\tcompare_files expected file\n+'\n+\n test_expect_success 'Detect CRLF/LF conflict after setting text=auto' '\n+\tgit config core.eol native &&\n \techo \"<<<<<<<\" >expected &&\n-\tif test_have_prereq NATIVE_CRLF; then\n-\t\techo first line | append_cr >>expected &&\n-\t\techo same line | append_cr >>expected &&\n-\t\techo ======= | append_cr >>expected\n-\telse\n-\t\techo first line >>expected &&\n-\t\techo same line >>expected &&\n-\t\techo ======= >>expected\n-\tfi &&\n+\techo first line >>expected &&\n+\techo same line >>expected &&\n+\techo ======= >>expected &&\n \techo first line | append_cr >>expected &&\n \techo same line | append_cr >>expected &&\n \techo \">>>>>>>\" >>expected &&\n@@ -135,15 +146,9 @@ test_expect_success 'Detect LF/CRLF conflict from addition of text=auto' '\n \techo \"<<<<<<<\" >expected &&\n \techo first line | append_cr >>expected &&\n \techo same line | append_cr >>expected &&\n-\tif test_have_prereq NATIVE_CRLF; then\n-\t\techo ======= | append_cr >>expected &&\n-\t\techo first line | append_cr >>expected &&\n-\t\techo same line | append_cr >>expected\n-\telse\n-\t\techo ======= >>expected &&\n-\t\techo first line >>expected &&\n-\t\techo same line >>expected\n-\tfi &&\n+\techo ======= >>expected &&\n+\techo first line >>expected &&\n+\techo same line >>expected &&\n \techo \">>>>>>>\" >>expected &&\n \tgit config merge.renormalize false &&\n \trm -f .gitattributes &&\n-- \n2.0.0.rc1.6318.g0c2c796\n"}]}