git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v3 1/2] merge-file: let conflict markers match end-of-line style of the context

From
Johannes Schindelin <johannes.schindelin@gmx.de>
Date
Jan 25, 2016, 08:07 UTC
Message-ID
<26c973eb3414a8634d515f3621c0ded77cf030ed.1453709205.git.johannes.schindelin@gmx.de>
In-Reply-To
<cover.1453709205.git.johannes.schindelin@gmx.de>

When merging files with CR/LF line endings, the conflict markers should match those, lest the output file has mixed line endings.

This is particularly of interest on Windows, where some editors get *really* confused by mixed line endings.

The original version of this patch by Beat Bolli respected core.eol, and a subsequent improvement by this developer also respected gitattributes. This approach was suboptimal, though: `git merge-file` was invented as a drop-in replacement for GNU merge and as such has no problem operating outside of any repository at all!

Another problem with the original approach was pointed out by Junio
Hamano: legacy repositories might have their text files committed using
CR/LF line endings (and core.eol and the gitattributes would give us a
false impression there). Therefore, the much superior approach is to
simply match the context's line endings, if any.

We actually do not have to look at the *entire* context at all: if the files are all LF-only, or if they all have CR/LF line endings, it is sufficient to look at just a *single* line to match that style. And if the line endings are mixed anyway, it is *still* okay to imitate just a single line's eol: we will just add to the pile of mixed line endings, and there is nothing we can do about that.

So what we do is: we look at the line preceding the conflict, falling back to the line preceding that in case it was the last line and had no line ending, falling back to the first line, first in the first post-image, then the second post-image, and finally the pre-image. If we find consistent CR/LF (or undecided) end-of-line style, we match that, otherwise we use LF-only line endings for the conflict markers.

Note that while it is true that there have to be at least two lines we can look at (otherwise there would be no conflict), the same is not true for line *endings*: the three files in question could all consist of a single line without any line ending, each. In this case we fall back to using LF-only.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 t/t6023-merge-file.sh | 12 +++++++++++
 xdiff/xmerge.c        | 57 +++++++++++++++++++++++++++++++++++++++++++++++----
 2 files changed, 65 insertions(+), 4 deletions(-)
diff --git a/t/t6023-merge-file.sh b/t/t6023-merge-file.sh
index 190ee90..bb20cbc 100755
--- a/t/t6023-merge-file.sh
+++ b/t/t6023-merge-file.sh
@@ -346,4 +346,16 @@ test_expect_success 'conflict at EOF without LF resolved by --union' \
 	 printf "line1\nline2\nline3x\nline3y" >expect.txt &&
 	 test_cmp expect.txt output.txt'
 
+test_expect_success 'conflict markers match existing line endings' '
+	printf "1\\r\\n2\\r\\n3" >crlf-orig.txt &&
+	printf "1\\r\\n2\\r\\n4" >crlf-diff1.txt &&
+	printf "1\\r\\n2\\r\\n5" >crlf-diff2.txt &&
+	test_must_fail git -c core.eol=crlf merge-file -p \
+		crlf-diff1.txt crlf-orig.txt crlf-diff2.txt >crlf.txt &&
+	test $(tr "\015" Q <crlf.txt | grep "\\.txtQ$" | wc -l) = 3 &&
+	test_must_fail git -c core.eol=crlf merge-file -p \
+		nolf-diff1.txt nolf-orig.txt nolf-diff2.txt >nolf.txt &&
+	test $(tr "\015" Q <nolf.txt | grep "\\.txtQ$" | wc -l) = 0
+'
+
 test_done
diff --git a/xdiff/xmerge.c b/xdiff/xmerge.c
index 625198e..c852acc 100644
--- a/xdiff/xmerge.c
+++ b/xdiff/xmerge.c
@@ -143,6 +143,35 @@ static int xdl_orig_copy(xdfenv_t *xe, int i, int count, int add_nl, char *dest)
 	return xdl_recs_copy_0(1, xe, i, count, add_nl, dest);
 }
 
+/*
+ * Returns 1 if the i'th line ends in CR/LF (if it is the last line and
+ * has no eol, the preceding line, if any), 0 if it ends in LF-only, and
+ * -1 if the line ending cannot be determined.
+ */
+static int is_eol_crlf(xdfile_t *file, int i)
+{
+	long size;
+
+	if (i < file->nrec - 1)
+		/* All lines before the last *must* end in LF */
+		return (size = file->recs[i]->size) > 1 &&
+			file->recs[i]->ptr[size - 2] == '\r';
+	if (!file->nrec)
+		/* Cannot determine eol style from empty file */
+		return -1;
+	if ((size = file->recs[i]->size) &&
+			file->recs[i]->ptr[size - 1] == '\n')
+		/* Last line; ends in LF; Is it CR/LF? */
+		return size > 1 &&
+			file->recs[i]->ptr[size - 2] == '\r';
+	if (!i)
+		/* The only line has no eol */
+		return -1;
+	/* Determine eol from second-to-last line */
+	return (size = file->recs[i - 1]->size) > 1 &&
+		file->recs[i - 1]->ptr[size - 2] == '\r';
+}
+
 static int fill_conflict_hunk(xdfenv_t *xe1, const char *name1,
 			      xdfenv_t *xe2, const char *name2,
 			      const char *name3,
@@ -152,6 +181,18 @@ static int fill_conflict_hunk(xdfenv_t *xe1, const char *name1,
 	int marker1_size = (name1 ? strlen(name1) + 1 : 0);
 	int marker2_size = (name2 ? strlen(name2) + 1 : 0);
 	int marker3_size = (name3 ? strlen(name3) + 1 : 0);
+	int needs_cr;
+
+	/* Match post-images' preceding, or first, lines' end-of-line style */
+	needs_cr = is_eol_crlf(&xe1->xdf2, m->i1 ? m->i1 - 1 : 0);
+	if (needs_cr)
+		needs_cr = is_eol_crlf(&xe2->xdf2, m->i2 ? m->i2 - 1 : 0);
+	/* Look at pre-image's first line, unless we already settled on LF */
+	if (needs_cr)
+		needs_cr = is_eol_crlf(&xe1->xdf1, 0);
+	/* If still undecided, use LF-only */
+	if (needs_cr < 0)
+		needs_cr = 0;
 
 	if (marker_size <= 0)
 		marker_size = DEFAULT_CONFLICT_MARKER_SIZE;
@@ -161,7 +202,7 @@ static int fill_conflict_hunk(xdfenv_t *xe1, const char *name1,
 			      dest ? dest + size : NULL);
 
 	if (!dest) {
-		size += marker_size + 1 + marker1_size;
+		size += marker_size + 1 + needs_cr + marker1_size;
 	} else {
 		memset(dest + size, '<', marker_size);
 		size += marker_size;
@@ -170,6 +211,8 @@ static int fill_conflict_hunk(xdfenv_t *xe1, const char *name1,
 			memcpy(dest + size + 1, name1, marker1_size - 1);
 			size += marker1_size;
 		}
+		if (needs_cr)
+			dest[size++] = '\r';
 		dest[size++] = '\n';
 	}
 
@@ -180,7 +223,7 @@ static int fill_conflict_hunk(xdfenv_t *xe1, const char *name1,
 	if (style == XDL_MERGE_DIFF3) {
 		/* Shared preimage */
 		if (!dest) {
-			size += marker_size + 1 + marker3_size;
+			size += marker_size + 1 + needs_cr + marker3_size;
 		} else {
 			memset(dest + size, '|', marker_size);
 			size += marker_size;
@@ -189,6 +232,8 @@ static int fill_conflict_hunk(xdfenv_t *xe1, const char *name1,
 				memcpy(dest + size + 1, name3, marker3_size - 1);
 				size += marker3_size;
 			}
+			if (needs_cr)
+				dest[size++] = '\r';
 			dest[size++] = '\n';
 		}
 		size += xdl_orig_copy(xe1, m->i0, m->chg0, 1,
@@ -196,10 +241,12 @@ static int fill_conflict_hunk(xdfenv_t *xe1, const char *name1,
 	}
 
 	if (!dest) {
-		size += marker_size + 1;
+		size += marker_size + 1 + needs_cr;
 	} else {
 		memset(dest + size, '=', marker_size);
 		size += marker_size;
+		if (needs_cr)
+			dest[size++] = '\r';
 		dest[size++] = '\n';
 	}
 
@@ -207,7 +254,7 @@ static int fill_conflict_hunk(xdfenv_t *xe1, const char *name1,
 	size += xdl_recs_copy(xe2, m->i2, m->chg2, 1,
 			      dest ? dest + size : NULL);
 	if (!dest) {
-		size += marker_size + 1 + marker2_size;
+		size += marker_size + 1 + needs_cr + marker2_size;
 	} else {
 		memset(dest + size, '>', marker_size);
 		size += marker_size;
@@ -216,6 +263,8 @@ static int fill_conflict_hunk(xdfenv_t *xe1, const char *name1,
 			memcpy(dest + size + 1, name2, marker2_size - 1);
 			size += marker2_size;
 		}
+		if (needs_cr)
+			dest[size++] = '\r';
 		dest[size++] = '\n';
 	}
 	return size;
-- 
2.7.0.windows.1.7.g55a05c8
Previous: Johannes SchindelinNext: Torsten Bögershausen
Message 34 of 55 in “Let merge-file write out conflict markers with correct EOLs”
  1. 0/2 Let merge-file write out conflict markers with correct EOLsJohannes Schindelin, Jan 22, 2016
  2. 1/2 convert: add a helper to determine the correct EOL for a given pathJohannes Schindelin, Jan 22, 2016
  3. Junio C HamanoJan 22, 2016
  4. Johannes SchindelinJan 22, 2016
  5. Torsten BögershausenJan 23, 2016
  6. Johannes SchindelinJan 24, 2016
  7. Torsten BögershausenJan 23, 2016
  8. Johannes SchindelinJan 24, 2016
  9. 2/2 merge-file: consider core.crlf when writing merge markersJohannes Schindelin, Jan 22, 2016
  10. Junio C HamanoJan 22, 2016
  11. Johannes SchindelinJan 22, 2016
  12. Junio C HamanoJan 22, 2016
  13. Johannes SchindelinJan 24, 2016
  14. Eric SunshineJan 22, 2016
  15. Eric SunshineJan 22, 2016
  16. Johannes SchindelinJan 24, 2016
  17. Eric SunshineJan 24, 2016
  18. Johannes SchindelinJan 25, 2016
  19. Torsten BögershausenJan 23, 2016
  20. Johannes SchindelinJan 24, 2016
  21. Beat BolliJan 22, 2016
  22. 0/1 Let merge-file write out conflict markers with correct EOLsJohannes Schindelin, Jan 24, 2016
  23. 1/1 merge-file: let conflict markers match end-of-line style of the contextJohannes Schindelin, Jan 24, 2016
  24. Torsten BögershausenJan 24, 2016
  25. Torsten BögershausenJan 24, 2016
  26. Johannes SchindelinJan 25, 2016
  27. Ramsay JonesJan 25, 2016
  28. Johannes SchindelinJan 26, 2016
  29. Ramsay JonesJan 26, 2016
  30. Johannes SchindelinJan 26, 2016
  31. Junio C HamanoJan 24, 2016
  32. Johannes SchindelinJan 25, 2016
  33. 0/2 Let merge-file write out conflict markers with correct EOLsJohannes Schindelin, Jan 25, 2016
  34. 1/2 merge-file: let conflict markers match end-of-line style of the contextJohannes Schindelin, Jan 25, 2016
  35. Torsten BögershausenJan 25, 2016
  36. Johannes SchindelinJan 25, 2016
  37. Junio C HamanoJan 25, 2016
  38. Johannes SchindelinJan 26, 2016
  39. Junio C HamanoJan 26, 2016
  40. 2/2 merge-file: ensure that conflict sections match eol styleJohannes Schindelin, Jan 25, 2016
  41. Johannes SchindelinJan 25, 2016
  42. 0/2 Let merge-file write out conflict markers with correct EOLsJohannes Schindelin, Jan 26, 2016
  43. 1/2 merge-file: let conflict markers match end-of-line style of the contextJohannes Schindelin, Jan 26, 2016
  44. Junio C HamanoJan 26, 2016
  45. Junio C HamanoJan 26, 2016
  46. Johannes SchindelinJan 27, 2016
  47. Junio C HamanoJan 27, 2016
  48. Johannes SchindelinJan 27, 2016
  49. Junio C HamanoJan 27, 2016
  50. Johannes SchindelinJan 28, 2016
  51. 2/2 merge-file: ensure that conflict sections match eol styleJohannes Schindelin, Jan 26, 2016
  52. 0/2 Let merge-file write out conflict markers with correct EOLsJohannes Schindelin, Jan 27, 2016
  53. 1/2 merge-file: let conflict markers match end-of-line style of the contextJohannes Schindelin, Jan 27, 2016
  54. 2/2 merge-file: ensure that conflict sections match eol styleJohannes Schindelin, Jan 27, 2016
  55. Junio C HamanoJan 27, 2016

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.