git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v3 4/8] refactor userdiff textconv code

From
Jeff King <peff@peff.net>
Date
Oct 26, 2008, 04:44 UTC
Message-ID
<20081026044453.GD21047@coredump.intra.peff.net>
In-Reply-To
<20081026043802.GA14530@coredump.intra.peff.net>

The original implementation of textconv put the conversion into fill_mmfile. This was a bad idea for a number of reasons:

 - it made the semantics of fill_mmfile unclear. In some
   cases, it was allocating data (if a text conversion
   occurred), and in some cases not (if we could use the
   data directly from the filespec). But the caller had
   no idea which had happened, and so didn't know whether
   the memory should be freed
 - similarly, the caller had no idea if a text conversion
   had occurred, and so didn't know whether the contents
   should be treated as binary or not. This meant that we
   incorrectly guessed that text-converted content was
   binary and didn't actually show it (unless the user
   overrode us with "diff.foo.binary = false", which then
   created problems in plumbing where the text conversion
   did _not_ occur)
 - not all callers of fill_mmfile want the text contents. In
   particular, we don't really want diffstat, whitespace
   checks, patch id generation, etc, to look at the
   converted contents.

This patch pulls the conversion code directly into builtin_diff, so that we only see the conversion when generating an actual patch. We also then know whether we are doing a conversion, so we can check the binary-ness and free the data from the mmfile appropriately (the previous version leaked quite badly when text conversion was used)

Signed-off-by: Jeff King <peff@peff.net>
---
This patch is the interesting one. I think your suggestion turned out
fairly clean (and this reverts fill_mmfile back to its state before the
original textconv series).
 diff.c                   |   48 +++++++++++++++++++++++++++++++++------------
 t/t4030-diff-textconv.sh |    6 ++--
 2 files changed, 38 insertions(+), 16 deletions(-)
diff --git a/diff.c b/diff.c
index d1fd594..6f01595 100644
--- a/diff.c
+++ b/diff.c
@@ -294,18 +294,8 @@ static int fill_mmfile(mmfile_t *mf, struct diff_filespec *one)
 	else if (diff_populate_filespec(one, 0))
 		return -1;
 
-	diff_filespec_load_driver(one);
-	if (one->driver->textconv) {
-		size_t size;
-		mf->ptr = run_textconv(one->driver->textconv, one, &size);
-		if (!mf->ptr)
-			return -1;
-		mf->size = size;
-	}
-	else {
-		mf->ptr = one->data;
-		mf->size = one->size;
-	}
+	mf->ptr = one->data;
+	mf->size = one->size;
 	return 0;
 }
 
@@ -1323,6 +1313,14 @@ void diff_set_mnemonic_prefix(struct diff_options *options, const char *a, const
 		options->b_prefix = b;
 }
 
+static const char *get_textconv(struct diff_filespec *one)
+{
+	if (!DIFF_FILE_VALID(one))
+		return NULL;
+	diff_filespec_load_driver(one);
+	return one->driver->textconv;
+}
+
 static void builtin_diff(const char *name_a,
 			 const char *name_b,
 			 struct diff_filespec *one,
@@ -1337,6 +1335,7 @@ static void builtin_diff(const char *name_a,
 	const char *set = diff_get_color_opt(o, DIFF_METAINFO);
 	const char *reset = diff_get_color_opt(o, DIFF_RESET);
 	const char *a_prefix, *b_prefix;
+	const char *textconv_one, *textconv_two;
 
 	diff_set_mnemonic_prefix(o, "a/", "b/");
 	if (DIFF_OPT_TST(o, REVERSE_DIFF)) {
@@ -1390,8 +1389,12 @@ static void builtin_diff(const char *name_a,
 	if (fill_mmfile(&mf1, one) < 0 || fill_mmfile(&mf2, two) < 0)
 		die("unable to read files to diff");
 
+	textconv_one = get_textconv(one);
+	textconv_two = get_textconv(two);
+
 	if (!DIFF_OPT_TST(o, TEXT) &&
-	    (diff_filespec_is_binary(one) || diff_filespec_is_binary(two))) {
+	    ( (diff_filespec_is_binary(one) && !textconv_one) ||
+	      (diff_filespec_is_binary(two) && !textconv_two) )) {
 		/* Quite common confusing case */
 		if (mf1.size == mf2.size &&
 		    !memcmp(mf1.ptr, mf2.ptr, mf1.size))
@@ -1412,6 +1415,21 @@ static void builtin_diff(const char *name_a,
 		struct emit_callback ecbdata;
 		const struct userdiff_funcname *pe;
 
+		if (textconv_one) {
+			size_t size;
+			mf1.ptr = run_textconv(textconv_one, one, &size);
+			if (!mf1.ptr)
+				die("unable to read files to diff");
+			mf1.size = size;
+		}
+		if (textconv_two) {
+			size_t size;
+			mf2.ptr = run_textconv(textconv_two, two, &size);
+			if (!mf2.ptr)
+				die("unable to read files to diff");
+			mf2.size = size;
+		}
+
 		pe = diff_funcname_pattern(one);
 		if (!pe)
 			pe = diff_funcname_pattern(two);
@@ -1443,6 +1461,10 @@ static void builtin_diff(const char *name_a,
 			      &xpp, &xecfg, &ecb);
 		if (DIFF_OPT_TST(o, COLOR_DIFF_WORDS))
 			free_diff_words_data(&ecbdata);
+		if (textconv_one)
+			free(mf1.ptr);
+		if (textconv_two)
+			free(mf2.ptr);
 	}
 
  free_ab_and_return:
diff --git a/t/t4030-diff-textconv.sh b/t/t4030-diff-textconv.sh
index 1b09648..090a21d 100755
--- a/t/t4030-diff-textconv.sh
+++ b/t/t4030-diff-textconv.sh
@@ -52,7 +52,7 @@ test_expect_success 'setup textconv filters' '
 	git config diff.fail.textconv false
 '
 
-test_expect_failure 'diff produces text' '
+test_expect_success 'diff produces text' '
 	git diff HEAD^ HEAD >diff &&
 	find_diff <diff >actual &&
 	test_cmp expect.text actual
@@ -64,7 +64,7 @@ test_expect_success 'diff-tree produces binary' '
 	test_cmp expect.binary actual
 '
 
-test_expect_failure 'log produces text' '
+test_expect_success 'log produces text' '
 	git log -1 -p >log &&
 	find_diff <log >actual &&
 	test_cmp expect.text actual
@@ -80,7 +80,7 @@ cat >expect.stat <<'EOF'
  file |  Bin 2 -> 4 bytes
  1 files changed, 0 insertions(+), 0 deletions(-)
 EOF
-test_expect_failure 'diffstat does not run textconv' '
+test_expect_success 'diffstat does not run textconv' '
 	echo file diff=fail >.gitattributes &&
 	git diff --stat HEAD^ HEAD >actual &&
 	test_cmp expect.stat actual
-- 
1.6.0.3.524.ge8b2e.dirty
Previous: Jeff KingNext: Jeff King
Message 59 of 85 in “Implementation of a "textconv" filter for easy custom diff.”
  1. Matthieu MoySep 28, 2008
  2. Facility to have multiple kinds of drivers for diff.Matthieu Moy, Sep 28, 2008
  3. Implement run_command_to_buf (spawn a process and reads its stdout)Matthieu Moy, Sep 28, 2008
  4. Implement a textconv filter for "git diff"Matthieu Moy, Sep 28, 2008
  5. Document the textconv filter.Matthieu Moy, Sep 28, 2008
  6. Add a basic test for the textconv filter.Matthieu Moy, Sep 28, 2008
  7. Johannes SixtSep 28, 2008
  8. Matthieu MoySep 28, 2008
  9. Jeff KingSep 28, 2008
  10. Matthieu MoySep 28, 2008
  11. Jeff KingSep 28, 2008
  12. Jeff KingSep 28, 2008
  13. Matthieu MoySep 28, 2008
  14. Jeff KingSep 28, 2008
  15. Matthieu MoySep 30, 2008
  16. Jeff KingSep 30, 2008
  17. 0/4 diff text conversion filterJeff King, Oct 5, 2008
  18. 1/4 t4012: use test_cmp instead of cmpJeff King, Oct 5, 2008
  19. 2/4 diff: unify external diff and funcname parsing codeJeff King, Oct 5, 2008
  20. 3/4 diff: introduce diff.<driver>.binaryJeff King, Oct 5, 2008
  21. Johannes SixtOct 7, 2008
  22. Jeff KingOct 7, 2008
  23. Johannes SixtOct 7, 2008
  24. Junio C HamanoOct 12, 2008
  25. Jeff KingOct 13, 2008
  26. Junio C HamanoOct 13, 2008
  27. Jeff KingOct 13, 2008
  28. Johannes SixtOct 13, 2008
  29. Junio C HamanoOct 13, 2008
  30. Matthieu MoyOct 13, 2008
  31. Jeff KingOct 24, 2008
  32. 1/5 diff: add missing static declarationJeff King, Oct 24, 2008
  33. 2/5 add userdiff textconv testsJeff King, Oct 24, 2008
  34. 3/5 refactor userdiff textconv codeJeff King, Oct 24, 2008
  35. Johannes SixtOct 24, 2008
  36. Jeff KingOct 24, 2008
  37. Jeff KingOct 24, 2008
  38. Johannes SixtOct 24, 2008
  39. Jeff KingOct 24, 2008
  40. Junio C HamanoOct 24, 2008
  41. Jeff KingOct 24, 2008
  42. Jeff KingOct 24, 2008
  43. Jeff KingOct 25, 2008
  44. 1/7 diff: add missing static declarationJeff King, Oct 25, 2008
  45. 2/7 add userdiff textconv testsJeff King, Oct 25, 2008
  46. 3/7 textconv: assume text-converted contents are not binaryJeff King, Oct 25, 2008
  47. 4/7 textconv: don't convert for every operationJeff King, Oct 25, 2008
  48. Junio C HamanoOct 25, 2008
  49. Jeff KingOct 25, 2008
  50. Junio C HamanoOct 25, 2008
  51. Jeff KingOct 25, 2008
  52. Junio C HamanoOct 25, 2008
  53. Junio C HamanoOct 25, 2008
  54. Jeff KingOct 26, 2008
  55. Jeff KingOct 26, 2008
  56. 1/8 diff: add missing static declarationJeff King, Oct 26, 2008
  57. 2/8 document the diff driver textconv featureJeff King, Oct 26, 2008
  58. 3/8 add userdiff textconv testsJeff King, Oct 26, 2008
  59. 4/8 refactor userdiff textconv codeJeff King, Oct 26, 2008
  60. 5/8 userdiff: require explicitly allowing textconvJeff King, Oct 26, 2008
  61. 6/8 only textconv regular filesJeff King, Oct 26, 2008
  62. 7/8 wt-status: load diff ui configJeff King, Oct 26, 2008
  63. Junio C HamanoOct 27, 2008
  64. Jeff KingOct 27, 2008
  65. 8/8 enable textconv for diff in verbose status/commitJeff King, Oct 26, 2008
  66. 5/7 userdiff: require explicitly allowing textconvJeff King, Oct 25, 2008
  67. 6/7 document the diff driver textconv featureJeff King, Oct 25, 2008
  68. 7/7 only textconv regular filesJeff King, Oct 25, 2008
  69. 4/5 userdiff: require explicitly allowing textconvJeff King, Oct 24, 2008
  70. Johannes SixtOct 24, 2008
  71. 5/5 document the diff driver textconv featureJeff King, Oct 24, 2008
  72. Johannes SixtOct 24, 2008
  73. 4/4 diff: add filter for converting binary to textJeff King, Oct 5, 2008
  74. Jakub NarebskiOct 5, 2008
  75. Johannes SixtOct 6, 2008
  76. Jeff KingOct 6, 2008
  77. Johannes SixtOct 6, 2008
  78. Matthieu MoyOct 6, 2008
  79. Jeff KingOct 7, 2008
  80. Johannes SixtOct 7, 2008
  81. Jeff KingOct 7, 2008
  82. Matthieu MoyOct 7, 2008
  83. Jeff KingOct 7, 2008
  84. Johannes SixtOct 7, 2008
  85. Jeff KingOct 13, 2008

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.