git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v5 2/9] xdiff: support external hunks via xpparam_t

From
Michael Montalbo via GitGitGadget <gitgitgadget@gmail.com>
Date
Jul 15, 2026, 21:01 UTC
Message-ID
<20045025497f08f3859646de8506d8f702bfc75f.1784149323.git.gitgitgadget@gmail.com>
In-Reply-To
<pull.2120.v5.git.1784149323.gitgitgadget@gmail.com>
From: Michael Montalbo <mmontalbo@gmail.com>

Add two new xpparam_t fields (external_hunks, external_hunks_nr) that let callers supply pre-computed hunks. When set, xdl_diff() populates the changed[] arrays from these hunks instead of running the diff algorithm, then continues through compaction and emission as usual.

Validate supplied hunks before use. Out-of-bounds line numbers, overlapping or out-of-order hunks, and misaligned unchanged runs are treated as a malformed tool response: xdl_populate_hunks_from_external() warns, returns -1, and xdl_diff() falls back to the builtin diff algorithm for that file. The run of unchanged lines between two hunks (and before the first and after the last) must be the same length on both sides; xdl_build_script() walks the two files in lockstep over unchanged lines, so a balanced total is not enough. Non-negative counts and 1-based starts are instead caller preconditions, checked with BUG(), since the caller normalizes hunks before this point.

On rejection xdl_diff() frees the environment it prepared and falls through to xdl_do_diff(), which prepares a fresh one for the builtin pass.

Skip trim_common_tail() in xdi_diff() when external hunks are present, since external hunks reference line numbers in the original content.

Signed-off-by: Michael Montalbo <mmontalbo@gmail.com>
---
 xdiff-interface.c |  7 +++-
 xdiff/xdiff.h     | 16 +++++++++
 xdiff/xdiffi.c    | 84 +++++++++++++++++++++++++++++++++++++++++++++--
 xdiff/xprepare.c  | 10 ++++++
 xdiff/xprepare.h  |  1 +
 5 files changed, 115 insertions(+), 3 deletions(-)
diff --git a/xdiff-interface.c b/xdiff-interface.c
index db6938689f..1fa16af668 100644
--- a/xdiff-interface.c
+++ b/xdiff-interface.c
@@ -124,7 +124,12 @@ int xdi_diff(mmfile_t *mf1, mmfile_t *mf2, xpparam_t const *xpp, xdemitconf_t co
 	if (mf1->size > MAX_XDIFF_SIZE || mf2->size > MAX_XDIFF_SIZE)
 		return -1;
 
-	if (!xecfg->ctxlen && !(xecfg->flags & XDL_EMIT_FUNCCONTEXT))
+	/*
+	 * External hunks reference line numbers in the original content;
+	 * trimming the tail would change line counts and invalidate them.
+	 */
+	if (!xpp->external_hunks &&
+	    !xecfg->ctxlen && !(xecfg->flags & XDL_EMIT_FUNCCONTEXT))
 		trim_common_tail(&a, &b);
 
 	return xdl_diff(&a, &b, xpp, xecfg, xecb);
diff --git a/xdiff/xdiff.h b/xdiff/xdiff.h
index dc370712e9..4736bcdb07 100644
--- a/xdiff/xdiff.h
+++ b/xdiff/xdiff.h
@@ -78,6 +78,18 @@ typedef struct s_mmbuffer {
 	long size;
 } mmbuffer_t;
 
+/*
+ * Hunk descriptor for externally computed diffs, in xdiff's own
+ * coordinates: line numbers are 1-based and a hunk's start is the
+ * first line it covers.  A caller translates any external "empty side"
+ * idiom (such as git diff's start-0/count-0) to a 1-based start before
+ * handing hunks over.
+ */
+struct xdl_hunk {
+	long old_start, old_count;
+	long new_start, new_count;
+};
+
 typedef struct s_xpparam {
 	unsigned long flags;
 
@@ -88,6 +100,10 @@ typedef struct s_xpparam {
 	/* See Documentation/diff-options.adoc. */
 	char **anchors;
 	size_t anchors_nr;
+
+	/* Externally computed hunks: bypass the diff algorithm.  Owned by caller. */
+	struct xdl_hunk *external_hunks;
+	size_t external_hunks_nr;
 } xpparam_t;
 
 typedef struct s_xdemitcb {
diff --git a/xdiff/xdiffi.c b/xdiff/xdiffi.c
index c5a892f91e..73a456f5dd 100644
--- a/xdiff/xdiffi.c
+++ b/xdiff/xdiffi.c
@@ -1085,16 +1085,96 @@ static void xdl_mark_ignorable_regex(xdchange_t *xscr, const xdfenv_t *xe,
 	}
 }
 
+/*
+ * Populate the changed[] arrays from externally supplied hunks,
+ * bypassing the diff algorithm.  The caller normalizes and validates
+ * the hunks first (order, overlap, and lockstep alignment), so this
+ * only marks lines changed after asserting the memory-safety
+ * preconditions it depends on: non-negative counts and 1-based starts
+ * (checked with BUG()), and an in-bounds range (a silent -1 so the
+ * caller can fall back to the builtin diff rather than index changed[]
+ * out of range).  Keeping this diagnostic-free leaves user-facing
+ * messages to the git layer.
+ *
+ * Returns 0 on success, -1 if a hunk is out of range.
+ */
+static int xdl_populate_hunks_from_external(xdfenv_t *xe,
+					    struct xdl_hunk *hunks,
+					    size_t nr_hunks)
+{
+	size_t i;
+	long j;
+
+	/*
+	 * xdl_prepare_env() may dirty changed[] via xdl_cleanup_records().
+	 * Clear them so only the external hunks are marked.
+	 */
+	xdl_clear_changed(&xe->xdf1);
+	xdl_clear_changed(&xe->xdf2);
+
+	for (i = 0; i < nr_hunks; i++) {
+		struct xdl_hunk *h = &hunks[i];
+
+		/*
+		 * Non-negative counts and 1-based starts are caller
+		 * preconditions (it normalizes hunks into xdiff coordinates
+		 * before this point), so a violation is a bug, not a bad
+		 * tool response.
+		 */
+		if (h->old_count < 0 || h->new_count < 0)
+			BUG("external hunk %"PRIuMAX": "
+				"negative count (old=%ld, new=%ld)",
+				(uintmax_t)(i + 1),
+				h->old_count, h->new_count);
+		if (h->old_start < 1 || h->new_start < 1)
+			BUG("external hunk %"PRIuMAX": "
+				"start not 1-based (old=%ld, new=%ld)",
+				(uintmax_t)(i + 1),
+				h->old_start, h->new_start);
+
+		/*
+		 * The caller validates ordering, overlap and lockstep
+		 * alignment (and diagnoses a bad response).  This is only a
+		 * silent in-bounds guard so the marking loop cannot index
+		 * changed[] out of range: start + count - 1 <= nrec,
+		 * rewritten to avoid overflow.  A count of 0 (pure
+		 * insert/delete) allows start == nrec + 1, the position
+		 * after the last line.  On a miss, return -1 and let the
+		 * caller fall back to the builtin diff.
+		 */
+		if (h->old_count > (long)xe->xdf1.nrec - h->old_start + 1 ||
+		    h->new_count > (long)xe->xdf2.nrec - h->new_start + 1)
+			return -1;
+
+		for (j = 0; j < h->old_count; j++)
+			xe->xdf1.changed[h->old_start - 1 + j] = true;
+		for (j = 0; j < h->new_count; j++)
+			xe->xdf2.changed[h->new_start - 1 + j] = true;
+	}
+
+	return 0;
+}
+
 int xdl_diff(mmfile_t *mf1, mmfile_t *mf2, xpparam_t const *xpp,
 	     xdemitconf_t const *xecfg, xdemitcb_t *ecb) {
 	xdchange_t *xscr;
 	xdfenv_t xe;
 	emit_func_t ef = xecfg->hunk_func ? xdl_call_hunk_func : xdl_emit_diff;
 
-	if (xdl_do_diff(mf1, mf2, xpp, &xe) < 0) {
+	if (xpp->external_hunks) {
+		if (xdl_prepare_env(mf1, mf2, xpp, &xe) < 0)
+			return -1;
+		if (xdl_populate_hunks_from_external(&xe,
+						     xpp->external_hunks,
+						     xpp->external_hunks_nr) == 0)
+			goto diff_done;
+		xdl_free_env(&xe);
+	}
 
+	if (xdl_do_diff(mf1, mf2, xpp, &xe) < 0)
 		return -1;
-	}
+
+diff_done:
 	if (xdl_change_compact(&xe.xdf1, &xe.xdf2, xpp->flags) < 0 ||
 	    xdl_change_compact(&xe.xdf2, &xe.xdf1, xpp->flags) < 0 ||
 	    xdl_build_script(&xe, &xscr) < 0) {
diff --git a/xdiff/xprepare.c b/xdiff/xprepare.c
index 11bada2608..f4ab935332 100644
--- a/xdiff/xprepare.c
+++ b/xdiff/xprepare.c
@@ -471,3 +471,13 @@ int xdl_prepare_env(mmfile_t *mf1, mmfile_t *mf2, xpparam_t const *xpp,
 
 	return 0;
 }
+
+/*
+ * Reset the changed[] array so that no lines are marked as changed.
+ * Also clears the sentinel slots at changed[-1] and changed[nrec]
+ * that xdl_change_compact() relies on during backward scans.
+ */
+void xdl_clear_changed(xdfile_t *xdf)
+{
+	memset(xdf->changed - 1, 0, (xdf->nrec + 2) * sizeof(bool));
+}
diff --git a/xdiff/xprepare.h b/xdiff/xprepare.h
index 947d9fc1bb..0413baf07b 100644
--- a/xdiff/xprepare.h
+++ b/xdiff/xprepare.h
@@ -28,6 +28,7 @@
 int xdl_prepare_env(mmfile_t *mf1, mmfile_t *mf2, xpparam_t const *xpp,
 		    xdfenv_t *xe);
 void xdl_free_env(xdfenv_t *xe);
+void xdl_clear_changed(xdfile_t *xdf);
 
 
 
-- 
gitgitgadget
Previous: Michael Montalbo via GitGitGadgetNext: Michael Montalbo via GitGitGadget
Message 46 of 81 in “[RFC] diff: add diff.<driver>.process for external hunk providers”
  1. 0/5 [RFC] diff: add diff.<driver>.process for external hunk providersMichael Montalbo via GitGitGadget, May 22, 2026
  2. 1/5 xdiff: support external hunks via xpparam_tMichael Montalbo via GitGitGadget, May 22, 2026
  3. Junio C HamanoMay 22, 2026
  4. Michael MontalboMay 22, 2026
  5. Junio C HamanoMay 24, 2026
  6. Michael MontalboMay 24, 2026
  7. 2/5 userdiff: add diff.<driver>.process configMichael Montalbo via GitGitGadget, May 22, 2026
  8. 3/5 diff: add long-running diff process via diff.<driver>.processMichael Montalbo via GitGitGadget, May 22, 2026
  9. 4/5 blame: consult diff process for zero-hunk detectionMichael Montalbo via GitGitGadget, May 22, 2026
  10. 5/5 diff-process-normalize: add built-in whitespace normalizerMichael Montalbo via GitGitGadget, May 22, 2026
  11. Junio C HamanoMay 22, 2026
  12. Michael MontalboMay 22, 2026
  13. 0/4 [RFC] diff: add diff.<driver>.process for external hunk providersMichael Montalbo via GitGitGadget, May 25, 2026
  14. 1/4 xdiff: support external hunks via xpparam_tMichael Montalbo via GitGitGadget, May 25, 2026
  15. 2/4 userdiff: add diff.<driver>.process configMichael Montalbo via GitGitGadget, May 25, 2026
  16. 3/4 diff: add long-running diff process via diff.<driver>.processMichael Montalbo via GitGitGadget, May 25, 2026
  17. Junio C HamanoMay 26, 2026
  18. Michael MontalboMay 29, 2026
  19. Junio C HamanoMay 26, 2026
  20. Michael MontalboMay 29, 2026
  21. 4/4 blame: consult diff process for zero-hunk detectionMichael Montalbo via GitGitGadget, May 25, 2026
  22. 0/6 [RFC] diff: add diff.<driver>.process for external hunk providersMichael Montalbo via GitGitGadget, May 29, 2026
  23. 1/6 xdiff: support external hunks via xpparam_tMichael Montalbo via GitGitGadget, May 29, 2026
  24. 2/6 userdiff: add diff.<driver>.process configMichael Montalbo via GitGitGadget, May 29, 2026
  25. 3/6 sub-process: separate process lifecycle from hashmap managementMichael Montalbo via GitGitGadget, May 29, 2026
  26. 4/6 diff: add long-running diff process via diff.<driver>.processMichael Montalbo via GitGitGadget, May 29, 2026
  27. Johannes SchindelinJun 7, 2026
  28. Michael MontalboJun 7, 2026
  29. Junio C HamanoJun 8, 2026
  30. Michael MontalboJun 7, 2026
  31. Junio C HamanoJun 8, 2026
  32. Junio C HamanoJun 8, 2026
  33. 5/6 diff: bypass diff process with --no-ext-diff and in format-patchMichael Montalbo via GitGitGadget, May 29, 2026
  34. 6/6 blame: consult diff process for no-hunk detectionMichael Montalbo via GitGitGadget, May 29, 2026
  35. Junio C HamanoMay 31, 2026
  36. Michael MontalboJun 1, 2026
  37. 0/6 [RFC] diff: add diff.<driver>.process for external hunk providersMichael Montalbo via GitGitGadget, Jun 14, 2026
  38. 1/6 xdiff: support external hunks via xpparam_tMichael Montalbo via GitGitGadget, Jun 14, 2026
  39. 2/6 userdiff: add diff.<driver>.process configMichael Montalbo via GitGitGadget, Jun 14, 2026
  40. 3/6 sub-process: separate process lifecycle from hashmap managementMichael Montalbo via GitGitGadget, Jun 14, 2026
  41. 4/6 diff: add long-running diff process via diff.<driver>.processMichael Montalbo via GitGitGadget, Jun 14, 2026
  42. 5/6 diff: bypass diff process with --no-ext-diff and in format-patchMichael Montalbo via GitGitGadget, Jun 14, 2026
  43. 6/6 blame: consult diff process for no-hunk detectionMichael Montalbo via GitGitGadget, Jun 14, 2026
  44. 0/9 [RFC] diff: add diff.<driver>.process for external hunk providersMichael Montalbo via GitGitGadget, Jul 15, 2026
  45. 1/9 gitattributes: document how external diff drivers relate to diff featuresMichael Montalbo via GitGitGadget, Jul 15, 2026
  46. 2/9 xdiff: support external hunks via xpparam_tMichael Montalbo via GitGitGadget, Jul 15, 2026
  47. 3/9 userdiff: add diff.<driver>.process configMichael Montalbo via GitGitGadget, Jul 15, 2026
  48. 4/9 sub-process: separate process lifecycle from hashmap managementMichael Montalbo via GitGitGadget, Jul 15, 2026
  49. 5/9 diff: add long-running diff process via diff.<driver>.processMichael Montalbo via GitGitGadget, Jul 15, 2026
  50. 6/9 diff: bypass diff process with --no-ext-diff and in format-patchMichael Montalbo via GitGitGadget, Jul 15, 2026
  51. 7/9 blame: consult diff process for no-hunk detectionMichael Montalbo via GitGitGadget, Jul 15, 2026
  52. 8/9 diff: consult diff process for --stat countsMichael Montalbo via GitGitGadget, Jul 15, 2026
  53. 9/9 line-log: consult diff process for range trackingMichael Montalbo via GitGitGadget, Jul 15, 2026
  54. Junio C HamanoJul 16, 2026
  55. Michael MontalboJul 16, 2026
  56. 0/9 [RFC] diff: add diff.<driver>.process for external hunk providersMichael Montalbo via GitGitGadget, Jul 26, 2026
  57. 1/9 gitattributes: document how external diff drivers relate to diff featuresMichael Montalbo via GitGitGadget, Jul 26, 2026
  58. 2/9 xdiff: support external hunks via xpparam_tMichael Montalbo via GitGitGadget, Jul 26, 2026
  59. 3/9 userdiff: add diff.<driver>.process configMichael Montalbo via GitGitGadget, Jul 26, 2026
  60. 4/9 sub-process: separate process lifecycle from hashmap managementMichael Montalbo via GitGitGadget, Jul 26, 2026
  61. 5/9 diff: add long-running diff process via diff.<driver>.processMichael Montalbo via GitGitGadget, Jul 26, 2026
  62. 6/9 diff: bypass diff process with --no-ext-diff and in format-patchMichael Montalbo via GitGitGadget, Jul 26, 2026
  63. 7/9 blame: consult diff process for no-hunk detectionMichael Montalbo via GitGitGadget, Jul 26, 2026
  64. 8/9 diff: consult diff process for --stat countsMichael Montalbo via GitGitGadget, Jul 26, 2026
  65. 9/9 line-log: consult diff process for range trackingMichael Montalbo via GitGitGadget, Jul 26, 2026
  66. 0/10 diff: add provider interface and initial providersMichael Montalbo, Aug 1, 2026
  67. 01/10 gitattributes: document how external diff drivers relate to diff featuresMichael Montalbo, Aug 1, 2026
  68. 02/10 diff: introduce a hunk provider interfaceMichael Montalbo, Aug 1, 2026
  69. 03/10 diff-hunks: add the store format, library, and commandMichael Montalbo, Aug 1, 2026
  70. 04/10 diff: record precomputed hunks during stat outputMichael Montalbo, Aug 1, 2026
  71. 05/10 diff: read precomputed hunks for stat outputMichael Montalbo, Aug 1, 2026
  72. 06/10 blame: read precomputed hunksMichael Montalbo, Aug 1, 2026
  73. 07/10 sub-process: separate process lifecycle from hashmap managementMichael Montalbo, Aug 1, 2026
  74. 08/10 sub-process: add a gentle status readMichael Montalbo, Aug 1, 2026
  75. 09/10 userdiff: add diff.<driver>.process configMichael Montalbo, Aug 1, 2026
  76. 10/10 diff: consult oid-only hunk providers via diff.<driver>.processMichael Montalbo, Aug 1, 2026
  77. Michael MontalboAug 4, 2026
  78. Junio C HamanoAug 13, 2026
  79. Phillip WoodAug 4, 2026
  80. Michael MontalboAug 23, 2026
  81. Michael MontalboJun 15, 2026

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.