git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 1/5] xdiff: support external hunks via xpparam_t

From
Michael Montalbo <mmontalbo@gmail.com>
Date
May 24, 2026, 18:01 UTC
Message-ID
<CAC2QwmLbFvMetGdVFdzJ3MmHo95LPx2Mohpj8NHk=uxKwt+ckA@mail.gmail.com>
In-Reply-To
<xmqqtsrxi43j.fsf@gitster.g>
On Sun, May 24, 2026 at 1:50 AM Junio C Hamano <gitster@pobox.com> wrote:
Show 48 quoted lines
>
> Michael Montalbo <mmontalbo@gmail.com> writes:
>
> >> > +      * Clear changed[] arrays.  xdl_prepare_env() may have dirtied
> >> > +      * them via xdl_cleanup_records().  The allocation is nrec + 2
> >> > +      * elements; changed points one past the start (see xprepare.c).
> >> > +      */
> >> > +     memset(xe->xdf1.changed - 1, 0,
> >> > +            (xe->xdf1.nrec + 2) * sizeof(bool));
> >> > +     memset(xe->xdf2.changed - 1, 0,
> >> > +            (xe->xdf2.nrec + 2) * sizeof(bool));
> >>
> >> This, especially the starting offset of -1, looks horrible.  The
> >> internal layout of xdfenv_t might happen to match the way the above
> >> code expects, which is how xdl_prepare_ctx() may have give you, but
> >> it somehow feels brittle.  I guess the assumption that changed[]
> >> does not point at the beginning of the allocated area (e.g., it is a
> >> no-no to free(xe->xdf1.changed) or realloc() it) is so pervasive that
> >> it cannot be helped.  Sigh.
> >>
> >
> > Agreed it is ugly. I wanted to make sure the entire changed[] including
> > sentinels were clear as a defensive measure for downstream callers
> > (xdl_change_compact). I agree this results in something that is ugly
> > and brittle, but in the end I thought it was superior to relying on the
> > fact that upstream zeroes the entire changed[] array. Maybe if the
> > comment was more explicit about why this is happening it would be
> > helpful?
>
> Perhaps make these memset() into calls to a helper function that is
> defined in xdiff/xprepare.c with a descriptive name and placed near
> where xdl_prepare_ctx() is.  That way, the patch in question does
> not even have to expose the strangeness of changed[] (i.e., it has 2
> more elements than it would normally contain to make the memory
> region for changed[-1] and changed[N] valid, and freeing it requires
> free(changed-1)) to the code path.  It only needs to say "Hey, I am
> clearing changed[] arrays because of XXX" without having to say "by
> the way, the memory layout of changed[] is strange this way", the
> latter of which is not exactly of interest for readers of this code.
>
> >     /*
> >      * Clear changed[] arrays including sentinels.
> >      * xdl_prepare_env() may have dirtied them via
> >      * xdl_cleanup_records(), and xdl_change_compact() reads
> >      * the sentinel at changed[-1] during backward scans.
> >      */
>
> And this belongs in xdiff/xprepare.c near that new helper function.
That sounds a lot nicer. Will update.
Previous: Junio C HamanoNext: Michael Montalbo via GitGitGadget
Message 6 of 81 in “[RFC] diff: add diff.<driver>.process for external hunk providers”
  1. 0/5 [RFC] diff: add diff.<driver>.process for external hunk providersMichael Montalbo via GitGitGadget, May 22, 2026
  2. 1/5 xdiff: support external hunks via xpparam_tMichael Montalbo via GitGitGadget, May 22, 2026
  3. Junio C HamanoMay 22, 2026
  4. Michael MontalboMay 22, 2026
  5. Junio C HamanoMay 24, 2026
  6. Michael MontalboMay 24, 2026
  7. 2/5 userdiff: add diff.<driver>.process configMichael Montalbo via GitGitGadget, May 22, 2026
  8. 3/5 diff: add long-running diff process via diff.<driver>.processMichael Montalbo via GitGitGadget, May 22, 2026
  9. 4/5 blame: consult diff process for zero-hunk detectionMichael Montalbo via GitGitGadget, May 22, 2026
  10. 5/5 diff-process-normalize: add built-in whitespace normalizerMichael Montalbo via GitGitGadget, May 22, 2026
  11. Junio C HamanoMay 22, 2026
  12. Michael MontalboMay 22, 2026
  13. 0/4 [RFC] diff: add diff.<driver>.process for external hunk providersMichael Montalbo via GitGitGadget, May 25, 2026
  14. 1/4 xdiff: support external hunks via xpparam_tMichael Montalbo via GitGitGadget, May 25, 2026
  15. 2/4 userdiff: add diff.<driver>.process configMichael Montalbo via GitGitGadget, May 25, 2026
  16. 3/4 diff: add long-running diff process via diff.<driver>.processMichael Montalbo via GitGitGadget, May 25, 2026
  17. Junio C HamanoMay 26, 2026
  18. Michael MontalboMay 29, 2026
  19. Junio C HamanoMay 26, 2026
  20. Michael MontalboMay 29, 2026
  21. 4/4 blame: consult diff process for zero-hunk detectionMichael Montalbo via GitGitGadget, May 25, 2026
  22. 0/6 [RFC] diff: add diff.<driver>.process for external hunk providersMichael Montalbo via GitGitGadget, May 29, 2026
  23. 1/6 xdiff: support external hunks via xpparam_tMichael Montalbo via GitGitGadget, May 29, 2026
  24. 2/6 userdiff: add diff.<driver>.process configMichael Montalbo via GitGitGadget, May 29, 2026
  25. 3/6 sub-process: separate process lifecycle from hashmap managementMichael Montalbo via GitGitGadget, May 29, 2026
  26. 4/6 diff: add long-running diff process via diff.<driver>.processMichael Montalbo via GitGitGadget, May 29, 2026
  27. Johannes SchindelinJun 7, 2026
  28. Michael MontalboJun 7, 2026
  29. Junio C HamanoJun 8, 2026
  30. Michael MontalboJun 7, 2026
  31. Junio C HamanoJun 8, 2026
  32. Junio C HamanoJun 8, 2026
  33. 5/6 diff: bypass diff process with --no-ext-diff and in format-patchMichael Montalbo via GitGitGadget, May 29, 2026
  34. 6/6 blame: consult diff process for no-hunk detectionMichael Montalbo via GitGitGadget, May 29, 2026
  35. Junio C HamanoMay 31, 2026
  36. Michael MontalboJun 1, 2026
  37. 0/6 [RFC] diff: add diff.<driver>.process for external hunk providersMichael Montalbo via GitGitGadget, Jun 14, 2026
  38. 1/6 xdiff: support external hunks via xpparam_tMichael Montalbo via GitGitGadget, Jun 14, 2026
  39. 2/6 userdiff: add diff.<driver>.process configMichael Montalbo via GitGitGadget, Jun 14, 2026
  40. 3/6 sub-process: separate process lifecycle from hashmap managementMichael Montalbo via GitGitGadget, Jun 14, 2026
  41. 4/6 diff: add long-running diff process via diff.<driver>.processMichael Montalbo via GitGitGadget, Jun 14, 2026
  42. 5/6 diff: bypass diff process with --no-ext-diff and in format-patchMichael Montalbo via GitGitGadget, Jun 14, 2026
  43. 6/6 blame: consult diff process for no-hunk detectionMichael Montalbo via GitGitGadget, Jun 14, 2026
  44. 0/9 [RFC] diff: add diff.<driver>.process for external hunk providersMichael Montalbo via GitGitGadget, Jul 15, 2026
  45. 1/9 gitattributes: document how external diff drivers relate to diff featuresMichael Montalbo via GitGitGadget, Jul 15, 2026
  46. 2/9 xdiff: support external hunks via xpparam_tMichael Montalbo via GitGitGadget, Jul 15, 2026
  47. 3/9 userdiff: add diff.<driver>.process configMichael Montalbo via GitGitGadget, Jul 15, 2026
  48. 4/9 sub-process: separate process lifecycle from hashmap managementMichael Montalbo via GitGitGadget, Jul 15, 2026
  49. 5/9 diff: add long-running diff process via diff.<driver>.processMichael Montalbo via GitGitGadget, Jul 15, 2026
  50. 6/9 diff: bypass diff process with --no-ext-diff and in format-patchMichael Montalbo via GitGitGadget, Jul 15, 2026
  51. 7/9 blame: consult diff process for no-hunk detectionMichael Montalbo via GitGitGadget, Jul 15, 2026
  52. 8/9 diff: consult diff process for --stat countsMichael Montalbo via GitGitGadget, Jul 15, 2026
  53. 9/9 line-log: consult diff process for range trackingMichael Montalbo via GitGitGadget, Jul 15, 2026
  54. Junio C HamanoJul 16, 2026
  55. Michael MontalboJul 16, 2026
  56. 0/9 [RFC] diff: add diff.<driver>.process for external hunk providersMichael Montalbo via GitGitGadget, Jul 26, 2026
  57. 1/9 gitattributes: document how external diff drivers relate to diff featuresMichael Montalbo via GitGitGadget, Jul 26, 2026
  58. 2/9 xdiff: support external hunks via xpparam_tMichael Montalbo via GitGitGadget, Jul 26, 2026
  59. 3/9 userdiff: add diff.<driver>.process configMichael Montalbo via GitGitGadget, Jul 26, 2026
  60. 4/9 sub-process: separate process lifecycle from hashmap managementMichael Montalbo via GitGitGadget, Jul 26, 2026
  61. 5/9 diff: add long-running diff process via diff.<driver>.processMichael Montalbo via GitGitGadget, Jul 26, 2026
  62. 6/9 diff: bypass diff process with --no-ext-diff and in format-patchMichael Montalbo via GitGitGadget, Jul 26, 2026
  63. 7/9 blame: consult diff process for no-hunk detectionMichael Montalbo via GitGitGadget, Jul 26, 2026
  64. 8/9 diff: consult diff process for --stat countsMichael Montalbo via GitGitGadget, Jul 26, 2026
  65. 9/9 line-log: consult diff process for range trackingMichael Montalbo via GitGitGadget, Jul 26, 2026
  66. 0/10 diff: add provider interface and initial providersMichael Montalbo, Aug 1, 2026
  67. 01/10 gitattributes: document how external diff drivers relate to diff featuresMichael Montalbo, Aug 1, 2026
  68. 02/10 diff: introduce a hunk provider interfaceMichael Montalbo, Aug 1, 2026
  69. 03/10 diff-hunks: add the store format, library, and commandMichael Montalbo, Aug 1, 2026
  70. 04/10 diff: record precomputed hunks during stat outputMichael Montalbo, Aug 1, 2026
  71. 05/10 diff: read precomputed hunks for stat outputMichael Montalbo, Aug 1, 2026
  72. 06/10 blame: read precomputed hunksMichael Montalbo, Aug 1, 2026
  73. 07/10 sub-process: separate process lifecycle from hashmap managementMichael Montalbo, Aug 1, 2026
  74. 08/10 sub-process: add a gentle status readMichael Montalbo, Aug 1, 2026
  75. 09/10 userdiff: add diff.<driver>.process configMichael Montalbo, Aug 1, 2026
  76. 10/10 diff: consult oid-only hunk providers via diff.<driver>.processMichael Montalbo, Aug 1, 2026
  77. Michael MontalboAug 4, 2026
  78. Junio C HamanoAug 13, 2026
  79. Phillip WoodAug 4, 2026
  80. Michael MontalboAug 23, 2026
  81. Michael MontalboJun 15, 2026

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.