diff --color-words breaks spacing

Phil Hord· Jan 24, 2017, 17:39 UTC · lore

I noticed some weird spacing when comparing files with git diff --color-words. The space before a colored word disappears sometimes.

$ git --version git version 2.11.0.485.g4e59582ff

echo "FOO foo; foo = bar" > a echo "FOO foo = baz" > b git diff --color-words --no-index a b FOOfoo; foo = barbaz

There should be a space after FOO in the diff, even if git doesn't think "foo" and "foo;" are the same word.

If I remove the semicolon, it looks better, but in fact it only moves the error later. The missing space is now between the two "foo" words.

echo "FOO foo foo = bar" > a echo "FOO foo = baz" > b git diff --color-words --no-index a b FOO foofoo = barbaz

Here's the same with the color codes changed to text for purposes of this email:

echo "FOO foo; foo = bar" > a echo "FOO foo = baz" > b git diff --color-words --no-index a b | tr \\033 E FOOE[31mfoo;E[m foo = E[31mbarE[mE[32mbazE[m

echo "FOO foo foo = bar" > a echo "FOO foo = baz" > b git diff --color-words --no-index a b | tr \\033 E FOO fooE[31mfooE[m = E[31mbarE[mE[32mbazE[m

Jeff King· Jan 25, 2017, 23:20 UTC · re: Phil Hord · lore

Re: diff --color-words breaks spacing

On Tue, Jan 24, 2017 at 09:39:31AM -0800, Phil Hord wrote:

> I noticed some weird spacing when comparing files with git diff
> --color-words.  The space before a colored word disappears sometimes.

I _think_ this is working as designed, though it is a bit tricky (and it may be possible to make it better).

> echo "FOO foo; foo = bar" > a
> echo "FOO foo = baz" > b
> git diff --color-words --no-index a b
> FOOfoo; foo = barbaz

It might be easier to see with --word-diff, which uses the same code but marks it with punctuation:

  $ git diff --word-diff
  FOO[-foo;-] foo = [-bar-]{+baz+}

The key thing is that what you are seeing is the post-image, plus word-based annotations. And the post-image does not have that space in its context, so we would not expect to see:

  FOO [-foo;-] foo = [-bar-]{+baz+}

(you would if we showed the pre-image plus annotations; but then I think you'd probably end up with the opposite problem when text was added).

However, we also don't see the space in the removed part. I.e., I think:

  FOO[- foo;-] foo = [-bar-]{+baz+}

would be correct. But I think because of the way word-diff is implemented, a non-word character will never be part of the annotation.

The way it works is basically to break the string down on word boundaries, so we have:

  pre: FOO, foo;, foo, =, bar
  post: FOO, foo, =, baz

as a set of tokens. We stick each of those on their own line and feed them back to xdiff, so we get a diff like:

  @@ -1,5 +1,4 @@
   FOO
  -foo;
   foo
   =
  -bar
  +baz

and then walk through the post-image buffer, either inserting chunks of context from the original buffer, or the changed lines. But the changed lines themselves do not include the non-word characters, so we have no idea that a space went away.

-Peff