{"thread":{"id":"61782","subject":"[PATCH] userdiff: add builtin diff driver for TypeScript language","startedAt":"2024-07-15T16:33:42Z","lastAt":"2024-07-16T21:10:27Z","messageCount":6,"participants":["Matthew Hughes via GitGitGadget","Matthew Hughes","Junio C Hamano","Johannes Sixt"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"498736","messageId":"pull.1746.git.git.1721061218993.gitgitgadget@gmail.com","threadId":"61782","inReplyTo":null,"subject":"[PATCH] userdiff: add builtin diff driver for TypeScript language","fromName":"Matthew Hughes via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2024-07-15T16:33:38Z","receivedAt":"2024-07-15T16:33:42Z","isPatch":true,"sender":{"key":"matthewhughes934@gmail.com","avatar":"https://avatars.githubusercontent.com/u/34972397?v=4"},"body":"From: Matthew Hughes \n\nTypeScript[1] is an open-source programming language that builds on\nJavaScript. This patch adds builtin diff driver support for this\nlanguage. As far as I can tell there is no official syntax specification\nfor the language (see[2] for some discussion) so this patch is based off\nsome existing work[3]. The docs[4] probably provide the best\nreference as to what this driver should satisfy. See[5] for\ndiscussion/motivation for this change from the TypeScript language team.\n\nThis is my first time developing a diff driver, so as such the\nimplementation borrows quite a bit from existing drivers. The funcname\nattribute matches function and class definitions, the list of keywords\nused to define functions was take from[3], I could not find an\nexhaustive list for these. The word-regex borrows much from other\nexisting diff engines, with the addition of the rather unique\nright-shifting operators (>>> and >>>=) available in JavaScript (and\nhence Typescript)[6]\n\n[1] https://www.typescriptlang.org/\n[2] https://github.com/Microsoft/TypeScript/issues/15711\n[3] https://github.com/git/git/pull/859\n[4] https://www.typescriptlang.org/docs/\n[5] https://github.com/microsoft/TypeScript/issues/36185\n[6] https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Unsigned_right_shift\n\nSigned-off-by: Matthew Hughes \n---\n userdiff: add builtin diff driver for TypeScript language\n\nPublished-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-1746%2Fmatthewhughes934%2Fadd-typescript-diff-driver-v1\nFetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-1746/matthewhughes934/add-typescript-diff-driver-v1\nPull-Request: https://github.com/git/git/pull/1746\n\n Documentation/gitattributes.txt | 2 ++\n t/t4018/typescript-arrow-func | 7 +++++\n t/t4018/typescript-class | 8 ++++++\n t/t4018/typescript-func | 7 +++++\n t/t4018/typescript-indented-func | 7 +++++\n t/t4018/typescript-interface | 5 ++++\n t/t4018/typescript-nested-arrow-func | 7 +++++\n t/t4034-diff-words.sh | 1 +\n t/t4034/typescript/expect | 42 ++++++++++++++++++++++++++++\n t/t4034/typescript/post | 24 ++++++++++++++++\n t/t4034/typescript/pre | 24 ++++++++++++++++\n userdiff.c | 11 ++++++++\n 12 files changed, 145 insertions(+)\n create mode 100644 t/t4018/typescript-arrow-func\n create mode 100644 t/t4018/typescript-class\n create mode 100644 t/t4018/typescript-func\n create mode 100644 t/t4018/typescript-indented-func\n create mode 100644 t/t4018/typescript-interface\n create mode 100644 t/t4018/typescript-nested-arrow-func\n create mode 100644 t/t4034/typescript/expect\n create mode 100644 t/t4034/typescript/post\n create mode 100644 t/t4034/typescript/pre\n\ndiff --git a/Documentation/gitattributes.txt b/Documentation/gitattributes.txt\nindex e6150595af8..88ebfb2e6c9 100644\n--- a/Documentation/gitattributes.txt\n+++ b/Documentation/gitattributes.txt\n@@ -907,6 +907,8 @@ patterns are available:\n \n - `tex` suitable for source code for LaTeX documents.\n \n+- `typescript` suitable for source code in the TypeScript language.\n+\n \n Customizing word diff\n ^^^^^^^^^^^^^^^^^^^^^\ndiff --git a/t/t4018/typescript-arrow-func b/t/t4018/typescript-arrow-func\nnew file mode 100644\nindex 00000000000..c884c2e6bdc\n--- /dev/null\n+++ b/t/t4018/typescript-arrow-func\n@@ -0,0 +1,7 @@\n+const RIGHT = (a: number) : number => {\n+ const res = a + 1\n+\n+ // some comment\n+ console.log(\"ChangeMe\")\n+ return res\n+}\ndiff --git a/t/t4018/typescript-class b/t/t4018/typescript-class\nnew file mode 100644\nindex 00000000000..a373f7af1dd\n--- /dev/null\n+++ b/t/t4018/typescript-class\n@@ -0,0 +1,8 @@\n+class RIGHT {\n+ // some\n+ // comments\n+ // ChangeMe\n+ constructor() {\n+ console.log(\"constructing...\")\n+ }\n+}\ndiff --git a/t/t4018/typescript-func b/t/t4018/typescript-func\nnew file mode 100644\nindex 00000000000..94cc659a452\n--- /dev/null\n+++ b/t/t4018/typescript-func\n@@ -0,0 +1,7 @@\n+function RIGHT(a: number) : number {\n+ const res = a + 1\n+\n+ // some comment\n+ console.log(\"ChangeMe\")\n+ return res\n+}\ndiff --git a/t/t4018/typescript-indented-func b/t/t4018/typescript-indented-func\nnew file mode 100644\nindex 00000000000..fee0373126e\n--- /dev/null\n+++ b/t/t4018/typescript-indented-func\n@@ -0,0 +1,7 @@\n+class Foo {\n+ public RIGHT() {\n+ // some\n+ // comments\n+ console.log(\"ChangeMe\")\n+ }\n+}\ndiff --git a/t/t4018/typescript-interface b/t/t4018/typescript-interface\nnew file mode 100644\nindex 00000000000..78d14523446\n--- /dev/null\n+++ b/t/t4018/typescript-interface\n@@ -0,0 +1,5 @@\n+interface RIGHT {\n+ name: string;\n+ // some comment \n+ ChangeMe: number;\n+}\ndiff --git a/t/t4018/typescript-nested-arrow-func b/t/t4018/typescript-nested-arrow-func\nnew file mode 100644\nindex 00000000000..52dbc6a92e5\n--- /dev/null\n+++ b/t/t4018/typescript-nested-arrow-func\n@@ -0,0 +1,7 @@\n+class Foo {\n+ RIGHT = () => {\n+ // some\n+ // comment\n+ console.log(\"ChangeMe\")\n+ }\n+}\ndiff --git a/t/t4034-diff-words.sh b/t/t4034-diff-words.sh\nindex 74586f3813c..4e3cf415c23 100755\n--- a/t/t4034-diff-words.sh\n+++ b/t/t4034-diff-words.sh\n@@ -338,6 +338,7 @@ test_language_driver python\n test_language_driver ruby\n test_language_driver scheme\n test_language_driver tex\n+test_language_driver typescript\n \n test_expect_success 'word-diff with diff.sbe' '\n \tcat >pre <<-\\EOF &&\ndiff --git a/t/t4034/typescript/expect b/t/t4034/typescript/expect\nnew file mode 100644\nindex 00000000000..fde27423339\n--- /dev/null\n+++ b/t/t4034/typescript/expect\n@@ -0,0 +1,42 @@\n+diff --git a/pre b/post\n+index 6a5ba06c60..db4788b1a6 100644\n+--- a/pre\n++++ b/post\n+@@ -1,24 +1,24 @@\n+console.log(\"Hello world!?\")\n+foo===barbuz\n+foo!==barbuz\n+function fn() : numberstring { return 1\"ok\" }\n+ax.someFunc()\n+const ax = () => {}\n+(1) (-1e10) (0xabcdef) 'xy'\n+\"xy\"\n+[ax] ax.by\n+!ax ~a ax x++ ax-- ax*b ay x&b\n+ay\n+x*b ay x/b ay x%b\n+ay\n+x+b ay x-b\n+ay\n+x<<b ay x>>b\n+ay\n+x>>>=b ay x>>>b\n+ay\n+x<b ay x<=b ay x>b ay x>=b\n+ay\n+x==b ay x!=b\n+ay\n+x&b\n+ay\n+x^b\n+ay\n+x|b\n+ay\n+x&&b\n+ay\n+x||b\n+ay\n+x?by:z\n+ax=b ay x+=b ay x-=b ay x*=b ay x/=b ay x%=b ay x<<=b ay x>>=b ay x&=b ay x^=b ay x|=b\n+ay\n+x,y\ndiff --git a/t/t4034/typescript/post b/t/t4034/typescript/post\nnew file mode 100644\nindex 00000000000..4aca03f8ce6\n--- /dev/null\n+++ b/t/t4034/typescript/post\n@@ -0,0 +1,24 @@\n+console.log(\"Hello world?\")\n+foo===buz\n+foo!==buz\n+function fn() : string { return \"ok\" }\n+x.someFunc()\n+const x = () => {}\n+(1) (-1e10) (0xabcdef) 'y'\n+\"y\"\n+[x] x.y\n+!x ~x x++ x-- x*y x&y\n+x*y x/y x%y\n+x+y x-y\n+x<>y\n+x>>>=y x>>>y\n+xy x>=y\n+x==y x!=y\n+x&y\n+x^y\n+x|y\n+x&&y\n+x||y\n+x?y:z\n+x=y x+=y x-=y x*=y x/=y x%=y x<<=y x>>=y x&=y x^=y x|=y\n+x,y\ndiff --git a/t/t4034/typescript/pre b/t/t4034/typescript/pre\nnew file mode 100644\nindex 00000000000..ef654a92c61\n--- /dev/null\n+++ b/t/t4034/typescript/pre\n@@ -0,0 +1,24 @@\n+console.log(\"Hello world!\")\n+foo===bar\n+foo!==bar\n+function fn() : number { return 1 }\n+a.someFunc()\n+const a = () => {}\n+1 -1e10 0xabcdef 'x'\n+\"x\"\n+[a] a.b\n+!a ~a a++ a-- a*b a&b\n+a*b a/b a%b\n+a+b a-b\n+a<>b\n+a>>>=b a>>>b\n+ab a>=b\n+a==b a!=b\n+a&b\n+a^b\n+a|b\n+a&&b\n+a||b\n+a?b:z\n+a=b a+=b a-=b a*=b a/=b a%=b a<<=b a>>=b a&=b a^=b a|=b\n+a,y\ndiff --git a/userdiff.c b/userdiff.c\nindex c4ebb9ff734..7247d351cde 100644\n--- a/userdiff.c\n+++ b/userdiff.c\n@@ -333,6 +333,17 @@ PATTERNS(\"scheme\",\n \t \"|([^][)(}{[ \\t])+\"),\n PATTERNS(\"tex\", \"^(\\\\\\\\((sub)*section|chapter|part)\\\\*{0,1}\\\\{.*)$\",\n \t \"\\\\\\\\[a-zA-Z@]+|\\\\\\\\.|([a-zA-Z0-9]|[^\\x01-\\x7f])+\"),\n+PATTERNS(\"typescript\",\n+\t\"^[\\t ]*((class|constructor|public|private|protected|function|interface)[ \\t].*)$\\n\"\n+\t// arrow funcs\n+\t\"^[\\t ]*((const|let|var)?[^()]*)=[\\t ]*\\\\([^()]*\\\\)[\\t ]*.*=>.*$\",\n+\t/* -- */\n+\t\"[a-zA-Z_][a-zA-Z0-9_]*\"\n+\t// numeric constants\n+\t\"|[-+0-9.e]+|0[xX]?[0-9a-fA-F]\"\n+\t// operators\n+\t\"|[-+*/<>%&^|=!]\"\n+\t\"|--|\\\\+\\\\+|//=?|<<=?|>>?=?\"),\n { .name = \"default\", .binary = -1 },\n };\n #undef PATTERNS\n\nbase-commit: a7dae3bdc8b516d36f630b12bb01e853a667e0d9\n-- \ngitgitgadget\n"},{"id":"498781","messageId":"20240716122112.zqauqgxmng2tk2j6@archP14s","threadId":"61782","inReplyTo":"pull.1746.git.git.1721061218993.gitgitgadget@gmail.com","subject":"Re: [PATCH] userdiff: add builtin diff driver for TypeScript language","fromName":"Matthew Hughes","fromEmail":"matthewhughes934@gmail.com","sentAt":"2024-07-16T12:21:12Z","receivedAt":"2024-07-16T12:21:21Z","isPatch":true,"sender":{"key":"matthewhughes934@gmail.com","avatar":"https://avatars.githubusercontent.com/u/34972397?v=4"},"body":"On Mon, Jul 15, 2024 at 04:33:38PM +0000, Matthew Hughes via GitGitGadget wrote:\n> diff --git a/userdiff.c b/userdiff.c\n> index c4ebb9ff734..7247d351cde 100644\n> --- a/userdiff.c\n> +++ b/userdiff.c\n> @@ -333,6 +333,17 @@ PATTERNS(\"scheme\",\n> \t \"|([^][)(}{[ \\t])+\"),\n> PATTERNS(\"tex\", \"^(\\\\\\\\((sub)*section|chapter|part)\\\\*{0,1}\\\\{.*)$\",\n> \t \"\\\\\\\\[a-zA-Z@]+|\\\\\\\\.|([a-zA-Z0-9]|[^\\x01-\\x7f])+\"),\n> +PATTERNS(\"typescript\",\n> +\t\"^[\\t ]*((class|constructor|public|private|protected|function|interface)[ \\t].*)$\\n\"\n> +\t// arrow funcs\n> +\t\"^[\\t ]*((const|let|var)?[^()]*)=[\\t ]*\\\\([^()]*\\\\)[\\t ]*.*=>.*$\",\n> +\t/* -- */\n> +\t\"[a-zA-Z_][a-zA-Z0-9_]*\"\n> +\t// numeric constants\n> +\t\"|[-+0-9.e]+|0[xX]?[0-9a-fA-F]\"\n> +\t// operators\n> +\t\"|[-+*/<>%&^|=!]\"\n> +\t\"|--|\\\\+\\\\+|//=?|<<=?|>>?=?\"),\n> { .name = \"default\", .binary = -1 },\n> };\n> #undef PATTERNS\n> \n> base-commit: a7dae3bdc8b516d36f630b12bb01e853a667e0d9\n> -- \n> gitgitgadget\n\nThis needs some updates. For the arrow function, definitions can cover multiple\nlines e.g.:\n\n const bar = (\n name: string\n ) => console.log(name)\n\nThe funcname pattern should also consider the `export` keyword, since both of\nthe following are valid:\n\n export const bar = (\n name: string\n ) => console.log(name)\n\n export function foo() {}\n\nSome docs: https://www.typescriptlang.org/docs/handbook/modules/reference.html#module-syntax\n"},{"id":"498790","messageId":"xmqq5xt5bat7.fsf@gitster.g","threadId":"61782","inReplyTo":"20240716122112.zqauqgxmng2tk2j6@archP14s","subject":"Re: [PATCH] userdiff: add builtin diff driver for TypeScript language","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2024-07-16T15:45:08Z","receivedAt":"2024-07-16T15:45:11Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Matthew Hughes writes:\n\n> This needs some updates.\n\nWhat does it mean?\n\nThe patterns that were posted were so broken that they are unusable\nand harm the users by giving misleading information?\n\nOr do the patterns work just fine in basic or tutorial cases, but\nwith more advanced or realistic uses of the language construct, they\nhighlight wrong lines as the function header and/or split at wrong\nword boundaries that are obviously much less optimal than ideal that\nany human users would find questionable?\n\nIn the latter case, how far from the ideal are the decisions done by\nthe current patterns, and what's the rough percentage of usual code\nwe see in the real world, for which the current patterns do not work\nwell?\n\nWhat I am trying to gauge is if it is so broken that it should not\nexist (in other words, you regret sending the patch to the list\nbefore doing these updates), or is \"already serviceable, but not\nperfect yet\". Waiting for perfection takes forever. If the latter,\nletting the general public to use it to gather feedbacks by waiting\nfor the dust to settle before making such updates is often better.\n\n"},{"id":"498799","messageId":"20240716193344.bjb62zsfnrfw3ngf@archP14s","threadId":"61782","inReplyTo":"xmqq5xt5bat7.fsf@gitster.g","subject":"Re: [PATCH] userdiff: add builtin diff driver for TypeScript language","fromName":"Matthew Hughes","fromEmail":"matthewhughes934@gmail.com","sentAt":"2024-07-16T19:33:44Z","receivedAt":"2024-07-16T19:33:51Z","isPatch":true,"sender":{"key":"matthewhughes934@gmail.com","avatar":"https://avatars.githubusercontent.com/u/34972397?v=4"},"body":"Oe Tue, Jul 16, 2024 at 08:45:08AM -0700, Junio C Hamano wrote:\n> What does it mean?\n> \n> The patterns that were posted were so broken that they are unusable\n> and harm the users by giving misleading information?\n\nI think this would be a good summary. It's sufficient for some simpler cases\nconsidered, and does even give some benefits e.g. for function headers for\nnested functions. However, the cases where it fails can be significant, e.g.\nhundreds of lines away from the correct function header for files with multiple\nconsecutive multi-line arrow functions.\n\n> In the latter case, how far from the ideal are the decisions done by\n> the current patterns, and what's the rough percentage of usual code\n> we see in the real world, for which the current patterns do not work\n> well?\n\nI think just the missing `export` keyword handling would be equivalent to\nmissing all public functions in other programming languages, so that alone\nwould be a decent percentage.\n\n> What I am trying to gauge is if it is so broken that it should not\n> exist (in other words, you regret sending the patch to the list\n> before doing these updates), or is \"already serviceable, but not\n> perfect yet\". Waiting for perfection takes forever. If the latter,\n> letting the general public to use it to gather feedbacks by waiting\n> for the dust to settle before making such updates is often better.\n\nI'm leaning towards the former case: that this patch was premature. I think\nit's far enough from perfect that it would greatly benefit from me more\nactively reaching out to the TypeScript language team and asking some devs\nthere try out the changes and gather some more input (and identify some more\nmissing cases, of which I now expect that are many) before getting\nsomething out to general users.\n"},{"id":"498804","messageId":"a1dc575b-a13e-4685-98ea-a6037c8e059c@kdbg.org","threadId":"61782","inReplyTo":"pull.1746.git.git.1721061218993.gitgitgadget@gmail.com","subject":"Re: [PATCH] userdiff: add builtin diff driver for TypeScript language","fromName":"Johannes Sixt","fromEmail":"j6t@kdbg.org","sentAt":"2024-07-16T20:53:15Z","receivedAt":"2024-07-16T20:53:24Z","isPatch":true,"sender":{"key":"j6t@kdbg.org","avatar":"https://avatars.githubusercontent.com/u/14810926?v=4"},"body":"We have had a submission for typescript just recently:\n\nhttps://lore.kernel.org/git/20240404163827.5855-1-utsavp0213@gmail.com/\n\nAnd two for Javascript\n\nhttps://lore.kernel.org/git/20240301074048.188835-1-sergiusnyah@gmail.com/\nhttps://lore.kernel.org/git/20220403132508.28196-1-a97410985new@gmail.com/\n\nPlease review these earlier submissions. If you think you can improve on\nthem, you are very welcome to do so. But you can also just resend one of\nthese series with a note that they are sufficiently mature and that you\nsupport the submitted version.\n\nI may very well mistaken, but I think that Typescript is a syntactical\nsuperset of Javascript, so we need just one language driver and mention\nin the documentation that it can be used for both languages.\n\nThanks,\n-- Hannes\n\n"},{"id":"498805","messageId":"xmqqjzhl6o1r.fsf@gitster.g","threadId":"61782","inReplyTo":"20240716193344.bjb62zsfnrfw3ngf@archP14s","subject":"Re: [PATCH] userdiff: add builtin diff driver for TypeScript language","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2024-07-16T21:10:24Z","receivedAt":"2024-07-16T21:10:27Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Matthew Hughes writes:\n\n> I'm leaning towards the former case: that this patch was premature.\n\nOK. Then let's take sufficient time. After all, we are never in a\nhurry ;-)\n\nThanks for giving an honest assessment. Will keep the topic in\n'seen' without marking it for 'next' (at least until it gets\nreplaced with a version that is more suitable to the public).\n\n"}]}