{"thread":{"id":"53268","subject":"[PATCH] userdiff: support Markdown","startedAt":"2020-04-21T01:03:15Z","lastAt":"2020-05-02T13:58:45Z","messageCount":15,"participants":["Ash Holland","Emma Brooks","Johannes Sixt","Junio C Hamano"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"395800","messageId":"20200421010035.13915-1-ash@sorrel.sh","threadId":"53268","inReplyTo":null,"subject":"[PATCH] userdiff: support Markdown","fromName":"Ash Holland","fromEmail":"ash@sorrel.sh","sentAt":"2020-04-21T01:00:35Z","receivedAt":"2020-04-21T01:03:15Z","isPatch":true,"sender":{"key":"ash@sorrel.sh","avatar":"https://avatars.githubusercontent.com/u/9433472?v=4"},"body":"It's typical to find Markdown documentation alongside source code, and\nhaving better context for documentation changes is useful; see also\ncommit 69f9c87d4 (userdiff: add support for Fountain documents,\n2015-07-21).\n\nThe pattern is based on the CommonMark specification 0.29, section 4.2:\nhttps://spec.commonmark.org/\n\nOnly ATX headings are supported, as detecting setext headings would\nrequire printing the line before a pattern matches, or matching a\nmultiline pattern.\n\nSigned-off-by: Ash Holland <ash@sorrel.sh>\n---\n\nIf it is indeed possible to match multiline patterns, let me know! I\nwould love to support setext (underlined) headings with this.\n\nI would also appreciate feedback on the word-diff pattern here, I have\nno real idea what should constitute a word in a Markdown document, apart\nfrom that it should probably be similar to the definition given for\nFountain, given that Fountain appears to have somewhat similar inline\nsyntax to Markdown.\n\n Documentation/gitattributes.txt       |  2 ++\n t/t4018-diff-funcname.sh              |  1 +\n t/t4018/markdown-heading-indented     |  6 ++++++\n t/t4018/markdown-heading-non-headings | 17 +++++++++++++++++\n userdiff.c                            |  3 +++\n 5 files changed, 29 insertions(+)\n create mode 100644 t/t4018/markdown-heading-indented\n create mode 100644 t/t4018/markdown-heading-non-headings\n\ndiff --git a/Documentation/gitattributes.txt b/Documentation/gitattributes.txt\nindex 508fe713c..2d0a03715 100644\n--- a/Documentation/gitattributes.txt\n+++ b/Documentation/gitattributes.txt\n@@ -824,6 +824,8 @@ patterns are available:\n \n - `java` suitable for source code in the Java language.\n \n+- `markdown` suitable for Markdown documents.\n+\n - `matlab` suitable for source code in the MATLAB and Octave languages.\n \n - `objc` suitable for source code in the Objective-C language.\ndiff --git a/t/t4018-diff-funcname.sh b/t/t4018-diff-funcname.sh\nindex 02255a08b..9d0779757 100755\n--- a/t/t4018-diff-funcname.sh\n+++ b/t/t4018-diff-funcname.sh\n@@ -38,6 +38,7 @@ diffpatterns=\"\n \tgolang\n \thtml\n \tjava\n+\tmarkdown\n \tmatlab\n \tobjc\n \tpascal\ndiff --git a/t/t4018/markdown-heading-indented b/t/t4018/markdown-heading-indented\nnew file mode 100644\nindex 000000000..1991c2bd4\n--- /dev/null\n+++ b/t/t4018/markdown-heading-indented\n@@ -0,0 +1,6 @@\n+Indented headings are allowed, as long as the indent is no more than 3 spaces.\n+\n+   ### RIGHT\n+\n+- something\n+- ChangeMe\ndiff --git a/t/t4018/markdown-heading-non-headings b/t/t4018/markdown-heading-non-headings\nnew file mode 100644\nindex 000000000..1f19b91d6\n--- /dev/null\n+++ b/t/t4018/markdown-heading-non-headings\n@@ -0,0 +1,17 @@\n+Headings can be right next to other lines of the file:\n+# RIGHT\n+Indents of more than four spaces make a code block:\n+\n+    # code comment, not heading\n+\n+If there's no space after the final hash, it's not a heading:\n+\n+#hashtag\n+\n+Sequences of more than 6 hashes don't make a heading:\n+\n+####### over-enthusiastic heading\n+\n+So the detected heading should be right up at the start of this file.\n+\n+ChangeMe\ndiff --git a/userdiff.c b/userdiff.c\nindex efbe05e5a..f79adb3a3 100644\n--- a/userdiff.c\n+++ b/userdiff.c\n@@ -79,6 +79,9 @@ PATTERNS(\"java\",\n \t \"|[-+0-9.e]+[fFlL]?|0[xXbB]?[0-9a-fA-F]+[lL]?\"\n \t \"|[-+*/<>%&^|=!]=\"\n \t \"|--|\\\\+\\\\+|<<=?|>>>?=?|&&|\\\\|\\\\|\"),\n+PATTERNS(\"markdown\",\n+\t \"^ {0,3}#{1,6}( .*)?$\",\n+\t \"[^ \\t-]+\"),\n PATTERNS(\"matlab\",\n \t /*\n \t  * Octave pattern is mostly the same as matlab, except that '%%%' and\n-- \n2.26.1\n\n"},{"id":"395806","messageId":"20200421022256.GA29071@pluvano.com","threadId":"53268","inReplyTo":"20200421010035.13915-1-ash@sorrel.sh","subject":"Re: [PATCH] userdiff: support Markdown","fromName":"Emma Brooks","fromEmail":"me@pluvano.com","sentAt":"2020-04-21T02:22:56Z","receivedAt":"2020-04-21T02:23:22Z","isPatch":true,"sender":{"key":"me@pluvano.com","avatar":"https://avatars.githubusercontent.com/u/50312486?v=4"},"body":"On 2020-04-21 02:00:35+0100, Ash Holland wrote:\n> I would also appreciate feedback on the word-diff pattern here, I have\n> no real idea what should constitute a word in a Markdown document, apart\n> from that it should probably be similar to the definition given for\n> Fountain, given that Fountain appears to have somewhat similar inline\n> syntax to Markdown.\n\nSince Markdown can have raw HTML tags in many variants, it may make\nsense to extend the word pattern to \"[^<>= \\t]+\" like HTML's pattern so\ntags starting/ending will not be considered part of a word.\n"},{"id":"396021","messageId":"33f037a9-d3d5-042a-d3ba-7e4b8364663a@kdbg.org","threadId":"53268","inReplyTo":"20200421010035.13915-1-ash@sorrel.sh","subject":"Re: [PATCH] userdiff: support Markdown","fromName":"Johannes Sixt","fromEmail":"j6t@kdbg.org","sentAt":"2020-04-23T18:17:41Z","receivedAt":"2020-04-23T18:17:46Z","isPatch":true,"sender":{"key":"j6t@kdbg.org","avatar":"https://avatars.githubusercontent.com/u/14810926?v=4"},"body":"Am 21.04.20 um 03:00 schrieb Ash Holland:\n> It's typical to find Markdown documentation alongside source code, and\n> having better context for documentation changes is useful; see also\n> commit 69f9c87d4 (userdiff: add support for Fountain documents,\n> 2015-07-21).\n> \n> The pattern is based on the CommonMark specification 0.29, section 4.2:\n> https://spec.commonmark.org/\n> \n> Only ATX headings are supported, as detecting setext headings would\n> require printing the line before a pattern matches, or matching a\n> multiline pattern.\n\nThe patch looks good. I have one question about the patthern, though\n(see below).\n\n> Signed-off-by: Ash Holland <ash@sorrel.sh>\n> ---\n> \n> If it is indeed possible to match multiline patterns, let me know! I\n> would love to support setext (underlined) headings with this.\n\nWe don't have multi-line matching, unfortunately.\n\n> I would also appreciate feedback on the word-diff pattern here, I have\n> no real idea what should constitute a word in a Markdown document, apart\n> from that it should probably be similar to the definition given for\n> Fountain, given that Fountain appears to have somewhat similar inline\n> syntax to Markdown.\n> \n>  Documentation/gitattributes.txt       |  2 ++\n>  t/t4018-diff-funcname.sh              |  1 +\n>  t/t4018/markdown-heading-indented     |  6 ++++++\n>  t/t4018/markdown-heading-non-headings | 17 +++++++++++++++++\n>  userdiff.c                            |  3 +++\n>  5 files changed, 29 insertions(+)\n>  create mode 100644 t/t4018/markdown-heading-indented\n>  create mode 100644 t/t4018/markdown-heading-non-headings\n> \n> diff --git a/Documentation/gitattributes.txt b/Documentation/gitattributes.txt\n> index 508fe713c..2d0a03715 100644\n> --- a/Documentation/gitattributes.txt\n> +++ b/Documentation/gitattributes.txt\n> @@ -824,6 +824,8 @@ patterns are available:\n>  \n>  - `java` suitable for source code in the Java language.\n>  \n> +- `markdown` suitable for Markdown documents.\n> +\n>  - `matlab` suitable for source code in the MATLAB and Octave languages.\n>  \n>  - `objc` suitable for source code in the Objective-C language.\n> diff --git a/t/t4018-diff-funcname.sh b/t/t4018-diff-funcname.sh\n> index 02255a08b..9d0779757 100755\n> --- a/t/t4018-diff-funcname.sh\n> +++ b/t/t4018-diff-funcname.sh\n> @@ -38,6 +38,7 @@ diffpatterns=\"\n>  \tgolang\n>  \thtml\n>  \tjava\n> +\tmarkdown\n>  \tmatlab\n>  \tobjc\n>  \tpascal\n> diff --git a/t/t4018/markdown-heading-indented b/t/t4018/markdown-heading-indented\n> new file mode 100644\n> index 000000000..1991c2bd4\n> --- /dev/null\n> +++ b/t/t4018/markdown-heading-indented\n> @@ -0,0 +1,6 @@\n> +Indented headings are allowed, as long as the indent is no more than 3 spaces.\n> +\n> +   ### RIGHT\n> +\n> +- something\n> +- ChangeMe\n> diff --git a/t/t4018/markdown-heading-non-headings b/t/t4018/markdown-heading-non-headings\n> new file mode 100644\n> index 000000000..1f19b91d6\n> --- /dev/null\n> +++ b/t/t4018/markdown-heading-non-headings\n> @@ -0,0 +1,17 @@\n> +Headings can be right next to other lines of the file:\n> +# RIGHT\n> +Indents of more than four spaces make a code block:\n> +\n> +    # code comment, not heading\n> +\n> +If there's no space after the final hash, it's not a heading:\n> +\n> +#hashtag\n> +\n> +Sequences of more than 6 hashes don't make a heading:\n> +\n> +####### over-enthusiastic heading\n> +\n> +So the detected heading should be right up at the start of this file.\n> +\n> +ChangeMe\n\nNicely done!\n\n> diff --git a/userdiff.c b/userdiff.c\n> index efbe05e5a..f79adb3a3 100644\n> --- a/userdiff.c\n> +++ b/userdiff.c\n> @@ -79,6 +79,9 @@ PATTERNS(\"java\",\n>  \t \"|[-+0-9.e]+[fFlL]?|0[xXbB]?[0-9a-fA-F]+[lL]?\"\n>  \t \"|[-+*/<>%&^|=!]=\"\n>  \t \"|--|\\\\+\\\\+|<<=?|>>>?=?|&&|\\\\|\\\\|\"),\n> +PATTERNS(\"markdown\",\n> +\t \"^ {0,3}#{1,6}( .*)?$\",\n\nWhat is the purpose of making the heading text optional? Why would you\nwant to match a sequence of hash marks without any text following it?\n\n> +\t \"[^ \\t-]+\"),\n>  PATTERNS(\"matlab\",\n>  \t /*\n>  \t  * Octave pattern is mostly the same as matlab, except that '%%%' and\n> \n\n-- Hannes\n"},{"id":"396071","messageId":"C28ZH8NBU8BS.1WKC7DSJANUIM@what","threadId":"53268","inReplyTo":"33f037a9-d3d5-042a-d3ba-7e4b8364663a@kdbg.org","subject":"Re: [PATCH] userdiff: support Markdown","fromName":"Ash Holland","fromEmail":"ash@sorrel.sh","sentAt":"2020-04-23T23:42:12Z","receivedAt":"2020-04-23T23:58:15Z","isPatch":true,"sender":{"key":"ash@sorrel.sh","avatar":"https://avatars.githubusercontent.com/u/9433472?v=4"},"body":"On Thu Apr 23, 2020 at 9:17 PM PST, Johannes Sixt wrote:\n> Am 21.04.20 um 03:00 schrieb Ash Holland:\n> > diff --git a/userdiff.c b/userdiff.c\n> > index efbe05e5a..f79adb3a3 100644\n> > --- a/userdiff.c\n> > +++ b/userdiff.c\n> > @@ -79,6 +79,9 @@ PATTERNS(\"java\",\n> >  \t \"|[-+0-9.e]+[fFlL]?|0[xXbB]?[0-9a-fA-F]+[lL]?\"\n> >  \t \"|[-+*/<>%&^|=!]=\"\n> >  \t \"|--|\\\\+\\\\+|<<=?|>>>?=?|&&|\\\\|\\\\|\"),\n> > +PATTERNS(\"markdown\",\n> > +\t \"^ {0,3}#{1,6}( .*)?$\",\n>\n> What is the purpose of making the heading text optional? Why would you\n> want to match a sequence of hash marks without any text following it?\n\nStrictly speaking, a markdown heading is allowed to be empty -- see for\nexample https://spec.commonmark.org/0.29/#example-49. I'm happy to\nchange it if you think it's more useful to show a previous heading which\ncontains text than an empty one, though.\n"},{"id":"396073","messageId":"C28ZA0MZKHK6.2CCZPZF15D32W@what","threadId":"53268","inReplyTo":"20200421022256.GA29071@pluvano.com","subject":"Re: [PATCH] userdiff: support Markdown","fromName":"Ash Holland","fromEmail":"ash@sorrel.sh","sentAt":"2020-04-23T23:32:46Z","receivedAt":"2020-04-24T00:05:27Z","isPatch":true,"sender":{"key":"ash@sorrel.sh","avatar":"https://avatars.githubusercontent.com/u/9433472?v=4"},"body":"On Tue Apr 21, 2020 at 2:22 AM, Emma Brooks wrote:\n> Since Markdown can have raw HTML tags in many variants, it may make\n> sense to extend the word pattern to \"[^<>= \\t]+\" like HTML's pattern so\n> tags starting/ending will not be considered part of a word.\n\nGood point, I'll update the pattern to that, thanks!\n"},{"id":"396159","messageId":"91ff4bdb-9b53-8fd1-6282-ac19a9caf1d8@kdbg.org","threadId":"53268","inReplyTo":"C28ZH8NBU8BS.1WKC7DSJANUIM@what","subject":"Re: [PATCH] userdiff: support Markdown","fromName":"Johannes Sixt","fromEmail":"j6t@kdbg.org","sentAt":"2020-04-24T17:21:34Z","receivedAt":"2020-04-24T17:21:41Z","isPatch":true,"sender":{"key":"j6t@kdbg.org","avatar":"https://avatars.githubusercontent.com/u/14810926?v=4"},"body":"Am 24.04.20 um 01:42 schrieb Ash Holland:\n> On Thu Apr 23, 2020 at 9:17 PM PST, Johannes Sixt wrote:\n>> Am 21.04.20 um 03:00 schrieb Ash Holland:\n>>> diff --git a/userdiff.c b/userdiff.c\n>>> index efbe05e5a..f79adb3a3 100644\n>>> --- a/userdiff.c\n>>> +++ b/userdiff.c\n>>> @@ -79,6 +79,9 @@ PATTERNS(\"java\",\n>>>  \t \"|[-+0-9.e]+[fFlL]?|0[xXbB]?[0-9a-fA-F]+[lL]?\"\n>>>  \t \"|[-+*/<>%&^|=!]=\"\n>>>  \t \"|--|\\\\+\\\\+|<<=?|>>>?=?|&&|\\\\|\\\\|\"),\n>>> +PATTERNS(\"markdown\",\n>>> +\t \"^ {0,3}#{1,6}( .*)?$\",\n>>\n>> What is the purpose of making the heading text optional? Why would you\n>> want to match a sequence of hash marks without any text following it?\n> \n> Strictly speaking, a markdown heading is allowed to be empty -- see for\n> example https://spec.commonmark.org/0.29/#example-49. I'm happy to\n> change it if you think it's more useful to show a previous heading which\n> contains text than an empty one, though.\n\nI don't know what makes sense, I don't write markdown regularly. A quick\ncheck shows that the sequence of hashmarks appears in the hunk header.\nIs that useful? (A genuine question!)\n\n-- Hannes\n"},{"id":"396513","messageId":"xmqqv9ljz32g.fsf@gitster.c.googlers.com","threadId":"53268","inReplyTo":"C28ZA0MZKHK6.2CCZPZF15D32W@what","subject":"Re: [PATCH] userdiff: support Markdown","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2020-04-28T21:57:11Z","receivedAt":"2020-04-28T21:57:19Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Ash Holland\" <ash@sorrel.sh> writes:\n\n> On Tue Apr 21, 2020 at 2:22 AM, Emma Brooks wrote:\n>> Since Markdown can have raw HTML tags in many variants, it may make\n>> sense to extend the word pattern to \"[^<>= \\t]+\" like HTML's pattern so\n>> tags starting/ending will not be considered part of a word.\n>\n> Good point, I'll update the pattern to that, thanks!\n\nI just marked the topic as \"expecting a reroll\" in the \"What's\ncooking\" report I have been preparing, but has something happened\nafter this exchange?\n\nNo need to rush, but we'll be closing the acceptance of new features\nin three weeks for this cycle, so we won't have infinite amount of\ntime, either.\n\nThanks.\n"},{"id":"396545","messageId":"C2DOKA77RJGH.3JJO8GPAGG9IN@what","threadId":"53268","inReplyTo":"xmqqv9ljz32g.fsf@gitster.c.googlers.com","subject":"Re: [PATCH] userdiff: support Markdown","fromName":"Ash Holland","fromEmail":"ash@sorrel.sh","sentAt":"2020-04-29T12:12:18Z","receivedAt":"2020-04-29T12:13:36Z","isPatch":true,"sender":{"key":"ash@sorrel.sh","avatar":"https://avatars.githubusercontent.com/u/9433472?v=4"},"body":"On Tue Apr 28, 2020 at 3:57 PM BST, Junio C Hamano wrote:\n> \"Ash Holland\" <ash@sorrel.sh> writes:\n>\n> > On Tue Apr 21, 2020 at 2:22 AM, Emma Brooks wrote:\n> >> Since Markdown can have raw HTML tags in many variants, it may make\n> >> sense to extend the word pattern to \"[^<>= \\t]+\" like HTML's pattern so\n> >> tags starting/ending will not be considered part of a word.\n> >\n> > Good point, I'll update the pattern to that, thanks!\n>\n> I just marked the topic as \"expecting a reroll\" in the \"What's\n> cooking\" report I have been preparing, but has something happened\n> after this exchange?\n\nYou've not missed anything, sorry, I've been busy with exams -- I'll\nprepare a v2 later today.\n"},{"id":"396551","messageId":"C2DOR3QPGHWC.2H494OQI75ZRW@what","threadId":"53268","inReplyTo":"91ff4bdb-9b53-8fd1-6282-ac19a9caf1d8@kdbg.org","subject":"Re: [PATCH] userdiff: support Markdown","fromName":"Ash Holland","fromEmail":"ash@sorrel.sh","sentAt":"2020-04-29T12:21:13Z","receivedAt":"2020-04-29T12:36:08Z","isPatch":true,"sender":{"key":"ash@sorrel.sh","avatar":"https://avatars.githubusercontent.com/u/9433472?v=4"},"body":"On Fri Apr 24, 2020 at 8:21 PM BST, Johannes Sixt wrote:\n> Am 24.04.20 um 01:42 schrieb Ash Holland:\n> > On Thu Apr 23, 2020 at 9:17 PM PST, Johannes Sixt wrote:\n> >> Am 21.04.20 um 03:00 schrieb Ash Holland:\n> >>> diff --git a/userdiff.c b/userdiff.c\n> >>> index efbe05e5a..f79adb3a3 100644\n> >>> --- a/userdiff.c\n> >>> +++ b/userdiff.c\n> >>> @@ -79,6 +79,9 @@ PATTERNS(\"java\",\n> >>>  \t \"|[-+0-9.e]+[fFlL]?|0[xXbB]?[0-9a-fA-F]+[lL]?\"\n> >>>  \t \"|[-+*/<>%&^|=!]=\"\n> >>>  \t \"|--|\\\\+\\\\+|<<=?|>>>?=?|&&|\\\\|\\\\|\"),\n> >>> +PATTERNS(\"markdown\",\n> >>> +\t \"^ {0,3}#{1,6}( .*)?$\",\n> >>\n> >> What is the purpose of making the heading text optional? Why would you\n> >> want to match a sequence of hash marks without any text following it?\n> > \n> > Strictly speaking, a markdown heading is allowed to be empty -- see for\n> > example https://spec.commonmark.org/0.29/#example-49. I'm happy to\n> > change it if you think it's more useful to show a previous heading which\n> > contains text than an empty one, though.\n>\n> I don't know what makes sense, I don't write markdown regularly. A quick\n> check shows that the sequence of hashmarks appears in the hunk header.\n> Is that useful? (A genuine question!)\n\nI think probably it would be more confusing to have Git silently ignore\nempty headings, having occasionally written documents with empty\nheadings in the past (e.g. when I know I want some different sections,\nbut I don't know what to call them yet). Probably not many people would\never run into this situation either way, though.\n"},{"id":"396629","messageId":"20200429230509.31017-1-ash@sorrel.sh","threadId":"53268","inReplyTo":"20200421010035.13915-1-ash@sorrel.sh","subject":"[PATCH v2] userdiff: support Markdown","fromName":"Ash Holland","fromEmail":"ash@sorrel.sh","sentAt":"2020-04-29T23:05:07Z","receivedAt":"2020-04-29T23:05:32Z","isPatch":true,"sender":{"key":"ash@sorrel.sh","avatar":"https://avatars.githubusercontent.com/u/9433472?v=4"},"body":"It's typical to find Markdown documentation alongside source code, and\nhaving better context for documentation changes is useful; see also\ncommit 69f9c87d4 (userdiff: add support for Fountain documents,\n2015-07-21).\n\nThe pattern is based on the CommonMark specification 0.29, section 4.2:\nhttps://spec.commonmark.org/\n\nOnly ATX headings are supported, as detecting setext headings would\nrequire printing the line before a pattern matches, or matching a\nmultiline pattern. The word-diff pattern is the same as the pattern for\nHTML, because many Markdown parsers accept inline HTML.\n\nSigned-off-by: Ash Holland <ash@sorrel.sh>\n---\nChanges since the previous patch:\n- changed the word-diff pattern to match the HTML pattern\n- fixed an off-by-one error in the wording of the test\n\n Documentation/gitattributes.txt       |  2 ++\n t/t4018-diff-funcname.sh              |  1 +\n t/t4018/markdown-heading-indented     |  6 ++++++\n t/t4018/markdown-heading-non-headings | 17 +++++++++++++++++\n userdiff.c                            |  3 +++\n 5 files changed, 29 insertions(+)\n create mode 100644 t/t4018/markdown-heading-indented\n create mode 100644 t/t4018/markdown-heading-non-headings\n\ndiff --git a/Documentation/gitattributes.txt b/Documentation/gitattributes.txt\nindex 508fe713c..2d0a03715 100644\n--- a/Documentation/gitattributes.txt\n+++ b/Documentation/gitattributes.txt\n@@ -824,6 +824,8 @@ patterns are available:\n \n - `java` suitable for source code in the Java language.\n \n+- `markdown` suitable for Markdown documents.\n+\n - `matlab` suitable for source code in the MATLAB and Octave languages.\n \n - `objc` suitable for source code in the Objective-C language.\ndiff --git a/t/t4018-diff-funcname.sh b/t/t4018-diff-funcname.sh\nindex 02255a08b..9d0779757 100755\n--- a/t/t4018-diff-funcname.sh\n+++ b/t/t4018-diff-funcname.sh\n@@ -38,6 +38,7 @@ diffpatterns=\"\n \tgolang\n \thtml\n \tjava\n+\tmarkdown\n \tmatlab\n \tobjc\n \tpascal\ndiff --git a/t/t4018/markdown-heading-indented b/t/t4018/markdown-heading-indented\nnew file mode 100644\nindex 000000000..1991c2bd4\n--- /dev/null\n+++ b/t/t4018/markdown-heading-indented\n@@ -0,0 +1,6 @@\n+Indented headings are allowed, as long as the indent is no more than 3 spaces.\n+\n+   ### RIGHT\n+\n+- something\n+- ChangeMe\ndiff --git a/t/t4018/markdown-heading-non-headings b/t/t4018/markdown-heading-non-headings\nnew file mode 100644\nindex 000000000..c479c1a3f\n--- /dev/null\n+++ b/t/t4018/markdown-heading-non-headings\n@@ -0,0 +1,17 @@\n+Headings can be right next to other lines of the file:\n+# RIGHT\n+Indents of four or more spaces make a code block:\n+\n+    # code comment, not heading\n+\n+If there's no space after the final hash, it's not a heading:\n+\n+#hashtag\n+\n+Sequences of more than 6 hashes don't make a heading:\n+\n+####### over-enthusiastic heading\n+\n+So the detected heading should be right up at the start of this file.\n+\n+ChangeMe\ndiff --git a/userdiff.c b/userdiff.c\nindex efbe05e5a..3eaa1df08 100644\n--- a/userdiff.c\n+++ b/userdiff.c\n@@ -79,6 +79,9 @@ PATTERNS(\"java\",\n \t \"|[-+0-9.e]+[fFlL]?|0[xXbB]?[0-9a-fA-F]+[lL]?\"\n \t \"|[-+*/<>%&^|=!]=\"\n \t \"|--|\\\\+\\\\+|<<=?|>>>?=?|&&|\\\\|\\\\|\"),\n+PATTERNS(\"markdown\",\n+\t \"^ {0,3}#{1,6}( .*)?$\",\n+\t \"[^<>= \\t]+\"),\n PATTERNS(\"matlab\",\n \t /*\n \t  * Octave pattern is mostly the same as matlab, except that '%%%' and\n-- \n2.26.2\n\n"},{"id":"396673","messageId":"xmqq4kt0ubgu.fsf@gitster.c.googlers.com","threadId":"53268","inReplyTo":"20200429230509.31017-1-ash@sorrel.sh","subject":"Re: [PATCH v2] userdiff: support Markdown","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2020-04-30T17:31:29Z","receivedAt":"2020-04-30T17:31:37Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Ash Holland <ash@sorrel.sh> writes:\n\n> It's typical to find Markdown documentation alongside source code, and\n> having better context for documentation changes is useful; see also\n> commit 69f9c87d4 (userdiff: add support for Fountain documents,\n> 2015-07-21).\n>\n> The pattern is based on the CommonMark specification 0.29, section 4.2:\n> https://spec.commonmark.org/\n>\n> Only ATX headings are supported, as detecting setext headings would\n> require printing the line before a pattern matches, or matching a\n> multiline pattern. The word-diff pattern is the same as the pattern for\n> HTML, because many Markdown parsers accept inline HTML.\n\n> +PATTERNS(\"markdown\",\n> +\t \"^ {0,3}#{1,6}( .*)?$\",\n\nThis is \"possibly just a bit indented run of up to 6 hashes, either\nending the line by itself or if some text follows, there must be a\nSP after the hashes\".\n\nIf I had a line that has a hash, HT and then \"Hello, world\", would\neverybody's markdown implementation reject it as a header, because\nthe whitespace after the run of hashes is not a SP?\n\nAlso, allowing only the hashes might be spec-compliant, but how\nuseful would it be to see just a sequence of 4 hashes without any\ntext after \"@@ -100,5, +100,6 @@\" in the diff output?\n\nTaking all that together, my suspicion is\n\n\t\"^ {0,3}#{1,6}[ \\t]\"\n\ni.e. \"possibly slightly indented run of 6 hashes, with a whitespace\nto catch the headers with real contents and nothing else\" might be\nmore practically useful.  I dunno.\n\n> +\t \"[^<>= \\t]+\"),\n\nThis does match the one for HTML.\n\nIn any case, let me queue this v2 as-is and see what happens.\n\nThanks.\n\n\n"},{"id":"396743","messageId":"C2FDC85CN8B8.8IVFPUOV0BHA@what","threadId":"53268","inReplyTo":"xmqq4kt0ubgu.fsf@gitster.c.googlers.com","subject":"Re: [PATCH v2] userdiff: support Markdown","fromName":"Ash Holland","fromEmail":"ash@sorrel.sh","sentAt":"2020-05-01T11:49:54Z","receivedAt":"2020-05-01T13:40:57Z","isPatch":true,"sender":{"key":"ash@sorrel.sh","avatar":"https://avatars.githubusercontent.com/u/9433472?v=4"},"body":"On Thu Apr 30, 2020 at 11:31 AM BST, Junio C Hamano wrote:\n> This is \"possibly just a bit indented run of up to 6 hashes, either\n> ending the line by itself or if some text follows, there must be a\n> SP after the hashes\".\n>\n> If I had a line that has a hash, HT and then \"Hello, world\", would\n> everybody's markdown implementation reject it as a header, because\n> the whitespace after the run of hashes is not a SP?\n\nGood point, Pandoc at least accepts a tab as well as a space there.\n(Some markdown implementations don't even require the whitespace, so any\nline starting with #{1,6} would be a heading, but that runs into\nproblems with paragraphs containing a line starting with \"#\", which is\nnot uncommon.)\n\n> Also, allowing only the hashes might be spec-compliant, but how\n> useful would it be to see just a sequence of 4 hashes without any\n> text after \"@@ -100,5, +100,6 @@\" in the diff output?\n>\n> Taking all that together, my suspicion is\n>\n> \"^ {0,3}#{1,6}[ \\t]\"\n>\n> i.e. \"possibly slightly indented run of 6 hashes, with a whitespace\n> to catch the headers with real contents and nothing else\" might be\n> more practically useful. I dunno.\n\nSure, that looks plausible. I don't have a strong opinion on whether\nit's more useful to be consistent (and show the last heading, even if it\ndoesn't contain any text) or to try as hard as possible to just show\n\"some text\", even if it's not the last heading, but two people have now\nsuggested changing it, so I'll submit a v3 with your suggested pattern.\n"},{"id":"396751","messageId":"ec6239c6-b619-e11d-9166-d968bf5601db@kdbg.org","threadId":"53268","inReplyTo":"C2FDC85CN8B8.8IVFPUOV0BHA@what","subject":"Re: [PATCH v2] userdiff: support Markdown","fromName":"Johannes Sixt","fromEmail":"j6t@kdbg.org","sentAt":"2020-05-01T14:26:20Z","receivedAt":"2020-05-01T14:26:31Z","isPatch":true,"sender":{"key":"j6t@kdbg.org","avatar":"https://avatars.githubusercontent.com/u/14810926?v=4"},"body":"Am 01.05.20 um 13:49 schrieb Ash Holland:\n> On Thu Apr 30, 2020 at 11:31 AM BST, Junio C Hamano wrote:\n>> Taking all that together, my suspicion is\n>>\n>> \"^ {0,3}#{1,6}[ \\t]\"\n>>\n>> i.e. \"possibly slightly indented run of 6 hashes, with a whitespace\n>> to catch the headers with real contents and nothing else\" might be\n>> more practically useful. I dunno.\n> \n> Sure, that looks plausible. I don't have a strong opinion on whether\n> it's more useful to be consistent (and show the last heading, even if it\n> doesn't contain any text) or to try as hard as possible to just show\n> \"some text\", even if it's not the last heading, but two people have now\n> suggested changing it, so I'll submit a v3 with your suggested pattern.\n\nThe pattern above captures only the hashmarks, but not the text of the\nheader. I suggest to append \".*\".\n\n-- Hannes\n"},{"id":"396821","messageId":"20200502131544.30878-1-ash@sorrel.sh","threadId":"53268","inReplyTo":"20200421010035.13915-1-ash@sorrel.sh","subject":"[PATCH v3] userdiff: support Markdown","fromName":"Ash Holland","fromEmail":"ash@sorrel.sh","sentAt":"2020-05-02T13:15:43Z","receivedAt":"2020-05-02T13:17:08Z","isPatch":true,"sender":{"key":"ash@sorrel.sh","avatar":"https://avatars.githubusercontent.com/u/9433472?v=4"},"body":"It's typical to find Markdown documentation alongside source code, and\nhaving better context for documentation changes is useful; see also\ncommit 69f9c87d4 (userdiff: add support for Fountain documents,\n2015-07-21).\n\nThe pattern is based on the CommonMark specification 0.29, section 4.2\n<https://spec.commonmark.org/> but doesn't match empty headings, as\nseeing them in a hunk header is unlikely to be useful.\n\nOnly ATX headings are supported, as detecting setext headings would\nrequire printing the line before a pattern matches, or matching a\nmultiline pattern. The word-diff pattern is the same as the pattern for\nHTML, because many Markdown parsers accept inline HTML.\n\nSigned-off-by: Ash Holland <ash@sorrel.sh>\n---\n Documentation/gitattributes.txt       |  2 ++\n t/t4018-diff-funcname.sh              |  1 +\n t/t4018/markdown-heading-indented     |  6 ++++++\n t/t4018/markdown-heading-non-headings | 17 +++++++++++++++++\n userdiff.c                            |  3 +++\n 5 files changed, 29 insertions(+)\n create mode 100644 t/t4018/markdown-heading-indented\n create mode 100644 t/t4018/markdown-heading-non-headings\n\ndiff --git a/Documentation/gitattributes.txt b/Documentation/gitattributes.txt\nindex 508fe713c..2d0a03715 100644\n--- a/Documentation/gitattributes.txt\n+++ b/Documentation/gitattributes.txt\n@@ -824,6 +824,8 @@ patterns are available:\n \n - `java` suitable for source code in the Java language.\n \n+- `markdown` suitable for Markdown documents.\n+\n - `matlab` suitable for source code in the MATLAB and Octave languages.\n \n - `objc` suitable for source code in the Objective-C language.\ndiff --git a/t/t4018-diff-funcname.sh b/t/t4018-diff-funcname.sh\nindex 02255a08b..9d0779757 100755\n--- a/t/t4018-diff-funcname.sh\n+++ b/t/t4018-diff-funcname.sh\n@@ -38,6 +38,7 @@ diffpatterns=\"\n \tgolang\n \thtml\n \tjava\n+\tmarkdown\n \tmatlab\n \tobjc\n \tpascal\ndiff --git a/t/t4018/markdown-heading-indented b/t/t4018/markdown-heading-indented\nnew file mode 100644\nindex 000000000..1991c2bd4\n--- /dev/null\n+++ b/t/t4018/markdown-heading-indented\n@@ -0,0 +1,6 @@\n+Indented headings are allowed, as long as the indent is no more than 3 spaces.\n+\n+   ### RIGHT\n+\n+- something\n+- ChangeMe\ndiff --git a/t/t4018/markdown-heading-non-headings b/t/t4018/markdown-heading-non-headings\nnew file mode 100644\nindex 000000000..c479c1a3f\n--- /dev/null\n+++ b/t/t4018/markdown-heading-non-headings\n@@ -0,0 +1,17 @@\n+Headings can be right next to other lines of the file:\n+# RIGHT\n+Indents of four or more spaces make a code block:\n+\n+    # code comment, not heading\n+\n+If there's no space after the final hash, it's not a heading:\n+\n+#hashtag\n+\n+Sequences of more than 6 hashes don't make a heading:\n+\n+####### over-enthusiastic heading\n+\n+So the detected heading should be right up at the start of this file.\n+\n+ChangeMe\ndiff --git a/userdiff.c b/userdiff.c\nindex efbe05e5a..069a8284c 100644\n--- a/userdiff.c\n+++ b/userdiff.c\n@@ -79,6 +79,9 @@ PATTERNS(\"java\",\n \t \"|[-+0-9.e]+[fFlL]?|0[xXbB]?[0-9a-fA-F]+[lL]?\"\n \t \"|[-+*/<>%&^|=!]=\"\n \t \"|--|\\\\+\\\\+|<<=?|>>>?=?|&&|\\\\|\\\\|\"),\n+PATTERNS(\"markdown\",\n+\t \"^ {0,3}#{1,6}[ \\t].*\",\n+\t \"[^<>= \\t]+\"),\n PATTERNS(\"matlab\",\n \t /*\n \t  * Octave pattern is mostly the same as matlab, except that '%%%' and\n-- \n2.26.2\n\n"},{"id":"396824","messageId":"40a1d950-d887-2518-f546-a2c2dc96e555@kdbg.org","threadId":"53268","inReplyTo":"20200502131544.30878-1-ash@sorrel.sh","subject":"Re: [PATCH v3] userdiff: support Markdown","fromName":"Johannes Sixt","fromEmail":"j6t@kdbg.org","sentAt":"2020-05-02T13:58:38Z","receivedAt":"2020-05-02T13:58:45Z","isPatch":true,"sender":{"key":"j6t@kdbg.org","avatar":"https://avatars.githubusercontent.com/u/14810926?v=4"},"body":"Am 02.05.20 um 15:15 schrieb Ash Holland:\n> It's typical to find Markdown documentation alongside source code, and\n> having better context for documentation changes is useful; see also\n> commit 69f9c87d4 (userdiff: add support for Fountain documents,\n> 2015-07-21).\n> \n> The pattern is based on the CommonMark specification 0.29, section 4.2\n> <https://spec.commonmark.org/> but doesn't match empty headings, as\n> seeing them in a hunk header is unlikely to be useful.\n> \n> Only ATX headings are supported, as detecting setext headings would\n> require printing the line before a pattern matches, or matching a\n> multiline pattern. The word-diff pattern is the same as the pattern for\n> HTML, because many Markdown parsers accept inline HTML.\n> \n> Signed-off-by: Ash Holland <ash@sorrel.sh>\n> ---\n>  Documentation/gitattributes.txt       |  2 ++\n>  t/t4018-diff-funcname.sh              |  1 +\n>  t/t4018/markdown-heading-indented     |  6 ++++++\n>  t/t4018/markdown-heading-non-headings | 17 +++++++++++++++++\n>  userdiff.c                            |  3 +++\n>  5 files changed, 29 insertions(+)\n>  create mode 100644 t/t4018/markdown-heading-indented\n>  create mode 100644 t/t4018/markdown-heading-non-headings\n> \n> diff --git a/Documentation/gitattributes.txt b/Documentation/gitattributes.txt\n> index 508fe713c..2d0a03715 100644\n> --- a/Documentation/gitattributes.txt\n> +++ b/Documentation/gitattributes.txt\n> @@ -824,6 +824,8 @@ patterns are available:\n>  \n>  - `java` suitable for source code in the Java language.\n>  \n> +- `markdown` suitable for Markdown documents.\n> +\n>  - `matlab` suitable for source code in the MATLAB and Octave languages.\n>  \n>  - `objc` suitable for source code in the Objective-C language.\n> diff --git a/t/t4018-diff-funcname.sh b/t/t4018-diff-funcname.sh\n> index 02255a08b..9d0779757 100755\n> --- a/t/t4018-diff-funcname.sh\n> +++ b/t/t4018-diff-funcname.sh\n> @@ -38,6 +38,7 @@ diffpatterns=\"\n>  \tgolang\n>  \thtml\n>  \tjava\n> +\tmarkdown\n>  \tmatlab\n>  \tobjc\n>  \tpascal\n> diff --git a/t/t4018/markdown-heading-indented b/t/t4018/markdown-heading-indented\n> new file mode 100644\n> index 000000000..1991c2bd4\n> --- /dev/null\n> +++ b/t/t4018/markdown-heading-indented\n> @@ -0,0 +1,6 @@\n> +Indented headings are allowed, as long as the indent is no more than 3 spaces.\n> +\n> +   ### RIGHT\n> +\n> +- something\n> +- ChangeMe\n> diff --git a/t/t4018/markdown-heading-non-headings b/t/t4018/markdown-heading-non-headings\n> new file mode 100644\n> index 000000000..c479c1a3f\n> --- /dev/null\n> +++ b/t/t4018/markdown-heading-non-headings\n> @@ -0,0 +1,17 @@\n> +Headings can be right next to other lines of the file:\n> +# RIGHT\n> +Indents of four or more spaces make a code block:\n> +\n> +    # code comment, not heading\n> +\n> +If there's no space after the final hash, it's not a heading:\n> +\n> +#hashtag\n> +\n> +Sequences of more than 6 hashes don't make a heading:\n> +\n> +####### over-enthusiastic heading\n> +\n> +So the detected heading should be right up at the start of this file.\n> +\n> +ChangeMe\n> diff --git a/userdiff.c b/userdiff.c\n> index efbe05e5a..069a8284c 100644\n> --- a/userdiff.c\n> +++ b/userdiff.c\n> @@ -79,6 +79,9 @@ PATTERNS(\"java\",\n>  \t \"|[-+0-9.e]+[fFlL]?|0[xXbB]?[0-9a-fA-F]+[lL]?\"\n>  \t \"|[-+*/<>%&^|=!]=\"\n>  \t \"|--|\\\\+\\\\+|<<=?|>>>?=?|&&|\\\\|\\\\|\"),\n> +PATTERNS(\"markdown\",\n> +\t \"^ {0,3}#{1,6}[ \\t].*\",\n> +\t \"[^<>= \\t]+\"),\n>  PATTERNS(\"matlab\",\n>  \t /*\n>  \t  * Octave pattern is mostly the same as matlab, except that '%%%' and\n> \n\nI tested this patch, and it looks good:\n\nAcked-by: Johannes Sixt <j6t@kdbg.org>\n\n-- Hannes\n"}]}