{"thread":{"id":"62486","subject":"Trailers Containing Underscore or Dot Characters","startedAt":"2024-11-13T07:40:36Z","lastAt":"2024-11-18T07:32:41Z","messageCount":8,"participants":["ふじを","Christian Couder","Junio C Hamano","Patrick Steinhardt","kristofferhaugsbakk@fastmail.com"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"507195","messageId":"CABpqQ_u4GG28L9KGX+HXiOv2AVAL7sckRBN4a99pCyeaQS+n_w@mail.gmail.com","threadId":"62486","inReplyTo":null,"subject":"Trailers Containing Underscore or Dot Characters","fromName":"ふじを","fromEmail":"ffjlabo@gmail.com","sentAt":"2024-11-13T07:40:22Z","receivedAt":"2024-11-13T07:40:36Z","isPatch":false,"sender":{"key":"ffjlabo@gmail.com","avatar":null},"body":"Hi everyone,\nFirst of all, thank you for making such a tool. I am grateful to use\nthis tool every day.\n\nI encountered the issue with trailers containing Underscore or Dot characters.\nI don't know if this behavior is OK as a specification, or not. So I\ncreated the post.\n\nWhat did you do before the bug happened? (Steps to reproduce your issue)\n\nI added coommit with trailers which have \"_\" or \".\" in each key.\n```\ngit commit -m \"Test\" --trailer \"test_hoge: fuga\" --trailer \"test.hoge:\nfuga\" --trailer \"test-hoge: fuga\"\n```\nWhat did you expect to happen? (Expected behavior)\nI expcted that they are shown like this.\n```\n# git show\n    Test\n    test_hoge: fuga\n    test.hoge: fuga\n    test-hoge: fuga\n```\n\nWhat happened instead? (Actual behavior)\n\nActually, some of them are unexpected value.\n```\n# git show\ncommit 70afae811def2345bead6660e0f3183662c9f4df (HEAD -> master)\nAuthor: Your Name <you@example.com>\nDate:   Wed Nov 13 08:28:19 2024 +0300\n    Test\n    test_hoge: fuga:\n    test.hoge: fuga:\n    test-hoge: fuga\n```\n\nWhat's different between what you expected and what actually happened?\n\nThe one for `--trailer \"test-hoge: fuga\"` is finished with nothing.\nBut The others for ones which have \"_\" or \".\" are finished with \":\".\n\nAnything else you want to add:\n\nI want to use such characters to add some key-value received other\nservice (e.g. GitHub Actions) as is.\nIs there any chance that characters like _ and . will be allowed in\nthe trailer's key?\n\nBy the way, I tried the investigation for the current specification.\nI hope this is helpful in some way.\n\n[1] This behavior occured when executing `git interpret-trailers`.\n```\n# echo msg > msg\n# git interpret-trailers --trailer \"test_hoge: test\" --trailer\n\"test.hoge: test\" --trailer \"test-hoge: test\" --in-place msg\n```\n```\n# cat msg\nmsg\ntest_hoge: test:\ntest.hoge: test:\ntest-hoge: test\n```\n[2] There might be the cause in the process of the persing trailers.\nif they have such characters, the whole of the trailer option values\nare interpreted as one token even though they have \":\" if they have\nsuch characters.\n\nI checked the source code at\n- parse_trailers_from_command_line_args\nhttps://github.com/git/git/blob/564d0252ca632e0264ed670534a51d18a689ef5d/trailer.c#L748C1-L764C4\n- -> find_separator\nhttps://github.com/git/git/blob/564d0252ca632e0264ed670534a51d18a689ef5d/trailer.c#L614C1-L642C1\n\n[System Info]\ngit version:\ngit version 2.43.0\ncpu: aarch64\nno commit associated with this build\nsizeof-long: 8\nsizeof-size_t: 8\nshell-path: /bin/sh\nuname: Linux 6.10.11-orbstack-00280-g1304bd068592 #21 SMP Sat Sep 21\n10:45:28 UTC 2024 aarch64\ncompiler info: gnuc: 13.2\nlibc info: glibc: 2.39\n$SHELL (typically, interactive shell): <unset>\n[Enabled Hooks]\n"},{"id":"507233","messageId":"CAP8UFD0_+=5xmw6y5qiO22KMZhExLyZ=5HRjYphLD66vt0LLNg@mail.gmail.com","threadId":"62486","inReplyTo":"CABpqQ_u4GG28L9KGX+HXiOv2AVAL7sckRBN4a99pCyeaQS+n_w@mail.gmail.com","subject":"Re: Trailers Containing Underscore or Dot Characters","fromName":"Christian Couder","fromEmail":"christian.couder@gmail.com","sentAt":"2024-11-13T21:57:34Z","receivedAt":"2024-11-13T21:57:48Z","isPatch":false,"sender":{"key":"christian.couder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/208954?v=4"},"body":"Hi,\n\nOn Wed, Nov 13, 2024 at 8:40 AM ふじを <ffjlabo@gmail.com> wrote:\n>\n> Hi everyone,\n> First of all, thank you for making such a tool. I am grateful to use\n> this tool every day.\n>\n> I encountered the issue with trailers containing Underscore or Dot characters.\n> I don't know if this behavior is OK as a specification, or not. So I\n> created the post.\n>\n> What did you do before the bug happened? (Steps to reproduce your issue)\n>\n> I added coommit with trailers which have \"_\" or \".\" in each key.\n> ```\n> git commit -m \"Test\" --trailer \"test_hoge: fuga\" --trailer \"test.hoge:\n> fuga\" --trailer \"test-hoge: fuga\"\n> ```\n> What did you expect to happen? (Expected behavior)\n> I expcted that they are shown like this.\n> ```\n> # git show\n>     Test\n>     test_hoge: fuga\n>     test.hoge: fuga\n>     test-hoge: fuga\n> ```\n>\n> What happened instead? (Actual behavior)\n>\n> Actually, some of them are unexpected value.\n> ```\n> # git show\n> commit 70afae811def2345bead6660e0f3183662c9f4df (HEAD -> master)\n> Author: Your Name <you@example.com>\n> Date:   Wed Nov 13 08:28:19 2024 +0300\n>     Test\n>     test_hoge: fuga:\n>     test.hoge: fuga:\n>     test-hoge: fuga\n> ```\n>\n> What's different between what you expected and what actually happened?\n>\n> The one for `--trailer \"test-hoge: fuga\"` is finished with nothing.\n> But The others for ones which have \"_\" or \".\" are finished with \":\".\n\nYeah, that's because '-' is allowed in trailer keys while '_' and '.' are not.\n\nThe underlying issue is that when we are given a full commit message\nthat might contain trailers and we have to parse it and find the\ntrailers in it, the more special characters we allow in trailer keys,\nthe higher the risk that regular text will be wrongly considered\ntrailer keys.\n\n> Anything else you want to add:\n>\n> I want to use such characters to add some key-value received other\n> service (e.g. GitHub Actions) as is.\n> Is there any chance that characters like _ and . will be allowed in\n> the trailer's key?\n\nIt's true that for trailers that are passed through --trailer options,\nwe could probably relax the restrictions for trailer keys. We could\nalso add config options to be more flexible in the general case. I\nthink patches implementing these could be well received.\n\n> By the way, I tried the investigation for the current specification.\n> I hope this is helpful in some way.\n>\n> [1] This behavior occured when executing `git interpret-trailers`.\n> ```\n> # echo msg > msg\n> # git interpret-trailers --trailer \"test_hoge: test\" --trailer\n> \"test.hoge: test\" --trailer \"test-hoge: test\" --in-place msg\n> ```\n> ```\n> # cat msg\n> msg\n> test_hoge: test:\n> test.hoge: test:\n> test-hoge: test\n> ```\n\nYeah `git commit --trailer ...` and `git interpret-trailers ...` use\nthe same underlying trailer parsing code.\n\n> [2] There might be the cause in the process of the persing trailers.\n> if they have such characters, the whole of the trailer option values\n> are interpreted as one token even though they have \":\" if they have\n> such characters.\n>\n> I checked the source code at\n> - parse_trailers_from_command_line_args\n> https://github.com/git/git/blob/564d0252ca632e0264ed670534a51d18a689ef5d/trailer.c#L748C1-L764C4\n> - -> find_separator\n> https://github.com/git/git/blob/564d0252ca632e0264ed670534a51d18a689ef5d/trailer.c#L614C1-L642C1\n\nYeah, find_separator() could definitely have options to be more\nrelaxed about what a trailer key is.\n\nBest,\nChristian.\n"},{"id":"507234","messageId":"xmqqa5e2ahqf.fsf@gitster.g","threadId":"62486","inReplyTo":"CAP8UFD0_+=5xmw6y5qiO22KMZhExLyZ=5HRjYphLD66vt0LLNg@mail.gmail.com","subject":"Re: Trailers Containing Underscore or Dot Characters","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2024-11-13T22:31:20Z","receivedAt":"2024-11-13T22:31:23Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Christian Couder <christian.couder@gmail.com> writes:\n\n>> Date:   Wed Nov 13 08:28:19 2024 +0300\n>>     Test\n>>     test_hoge: fuga:\n>>     test.hoge: fuga:\n>>     test-hoge: fuga\n>> ```\n>>\n>> What's different between what you expected and what actually happened?\n>>\n>> The one for `--trailer \"test-hoge: fuga\"` is finished with nothing.\n>> But The others for ones which have \"_\" or \".\" are finished with \":\".\n>\n> Yeah, that's because '-' is allowed in trailer keys while '_' and '.' are not.\n\nThanks for responding.\n\nI did not offhand recall seeing anywhere in our documentation set\nthat defines what a valid trailer key looks like, so I went and read\nthe interpret-trailers manual page and did not find any.  For\nexample, is this a valid trailer line, even if we know '-' is\n\"allowed in trailer keys\"?\n\n\t-test: fuga\n\nIs this a valid trailer line, when your configuration adds '-' to\nthe set of separator characters?\n\n\ttest- fuga\n\nWe do not even have an entry in the glossary for \"trailer\", and that\nprobably is the first thing we need to fix.\n\n"},{"id":"507349","messageId":"Zzb7aI2C2uh_nwuc@pks.im","threadId":"62486","inReplyTo":"xmqqa5e2ahqf.fsf@gitster.g","subject":"Re: Trailers Containing Underscore or Dot Characters","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2024-11-15T07:42:37Z","receivedAt":"2024-11-15T07:42:50Z","isPatch":false,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Thu, Nov 14, 2024 at 07:31:20AM +0900, Junio C Hamano wrote:\n> Christian Couder <christian.couder@gmail.com> writes:\n> \n> >> Date:   Wed Nov 13 08:28:19 2024 +0300\n> >>     Test\n> >>     test_hoge: fuga:\n> >>     test.hoge: fuga:\n> >>     test-hoge: fuga\n> >> ```\n> >>\n> >> What's different between what you expected and what actually happened?\n> >>\n> >> The one for `--trailer \"test-hoge: fuga\"` is finished with nothing.\n> >> But The others for ones which have \"_\" or \".\" are finished with \":\".\n> >\n> > Yeah, that's because '-' is allowed in trailer keys while '_' and '.' are not.\n> \n> Thanks for responding.\n> \n> I did not offhand recall seeing anywhere in our documentation set\n> that defines what a valid trailer key looks like, so I went and read\n> the interpret-trailers manual page and did not find any.  For\n> example, is this a valid trailer line, even if we know '-' is\n> \"allowed in trailer keys\"?\n> \n> \t-test: fuga\n> \n> Is this a valid trailer line, when your configuration adds '-' to\n> the set of separator characters?\n> \n> \ttest- fuga\n> \n> We do not even have an entry in the glossary for \"trailer\", and that\n> probably is the first thing we need to fix.\n\nThe second thing we should be fixing is that git-interpret-trailers(1)\nallows us to add invalid trailers:\n\n    $ touch file\n    $ git interpret-trailers --in-place file --trailer 'Valid-trailer: bar'\n    $ git interpret-trailers --parse file\n    Valid-trailer: bar\n    $ git interpret-trailers --in-place file --trailer 'Invalid_trailer: bar'\n    $ git interpret-trailers --parse file\n    $ cat file\n\n    Valid-trailer: bar\n    Invalid_trailer: bar:\n\nAfter the second invocation of git-interpret-trailers(1) it is unable to\nfind any trailers anymore due to the bogus format of the second trailer\nline.\n\nPatrick\n"},{"id":"507445","messageId":"ba35ba3ec212860a3151cfad582a384a50c55e6a.1731871683.git.code@khaugsbakk.name","threadId":"62486","inReplyTo":"xmqqa5e2ahqf.fsf@gitster.g","subject":"[PATCH] Documentation/glossary: describe \"trailer\"","fromName":"","fromEmail":"kristofferhaugsbakk@fastmail.com","sentAt":"2024-11-17T19:33:49Z","receivedAt":"2024-11-17T19:34:44Z","isPatch":true,"sender":{"key":"kristofferhaugsbakk@fastmail.com","avatar":null},"body":"From: Kristoffer Haugsbakk <code@khaugsbakk.name>\n\nReported-by: Junio C Hamano <gitster@pobox.com>\nSigned-off-by: Kristoffer Haugsbakk <code@khaugsbakk.name>\n---\n\nNotes (series):\n    On Wed, Nov 13, 2024, at 23:31, Junio C Hamano wrote:\n    > Christian Couder <christian.couder@gmail.com> writes:\n    > > [...]\n    > We do not even have an entry in the glossary for \"trailer\", and that\n    > probably is the first thing we need to fix.\n    \n    ---\n    \n    • Tags: What the Linux Kernel uses\n    • Footers: Lots of people around the Internet apparently. Like on\n      Stackoverflow. Or Chromium: https://commondatastorage.googleapis.com/chrome-infra-docs/flat/depot_tools/docs/html/git-footers.html\n\n Documentation/glossary-content.txt | 5 +++++\n 1 file changed, 5 insertions(+)\n\ndiff --git a/Documentation/glossary-content.txt b/Documentation/glossary-content.txt\nindex 42afe048691..575c18f776e 100644\n--- a/Documentation/glossary-content.txt\n+++ b/Documentation/glossary-content.txt\n@@ -696,6 +696,11 @@ the `refs/tags/` hierarchy is used to represent local tags..\n \tthat each contain very well defined concepts or small incremental yet\n \trelated changes.\n \n+[[def_trailer]]trailer::\n+\tKey-value metadata.  Trailers are optionally found at the end of\n+\ta commit message.  Might be called \"footers\" or \"tags\" in other\n+\tcommunities.  See linkgit:git-interpret-trailers[1].\n+\n [[def_tree]]tree::\n \tEither a <<def_working_tree,working tree>>, or a <<def_tree_object,tree\n \tobject>> together with the dependent <<def_blob_object,blob>> and tree objects\n-- \n2.47.0\n\n"},{"id":"507453","messageId":"xmqqcyitxtkj.fsf@gitster.g","threadId":"62486","inReplyTo":"ba35ba3ec212860a3151cfad582a384a50c55e6a.1731871683.git.code@khaugsbakk.name","subject":"Re: [PATCH] Documentation/glossary: describe \"trailer\"","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2024-11-18T00:40:44Z","receivedAt":"2024-11-18T00:40:46Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"kristofferhaugsbakk@fastmail.com writes:\n\n> diff --git a/Documentation/glossary-content.txt b/Documentation/glossary-content.txt\n> index 42afe048691..575c18f776e 100644\n> --- a/Documentation/glossary-content.txt\n> +++ b/Documentation/glossary-content.txt\n> @@ -696,6 +696,11 @@ the `refs/tags/` hierarchy is used to represent local tags..\n>  \tthat each contain very well defined concepts or small incremental yet\n>  \trelated changes.\n>  \n> +[[def_trailer]]trailer::\n> +\tKey-value metadata.  Trailers are optionally found at the end of\n> +\ta commit message.  Might be called \"footers\" or \"tags\" in other\n> +\tcommunities.  See linkgit:git-interpret-trailers[1].\n\nSounds sensible.\n"},{"id":"507455","messageId":"xmqqy11hjqnk.fsf@gitster.g","threadId":"62486","inReplyTo":"Zzb7aI2C2uh_nwuc@pks.im","subject":"Re: Trailers Containing Underscore or Dot Characters","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2024-11-18T01:07:27Z","receivedAt":"2024-11-18T01:07:30Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Patrick Steinhardt <ps@pks.im> writes:\n\n> The second thing we should be fixing is that git-interpret-trailers(1)\n> allows us to add invalid trailers:\n>\n>     $ touch file\n>     $ git interpret-trailers --in-place file --trailer 'Valid-trailer: bar'\n>     $ git interpret-trailers --parse file\n>     Valid-trailer: bar\n>     $ git interpret-trailers --in-place file --trailer 'Invalid_trailer: bar'\n>     $ git interpret-trailers --parse file\n>     $ cat file\n>\n>     Valid-trailer: bar\n>     Invalid_trailer: bar:\n>\n> After the second invocation of git-interpret-trailers(1) it is unable to\n> find any trailers anymore due to the bogus format of the second trailer\n> line.\n\nA good point.\nThanks.\n"},{"id":"507461","messageId":"CAP8UFD3p3ts_7kvk_Zm2efhcPUcEFnLcLFPAqzb40d46BTVzyA@mail.gmail.com","threadId":"62486","inReplyTo":"ba35ba3ec212860a3151cfad582a384a50c55e6a.1731871683.git.code@khaugsbakk.name","subject":"Re: [PATCH] Documentation/glossary: describe \"trailer\"","fromName":"Christian Couder","fromEmail":"christian.couder@gmail.com","sentAt":"2024-11-18T07:32:27Z","receivedAt":"2024-11-18T07:32:41Z","isPatch":true,"sender":{"key":"christian.couder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/208954?v=4"},"body":"On Sun, Nov 17, 2024 at 8:34 PM <kristofferhaugsbakk@fastmail.com> wrote:\n>\n>     • Tags: What the Linux Kernel uses\n>     • Footers: Lots of people around the Internet apparently. Like on\n>       Stackoverflow. Or Chromium: https://commondatastorage.googleapis.com/chrome-infra-docs/flat/depot_tools/docs/html/git-footers.html\n>\n>  Documentation/glossary-content.txt | 5 +++++\n>  1 file changed, 5 insertions(+)\n>\n> diff --git a/Documentation/glossary-content.txt b/Documentation/glossary-content.txt\n> index 42afe048691..575c18f776e 100644\n> --- a/Documentation/glossary-content.txt\n> +++ b/Documentation/glossary-content.txt\n> @@ -696,6 +696,11 @@ the `refs/tags/` hierarchy is used to represent local tags..\n>         that each contain very well defined concepts or small incremental yet\n>         related changes.\n>\n> +[[def_trailer]]trailer::\n> +       Key-value metadata.  Trailers are optionally found at the end of\n> +       a commit message.  Might be called \"footers\" or \"tags\" in other\n> +       communities.  See linkgit:git-interpret-trailers[1].\n> +\n\nAck, thanks!\n"}]}