{"thread":{"id":"63972","subject":"[RFE] Add JSON output to git log commands","startedAt":"2025-08-17T20:17:59Z","lastAt":"2025-08-17T22:54:25Z","messageCount":5,"participants":["Ron Ziroby Romero","Kristoffer Haugsbakk","D. Ben Knoble","brian m. carlson","Junio C Hamano"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"524327","messageId":"CAGW8g7mV6az3ybYf3uzHYQTGLDwc34eebZnha1EQ3Sb6B8E-fQ@mail.gmail.com","threadId":"63972","inReplyTo":null,"subject":"[RFE] Add JSON output to git log commands","fromName":"Ron Ziroby Romero","fromEmail":"ziroby@gmail.com","sentAt":"2025-08-17T20:17:46Z","receivedAt":"2025-08-17T20:17:59Z","isPatch":false,"sender":{"key":"ziroby@gmail.com","avatar":"https://gravatar.com/avatar/5b42738b00e3d753c7373bde65683bea6fc16cf5a2fc922fc34039e88c0ae4bf?d=mp&s=160"},"body":"I would like to add JSON output to the git log command.\n\n## Motivation\n\nMachine parsing of git log output is prevalent, but git only provides\nhuman-readable output. Having git output JSON directly solves problems\nwith the format option or third-party tools. Git has the information\nin a machine-readable format. It should output in a machine-readable\nformat. JSON is ubiquitous and easy to generate, and therefore, it\nmakes sense to output JSON.\n\nThe author of one of the third-party tools says that JSON output is\nthe natural evolution of the Unix philosophy and should be done\nnatively for all tools[4].\n\n## Current behaviour\n\nGit log can output human-readable output in several ways. However,\noutputting in JSON requires third-party tools or hacking pretty\noutput.\n\n## Proposed enhancement\n\nAdd a –pretty=json flag to output logs in JSON format.\n\n## Alternatives\n\n### Why natively?\n\nThe `jc` command parses git log output to convert to JSON[3]. However,\nit post processes and has some difficulty with time zones in dates.\nAlso, the author considers it to be a stopgap until Unix tools can be\nadopted to output JSON natively[4], which is what I'm proposing.\n\nIn a TIL post[2], Simon Willison showcases a method using pretty\noutput with nulls piped to jq. This method uses the pretty command to\nget delimited output. However, this script doesn't handle all the\noutput from git log.\n\nTools like git-log2json[1] parse git output post-hoc, rather than\nproducing structured output from git.\n\nProviding git log in JSON format will allow us to go to the source,\nwhere we have the log output in machine readable form, and output\ndirectly in machine readable form directly, without going through an\nintermediate format\n\n### Why JSON?\n\nJSON is a sufficient and popular output format. It is sufficient in\nthat it can represent all the fields of git log in a way that allows\nfor special characters like quotes, newlines, and control characters.\nIt is also popular. Every language has libraries to parse JSON,\nincluding the command-line utility jq, which can read and mutate JSON.\n\n## Use cases\n\nThe JSON can be used by tools or piped into jq to extract and\nmanipulate the data. Scripts can be written to work with the JSON\noutput.\n\n## Design outline\n\n* Add a `PRETTY_JSON` constant.\n* Create a pretty-json.c file to output JSON log information\n* Modify pretty.c to call pretty-json to output JSON when the flag is set.\n* Use existing utility functions written in the existing source to\noutput the JSON.\n\n## Example output\n\nHere’s a sample with two commits:\n\n```JSON\n{\n  \"commits\": [\n    {\n      \"commit\": \"3857aae53f3633b7de63ad640737c657387ae0c6\",\n      \"refs\": [\n        \"HEAD\",\n        \"refs/remotes/origin/main\",\n        \"refs/remotes/origin/HEAD\"\n      ],\n      \"author\": {\n        \"name\": \"Somebody J. Example\",\n        \"email\": \"somebody@example.com\",\n        \"date\": \"2024-09-25T18:23:49-07:00\",\n        \"timestamp\": \"1727313829\"\n      },\n      \"committer\": {\n        \"name\": \"Somebody Else\",\n        \"email\": \"somebody.else@example.org\",\n        \"date\": \"2024-09-25T18:24:52-07:00\",\n        \"timestamp\": \"1727313892\"\n      },\n      \"message\": \"Do a thing\\n\"\n    },\n    {\n      \"commit\": \"1522467d13a8fe29eb32209f175722df41e224b6\",\n      \"merge\": [\n        \"f92c61aef0190641e01294dad3b891b28113e1d5\",\n        \"7ffcbafbf32185da7dccb4b3f49b871f24ab58c4\"\n      ],\n      \"author\": {\n        \"name\": \"Somebody J. Example\",\n        \"email\": \"somebody@example.com\",\n        \"date\": \"2024-09-25T18:24:52-07:00\",\n        \"timestamp\": \"1727313892\"\n      },\n      \"committer\": {\n        \"name\": \"Somebody Else\",\n        \"email\": \"somebody.else@example.org\",\n        \"date\": \"2024-09-25T18:24:52-07:00\",\n        \"timestamp\": \"1727313892\"\n      },\n      \"message\": \"Merge something\\n\\n* This,\\n* That, and\\n* The other\\n\"\n    }\n  ]\n}\n```\n\n## References\n\n> [1] Context-Driven Testing Toolkit, git-log2json: Convert git log to JSON, GitHub repository, https://github.com/context-driven-testing-toolkit/git-log2json\n\n> [2] Simon Willison, “Convert git log output to JSON using jq,” til.simonwillison.net, March 22 2023. https://til.simonwillison.net/jq/git-log-json\n\n> [3] Kelly Brazil, jc.parsers.git_log: JSON parser for git log, jc documentation, version 1.5. Retrieved via GitHub Pages, https://kellyjonbrazil.github.io/jc/docs/parsers/git_log.html\n\n> [4] Kelly Brazil, Bringing the Unix Philosophy to the 21st Century, Brazil’s Blog, November 26 2019. https://blog.kellybrazil.com/2019/11/26/bringing-the-unix-philosophy-to-the-21st-century/\n"},{"id":"524328","messageId":"67621563-9441-4ae9-a23b-51af7d3ce666@app.fastmail.com","threadId":"63972","inReplyTo":"CAGW8g7mV6az3ybYf3uzHYQTGLDwc34eebZnha1EQ3Sb6B8E-fQ@mail.gmail.com","subject":"Re: [RFE] Add JSON output to git log commands","fromName":"Kristoffer Haugsbakk","fromEmail":"kristofferhaugsbakk@fastmail.com","sentAt":"2025-08-17T21:09:00Z","receivedAt":"2025-08-17T21:09:42Z","isPatch":false,"sender":{"key":"kristofferhaugsbakk@fastmail.com","avatar":null},"body":"On Sun, Aug 17, 2025, at 22:17, Ron Ziroby Romero wrote:\n> I would like to add JSON output to the git log command.\n\nPreviously: https://lore.kernel.org/git/CAGW8g7=21pPAgCixjpayEvmw_ns-hcB4e59NP476TKtCRXHPXQ@mail.gmail.com/\n\n> ...\n> ## Design outline\n>\n> * Add a `PRETTY_JSON` constant.\n> * Create a pretty-json.c file to output JSON log information\n> * Modify pretty.c to call pretty-json to output JSON when the flag is set.\n> * Use existing utility functions written in the existing source to\n> output the JSON.\n\nI’m guessing that the existing `json-writer.h` is relevant.\n\n-- \nKristoffer Haugsbakk\n\n"},{"id":"524330","messageId":"CALnO6CAkTt0brPbnKDu9rOAHeLRtk2hm3xJT5+HvZOfHSL-Szg@mail.gmail.com","threadId":"63972","inReplyTo":"67621563-9441-4ae9-a23b-51af7d3ce666@app.fastmail.com","subject":"Re: [RFE] Add JSON output to git log commands","fromName":"D. Ben Knoble","fromEmail":"ben.knoble@gmail.com","sentAt":"2025-08-17T21:28:15Z","receivedAt":"2025-08-17T21:28:28Z","isPatch":false,"sender":{"key":"ben.knoble@gmail.com","avatar":"https://avatars.githubusercontent.com/u/22802209?v=4"},"body":"On Sun, Aug 17, 2025 at 5:09 PM Kristoffer Haugsbakk\n<kristofferhaugsbakk@fastmail.com> wrote:\n>\n> On Sun, Aug 17, 2025, at 22:17, Ron Ziroby Romero wrote:\n> > I would like to add JSON output to the git log command.\n>\n> Previously: https://lore.kernel.org/git/CAGW8g7=21pPAgCixjpayEvmw_ns-hcB4e59NP476TKtCRXHPXQ@mail.gmail.com/\n\nIn particular, I would guess that deciding how to handle \"raw bytes\"\n(e.g., paths that are not necessarily UTF-8 strings) is important.\nJSON uses Unicode characters:\n\n     A string is a sequence of zero or more Unicode characters,\nwrapped in double quotes, using backslash escapes. A character is\nrepresented as a single character string. A string is very much like a\nC or Java string.\n\nhttps://www.json.org/json-en.html\n\n(even though the railroad diagram says \"codepoints\"; I find the whole\nthing a bit muddy—what exactly is representable in JSON strings?\n\n    Excepting a few encoding details, that completely describes the language.\n\n??? which details?)\n\n\n-- \nD. Ben Knoble\n"},{"id":"524332","messageId":"aKJTpJL3kEyqDA6P@fruit.crustytoothpaste.net","threadId":"63972","inReplyTo":"CAGW8g7mV6az3ybYf3uzHYQTGLDwc34eebZnha1EQ3Sb6B8E-fQ@mail.gmail.com","subject":"Re: [RFE] Add JSON output to git log commands","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2025-08-17T22:11:48Z","receivedAt":"2025-08-17T22:11:57Z","isPatch":false,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On 2025-08-17 at 20:17:46, Ron Ziroby Romero wrote:\n> I would like to add JSON output to the git log command.\n> \n> ## Motivation\n> \n> Machine parsing of git log output is prevalent, but git only provides\n> human-readable output. Having git output JSON directly solves problems\n> with the format option or third-party tools. Git has the information\n> in a machine-readable format. It should output in a machine-readable\n> format. JSON is ubiquitous and easy to generate, and therefore, it\n> makes sense to output JSON.\n\nGit provides plenty of machine-readable formats, to be clear.  They're\nnot typically structured in a standard way like JSON or CBOR, but many\nforges and other tools do successfully parse Git output with a variety\nof tools.\n\n> The author of one of the third-party tools says that JSON output is\n> the natural evolution of the Unix philosophy and should be done\n> natively for all tools[4].\n> \n> ## Current behaviour\n> \n> Git log can output human-readable output in several ways. However,\n> outputting in JSON requires third-party tools or hacking pretty\n> output.\n> \n> ## Proposed enhancement\n> \n> Add a –pretty=json flag to output logs in JSON format.\n\nI'd like to hear how you plan to deal with non-UTF-8 byte strings since\nJSON must always be valid Unicode.  Most data in Git is only by\nconvention UTF-8 and can actually be in other encodings or no encoding\nat all: refs, commit messages[0], and author and committer idents.\n\nWhat would be a good idea is to add a byte string entry to the JSON\nwriter and use it for these formats.  If the data is not valid UTF-8, or\nif it contains a % sign, then you URL-encode it.  Other encodings are\npossible as well, but not JSON escapes[1].\n\nOther good options would be to use CBOR instead, since it provides\nnative byte strings.\n\nBad options would be to use U+FFFD, since that makes the output useless\nwhen you hit one of these cases (and I can tell you from $DAYJOB that\nthey're not that uncommon) and to just shovel bytes into the output and\nlet the reader be sad (which will definitely make the output useless as\nwell as result in angry bug reports to the list).\n\nAs a note, I think you want `--pretty`, not `-pretty` (we use two dashes\nfor long options).\n\n[0] Yes, they declare an encoding, but it isn't always correct and the\nencoding someone used is not always available on every system.  I saw\nsomeone in the Linux kernel history write \"latin1\", which is not a valid\nencoding according to Ruby, which I was using to parse it.\n[1] `\\u00ff` represents U+00FF, which is equivalent to the byte sequence\n0xc3 0xbf, not 0xff.\n-- \nbrian m. carlson (they/them)\nToronto, Ontario, CA\n"},{"id":"524333","messageId":"xmqqqzx9k19d.fsf@gitster.g","threadId":"63972","inReplyTo":"CAGW8g7mV6az3ybYf3uzHYQTGLDwc34eebZnha1EQ3Sb6B8E-fQ@mail.gmail.com","subject":"Re: [RFE] Add JSON output to git log commands","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-08-17T22:54:22Z","receivedAt":"2025-08-17T22:54:25Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Ron Ziroby Romero <ziroby@gmail.com> writes:\n\n> ## Design outline\n>\n> * Add a `PRETTY_JSON` constant.\n> * Create a pretty-json.c file to output JSON log information\n> * Modify pretty.c to call pretty-json to output JSON when the flag is set.\n> * Use existing utility functions written in the existing source to\n> output the JSON.\n\nIs this limited to only giving another serialization format to what\nis in `git cat-file commit` output for sequence of commits, which is\nwhat I see in the example below?\n\nWithin that limited scope, I am curious what your plan is to deal\nwith header elements like \"encoding\", \"gpgsig\", \"mergetag\", etc.\n\nAnd outside that scope, I am not sure what the most useful output\nwould be for things outside what is in each of the commit object.\nE.g., various \"diff\" output, e.g. --stat, -p, --name-status...\n\nLeaving that outside the scope would be a very clean way out to\navoid confusing design issues ;-)\n"}]}