$ git log --format=json
[{
"commit": "64eabf050e315a4c7a11e0c05ca163be7cf9075e",
"tree": "b1e977800f40bbf6de906b1fe4f2de4b4b14f0fd",
"author": "Tux <tux@example.com> 1490981516 +0200",
"committer": "Tux <tux@example.com> 1490981516 +0200",
"message": "This is a test commit",
"long_message": "This explains in more details the commit"
}]This would make it easy to parse the output.
Ævar Arnfjörð BjarmasonRe: Feature request: --format=json
On Sat, Apr 8, 2017 at 6:07 PM, Fred .Flintstone <eldmannen@gmail.com> wrote:
> $ git log --format=json
> [{
> "commit": "64eabf050e315a4c7a11e0c05ca163be7cf9075e",
> "tree": "b1e977800f40bbf6de906b1fe4f2de4b4b14f0fd",
> "author": "Tux <tux@example.com> 1490981516 +0200",
> "committer": "Tux <tux@example.com> 1490981516 +0200",
> "message": "This is a test commit",
> "long_message": "This explains in more details the commit"
> }]
>
> This would make it easy to parse the output.The git-log command isn't plumbing that's meant for machines, but the git-for-each-ref command is what you're most likely looking for.
It doesn't have JSON output, but you can make e.g. --format emit something even more easily parsable, e.g. a version of what you have with each field delimited by a custom delimiter, and then split on that.
It does have --perl, --tcl etc. options to make it easy to quote the fields, however there's no logic to manage the state machine JSON would need to omit trailing commas, whereas emitting output for languages like Perl where trailing commas don't matter is much easier.
Junio C HamanoRe: Feature request: --format=json
Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:
> On Sat, Apr 8, 2017 at 6:07 PM, Fred .Flintstone <eldmannen@gmail.com> wrote:
>> $ git log --format=json
>> [{
>> "commit": "64eabf050e315a4c7a11e0c05ca163be7cf9075e",
>> "tree": "b1e977800f40bbf6de906b1fe4f2de4b4b14f0fd",
>> "author": "Tux <tux@example.com> 1490981516 +0200",
>> "committer": "Tux <tux@example.com> 1490981516 +0200",
>> "message": "This is a test commit",
>> "long_message": "This explains in more details the commit"
>> }]
>>
>> This would make it easy to parse the output.
>
> The git-log command isn't plumbing that's meant for machines, but the
> git-for-each-ref command is what you're most likely looking for.They are apples and oranges. log is about traversing the history. "for-each-ref" is about listing the tips of refs. It doesn't and it shouldn't traverse the history.
The plumbing to use when you want to reimplement "git log" lookalike is "rev-list".
Re: Feature request: --format=json
So I did "git rev-list --all --pretty" and it looks like "git log". Which outputs a human-readable format.
However, if I want something more suitable for machine parsing, is there any way to get that output?
Example maybe I want another date format like ISO dates, or maybe a serializable format like JSON or CSV or something. Maybe I want more data than commit, auhor, date, subject and body?
On Mon, Apr 17, 2017 at 2:38 AM, Junio C Hamano <gitster@pobox.com> wrote:
> Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:
>
>> On Sat, Apr 8, 2017 at 6:07 PM, Fred .Flintstone <eldmannen@gmail.com> wrote:
>>> $ git log --format=json
>>> [{
>>> "commit": "64eabf050e315a4c7a11e0c05ca163be7cf9075e",
>>> "tree": "b1e977800f40bbf6de906b1fe4f2de4b4b14f0fd",
>>> "author": "Tux <tux@example.com> 1490981516 +0200",
>>> "committer": "Tux <tux@example.com> 1490981516 +0200",
>>> "message": "This is a test commit",
>>> "long_message": "This explains in more details the commit"
>>> }]
>>>
>>> This would make it easy to parse the output.
>>
>> The git-log command isn't plumbing that's meant for machines, but the
>> git-for-each-ref command is what you're most likely looking for.
>
> They are apples and oranges. log is about traversing the history.
> "for-each-ref" is about listing the tips of refs. It doesn't and it
> shouldn't traverse the history.
>
> The plumbing to use when you want to reimplement "git log" lookalike
> is "rev-list".
Duy NguyenRe: Feature request: --format=json
On Mon, Apr 17, 2017 at 7:44 PM, Fred .Flintstone <eldmannen@gmail.com> wrote:
> So I did "git rev-list --all --pretty" and it looks like "git log". > Which outputs a human-readable format. > > However, if I want something more suitable for machine parsing, is > there any way to get that output? > > Example maybe I want another date format like ISO dates, or maybe a > serializable format like JSON or CSV or something. Maybe I want more > data than commit, auhor, date, subject and body?
"git cat-file commit <commit-id>" should give you a machine-readable format of everything (it's the same thing that git-log parses and shows you; not counting options like --diff, --stat...). <commit-id> is from rev-list output (without --pretty, that's not for machine processing). You probably can use "git cat-file --batch" too, just pipe the whole rev-list output through it. You don't get to choose a convenient format this way though.
-- Duy
Re: Feature request: --format=json
So I would either have to do: git rev-list --all Then iterate over each line and do git-cat-file commit <commit-id>.
Or do: git rev-list --all | git cat-file --batch
If I do it in a batch, then it will be tricky to reliably parse since I don't know when the message body ends and when the next commit starts.
JSON output would have been very handy.
On Mon, Apr 17, 2017 at 2:59 PM, Duy Nguyen <pclouds@gmail.com> wrote:
> On Mon, Apr 17, 2017 at 7:44 PM, Fred .Flintstone <eldmannen@gmail.com> wrote: >> So I did "git rev-list --all --pretty" and it looks like "git log". >> Which outputs a human-readable format. >> >> However, if I want something more suitable for machine parsing, is >> there any way to get that output? >> >> Example maybe I want another date format like ISO dates, or maybe a >> serializable format like JSON or CSV or something. Maybe I want more >> data than commit, auhor, date, subject and body? > > "git cat-file commit <commit-id>" should give you a machine-readable > format of everything (it's the same thing that git-log parses and > shows you; not counting options like --diff, --stat...). <commit-id> > is from rev-list output (without --pretty, that's not for machine > processing). You probably can use "git cat-file --batch" too, just > pipe the whole rev-list output through it. You don't get to choose a > convenient format this way though. > -- > Duy
Junio C HamanoRe: Feature request: --format=json
"Fred .Flintstone" <eldmannen@gmail.com> writes:
> So I would either have to do: > git rev-list --all > Then iterate over each line and do git-cat-file commit <commit-id>. > > Or do: > git rev-list --all | git cat-file --batch > > If I do it in a batch, then it will be tricky to reliably parse since > I don't know when the message body ends and when the next commit > starts. > > JSON output would have been very handy.
I am somewhat puzzled. I thought that you were trying to come up with a way to produce JSON output and people are trying to help you by pointing out tools that you can use for that.
Re: Feature request: --format=json
Well the easiest way to work with that would be JSON. So the best would be if Git could output the data I want in JSON format. Then it would be easy for me to work with data.
With git rev-list and git-cat file, its not so easy to reliably parse that output.
On Tue, Apr 18, 2017 at 2:38 AM, Junio C Hamano <gitster@pobox.com> wrote:
> "Fred .Flintstone" <eldmannen@gmail.com> writes: > >> So I would either have to do: >> git rev-list --all >> Then iterate over each line and do git-cat-file commit <commit-id>. >> >> Or do: >> git rev-list --all | git cat-file --batch >> >> If I do it in a batch, then it will be tricky to reliably parse since >> I don't know when the message body ends and when the next commit >> starts. >> >> JSON output would have been very handy. > > I am somewhat puzzled. I thought that you were trying to come up > with a way to produce JSON output and people are trying to help you > by pointing out tools that you can use for that.
Samuel LijinRe: Feature request: --format=json
If for some reason your use case is so performance intensive that you can't just `git cat-file commit` every entry in `git rev-list --all` individually, then you can also pipe input into `git cat-file --batch` and read output as you pipe input in, which will give you a very simple mechanism for delimiting the cat-file output.
In any case, as developers, it's rare to have our job done for us. That's why we write code.
I'm sure people would be happy to help if you submitted patches to support --format=json.
On Tue, Apr 18, 2017 at 3:44 AM, Fred .Flintstone <eldmannen@gmail.com> wrote:
> Well the easiest way to work with that would be JSON. > So the best would be if Git could output the data I want in JSON format. > Then it would be easy for me to work with data. > > With git rev-list and git-cat file, its not so easy to reliably parse > that output. > > On Tue, Apr 18, 2017 at 2:38 AM, Junio C Hamano <gitster@pobox.com> wrote: >> "Fred .Flintstone" <eldmannen@gmail.com> writes: >> >>> So I would either have to do: >>> git rev-list --all >>> Then iterate over each line and do git-cat-file commit <commit-id>. >>> >>> Or do: >>> git rev-list --all | git cat-file --batch >>> >>> If I do it in a batch, then it will be tricky to reliably parse since >>> I don't know when the message body ends and when the next commit >>> starts. >>> >>> JSON output would have been very handy. >> >> I am somewhat puzzled. I thought that you were trying to come up >> with a way to produce JSON output and people are trying to help you >> by pointing out tools that you can use for that.
Re: Feature request: --format=json
But a repository or branch can have thousands of commits, so running `git commit-file <commit>` seems maybe not be a wide idea. But parsing `git cat-file --batch` is difficult, because there seems to be no reliable way to discern when a commit starts and ends.
I don't code in C though. A JSON formatter option would need a JSON library. But maybe there should be raised a discussion about JSON in Git if there are other people interested in this?
On Tue, Apr 18, 2017 at 11:39 AM, Samuel Lijin <sxlijin@gmail.com> wrote:
> If for some reason your use case is so performance intensive that you > can't just `git cat-file commit` every entry in `git rev-list --all` > individually, then you can also pipe input into `git cat-file --batch` > and read output as you pipe input in, which will give you a very > simple mechanism for delimiting the cat-file output. > > In any case, as developers, it's rare to have our job done for us. > That's why we write code. > > I'm sure people would be happy to help if you submitted patches to > support --format=json. > > On Tue, Apr 18, 2017 at 3:44 AM, Fred .Flintstone <eldmannen@gmail.com> wrote: >> Well the easiest way to work with that would be JSON. >> So the best would be if Git could output the data I want in JSON format. >> Then it would be easy for me to work with data. >> >> With git rev-list and git-cat file, its not so easy to reliably parse >> that output. >> >> On Tue, Apr 18, 2017 at 2:38 AM, Junio C Hamano <gitster@pobox.com> wrote: >>> "Fred .Flintstone" <eldmannen@gmail.com> writes: >>> >>>> So I would either have to do: >>>> git rev-list --all >>>> Then iterate over each line and do git-cat-file commit <commit-id>. >>>> >>>> Or do: >>>> git rev-list --all | git cat-file --batch >>>> >>>> If I do it in a batch, then it will be tricky to reliably parse since >>>> I don't know when the message body ends and when the next commit >>>> starts. >>>> >>>> JSON output would have been very handy. >>> >>> I am somewhat puzzled. I thought that you were trying to come up >>> with a way to produce JSON output and people are trying to help you >>> by pointing out tools that you can use for that.
Re: Feature request: --format=json
On 18 April 2017 at 10:44, Fred .Flintstone <eldmannen@gmail.com> wrote:
> Well the easiest way to work with that would be JSON. > So the best would be if Git could output the data I want in JSON format. > Then it would be easy for me to work with data. > > With git rev-list and git-cat file, its not so easy to reliably parse > that output.
Doesn't seem too hard to work with rev-list to me. As far as I can tell the following produces what you want. You need perl installed obviously, and the JSON::PP module is required, but should come bundled with recent perls.
git rev-list master --pretty=raw | perl -MJSON::PP=encode_json -ane'if(/^(\w+) (.*)/) { if ($1 eq "commit") { push @objs, $o if $o; $o={}; } $o->{$1} = $2; } else { $o->{text} .= $_; } END{ push @objs, $o if $o; for $o (@objs) { s/^ //mg, s/^\n// for $o->{text}; ($o->{message},$o->{long_message})= split /\n\n/, delete $o->{text}; } print JSON::PP->new->pretty->encode(\@objs);}'
You might consider an alternative approach than stating that working with JSON is "the easiest", especially to people who clearly are making do without it. :-)
A better argument might be that exposing data through a well defined and widely used and simple data format would trivially expand the range of projects that might interoperate with git or enhance the git ecosystem. For instance you could argue that having clean JSON output would make it easier to integrate into search engines and other indexing tools that already know how to speak JSON. Maybe a regular contributor on this list might agree with your arguments and make it happen.
Until then you can parse rev-list like the rest of us. :-)
cheers, Yves
Sebastian SchuberthRe: Feature request: --format=json
On 2017-04-17 14:44, Fred .Flintstone wrote:
> However, if I want something more suitable for machine parsing, is > there any way to get that output?
Instead of machine parsing, why not directly get what you want via libgit2 (or one of its language bindings), or jgit?
[1] https://github.com/libgit2/libgit2 [2] https://github.com/eclipse/jgit
-- Sebastian Schuberth
Duy NguyenRe: Feature request: --format=json
On Mon, Apr 24, 2017 at 3:33 PM, shawn wilson <ag4ve.us@gmail.com> wrote:
> Late to the party, but I too would also like json format output (mainly so I > could pipe stuff to jq instead of looking at the man page for which %thing > I'm looking for). That said, it's not at the PR level of want for me. > > OTOH, format=xml would be even more handy IMHO... Which I see has hit both > SO and this ml in the past. Either way /some/ machine output would be a good > thing :)
Personally I'd rather avoid linking to another library just for json/xml formatting. libgit2 would be a great place to have functionality like this and it looks like you don't even have to touch C [1] to do that.
[1] https://gist.github.com/m1el/42472327b4be382b02eb
-- Duy
Re: Feature request: --format=json
On Mon, Apr 24, 2017 at 5:28 AM, Duy Nguyen <pclouds@gmail.com> wrote:
> On Mon, Apr 24, 2017 at 3:33 PM, shawn wilson <ag4ve.us@gmail.com> wrote: >> Late to the party, but I too would also like json format output (mainly so I >> could pipe stuff to jq instead of looking at the man page for which %thing >> I'm looking for). That said, it's not at the PR level of want for me. >> >> OTOH, format=xml would be even more handy IMHO... Which I see has hit both >> SO and this ml in the past. Either way /some/ machine output would be a good >> thing :) > > Personally I'd rather avoid linking to another library just for > json/xml formatting. libgit2 would be a great place to have > functionality like this and it looks like you don't even have to touch > C [1] to do that. > > [1] https://gist.github.com/m1el/42472327b4be382b02eb
Heh, well, I guess if it's like that (simple), I don't really care :) I was under the impression (no idea why) that it was limited to C or java (and friends). Given this means I don't have to figure out someone's json structure or refer to an xsd, I'll live w/ this.