From: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Date: Wed, 12 Jun 2019 13:28:53 GMT
Subject: Re: [RFC PATCH 3/3] trace2: add a schema validator for trace2 events
Message-ID: <87v9xb7x0q.fsf@evledraar.gmail.com>
In-Reply-To: <7475c6220895d96cdc7d25d6edea70e2f978526b.1560295286.git.steadmon@google.com>


On Wed, Jun 12 2019, Josh Steadmon wrote:

> trace_schema_validator can be used to verify that trace2 event output
> conforms to the expectations set by the API documentation and codified
> in event_schema.json (or strict_schema.json). This allows us to build a
> regression test to verify that trace2 output does not change
> unexpectedly.

Does this actually work for you? As seen in my code at
https://public-inbox.org/git/87zhnuwdkp.fsf@evledraar.gmail.com/ our
test suite emits various lines of JSON that aren't even validly encoded,
so I can't imagine we're passing any sort of proper parser validatior,
let alone a schema validator.

In terms of implementation I think it would make sense to have a *.sh
wrapper for this already, then we could test via prereqs if we have some
of the existing validators (seems there's a list at
https://json-schema.org/implementations.html) and e.g. run a dummy test
against some small list of git commands, and then you could also pass it
an env variable with "here's the trace file" so you could do:

    GIT_TRACE2_EVENT=/tmp/git.events prove <all testss> && VALIDATE_THIS=/tmp/git.events ./<that new test>.sh

And it would validate that file, if set.