Re: [RFC PATCH 3/3] trace2: add a schema validator for trace2 events

From: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Date: Jun 12, 2019, 13:28 UTC
Message-ID: <87v9xb7x0q.fsf@evledraar.gmail.com>
In-Reply-To: <7475c6220895d96cdc7d25d6edea70e2f978526b.1560295286.git.steadmon@google.com>

On Wed, Jun 12 2019, Josh Steadmon wrote:

Show 5 quoted lines

> trace_schema_validator can be used to verify that trace2 event output
> conforms to the expectations set by the API documentation and codified
> in event_schema.json (or strict_schema.json). This allows us to build a
> regression test to verify that trace2 output does not change
> unexpectedly.

Does this actually work for you? As seen in my code at https://public-inbox.org/git/87zhnuwdkp.fsf@evledraar.gmail.com/ our test suite emits various lines of JSON that aren't even validly encoded, so I can't imagine we're passing any sort of proper parser validatior, let alone a schema validator.

In terms of implementation I think it would make sense to have a *.sh wrapper for this already, then we could test via prereqs if we have some of the existing validators (seems there's a list at https://json-schema.org/implementations.html) and e.g. run a dummy test against some small list of git commands, and then you could also pass it an env variable with "here's the trace file" so you could do:

    GIT_TRACE2_EVENT=/tmp/git.events prove <all testss> && VALIDATE_THIS=/tmp/git.events ./<that new test>.sh

And it would validate that file, if set.