From: Jeff King <peff@peff.net>
Date: Wed, 12 Jun 2019 19:18:19 GMT
Subject: Re: [RFC PATCH 3/3] trace2: add a schema validator for trace2 events
Message-ID: <20190612191818.GB14949@sigill.intra.peff.net>
In-Reply-To: <20190612162341.GA42943@google.com>

On Wed, Jun 12, 2019 at 09:23:41AM -0700, Josh Steadmon wrote:

> The problem with the existing validators is that they expect each file to be a
> complete JSON entity, whereas the trace output is one object per line. You can
> of course loop over the lines in a shell script, but in my testing this approach
> took multiple hours on the full test suite trace output, vs. 15 minutes for the
> implementation in this patch.

It seems like it should be easy to turn a sequence of entities into a
single entity, with something like:

  echo '['
  sed 's/$/,/' <one-per-line
  echo ']'

You could even turn a sequence of files into a single entity (which
might be even faster to validate, since it would be one invocation for
the entire test suite) with something like:

  echo '{'
  for fn in $FILES; do
	echo "\"$fn\": "
	cat $fn
	echo ","
  done
  echo '}'

though I suspect the resulting error messages might not be as good.

Obviously neither of those is particularly robust if the individual JSON
is not well-formed. But then, if we are mostly interested in testing
whether it's well-formed and expect it to be in the normal case, that
might be a good optimization.

I also wouldn't be surprised if "jq" could do this in a more robust way.

-Peff