Re: [RFC PATCH 3/3] trace2: add a schema validator for trace2 events

From: Jeff King <peff@peff.net>
Date: Jun 12, 2019, 19:18 UTC
Message-ID: <20190612191818.GB14949@sigill.intra.peff.net>
In-Reply-To: <20190612162341.GA42943@google.com>

On Wed, Jun 12, 2019 at 09:23:41AM -0700, Josh Steadmon wrote:

Show 5 quoted lines

> The problem with the existing validators is that they expect each file to be a
> complete JSON entity, whereas the trace output is one object per line. You can
> of course loop over the lines in a shell script, but in my testing this approach
> took multiple hours on the full test suite trace output, vs. 15 minutes for the
> implementation in this patch.

It seems like it should be easy to turn a sequence of entities into a single entity, with something like:

  echo '['
  sed 's/$/,/' <one-per-line
  echo ']'

You could even turn a sequence of files into a single entity (which might be even faster to validate, since it would be one invocation for the entire test suite) with something like:

  echo '{'
  for fn in $FILES; do
	echo "\"$fn\": "
	cat $fn
	echo ","
  done
  echo '}'

though I suspect the resulting error messages might not be as good.

Obviously neither of those is particularly robust if the individual JSON is not well-formed. But then, if we are mostly interested in testing whether it's well-formed and expect it to be in the normal case, that might be a good optimization.

I also wouldn't be surprised if "jq" could do this in a more robust way.

-Peff