git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v3 01/11] t: add tool to translate hash-related values

From
brian m. carlson <sandals@crustytoothpaste.net>
Date
Aug 29, 2018, 23:22 UTC
Message-ID
<20180829232211.GF432229@genre.crustytoothpaste.net>
In-Reply-To
<20180829040548.GB197350@aiede.svl.corp.google.com>
On Tue, Aug 28, 2018 at 09:05:48PM -0700, Jonathan Nieder wrote:
Show 45 quoted lines
> >                                Add two additional helpers,
> > test_oid_cache, which can be used to load data for test_oid from
> > standard input, and test_oid_init, which can be used to load certain
> > fixed values from lookup charts.  Check that these functions work in
> > t0000, as the rest of the testsuite will soon come to depend on them.
> >
> > Implement two basic lookup charts, one for common invalid or synthesized
> > object IDs, and one for various facts about the hash function in use.
> > Provide versions for both SHA-1 and SHA-256.
> 
> What do test_oid_cache and test_oid_init do?  How can I use them?
> 
> Judging from t0000-basic.sh, the idea looks something like
> 
> 	Add a test function helper, test_oid, that ...
> 
> 	test_oid allows looking up arbitrary information about an object format:
> 	the length of object ids, values of well known object ids, etc.  Before
> 	calling it, a test script must invoke test_oid_cache (either directly
> 	or indirectly through test_oid_init) to load the lookup charts.
> 
> 	See t0000 for an example, which also serves as a sanity-check that
> 	these functions work in preparation for using them in the rest of the
> 	test suite.
> 
> 	There are two basic lookup charts for now: oid-info/oid, with common
> 	invalid or synthesized object IDs; and oid-info/hash-info, with facts
> 	such as object id length about the formats in use.  The charts contain
> 	information about both SHA-1 and SHA-256.
> 
> 	So now you can update existing tests to be format-independent by (1)
> 	adding an invocation of test_oid_init to test setup and (2) replacing
> 	format dependencies with $(test_oid foo).
> 
> 	Since values are stored as shell variables, names used for lookup can
> 	only consist of shell identifier characters.  If this is a problem in
> 	the future, we can hash the names before use.
> 
> 	Improved-by: Eric Sunshine <sunshine@sunshineco.com>
> 
> Do these always use sha1 for now?  Ah, t0000 answers but it might be
> worth mentioning in the commit message, too:
> 
> 	test_set_hash allows setting which object format test_oid should look
> 	up information for, and test_detect_hash returns to the default format.

I'll expand the commit message. They do use SHA-1 for now, but I have a branch that makes them use SHA-256 instead.

Show 8 quoted lines
> [...]
> > --- /dev/null
> > +++ b/t/oid-info/hash-info
> > @@ -0,0 +1,8 @@
> > +rawsz sha1:20
> > +rawsz sha256:32
> 
> Can there be a README in this directory describing the files and format?
Sure, if you like.
Show 11 quoted lines
> [...]
> > --- a/t/t0000-basic.sh
> > +++ b/t/t0000-basic.sh
> > @@ -821,6 +821,41 @@ test_expect_success 'tests clean up even on failures' "
> >  	EOF
> >  "
> >  
> > +test_oid_init
> 
> Can this be wrapped in test_expect_success?  That way, if it fails or
> prints an error message then the usual test machinery would handle it.
Sure.
Show 6 quoted lines
> > +
> > +test_expect_success 'test_oid provides sane info by default' '
> > +	test_oid zero >actual &&
> > +	grep "^00*$" actual &&
> 
> nit: can save the reader some confusion by escaping the $.
Good point.
Show 5 quoted lines
> > +	rawsz="$(test_oid rawsz)" &&
> > +	hexsz="$(test_oid hexsz)" &&
> 
> optional: no need for these quotation marks --- a command substitution
> assigned to a shell variable is treated as if it were quoted.

That's good to know. I will honestly say that looking through the Git codebase and getting reviews on the list has taught me huge amounts about the intricacies of shell.

Show 9 quoted lines
> [...]
> > --- a/t/test-lib-functions.sh
> > +++ b/t/test-lib-functions.sh
> > @@ -1155,3 +1155,47 @@ depacketize () {
> [...]
> > +test_oid_cache () {
> > +	test -n "$test_hash_algo" || test_detect_hash
> 
> Should this use an uninterrupted &&-chain?
Yes.  Will fix.
> > +	while read _tag _rest
> 
> This appears to be the first use of this naming convention.  I wonder
> if we can use "local" instead.

We probably can. There was a discussion about this elsewhere, and we determined that it's probably safe, and if it's not, it should be relatively easy to back out.

Show 20 quoted lines
> > +		esac &&
> > +
> > +		_k="${_rest%:*}" &&
> > +		_v="${_rest#*:}" &&
> > +		{ echo "$_k" | egrep '^[a-z0-9]+$' >/dev/null ||
> > +			error 'bug in the test script: bad hash algorithm'; } &&
> > +		eval "test_oid_${_k}_$_tag=\"\$_v\"" || return 1
> 
> This is dense, so I'm having trouble taking it in at a glance.
> 
> I think the idea is
> 
> 		key=${rest%%:*} &&
> 		val=${rest#*:} &&
> 
> 		if ! expr "$key" : '[a-z0-9]*$' >/dev/null
> 		then
> 			error ...
> 		fi &&
> 		eval "test_oid_${key}_${tag}=\${val}"

Yes. I take it that you think that's easier to read, so I'll rewrite it that way. I will admit a tendency to write code that is more compact, sometimes (unintentionally) at the cost of readability. Thanks for providing a sanity check.

I agree that expr is probably better than the echo and egrep.
Show 22 quoted lines
> > +	done
> > +}
> > +
> > +test_oid () {
> > +	eval "
> > +		test -n \"\${test_oid_${test_hash_algo}_$1+set}\" &&
> > +		printf '%s' \"\${test_oid_${test_hash_algo}_$1}\"
> > +	"
> 
> I'm also having trouble taking this one in.  Maybe splitting into two
> evals would work?
> 
> 	var=test_oid_${test_hash_algo}_$1 &&
> 
> 	eval "test -n \"\${$var+set}\"" &&
> 	eval "printf '%s\n' \"\${$var}\""
> 
> What is the initial test meant to do?  Can this function get a
> documentation comment?  Are we relying on "test -n" to return a failing
> result if the variable is unset, or could the test be omitted (relying
> on "\${$var}" to evaluate to "" when the variable is unset)?  Should
> this call 'error' when the variable is unset?

Yes. The test -n will return false if the variable is unset, since ${$var+set} evaluates to nothing if the variable is unset and "set" if it is set. I will admit that I had to look this up in the shell documentation, so I'm not surprised that this is confusing at first glance.

Switching to error is probably better.
> Can t/README describe the new test helpers?

Sure. I wasn't aware that there was one to add this to, but now that you've pointed it out, it makes sense to add them.

-- 
brian m. carlson: Houston, Texas, US
OpenPGP: https://keybase.io/bk2204
Previous: Jonathan NiederNext: Derrick Stolee
Message 4 of 19 in “Hash-independent tests (part 3)”
  1. 00/11 Hash-independent tests (part 3)brian m. carlson, Aug 29, 2018
  2. 01/11 t: add tool to translate hash-related valuesbrian m. carlson, Aug 29, 2018
  3. Jonathan NiederAug 29, 2018
  4. brian m. carlsonAug 29, 2018
  5. Derrick StoleeAug 29, 2018
  6. brian m. carlsonAug 29, 2018
  7. 02/11 t0000: use hash translation tablebrian m. carlson, Aug 29, 2018
  8. 03/11 t0000: update tests for SHA-256brian m. carlson, Aug 29, 2018
  9. 05/11 t0027: make hash size independentbrian m. carlson, Aug 29, 2018
  10. Torsten BögershausenAug 31, 2018
  11. Eric SunshineAug 31, 2018
  12. brian m. carlsonSep 1, 2018
  13. 04/11 t0002: abstract away SHA-1 specific constantsbrian m. carlson, Aug 29, 2018
  14. 07/11 t1006: make hash size independentbrian m. carlson, Aug 29, 2018
  15. 06/11 t0064: make hash size independentbrian m. carlson, Aug 29, 2018
  16. 08/11 t1400: switch hard-coded object ID to variablebrian m. carlson, Aug 29, 2018
  17. 09/11 t1405: make hash size independentbrian m. carlson, Aug 29, 2018
  18. 10/11 t1406: make hash-size independentbrian m. carlson, Aug 29, 2018
  19. 11/11 t1407: make hash size independentbrian m. carlson, Aug 29, 2018

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.