git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 7/7] p0013: new test to compare SHA1DC vs OpenSSL

From
Johannes Schindelin <johannes.schindelin@gmx.de>
Date
Mar 24, 2017, 23:28 UTC
Message-ID
<4c2e3b2abc84a4b4eb7b41b424f76defda550646.1490397869.git.johannes.schindelin@gmx.de>
In-Reply-To
<cover.1490397869.git.johannes.schindelin@gmx.de>

To demonstrate the need for the core.enableSHA1DC knob, this test compares the performance of the SHA-1 algorithms with collision detection vs OpenSSL's (that does not detect attempted collision attacks).

The payload size of 300MB was actually not concocted from thin air, but is based on the massive Windows monorepo, whose index file weighs in with roughly that size (and which is SHA-1'ed upon every single read and write).

On this developer's machine, this comparison shows a hefty difference:
	0013.1: calculate SHA-1 for 300MB (SHA1DC)    3.03(0.03+0.17)
	0013.2: calculate SHA-1 for 300MB (OpenSSL)   0.58(0.06+0.16)

It is not only that ~6x slower performance is a pretty tall order, the absolute numbers themselves speak a very clear language: having to wait one second every time a file is `git add`ed is noticeable, but one can handwave it away. Having to wait six seconds (3 to read the index, a fraction of a millisecond to hash the new contents and update the in-memory index, then 3 seconds to write out the index) is outright annoying. And unnecessary, too: the content of the index is never crafted to cause SHA-1 collisions.

Obviously, this test requires that Git was built with the new DC_AND_OPENSSL_SHA1 make flag; it is skipped otherwise.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 t/perf/p0013-sha1dc.sh | 23 +++++++++++++++++++++++
 1 file changed, 23 insertions(+)
 create mode 100644 t/perf/p0013-sha1dc.sh
diff --git a/t/perf/p0013-sha1dc.sh b/t/perf/p0013-sha1dc.sh
new file mode 100644
index 00000000000..e08473ac969
--- /dev/null
+++ b/t/perf/p0013-sha1dc.sh
@@ -0,0 +1,23 @@
+#!/bin/sh
+
+test_description="Tests performance of SHA1DC vs OpenSSL"
+
+. ./perf-lib.sh
+
+test -n "$DC_AND_OPENSSL_SHA1" || {
+	skip_all='DC_AND_OPENSSL_SHA1 required for this test'
+	test_done
+}
+
+test_perf 'calculate SHA-1 for 300MB (SHA1DC)' '
+	dd if=/dev/zero bs=1M count=300 |
+	test-sha1
+'
+
+test_perf 'calculate SHA-1 for 300MB (OpenSSL)' '
+	dd if=/dev/zero bs=1M count=300 |
+	test-sha1 --disable-sha1dc
+'
+
+test_done
+
-- 
2.12.1.windows.1
Previous: Johannes SchindelinNext: Junio C Hamano
Message 12 of 23 in “PREVIEW: Introduce DC_AND_OPENSSL_SHA1 make flag”
  1. 0/7 PREVIEW: Introduce DC_AND_OPENSSL_SHA1 make flagJohannes Schindelin, Mar 24, 2017
  2. 1/7 sha1dc: safeguard against outside definitions of BIGENDIANJohannes Schindelin, Mar 24, 2017
  3. 2/7 Makefile: optionally compile with both SHA1DC and SHA1_OPENSSLJohannes Schindelin, Mar 24, 2017
  4. Ævar Arnfjörð BjarmasonMar 25, 2017
  5. Junio C HamanoMar 30, 2017
  6. Junio C HamanoMar 30, 2017
  7. Johannes SchindelinApr 18, 2017
  8. 3/7 config: add the core.enablesha1dc settingJohannes Schindelin, Mar 24, 2017
  9. 4/7 t0013: do not skip the entire file wholesale without DC_SHA1Johannes Schindelin, Mar 24, 2017
  10. 5/7 t0013: test DC_AND_OPENSSL_SHA1, tooJohannes Schindelin, Mar 24, 2017
  11. 6/7 mingw: enable DC_AND_OPENSSL_SHA1 by defaultJohannes Schindelin, Mar 24, 2017
  12. 7/7 p0013: new test to compare SHA1DC vs OpenSSLJohannes Schindelin, Mar 24, 2017
  13. Junio C HamanoMar 25, 2017
  14. Junio C HamanoMar 25, 2017
  15. Jeff KingMar 26, 2017
  16. Junio C HamanoMar 26, 2017
  17. Jeff KingMar 27, 2017
  18. Junio C HamanoMar 27, 2017
  19. Jeff KingMar 27, 2017
  20. Junio C HamanoMar 27, 2017
  21. Johannes SchindelinMar 29, 2017
  22. Junio C HamanoMar 30, 2017
  23. Johannes SchindelinApr 18, 2017

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.