git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH] Limit the size of the data block passed to SHA1_Update()

From
Atousa Pahlevan Duprat <atousa.p@gmail.com>
Date
Oct 30, 2015, 22:18 UTC
Message-ID
<1446243500-21580-1-git-send-email-apahlevan@ieee.org>
In-Reply-To
<CA+izobsBmYHHepYka795K2VnVLYBmN2tFqEyzSweMoS9gvuRVw@mail.gmail.com>
Some implementations of SHA_Updates have inherent limits
on the max chunk size. SHA1_MAX_BLOCK_SIZE can be defined
to set the max chunk size supported, if required.  This is
enabled for OSX CommonCrypto library and set to 1GiB.
---
 Makefile                     |  9 +++++++++
 cache.h                      |  7 ++++++-
 compat/apple-common-crypto.h |  4 ++++
 compat/sha1_chunked.c        | 20 ++++++++++++++++++++
 4 files changed, 39 insertions(+), 1 deletion(-)
 create mode 100644 compat/sha1_chunked.c
diff --git a/Makefile b/Makefile
index 04c2231..5955542 100644
--- a/Makefile
+++ b/Makefile
@@ -141,6 +141,10 @@ all::
 # Define PPC_SHA1 environment variable when running make to make use of
 # a bundled SHA1 routine optimized for PowerPC.
 #
+# Define SHA1_MAX_BLOCK_SIZE if your SSH1_Update() implementation can
+# hash only a limited amount of data in one call (e.g. APPLE_COMMON_CRYPTO
+# may want 'SHA1_MAX_BLOCK_SIZE=1024L*1024L*1024L' defined).
+#
 # Define NEEDS_CRYPTO_WITH_SSL if you need -lcrypto when using -lssl (Darwin).
 #
 # Define NEEDS_SSL_WITH_CRYPTO if you need -lssl when using -lcrypto (Darwin).
@@ -1346,6 +1350,7 @@ else
 ifdef APPLE_COMMON_CRYPTO
 	COMPAT_CFLAGS += -DCOMMON_DIGEST_FOR_OPENSSL
 	SHA1_HEADER = <CommonCrypto/CommonDigest.h>
+	SHA1_MAX_BLOCK_SIZE = 1024L*1024L*1024L
 else
 	SHA1_HEADER = <openssl/sha.h>
 	EXTLIBS += $(LIB_4_CRYPTO)
@@ -1353,6 +1358,10 @@ endif
 endif
 endif
 
+ifdef SHA1_MAX_BLOCK_SIZE
+	LIB_OBJS += compat/sha1_chunked.o
+	BASIC_CFLAGS += -DSHA1_MAX_BLOCK_SIZE="$(SHA1_MAX_BLOCK_SIZE)"
+endif
 ifdef NO_PERL_MAKEMAKER
 	export NO_PERL_MAKEMAKER
 endif
diff --git a/cache.h b/cache.h
index 79066e5..ec84b16 100644
--- a/cache.h
+++ b/cache.h
@@ -14,7 +14,12 @@
 #ifndef git_SHA_CTX
 #define git_SHA_CTX	SHA_CTX
 #define git_SHA1_Init	SHA1_Init
-#define git_SHA1_Update	SHA1_Update
+#ifdef SHA1_MAX_BLOCK_SIZE
+extern int SHA1_Update_Chunked(SHA_CTX *, const void *, size_t);
+#define git_SHA1_Update SHA1_Update_Chunked
+#else
+#define git_SHA1_Update SHA1_Update
+#endif
 #define git_SHA1_Final	SHA1_Final
 #endif
 
diff --git a/compat/apple-common-crypto.h b/compat/apple-common-crypto.h
index c8b9b0e..83668fd 100644
--- a/compat/apple-common-crypto.h
+++ b/compat/apple-common-crypto.h
@@ -16,6 +16,10 @@
 #undef TYPE_BOOL
 #endif
 
+#ifndef SHA1_MAX_BLOCK_SIZE
+#error "Using Apple Common Crypto library requires setting SHA1_MAX_BLOCK_SIZE"
+#endif
+
 #ifdef APPLE_LION_OR_NEWER
 #define git_CC_error_check(pattern, err) \
 	do { \
diff --git a/compat/sha1_chunked.c b/compat/sha1_chunked.c
new file mode 100644
index 0000000..4a8e4f7
--- /dev/null
+++ b/compat/sha1_chunked.c
@@ -0,0 +1,20 @@
+#include "cache.h"
+
+#ifdef SHA1_MAX_BLOCK_SIZE
+int git_SHA1_Update(SHA_CTX *c, const void *data, size_t len)
+{
+	size_t nr;
+	size_t total = 0;
+	char *cdata = (char*)data;
+	while(len > 0) {
+		nr = len;
+		if(nr > SHA1_MAX_BLOCK_SIZE)
+			nr = SHA1_MAX_BLOCK_SIZE;
+		SHA1_Update(c, cdata, nr);
+		total += nr;
+		cdata += nr;
+		len -= nr;
+	}
+	return total;
+}
+#endif
-- 
2.4.9 (Apple Git-60)
Previous: Junio C HamanoNext: Randall S. Becker
Message 24 of 27 in “git fsck failure on OS X with files >= 4 GiB”
  1. Rafael EspíndolaOct 28, 2015
  2. Filipe CabecinhasOct 29, 2015
  3. Rafael EspíndolaOct 29, 2015
  4. Filipe CabecinhasOct 29, 2015
  5. Atousa DupratOct 29, 2015
  6. Junio C HamanoOct 29, 2015
  7. Atousa DupratOct 30, 2015
  8. Limit the size of the data block passed to SHA1_Update()Atousa Pahlevan Duprat, Oct 30, 2015
  9. Junio C HamanoOct 30, 2015
  10. Atousa DupratNov 1, 2015
  11. Junio C HamanoNov 1, 2015
  12. Eric SunshineNov 1, 2015
  13. Limit the size of the data block passed to SHA1_Update()atousa.p@gmail.com, Nov 1, 2015
  14. Eric SunshineNov 1, 2015
  15. Junio C HamanoNov 1, 2015
  16. Atousa DupratNov 2, 2015
  17. Junio C HamanoNov 2, 2015
  18. 1/2 Limit the size of the data block passed to SHA1_Update()atousa.p@gmail.com, Nov 3, 2015
  19. Torsten BögershausenNov 3, 2015
  20. Limit the size of the data block passed to SHA1_Update()atousa.p@gmail.com, Nov 4, 2015
  21. Eric SunshineNov 4, 2015
  22. Atousa DupratNov 4, 2015
  23. Junio C HamanoNov 4, 2015
  24. Limit the size of the data block passed to SHA1_Update()Atousa Pahlevan Duprat, Oct 30, 2015
  25. Randall S. BeckerOct 30, 2015
  26. Junio C HamanoOct 31, 2015
  27. Atousa DupratNov 1, 2015

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.