{"thread":{"id":"57779","subject":"[PATCH 1/5] Makefile: create and use sections for \"define\" flag listing","startedAt":"2022-04-22T09:53:30Z","lastAt":"2022-11-07T21:24:02Z","messageCount":55,"participants":["Ævar Arnfjörð Bjarmason","Junio C Hamano","Eric Sunshine","brian m. carlson"],"isPatch":true,"patchVersion":1,"patchTotal":5},"messages":[{"id":"454244","messageId":"patch-1.5-f799d30e82e-20220422T094624Z-avarab@gmail.com","threadId":"57779","inReplyTo":"cover-0.5-00000000000-20220422T094624Z-avarab@gmail.com","subject":"[PATCH 1/5] Makefile: create and use sections for \"define\" flag listing","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-04-22T09:53:12Z","receivedAt":"2022-04-22T09:53:30Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Since the \"Define ...\" template of comments at the top of the Makefile\nwas started in 5bdac8b3269 ([PATCH] Improve the compilation-time\nsettings interface, 2005-07-29) we've had a lot more flags added,\nincluding flags that come in \"groups\". Not having any obvious\nstructure to the >500 line comment at the top of the Makefile has made\nit hard to follow.\n\nThis change is almost entirely a move-only change, the two paragraphs\nat the start of the first two sections are new, and so are the added\nsections themselves, but other than that no lines are changed, only\nmoved.\n\nWe now list Makefile-only flags at the start, followed by stand-alone\nflags, and then cover \"optional library\" flags in their respective\ngroups, followed by SHA-1 and SHA-256 flags, and finally\nDEVELOPER-specific flags.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n Makefile | 212 +++++++++++++++++++++++++++++++------------------------\n 1 file changed, 118 insertions(+), 94 deletions(-)\n\ndiff --git a/Makefile b/Makefile\nindex f8bccfab5e9..246f6729022 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -4,8 +4,20 @@ all::\n # Import tree-wide shared Makefile behavior and libraries\n include shared.mak\n \n+# == Makefile defines ==\n+#\n+# These defines change the behavior of the Makefile itself, but have\n+# no impact on what it builds:\n+#\n # Define V=1 to have a more verbose compile.\n #\n+# == Portability and optional library defines ==\n+#\n+# These defines indicate what Git can expect from the OS, what\n+# libraries are available etc. Much of this is auto-detected in\n+# config.mak.uname, or in configure.ac when using the optional \"make\n+# configure && ./configure\" (see INSTALL).\n+#\n # Define SHELL_PATH to a POSIX shell if your /bin/sh is broken.\n #\n # Define SANE_TOOL_PATH to a colon-separated list of paths to prepend\n@@ -30,68 +42,8 @@ include shared.mak\n #\n # Define NO_OPENSSL environment variable if you do not have OpenSSL.\n #\n-# Define USE_LIBPCRE if you have and want to use libpcre. Various\n-# commands such as log and grep offer runtime options to use\n-# Perl-compatible regular expressions instead of standard or extended\n-# POSIX regular expressions.\n-#\n-# Only libpcre version 2 is supported. USE_LIBPCRE2 is a synonym for\n-# USE_LIBPCRE, support for the old USE_LIBPCRE1 has been removed.\n-#\n-# Define LIBPCREDIR=/foo/bar if your PCRE header and library files are\n-# in /foo/bar/include and /foo/bar/lib directories.\n-#\n # Define HAVE_ALLOCA_H if you have working alloca(3) defined in that header.\n #\n-# Define NO_CURL if you do not have libcurl installed. git-http-fetch and\n-# git-http-push are not built, and you cannot use http:// and https://\n-# transports (neither smart nor dumb).\n-#\n-# Define CURLDIR=/foo/bar if your curl header and library files are in\n-# /foo/bar/include and /foo/bar/lib directories.\n-#\n-# Define CURL_CONFIG to curl's configuration program that prints information\n-# about the library (e.g., its version number). The default is 'curl-config'.\n-#\n-# Define CURL_LDFLAGS to specify flags that you need to link when using libcurl,\n-# if you do not want to rely on the libraries provided by CURL_CONFIG. The\n-# default value is a result of `curl-config --libs`. An example value for\n-# CURL_LDFLAGS is as follows:\n-#\n-# CURL_LDFLAGS=-lcurl\n-#\n-# Define NO_EXPAT if you do not have expat installed. git-http-push is\n-# not built, and you cannot push using http:// and https:// transports (dumb).\n-#\n-# Define EXPATDIR=/foo/bar if your expat header and library files are in\n-# /foo/bar/include and /foo/bar/lib directories.\n-#\n-# Define EXPAT_NEEDS_XMLPARSE_H if you have an old version of expat (e.g.,\n-# 1.1 or 1.2) that provides xmlparse.h instead of expat.h.\n-#\n-# Define NO_GETTEXT if you don't want Git output to be translated.\n-# A translated Git requires GNU libintl or another gettext implementation,\n-# plus libintl-perl at runtime.\n-#\n-# Define USE_GETTEXT_SCHEME and set it to 'fallthrough', if you don't trust\n-# the installed gettext translation of the shell scripts output.\n-#\n-# Define HAVE_LIBCHARSET_H if you haven't set NO_GETTEXT and you can't\n-# trust the langinfo.h's nl_langinfo(CODESET) function to return the\n-# current character set. GNU and Solaris have a nl_langinfo(CODESET),\n-# FreeBSD can use either, but MinGW and some others need to use\n-# libcharset.h's locale_charset() instead.\n-#\n-# Define CHARSET_LIB to the library you need to link with in order to\n-# use locale_charset() function. On some platforms this needs to set to\n-# -lcharset, on others to -liconv .\n-#\n-# Define LIBC_CONTAINS_LIBINTL if your gettext implementation doesn't\n-# need -lintl when linking.\n-#\n-# Define NO_MSGFMT_EXTENDED_OPTIONS if your implementation of msgfmt\n-# doesn't support GNU extensions like --check and --statistics\n-#\n # Define HAVE_PATHS_H if you have paths.h and want to use the default PATH\n # it specifies.\n #\n@@ -152,40 +104,6 @@ include shared.mak\n # and do not want to use Apple's CommonCrypto library. This allows you\n # to provide your own OpenSSL library, for example from MacPorts.\n #\n-# Define BLK_SHA1 environment variable to make use of the bundled\n-# optimized C SHA1 routine.\n-#\n-# Define PPC_SHA1 environment variable when running make to make use of\n-# a bundled SHA1 routine optimized for PowerPC.\n-#\n-# Define DC_SHA1 to unconditionally enable the collision-detecting sha1\n-# algorithm. This is slower, but may detect attempted collision attacks.\n-# Takes priority over other *_SHA1 knobs.\n-#\n-# Define DC_SHA1_EXTERNAL in addition to DC_SHA1 if you want to build / link\n-# git with the external SHA1 collision-detect library.\n-# Without this option, i.e. the default behavior is to build git with its\n-# own built-in code (or submodule).\n-#\n-# Define DC_SHA1_SUBMODULE in addition to DC_SHA1 to use the\n-# sha1collisiondetection shipped as a submodule instead of the\n-# non-submodule copy in sha1dc/. This is an experimental option used\n-# by the git project to migrate to using sha1collisiondetection as a\n-# submodule.\n-#\n-# Define OPENSSL_SHA1 environment variable when running make to link\n-# with the SHA1 routine from openssl library.\n-#\n-# Define SHA1_MAX_BLOCK_SIZE to limit the amount of data that will be hashed\n-# in one call to the platform's SHA1_Update(). e.g. APPLE_COMMON_CRYPTO\n-# wants 'SHA1_MAX_BLOCK_SIZE=1024L*1024L*1024L' defined.\n-#\n-# Define BLK_SHA256 to use the built-in SHA-256 routines.\n-#\n-# Define GCRYPT_SHA256 to use the SHA-256 routines in libgcrypt.\n-#\n-# Define OPENSSL_SHA256 to use the SHA-256 routines in OpenSSL.\n-#\n # Define NEEDS_CRYPTO_WITH_SSL if you need -lcrypto when using -lssl (Darwin).\n #\n # Define NEEDS_SSL_WITH_CRYPTO if you need -lssl when using -lcrypto (Darwin).\n@@ -480,6 +398,112 @@ include shared.mak\n # `compat/fsmonitor/fsm-listen-.c` that implements the\n # `fsm_listen__*()` routines.\n #\n+# === Optional library: libintl ===\n+#\n+# Define NO_GETTEXT if you don't want Git output to be translated.\n+# A translated Git requires GNU libintl or another gettext implementation,\n+# plus libintl-perl at runtime.\n+#\n+# Define USE_GETTEXT_SCHEME and set it to 'fallthrough', if you don't trust\n+# the installed gettext translation of the shell scripts output.\n+#\n+# Define HAVE_LIBCHARSET_H if you haven't set NO_GETTEXT and you can't\n+# trust the langinfo.h's nl_langinfo(CODESET) function to return the\n+# current character set. GNU and Solaris have a nl_langinfo(CODESET),\n+# FreeBSD can use either, but MinGW and some others need to use\n+# libcharset.h's locale_charset() instead.\n+#\n+# Define CHARSET_LIB to the library you need to link with in order to\n+# use locale_charset() function. On some platforms this needs to set to\n+# -lcharset, on others to -liconv .\n+#\n+# Define LIBC_CONTAINS_LIBINTL if your gettext implementation doesn't\n+# need -lintl when linking.\n+#\n+# Define NO_MSGFMT_EXTENDED_OPTIONS if your implementation of msgfmt\n+# doesn't support GNU extensions like --check and --statistics\n+#\n+# === Optional library: libexpat ===\n+#\n+# Define NO_EXPAT if you do not have expat installed. git-http-push is\n+# not built, and you cannot push using http:// and https:// transports (dumb).\n+#\n+# Define EXPATDIR=/foo/bar if your expat header and library files are in\n+# /foo/bar/include and /foo/bar/lib directories.\n+#\n+# Define EXPAT_NEEDS_XMLPARSE_H if you have an old version of expat (e.g.,\n+# 1.1 or 1.2) that provides xmlparse.h instead of expat.h.\n+\n+# === Optional library: libcurl ===\n+#\n+# Define NO_CURL if you do not have libcurl installed. git-http-fetch and\n+# git-http-push are not built, and you cannot use http:// and https://\n+# transports (neither smart nor dumb).\n+#\n+# Define CURLDIR=/foo/bar if your curl header and library files are in\n+# /foo/bar/include and /foo/bar/lib directories.\n+#\n+# Define CURL_CONFIG to curl's configuration program that prints information\n+# about the library (e.g., its version number). The default is 'curl-config'.\n+#\n+# Define CURL_LDFLAGS to specify flags that you need to link when using libcurl,\n+# if you do not want to rely on the libraries provided by CURL_CONFIG. The\n+# default value is a result of `curl-config --libs`. An example value for\n+# CURL_LDFLAGS is as follows:\n+#\n+# CURL_LDFLAGS=-lcurl\n+#\n+# === Optional library: libpcre2 ===\n+#\n+# Define USE_LIBPCRE if you have and want to use libpcre. Various\n+# commands such as log and grep offer runtime options to use\n+# Perl-compatible regular expressions instead of standard or extended\n+# POSIX regular expressions.\n+#\n+# Only libpcre version 2 is supported. USE_LIBPCRE2 is a synonym for\n+# USE_LIBPCRE, support for the old USE_LIBPCRE1 has been removed.\n+#\n+# Define LIBPCREDIR=/foo/bar if your PCRE header and library files are\n+# in /foo/bar/include and /foo/bar/lib directories.\n+#\n+# == SHA-1 and SHA-256 defines ==\n+#\n+# Define BLK_SHA1 environment variable to make use of the bundled\n+# optimized C SHA1 routine.\n+#\n+# Define PPC_SHA1 environment variable when running make to make use of\n+# a bundled SHA1 routine optimized for PowerPC.\n+#\n+# Define DC_SHA1 to unconditionally enable the collision-detecting sha1\n+# algorithm. This is slower, but may detect attempted collision attacks.\n+# Takes priority over other *_SHA1 knobs.\n+#\n+# Define DC_SHA1_EXTERNAL in addition to DC_SHA1 if you want to build / link\n+# git with the external SHA1 collision-detect library.\n+# Without this option, i.e. the default behavior is to build git with its\n+# own built-in code (or submodule).\n+#\n+# Define DC_SHA1_SUBMODULE in addition to DC_SHA1 to use the\n+# sha1collisiondetection shipped as a submodule instead of the\n+# non-submodule copy in sha1dc/. This is an experimental option used\n+# by the git project to migrate to using sha1collisiondetection as a\n+# submodule.\n+#\n+# Define OPENSSL_SHA1 environment variable when running make to link\n+# with the SHA1 routine from openssl library.\n+#\n+# Define SHA1_MAX_BLOCK_SIZE to limit the amount of data that will be hashed\n+# in one call to the platform's SHA1_Update(). e.g. APPLE_COMMON_CRYPTO\n+# wants 'SHA1_MAX_BLOCK_SIZE=1024L*1024L*1024L' defined.\n+#\n+# Define BLK_SHA256 to use the built-in SHA-256 routines.\n+#\n+# Define GCRYPT_SHA256 to use the SHA-256 routines in libgcrypt.\n+#\n+# Define OPENSSL_SHA256 to use the SHA-256 routines in OpenSSL.\n+#\n+# == DEVELOPER defines ==\n+#\n # Define DEVELOPER to enable more compiler warnings. Compiler version\n # and family are auto detected, but could be overridden by defining\n # COMPILER_FEATURES (see config.mak.dev). You can still set\n-- \n2.36.0.879.g56a83971f3f\n\n"},{"id":"454245","messageId":"cover-0.5-00000000000-20220422T094624Z-avarab@gmail.com","threadId":"57779","inReplyTo":null,"subject":"[PATCH 0/5] core: update our SHA-1 docs, use sha1collisiondetection on OSX too","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-04-22T09:53:11Z","receivedAt":"2022-04-22T09:53:35Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"I wasn't able to find any on-list references to it being intentional,\nbut it appears that while we made the sha1collisiondetection variant\nof SHA-1 the default in early 2017 we've never updated the OSX builds\nto do likewise.\n\nI don't know what various git packages for OSX to, but our vanilla OSX\ndistribution definitely uses Apple Common Crypto, and won't detect the\nhttps://shattered.io attack.\n\nThis series changes that, and while doing so in 2/5 updates our\ndocumentation and Makefile interface for the SHA-1 selection. Our\nINSTALL file was still claiming we used OpenSSL's SHA-1 by default.\n\nThen since we'd made sha1collisiondetection the default we hadn't\nchanged the code's default fallback to be that, it was still\nblock-sha1. Now our fallback behavior is \"error\" instead, which makes\nit less likely that we'll get some foot-gun like the \"OSX not using\nsha1collisiondetection\" again.\n\nThe 4/5 and 5/5 then remove the PPC_SHA1 implementation. I submitted\nthis before as [1], and the range-diff is to that submission (it\nwasn't picked up). I think it makes sense as part of this general\nSHA-1 cleanup.\n\n1. https://lore.kernel.org/git/patch-1.1-05dcdca3877-20220319T005952Z-avarab@gmail.com/\n\nÆvar Arnfjörð Bjarmason (5):\n Makefile: create and use sections for \"define\" flag listing\n Makefile: really use and document sha1collisiondetection by default\n Makefile: rephrase the discussion of *_SHA1 knobs\n Makefile + hash.h: remove PPC_SHA1 implementation\n Makefile: use $(OBJECTS) instead of $(C_OBJ)\n\n INSTALL | 11 +-\n Makefile | 301 ++++++++++++++++------------\n configure.ac | 3 -\n contrib/buildsystems/CMakeLists.txt | 3 +-\n hash.h | 12 +-\n ppc/sha1.c | 72 -------\n ppc/sha1.h | 25 ---\n ppc/sha1ppc.S | 224 ---------------------\n t/t0013-sha1dc.sh | 4 +-\n 9 files changed, 191 insertions(+), 464 deletions(-)\n delete mode 100644 ppc/sha1.c\n delete mode 100644 ppc/sha1.h\n delete mode 100644 ppc/sha1ppc.S\n\nRange-diff:\n-: ----------- > 1: f799d30e82e Makefile: create and use sections for \"define\" flag listing\n-: ----------- > 2: 5ffb68dc77b Makefile: really use and document sha1collisiondetection by default\n-: ----------- > 3: d559e5212bc Makefile: rephrase the discussion of *_SHA1 knobs\n1: 3a8caf62137 ! 4: 4b2d0b7b51a ppc: remove custom SHA-1 implementation\n @@ Metadata\n Author: Ævar Arnfjörð Bjarmason \n \n ## Commit message ##\n - ppc: remove custom SHA-1 implementation\n + Makefile + hash.h: remove PPC_SHA1 implementation\n \n Remove the PPC_SHA1 implementation added in a6ef3518f9a ([PATCH] PPC\n assembly implementation of SHA1, 2005-04-22). When this was added\n Apple consumer hardware used the PPC architecture, and the\n implementation was intended to improve SHA-1 speed there.\n \n - Since it was added we've moved to DC_SHA1 by default, and anyone\n - wanting hard-rolled non-DC SHA-1 implementation can use OpenSSL's via\n - the OPENSSL_SHA1 knob.\n + Since it was added we've moved to using sha1collisiondetection by\n + default, and anyone wanting hard-rolled non-DC SHA-1 implementation\n + can use OpenSSL's via the OPENSSL_SHA1 knob.\n \n - I'm unsure if this was ever supposed to work on 64-bit PPC. It clearly\n - originally targeted 32 bit PPC, but there's some mailing list\n - references to this being tried on G5 (PPC 970). I can't get it to do\n - anything but segfault on the BE POWER8 machine in the GCC compile\n - farm. Anyone caring about speed on PPC these days is likely to be\n - using IBM's POWER, not PPC 970.\n + Furthermore this doesn't run on the modern PPC processors which anyone\n + who's concerned about performance on PPC is likely to be using these\n + days, i.e. the IBM POWER series. It originally originally targeted 32\n + bit PPC, but there's some mailing list references to this being tried\n + on G5 (PPC 970).\n \n - There have been proposals to entirely remove non-DC_SHA1\n + I can't get it to do anything but segfault on both the BE and LE POWER\n + machines in the GCC compile farm.\n +\n + There have been proposals to entirely remove non-sha1collisiondetection\n implementations from the tree[1]. I think per [2] that would be a bit\n overzealous. I.e. there are various set-ups git's speed is going to be\n more important than the relatively implausible SHA-1 collision attack,\n or where such attacks are entirely mitigated by other means (e.g. by\n incoming objects being checked with DC_SHA1).\n \n - The main reason for doing so at this point is to simplify follow-up\n - Makefile change. Since PPC_SHA1 included the only in-tree *.S assembly\n - file we needed to keep around special support for building objects\n - from it. By getting rid of it we know we'll always build *.o from *.c\n - files, which makes the build process simpler.\n + But that really doesn't apply to PPC_SHA1 in particular, which seems\n + to have outlived its usefulness.\n +\n + As this gets rid of the only in-tree *.S assembly file we can remove\n + the small bits of logic from the Makefile needed to build objects\n + from *.S (as opposed to *.c)\n \n - As an aside the code being removed here was also throwing warnings\n - with the \"-pedantic\" flag, but let's remove it instead of fixing it,\n - as 544d93bc3b4 (block-sha1: remove use of obsolete x86 assembly,\n - 2022-03-10) did for block-sha1/*.\n + The code being removed here was also throwing warnings with the\n + \"-pedantic\" flag, it could have been fixed as 544d93bc3b4 (block-sha1:\n + remove use of obsolete x86 assembly, 2022-03-10) did for block-sha1/*,\n + but as noted above let's remove it instead.\n \n 1. https://lore.kernel.org/git/20200223223758.120941-1-mh@glandium.org/\n 2. https://lore.kernel.org/git/20200224044732.GK1018190@coredump.intra.peff.net/\n \n Signed-off-by: Ævar Arnfjörð Bjarmason \n \n - ## INSTALL ##\n -@@ INSTALL: Issues of note:\n - \n - \t By default, git uses OpenSSL for SHA1 but it will use its own\n - \t library (inspired by Mozilla's) with either NO_OPENSSL or\n --\t BLK_SHA1. Also included is a version optimized for PowerPC\n --\t (PPC_SHA1).\n -+\t BLK_SHA1.\n - \n - \t- \"libcurl\" library is used for fetching and pushing\n - \t repositories over http:// or https://, as well as by\n -\n ## Makefile ##\n @@ Makefile: include shared.mak\n - # Define BLK_SHA1 environment variable to make use of the bundled\n - # optimized C SHA1 routine.\n + # Define OPENSSL_SHA1 to link to the the SHA-1 routines from\n + # the OpenSSL library.\n #\n --# Define PPC_SHA1 environment variable when running make to make use of\n --# a bundled SHA1 routine optimized for PowerPC.\n +-# Define PPC_SHA1 to make use of optimized (in assembly)\n +-# PowerPC SHA-1 routines.\n -#\n - # Define DC_SHA1 to unconditionally enable the collision-detecting sha1\n - # algorithm. This is slower, but may detect attempted collision attacks.\n - # Takes priority over other *_SHA1 knobs.\n -@@ Makefile: ifdef OPENSSL_SHA1\n - \tEXTLIBS += $(LIB_4_CRYPTO)\n - \tBASIC_CFLAGS += -DSHA1_OPENSSL\n - else\n + # Define APPLE_SHA1 to use Apple's CommonCrypto SHA-1 routines on\n + # Darwin/Mac OS X.\n + #\n +@@ Makefile: endif\n + ifdef DC_SHA1\n + $(error the DC_SHA1 flag is no longer used, and has become the default. Adjust your build scripts accordingly)\n + endif\n +ifdef PPC_SHA1\n -+$(error PPC_SHA1 has been removed! Use DC_SHA1 instead, which is the default)\n ++$(error the PPC_SHA1 flag has been removed along with the PowerPC-specific SHA-1 implementation.)\n +endif\n - ifdef BLK_SHA1\n + ifndef NO_DC_SHA1\n +-\tifneq ($(OPENSSL_SHA1)$(BLK_SHA1)$(PPC_SHA1)$(APPLE_SHA1),)\n ++\tifneq ($(OPENSSL_SHA1)$(BLK_SHA1)$(APPLE_SHA1),)\n + $(error no other *_SHA1 option can be defined unless NO_DC_SHA1 is defined)\n + \tendif\n + \tLIB_OBJS += sha1dc_git.o\n +@@ Makefile: ifdef BLK_SHA1\n \tLIB_OBJS += block-sha1/sha1.o\n \tBASIC_CFLAGS += -DSHA1_BLK\n else\n @@ Makefile: ifdef OPENSSL_SHA1\n -\tLIB_OBJS += ppc/sha1.o ppc/sha1ppc.o\n -\tBASIC_CFLAGS += -DSHA1_PPC\n -else\n - ifdef APPLE_COMMON_CRYPTO\n + ifdef APPLE_SHA1\n \tCOMPAT_CFLAGS += -DCOMMON_DIGEST_FOR_OPENSSL\n \tBASIC_CFLAGS += -DSHA1_APPLE\n @@ Makefile: endif\n @@ configure.ac: AC_MSG_NOTICE([CHECKS for site configuration])\n \n ## hash.h ##\n @@\n - #include \"git-compat-util.h\"\n - #include \"repository.h\"\n \n --#if defined(SHA1_PPC)\n + #if !defined(NO_SHA1_DC)\n + #include \"sha1dc_git.h\"\n +-#elif defined(SHA1_PPC)\n -#include \"ppc/sha1.h\"\n --#elif defined(SHA1_APPLE)\n -+#if defined(SHA1_APPLE)\n + #elif defined(SHA1_APPLE)\n #include \n #elif defined(SHA1_OPENSSL)\n - #include \n @@\n * platform's underlying implementation of SHA-1; could be OpenSSL,\n * blk_SHA, Apple CommonCrypto, etc... Note that the relevant\n-: ----------- > 5: 0575faebc30 Makefile: use $(OBJECTS) instead of $(C_OBJ)\n-- \n2.36.0.879.g56a83971f3f\n\n"},{"id":"454246","messageId":"patch-3.5-d559e5212bc-20220422T094624Z-avarab@gmail.com","threadId":"57779","inReplyTo":"cover-0.5-00000000000-20220422T094624Z-avarab@gmail.com","subject":"[PATCH 3/5] Makefile: rephrase the discussion of *_SHA1 knobs","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-04-22T09:53:14Z","receivedAt":"2022-04-22T09:53:47Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"In the preceding commit the discussion of the *_SHA1 knobs was left\nas-is to benefit from a smaller diff, but since we're changing these\nlet's use the same phrasing we use for most other knobs. E.g. \"define\nX\", not \"define X environment variable\", and get rid of the \"when\nrunning make to link with\" entirely.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n Makefile | 12 ++++++------\n 1 file changed, 6 insertions(+), 6 deletions(-)\n\ndiff --git a/Makefile b/Makefile\nindex 12bb9edd70a..d317c6aba06 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -506,14 +506,14 @@ include shared.mak\n # To use them you must define NO_DC_SHA1 and one of the *_SHA1\n # variables below:\n #\n-# Define BLK_SHA1 environment variable to make use of the bundled\n-# optimized C SHA1 routine.\n+# Define BLK_SHA1 to make use of optimized C SHA-1 routines bundled\n+# with git (in the block-sha1/ directory).\n #\n-# Define OPENSSL_SHA1 environment variable when running make to link\n-# with the SHA1 routine from openssl library.\n+# Define OPENSSL_SHA1 to link to the the SHA-1 routines from\n+# the OpenSSL library.\n #\n-# Define PPC_SHA1 environment variable when running make to make use of\n-# a bundled SHA1 routine optimized for PowerPC.\n+# Define PPC_SHA1 to make use of optimized (in assembly)\n+# PowerPC SHA-1 routines.\n #\n # Define APPLE_SHA1 to use Apple's CommonCrypto SHA-1 routines on\n # Darwin/Mac OS X.\n-- \n2.36.0.879.g56a83971f3f\n\n"},{"id":"454247","messageId":"patch-5.5-0575faebc30-20220422T094624Z-avarab@gmail.com","threadId":"57779","inReplyTo":"cover-0.5-00000000000-20220422T094624Z-avarab@gmail.com","subject":"[PATCH 5/5] Makefile: use $(OBJECTS) instead of $(C_OBJ)","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-04-22T09:53:16Z","receivedAt":"2022-04-22T09:53:47Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"In the preceding commit $(C_OBJ) added in c373991375a (Makefile: list\ngenerated object files in OBJECTS, 2010-01-26) became synonymous with\n$(OBJECTS). Let's avoid the indirection and use the $(OBJECTS)\nvariable directly instead.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n Makefile | 8 +++-----\n 1 file changed, 3 insertions(+), 5 deletions(-)\n\ndiff --git a/Makefile b/Makefile\nindex 6f8272ae4a4..ff6bab134d2 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -2616,9 +2616,7 @@ missing_compdb_dir =\n compdb_args =\n endif\n \n-C_OBJ = $(OBJECTS)\n-\n-$(C_OBJ): %.o: %.c GIT-CFLAGS $(missing_dep_dirs) $(missing_compdb_dir)\n+$(OBJECTS): %.o: %.c GIT-CFLAGS $(missing_dep_dirs) $(missing_compdb_dir)\n \t$(QUIET_CC)$(CC) -o $*.o -c $(dep_args) $(compdb_args) $(ALL_CFLAGS) $(EXTRA_CPPFLAGS) $<\n \n %.s: %.c GIT-CFLAGS FORCE\n@@ -2764,7 +2762,7 @@ XGETTEXT_FLAGS_SH = $(XGETTEXT_FLAGS) --language=Shell \\\n \t--keyword=gettextln --keyword=eval_gettextln\n XGETTEXT_FLAGS_PERL = $(XGETTEXT_FLAGS) --language=Perl \\\n \t--keyword=__ --keyword=N__ --keyword=\"__n:1,2\"\n-LOCALIZED_C = $(C_OBJ:o=c) $(LIB_H) $(GENERATED_H)\n+LOCALIZED_C = $(OBJECTS:o=c) $(LIB_H) $(GENERATED_H)\n LOCALIZED_SH = $(SCRIPT_SH)\n LOCALIZED_SH += git-sh-setup.sh\n LOCALIZED_PERL = $(SCRIPT_PERL)\n@@ -3014,7 +3012,7 @@ t/helper/test-%$X: t/helper/test-%.o GIT-LDFLAGS $(GITLIBS) $(REFTABLE_TEST_LIB)\n check-sha1:: t/helper/test-tool$X\n \tt/helper/test-sha1.sh\n \n-SP_OBJ = $(patsubst %.o,%.sp,$(C_OBJ))\n+SP_OBJ = $(patsubst %.o,%.sp,$(OBJECTS))\n \n $(SP_OBJ): %.sp: %.c %.o\n \t$(QUIET_SP)cgcc -no-compile $(ALL_CFLAGS) $(EXTRA_CPPFLAGS) \\\n-- \n2.36.0.879.g56a83971f3f\n\n"},{"id":"454248","messageId":"patch-2.5-5ffb68dc77b-20220422T094624Z-avarab@gmail.com","threadId":"57779","inReplyTo":"cover-0.5-00000000000-20220422T094624Z-avarab@gmail.com","subject":"[PATCH 2/5] Makefile: really use and document sha1collisiondetection by default","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-04-22T09:53:13Z","receivedAt":"2022-04-22T09:53:50Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"When the sha1collisiondetection library was added and made the default\nin [1] we never updated the documentation added in [2] early in that\nseries once the default was flipped to DC_SHA1 in [3]. Furthermore the\nINSTALL file has been claiming that we use OpenSSL by default since\n[4], and hadn't been updated for the sha1collisiondetection switch.\n\nThe interaction between NO_APPLE_COMMON_CRYPTO and DC_SHA1 seems to\nhave been missed in [3], so ever since DC_SHA1 was made the default\nwe've still used Apple's CommonCrypto instead of\nsha1collisiondetection on Darwin and Mac OS X.\n\nInstead off all of this we now:\n\n* Don't have a DC_SHA1 know anymore (using it is an error), you need\n to set NO_DC_SHA1 instead to use any optional *_SHA1 implementation.\n* Re-arranged the algorithm inclusion in hash.h to correspond to\n NO_DC_SHA1, and \"#error\" if we have no defined SHA_*, rather than\n silently picking block-sha1/sha1.h as a fallback.\n* Have an INSTALL that reflects reality. We were still claiming to use\n OpenSSL's SHA-1 hashing by default.\n* Have Darwin and Mac OS X use sha1collisiondetection, like everywhere\n else. There is still a NO_APPLE_COMMON_CRYPTO knob, but it's used for\n things unrelated to SHA-1 (see [6]).\n* Have a rewritten discussion of SHA-1 and SHA-256 in the Makefile\n which covers all of this.\n\n1. 48b3693d3ce (Merge branch 'jk/sha1dc', 2017-03-24)\n2. 8325e43b82d (Makefile: add DC_SHA1 knob, 2017-03-16)\n3. e6b07da2780 (Makefile: make DC_SHA1 the default, 2017-03-17)\n4. 5beb577db8c (INSTALL: Describe dependency knobs from Makefile,\n 2009-09-10)\n5. 4dcd7732db0 (Makefile: add support for Apple CommonCrypto facility,\n 2013-05-19)\n6. 3ef2bcad02e (imap-send: use Apple's Security framework for base64\n encoding, 2013-07-29)\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n INSTALL | 11 +--\n Makefile | 108 ++++++++++++++++++----------\n contrib/buildsystems/CMakeLists.txt | 3 +-\n hash.h | 10 +--\n t/t0013-sha1dc.sh | 4 +-\n 5 files changed, 88 insertions(+), 48 deletions(-)\n\ndiff --git a/INSTALL b/INSTALL\nindex 4140a3f5c8b..8fc350fc124 100644\n--- a/INSTALL\n+++ b/INSTALL\n@@ -133,10 +133,13 @@ Issues of note:\n \t you are using libcurl older than 7.34.0. Otherwise you can use\n \t NO_OPENSSL without losing git-imap-send.\n \n-\t By default, git uses OpenSSL for SHA1 but it will use its own\n-\t library (inspired by Mozilla's) with either NO_OPENSSL or\n-\t BLK_SHA1. Also included is a version optimized for PowerPC\n-\t (PPC_SHA1).\n+\t- Git uses an altered version of SHA-1 by default which\n+ detects the SHAttered attack via the sha1collisiondetection\n+ counter-cryptanalysis library. For SHA-256 we'll select a\n+ working implementation (and ship a fallback\n+ implementation). See the \"SHA-1 and SHA-256 defines\" section\n+ in the Makefile for details. You should not need to tweak\n+ those settings.\n \n \t- \"libcurl\" library is used for fetching and pushing\n \t repositories over http:// or https://, as well as by\ndiff --git a/Makefile b/Makefile\nindex 246f6729022..12bb9edd70a 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -468,33 +468,65 @@ include shared.mak\n #\n # == SHA-1 and SHA-256 defines ==\n #\n-# Define BLK_SHA1 environment variable to make use of the bundled\n-# optimized C SHA1 routine.\n+# === SHA-1 backend ===\n #\n-# Define PPC_SHA1 environment variable when running make to make use of\n-# a bundled SHA1 routine optimized for PowerPC.\n+# Due to the SHAttered (https://shattered.io) attack vector on SHA-1\n+# Git uses the sha1collisiondetection counter-cryptanalysis library\n+# for SHA-1 hashing.\n+#\n+# You're strongly advised not to override this for any usage of Git\n+# where you don't 100% trust the repository content.\n+#\n+# ==== Options common to all SHA-1 implementations ====\n+#\n+# Define SHA1_MAX_BLOCK_SIZE to limit the amount of data that will be hashed\n+# in one call to the platform's SHA1_Update(). e.g. APPLE_COMMON_CRYPTO\n+# wants 'SHA1_MAX_BLOCK_SIZE=1024L*1024L*1024L' defined.\n #\n-# Define DC_SHA1 to unconditionally enable the collision-detecting sha1\n-# algorithm. This is slower, but may detect attempted collision attacks.\n-# Takes priority over other *_SHA1 knobs.\n+# ===== Options for the default sha1collisiondetection implementations =====\n #\n-# Define DC_SHA1_EXTERNAL in addition to DC_SHA1 if you want to build / link\n+# Define DC_SHA1_EXTERNAL if you want to build / link\n # git with the external SHA1 collision-detect library.\n # Without this option, i.e. the default behavior is to build git with its\n # own built-in code (or submodule).\n #\n-# Define DC_SHA1_SUBMODULE in addition to DC_SHA1 to use the\n+# Define DC_SHA1_SUBMODULE to use the\n # sha1collisiondetection shipped as a submodule instead of the\n # non-submodule copy in sha1dc/. This is an experimental option used\n # by the git project to migrate to using sha1collisiondetection as a\n # submodule.\n #\n+# ==== Alternate implementations ====\n+#\n+# Git still ships with alternate SHA-1 implementations. These are\n+# faster than the default, which is useful when hashing speed\n+# is imperative, consider using them if you're confident that you\n+# won't need to worry about SHA-1 collision attacks.\n+#\n+# To use them you must define NO_DC_SHA1 and one of the *_SHA1\n+# variables below:\n+#\n+# Define BLK_SHA1 environment variable to make use of the bundled\n+# optimized C SHA1 routine.\n+#\n # Define OPENSSL_SHA1 environment variable when running make to link\n # with the SHA1 routine from openssl library.\n #\n-# Define SHA1_MAX_BLOCK_SIZE to limit the amount of data that will be hashed\n-# in one call to the platform's SHA1_Update(). e.g. APPLE_COMMON_CRYPTO\n-# wants 'SHA1_MAX_BLOCK_SIZE=1024L*1024L*1024L' defined.\n+# Define PPC_SHA1 environment variable when running make to make use of\n+# a bundled SHA1 routine optimized for PowerPC.\n+#\n+# Define APPLE_SHA1 to use Apple's CommonCrypto SHA-1 routines on\n+# Darwin/Mac OS X.\n+#\n+# The APPLE_SHA1 option is unrelated to the NO_APPLE_COMMON_CRYPTO\n+# flag, which determines if Apple's crypto libraries are used for\n+# things that aren't SHA-1.\n+#\n+# === SHA-256 backend ===\n+#\n+# Unlike SHA-1 the SHA-256 algorithm does not suffer from any known\n+# vulnerabilities, so any implementation will do. BLK_SHA256 is\n+# currently the default implementation (but that may change).\n #\n # Define BLK_SHA256 to use the built-in SHA-256 routines.\n #\n@@ -1434,7 +1466,6 @@ ifeq ($(uname_S),Darwin)\n \tendif\n \tifndef NO_APPLE_COMMON_CRYPTO\n \t\tNO_OPENSSL = YesPlease\n-\t\tAPPLE_COMMON_CRYPTO = YesPlease\n \t\tCOMPAT_CFLAGS += -DAPPLE_COMMON_CRYPTO\n \tendif\n \tNO_REGEX = YesPlease\n@@ -1796,30 +1827,15 @@ ifdef NO_POSIX_GOODIES\n \tBASIC_CFLAGS += -DNO_POSIX_GOODIES\n endif\n \n-ifdef APPLE_COMMON_CRYPTO\n-\t# Apple CommonCrypto requires chunking\n-\tSHA1_MAX_BLOCK_SIZE = 1024L*1024L*1024L\n+ifdef DC_SHA1\n+$(error the DC_SHA1 flag is no longer used, and has become the default. Adjust your build scripts accordingly)\n endif\n-\n-ifdef OPENSSL_SHA1\n-\tEXTLIBS += $(LIB_4_CRYPTO)\n-\tBASIC_CFLAGS += -DSHA1_OPENSSL\n-else\n-ifdef BLK_SHA1\n-\tLIB_OBJS += block-sha1/sha1.o\n-\tBASIC_CFLAGS += -DSHA1_BLK\n-else\n-ifdef PPC_SHA1\n-\tLIB_OBJS += ppc/sha1.o ppc/sha1ppc.o\n-\tBASIC_CFLAGS += -DSHA1_PPC\n-else\n-ifdef APPLE_COMMON_CRYPTO\n-\tCOMPAT_CFLAGS += -DCOMMON_DIGEST_FOR_OPENSSL\n-\tBASIC_CFLAGS += -DSHA1_APPLE\n-else\n-\tDC_SHA1 := YesPlease\n-\tBASIC_CFLAGS += -DSHA1_DC\n+ifndef NO_DC_SHA1\n+\tifneq ($(OPENSSL_SHA1)$(BLK_SHA1)$(PPC_SHA1)$(APPLE_SHA1),)\n+$(error no other *_SHA1 option can be defined unless NO_DC_SHA1 is defined)\n+\tendif\n \tLIB_OBJS += sha1dc_git.o\n+\n ifdef DC_SHA1_EXTERNAL\n \tifdef DC_SHA1_SUBMODULE\n \t\tifneq ($(DC_SHA1_SUBMODULE),auto)\n@@ -1843,6 +1859,26 @@ endif\n \t\t-DSHA1DC_CUSTOM_INCLUDE_SHA1_C=\"\\\"cache.h\\\"\" \\\n \t\t-DSHA1DC_CUSTOM_INCLUDE_UBC_CHECK_C=\"\\\"git-compat-util.h\\\"\"\n endif\n+else # !NO_DC_SHA1\n+BASIC_CFLAGS += -DNO_SHA1_DC\n+ifdef OPENSSL_SHA1\n+\tEXTLIBS += $(LIB_4_CRYPTO)\n+\tBASIC_CFLAGS += -DSHA1_OPENSSL\n+else\n+ifdef BLK_SHA1\n+\tLIB_OBJS += block-sha1/sha1.o\n+\tBASIC_CFLAGS += -DSHA1_BLK\n+else\n+ifdef PPC_SHA1\n+\tLIB_OBJS += ppc/sha1.o ppc/sha1ppc.o\n+\tBASIC_CFLAGS += -DSHA1_PPC\n+else\n+ifdef APPLE_SHA1\n+\tCOMPAT_CFLAGS += -DCOMMON_DIGEST_FOR_OPENSSL\n+\tBASIC_CFLAGS += -DSHA1_APPLE\n+else\n+$(error when defining NO_DC_SHA1 another valid *_SHA1 variable must be defined!)\n+endif\n endif\n endif\n endif\n@@ -2884,7 +2920,7 @@ GIT-BUILD-OPTIONS: FORCE\n \t@echo NO_PYTHON=\\''$(subst ','\\'',$(subst ','\\'',$(NO_PYTHON)))'\\' >>$@+\n \t@echo NO_UNIX_SOCKETS=\\''$(subst ','\\'',$(subst ','\\'',$(NO_UNIX_SOCKETS)))'\\' >>$@+\n \t@echo PAGER_ENV=\\''$(subst ','\\'',$(subst ','\\'',$(PAGER_ENV)))'\\' >>$@+\n-\t@echo DC_SHA1=\\''$(subst ','\\'',$(subst ','\\'',$(DC_SHA1)))'\\' >>$@+\n+\t@echo NO_DC_SHA1=\\''$(subst ','\\'',$(subst ','\\'',$(NO_DC_SHA1)))'\\' >>$@+\n \t@echo SANITIZE_LEAK=\\''$(subst ','\\'',$(subst ','\\'',$(SANITIZE_LEAK)))'\\' >>$@+\n \t@echo X=\\'$(X)\\' >>$@+\n ifdef FSMONITOR_DAEMON_BACKEND\ndiff --git a/contrib/buildsystems/CMakeLists.txt b/contrib/buildsystems/CMakeLists.txt\nindex 185f56f414f..10616188e4e 100644\n--- a/contrib/buildsystems/CMakeLists.txt\n+++ b/contrib/buildsystems/CMakeLists.txt\n@@ -996,7 +996,6 @@ set(NO_PERL )\n set(NO_PTHREADS )\n set(NO_PYTHON )\n set(PAGER_ENV \"LESS=FRX LV=-c\")\n-set(DC_SHA1 YesPlease)\n set(RUNTIME_PREFIX true)\n set(NO_GETTEXT )\n \n@@ -1032,7 +1031,7 @@ file(APPEND ${CMAKE_BINARY_DIR}/GIT-BUILD-OPTIONS \"NO_PERL='${NO_PERL}'\\n\")\n file(APPEND ${CMAKE_BINARY_DIR}/GIT-BUILD-OPTIONS \"NO_PTHREADS='${NO_PTHREADS}'\\n\")\n file(APPEND ${CMAKE_BINARY_DIR}/GIT-BUILD-OPTIONS \"NO_UNIX_SOCKETS='${NO_UNIX_SOCKETS}'\\n\")\n file(APPEND ${CMAKE_BINARY_DIR}/GIT-BUILD-OPTIONS \"PAGER_ENV='${PAGER_ENV}'\\n\")\n-file(APPEND ${CMAKE_BINARY_DIR}/GIT-BUILD-OPTIONS \"DC_SHA1='${DC_SHA1}'\\n\")\n+file(APPEND ${CMAKE_BINARY_DIR}/GIT-BUILD-OPTIONS \"NO_DC_SHA1=''\\n\")\n file(APPEND ${CMAKE_BINARY_DIR}/GIT-BUILD-OPTIONS \"X='${EXE_EXTENSION}'\\n\")\n file(APPEND ${CMAKE_BINARY_DIR}/GIT-BUILD-OPTIONS \"NO_GETTEXT='${NO_GETTEXT}'\\n\")\n file(APPEND ${CMAKE_BINARY_DIR}/GIT-BUILD-OPTIONS \"RUNTIME_PREFIX='${RUNTIME_PREFIX}'\\n\")\ndiff --git a/hash.h b/hash.h\nindex 5d40368f18a..20012167ce4 100644\n--- a/hash.h\n+++ b/hash.h\n@@ -4,16 +4,18 @@\n #include \"git-compat-util.h\"\n #include \"repository.h\"\n \n-#if defined(SHA1_PPC)\n+#if !defined(NO_SHA1_DC)\n+#include \"sha1dc_git.h\"\n+#elif defined(SHA1_PPC)\n #include \"ppc/sha1.h\"\n #elif defined(SHA1_APPLE)\n #include \n #elif defined(SHA1_OPENSSL)\n #include \n-#elif defined(SHA1_DC)\n-#include \"sha1dc_git.h\"\n-#else /* SHA1_BLK */\n+#elif defined(SHA1_BLK)\n #include \"block-sha1/sha1.h\"\n+#else\n+#error \"need a SHA1_* implementation defined\"\n #endif\n \n #if defined(SHA256_GCRYPT)\ndiff --git a/t/t0013-sha1dc.sh b/t/t0013-sha1dc.sh\nindex 9ad76080aa4..539270a2665 100755\n--- a/t/t0013-sha1dc.sh\n+++ b/t/t0013-sha1dc.sh\n@@ -6,9 +6,9 @@ TEST_PASSES_SANITIZE_LEAK=true\n . ./test-lib.sh\n TEST_DATA=\"$TEST_DIRECTORY/t0013\"\n \n-if test -z \"$DC_SHA1\"\n+if test -n \"$NO_DC_SHA1\"\n then\n-\tskip_all='skipping sha1 collision tests, DC_SHA1 not set'\n+\tskip_all='skipping sha1 collision tests, NO_DC_SHA1 set'\n \ttest_done\n fi\n \n-- \n2.36.0.879.g56a83971f3f\n\n"},{"id":"454249","messageId":"patch-4.5-4b2d0b7b51a-20220422T094624Z-avarab@gmail.com","threadId":"57779","inReplyTo":"cover-0.5-00000000000-20220422T094624Z-avarab@gmail.com","subject":"[PATCH 4/5] Makefile + hash.h: remove PPC_SHA1 implementation","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-04-22T09:53:15Z","receivedAt":"2022-04-22T09:53:50Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Remove the PPC_SHA1 implementation added in a6ef3518f9a ([PATCH] PPC\nassembly implementation of SHA1, 2005-04-22). When this was added\nApple consumer hardware used the PPC architecture, and the\nimplementation was intended to improve SHA-1 speed there.\n\nSince it was added we've moved to using sha1collisiondetection by\ndefault, and anyone wanting hard-rolled non-DC SHA-1 implementation\ncan use OpenSSL's via the OPENSSL_SHA1 knob.\n\nFurthermore this doesn't run on the modern PPC processors which anyone\nwho's concerned about performance on PPC is likely to be using these\ndays, i.e. the IBM POWER series. It originally originally targeted 32\nbit PPC, but there's some mailing list references to this being tried\non G5 (PPC 970).\n\nI can't get it to do anything but segfault on both the BE and LE POWER\nmachines in the GCC compile farm.\n\nThere have been proposals to entirely remove non-sha1collisiondetection\nimplementations from the tree[1]. I think per [2] that would be a bit\noverzealous. I.e. there are various set-ups git's speed is going to be\nmore important than the relatively implausible SHA-1 collision attack,\nor where such attacks are entirely mitigated by other means (e.g. by\nincoming objects being checked with DC_SHA1).\n\nBut that really doesn't apply to PPC_SHA1 in particular, which seems\nto have outlived its usefulness.\n\nAs this gets rid of the only in-tree *.S assembly file we can remove\nthe small bits of logic from the Makefile needed to build objects\nfrom *.S (as opposed to *.c)\n\nThe code being removed here was also throwing warnings with the\n\"-pedantic\" flag, it could have been fixed as 544d93bc3b4 (block-sha1:\nremove use of obsolete x86 assembly, 2022-03-10) did for block-sha1/*,\nbut as noted above let's remove it instead.\n\n1. https://lore.kernel.org/git/20200223223758.120941-1-mh@glandium.org/\n2. https://lore.kernel.org/git/20200224044732.GK1018190@coredump.intra.peff.net/\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n Makefile | 19 ++---\n configure.ac | 3 -\n hash.h | 4 +-\n ppc/sha1.c | 72 ----------------\n ppc/sha1.h | 25 ------\n ppc/sha1ppc.S | 224 --------------------------------------------------\n 6 files changed, 6 insertions(+), 341 deletions(-)\n delete mode 100644 ppc/sha1.c\n delete mode 100644 ppc/sha1.h\n delete mode 100644 ppc/sha1ppc.S\n\ndiff --git a/Makefile b/Makefile\nindex d317c6aba06..6f8272ae4a4 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -512,9 +512,6 @@ include shared.mak\n # Define OPENSSL_SHA1 to link to the the SHA-1 routines from\n # the OpenSSL library.\n #\n-# Define PPC_SHA1 to make use of optimized (in assembly)\n-# PowerPC SHA-1 routines.\n-#\n # Define APPLE_SHA1 to use Apple's CommonCrypto SHA-1 routines on\n # Darwin/Mac OS X.\n #\n@@ -1830,8 +1827,11 @@ endif\n ifdef DC_SHA1\n $(error the DC_SHA1 flag is no longer used, and has become the default. Adjust your build scripts accordingly)\n endif\n+ifdef PPC_SHA1\n+$(error the PPC_SHA1 flag has been removed along with the PowerPC-specific SHA-1 implementation.)\n+endif\n ifndef NO_DC_SHA1\n-\tifneq ($(OPENSSL_SHA1)$(BLK_SHA1)$(PPC_SHA1)$(APPLE_SHA1),)\n+\tifneq ($(OPENSSL_SHA1)$(BLK_SHA1)$(APPLE_SHA1),)\n $(error no other *_SHA1 option can be defined unless NO_DC_SHA1 is defined)\n \tendif\n \tLIB_OBJS += sha1dc_git.o\n@@ -1869,10 +1869,6 @@ ifdef BLK_SHA1\n \tLIB_OBJS += block-sha1/sha1.o\n \tBASIC_CFLAGS += -DSHA1_BLK\n else\n-ifdef PPC_SHA1\n-\tLIB_OBJS += ppc/sha1.o ppc/sha1ppc.o\n-\tBASIC_CFLAGS += -DSHA1_PPC\n-else\n ifdef APPLE_SHA1\n \tCOMPAT_CFLAGS += -DCOMMON_DIGEST_FOR_OPENSSL\n \tBASIC_CFLAGS += -DSHA1_APPLE\n@@ -1882,7 +1878,6 @@ endif\n endif\n endif\n endif\n-endif\n \n ifdef OPENSSL_SHA256\n \tEXTLIBS += $(LIB_4_CRYPTO)\n@@ -2621,14 +2616,10 @@ missing_compdb_dir =\n compdb_args =\n endif\n \n-ASM_SRC := $(wildcard $(OBJECTS:o=S))\n-ASM_OBJ := $(ASM_SRC:S=o)\n-C_OBJ := $(filter-out $(ASM_OBJ),$(OBJECTS))\n+C_OBJ = $(OBJECTS)\n \n $(C_OBJ): %.o: %.c GIT-CFLAGS $(missing_dep_dirs) $(missing_compdb_dir)\n \t$(QUIET_CC)$(CC) -o $*.o -c $(dep_args) $(compdb_args) $(ALL_CFLAGS) $(EXTRA_CPPFLAGS) $<\n-$(ASM_OBJ): %.o: %.S GIT-CFLAGS $(missing_dep_dirs) $(missing_compdb_dir)\n-\t$(QUIET_CC)$(CC) -o $*.o -c $(dep_args) $(compdb_args) $(ALL_CFLAGS) $(EXTRA_CPPFLAGS) $<\n \n %.s: %.c GIT-CFLAGS FORCE\n \t$(QUIET_CC)$(CC) -o $@ -S $(ALL_CFLAGS) $(EXTRA_CPPFLAGS) $<\ndiff --git a/configure.ac b/configure.ac\nindex 316a31d2313..bbfbcc63af0 100644\n--- a/configure.ac\n+++ b/configure.ac\n@@ -237,9 +237,6 @@ AC_MSG_NOTICE([CHECKS for site configuration])\n # tests. These tests take up a significant amount of the total test time\n # but are not needed unless you plan to talk to SVN repos.\n #\n-# Define PPC_SHA1 environment variable when running make to make use of\n-# a bundled SHA1 routine optimized for PowerPC.\n-#\n # Define NO_OPENSSL environment variable if you do not have OpenSSL.\n #\n # Define OPENSSLDIR=/foo/bar if your openssl header and library files are in\ndiff --git a/hash.h b/hash.h\nindex 20012167ce4..248615df11d 100644\n--- a/hash.h\n+++ b/hash.h\n@@ -6,8 +6,6 @@\n \n #if !defined(NO_SHA1_DC)\n #include \"sha1dc_git.h\"\n-#elif defined(SHA1_PPC)\n-#include \"ppc/sha1.h\"\n #elif defined(SHA1_APPLE)\n #include \n #elif defined(SHA1_OPENSSL)\n@@ -32,7 +30,7 @@\n * platform's underlying implementation of SHA-1; could be OpenSSL,\n * blk_SHA, Apple CommonCrypto, etc... Note that the relevant\n * SHA-1 header may have already defined platform_SHA_CTX for our\n- * own implementations like block-sha1 and ppc-sha1, so we list\n+ * own implementations like block-sha1, so we list\n * the default for OpenSSL compatible SHA-1 implementations here.\n */\n #define platform_SHA_CTX\tSHA_CTX\ndiff --git a/ppc/sha1.c b/ppc/sha1.c\ndeleted file mode 100644\nindex 1b705cee1fe..00000000000\n--- a/ppc/sha1.c\n+++ /dev/null\n@@ -1,72 +0,0 @@\n-/*\n- * SHA-1 implementation.\n- *\n- * Copyright (C) 2005 Paul Mackerras \n- *\n- * This version assumes we are running on a big-endian machine.\n- * It calls an external sha1_core() to process blocks of 64 bytes.\n- */\n-#include \n-#include \n-#include \"sha1.h\"\n-\n-void ppc_sha1_core(uint32_t *hash, const unsigned char *p,\n-\t\t unsigned int nblocks);\n-\n-int ppc_SHA1_Init(ppc_SHA_CTX *c)\n-{\n-\tc->hash[0] = 0x67452301;\n-\tc->hash[1] = 0xEFCDAB89;\n-\tc->hash[2] = 0x98BADCFE;\n-\tc->hash[3] = 0x10325476;\n-\tc->hash[4] = 0xC3D2E1F0;\n-\tc->len = 0;\n-\tc->cnt = 0;\n-\treturn 0;\n-}\n-\n-int ppc_SHA1_Update(ppc_SHA_CTX *c, const void *ptr, unsigned long n)\n-{\n-\tunsigned long nb;\n-\tconst unsigned char *p = ptr;\n-\n-\tc->len += (uint64_t) n << 3;\n-\twhile (n != 0) {\n-\t\tif (c->cnt || n < 64) {\n-\t\t\tnb = 64 - c->cnt;\n-\t\t\tif (nb > n)\n-\t\t\t\tnb = n;\n-\t\t\tmemcpy(&c->buf.b[c->cnt], p, nb);\n-\t\t\tif ((c->cnt += nb) == 64) {\n-\t\t\t\tppc_sha1_core(c->hash, c->buf.b, 1);\n-\t\t\t\tc->cnt = 0;\n-\t\t\t}\n-\t\t} else {\n-\t\t\tnb = n >> 6;\n-\t\t\tppc_sha1_core(c->hash, p, nb);\n-\t\t\tnb <<= 6;\n-\t\t}\n-\t\tn -= nb;\n-\t\tp += nb;\n-\t}\n-\treturn 0;\n-}\n-\n-int ppc_SHA1_Final(unsigned char *hash, ppc_SHA_CTX *c)\n-{\n-\tunsigned int cnt = c->cnt;\n-\n-\tc->buf.b[cnt++] = 0x80;\n-\tif (cnt > 56) {\n-\t\tif (cnt < 64)\n-\t\t\tmemset(&c->buf.b[cnt], 0, 64 - cnt);\n-\t\tppc_sha1_core(c->hash, c->buf.b, 1);\n-\t\tcnt = 0;\n-\t}\n-\tif (cnt < 56)\n-\t\tmemset(&c->buf.b[cnt], 0, 56 - cnt);\n-\tc->buf.l[7] = c->len;\n-\tppc_sha1_core(c->hash, c->buf.b, 1);\n-\tmemcpy(hash, c->hash, 20);\n-\treturn 0;\n-}\ndiff --git a/ppc/sha1.h b/ppc/sha1.h\ndeleted file mode 100644\nindex 9b24b326159..00000000000\n--- a/ppc/sha1.h\n+++ /dev/null\n@@ -1,25 +0,0 @@\n-/*\n- * SHA-1 implementation.\n- *\n- * Copyright (C) 2005 Paul Mackerras \n- */\n-#include \n-\n-typedef struct {\n-\tuint32_t hash[5];\n-\tuint32_t cnt;\n-\tuint64_t len;\n-\tunion {\n-\t\tunsigned char b[64];\n-\t\tuint64_t l[8];\n-\t} buf;\n-} ppc_SHA_CTX;\n-\n-int ppc_SHA1_Init(ppc_SHA_CTX *c);\n-int ppc_SHA1_Update(ppc_SHA_CTX *c, const void *p, unsigned long n);\n-int ppc_SHA1_Final(unsigned char *hash, ppc_SHA_CTX *c);\n-\n-#define platform_SHA_CTX\tppc_SHA_CTX\n-#define platform_SHA1_Init\tppc_SHA1_Init\n-#define platform_SHA1_Update\tppc_SHA1_Update\n-#define platform_SHA1_Final\tppc_SHA1_Final\ndiff --git a/ppc/sha1ppc.S b/ppc/sha1ppc.S\ndeleted file mode 100644\nindex 1711eef6e71..00000000000\n--- a/ppc/sha1ppc.S\n+++ /dev/null\n@@ -1,224 +0,0 @@\n-/*\n- * SHA-1 implementation for PowerPC.\n- *\n- * Copyright (C) 2005 Paul Mackerras \n- */\n-\n-/*\n- * PowerPC calling convention:\n- * %r0 - volatile temp\n- * %r1 - stack pointer.\n- * %r2 - reserved\n- * %r3-%r12 - Incoming arguments & return values; volatile.\n- * %r13-%r31 - Callee-save registers\n- * %lr - Return address, volatile\n- * %ctr - volatile\n- *\n- * Register usage in this routine:\n- * %r0 - temp\n- * %r3 - argument (pointer to 5 words of SHA state)\n- * %r4 - argument (pointer to data to hash)\n- * %r5 - Constant K in SHA round (initially number of blocks to hash)\n- * %r6-%r10 - Working copies of SHA variables A..E (actually E..A order)\n- * %r11-%r26 - Data being hashed W[].\n- * %r27-%r31 - Previous copies of A..E, for final add back.\n- * %ctr - loop count\n- */\n-\n-\n-/*\n- * We roll the registers for A, B, C, D, E around on each\n- * iteration; E on iteration t is D on iteration t+1, and so on.\n- * We use registers 6 - 10 for this. (Registers 27 - 31 hold\n- * the previous values.)\n- */\n-#define RA(t)\t(((t)+4)%5+6)\n-#define RB(t)\t(((t)+3)%5+6)\n-#define RC(t)\t(((t)+2)%5+6)\n-#define RD(t)\t(((t)+1)%5+6)\n-#define RE(t)\t(((t)+0)%5+6)\n-\n-/* We use registers 11 - 26 for the W values */\n-#define W(t)\t((t)%16+11)\n-\n-/* Register 5 is used for the constant k */\n-\n-/*\n- * The basic SHA-1 round function is:\n- * E += ROTL(A,5) + F(B,C,D) + W[i] + K; B = ROTL(B,30)\n- * Then the variables are renamed: (A,B,C,D,E) = (E,A,B,C,D).\n- *\n- * Every 20 rounds, the function F() and the constant K changes:\n- * - 20 rounds of f0(b,c,d) = \"bit wise b ? c : d\" = (^b & d) + (b & c)\n- * - 20 rounds of f1(b,c,d) = b^c^d = (b^d)^c\n- * - 20 rounds of f2(b,c,d) = majority(b,c,d) = (b&d) + ((b^d)&c)\n- * - 20 more rounds of f1(b,c,d)\n- *\n- * These are all scheduled for near-optimal performance on a G4.\n- * The G4 is a 3-issue out-of-order machine with 3 ALUs, but it can only\n- * *consider* starting the oldest 3 instructions per cycle. So to get\n- * maximum performance out of it, you have to treat it as an in-order\n- * machine. Which means interleaving the computation round t with the\n- * computation of W[t+4].\n- *\n- * The first 16 rounds use W values loaded directly from memory, while the\n- * remaining 64 use values computed from those first 16. We preload\n- * 4 values before starting, so there are three kinds of rounds:\n- * - The first 12 (all f0) also load the W values from memory.\n- * - The next 64 compute W(i+4) in parallel. 8*f0, 20*f1, 20*f2, 16*f1.\n- * - The last 4 (all f1) do not do anything with W.\n- *\n- * Therefore, we have 6 different round functions:\n- * STEPD0_LOAD(t,s) - Perform round t and load W(s). s < 16\n- * STEPD0_UPDATE(t,s) - Perform round t and compute W(s). s >= 16.\n- * STEPD1_UPDATE(t,s)\n- * STEPD2_UPDATE(t,s)\n- * STEPD1(t) - Perform round t with no load or update.\n- *\n- * The G5 is more fully out-of-order, and can find the parallelism\n- * by itself. The big limit is that it has a 2-cycle ALU latency, so\n- * even though it's 2-way, the code has to be scheduled as if it's\n- * 4-way, which can be a limit. To help it, we try to schedule the\n- * read of RA(t) as late as possible so it doesn't stall waiting for\n- * the previous round's RE(t-1), and we try to rotate RB(t) as early\n- * as possible while reading RC(t) (= RB(t-1)) as late as possible.\n- */\n-\n-/* the initial loads. */\n-#define LOADW(s) \\\n-\tlwz\tW(s),(s)*4(%r4)\n-\n-/*\n- * Perform a step with F0, and load W(s). Uses W(s) as a temporary\n- * before loading it.\n- * This is actually 10 instructions, which is an awkward fit.\n- * It can execute grouped as listed, or delayed one instruction.\n- * (If delayed two instructions, there is a stall before the start of the\n- * second line.) Thus, two iterations take 7 cycles, 3.5 cycles per round.\n- */\n-#define STEPD0_LOAD(t,s) \\\n-add RE(t),RE(t),W(t); andc %r0,RD(t),RB(t); and W(s),RC(t),RB(t); \\\n-add RE(t),RE(t),%r0; rotlwi %r0,RA(t),5; rotlwi RB(t),RB(t),30; \\\n-add RE(t),RE(t),W(s); add %r0,%r0,%r5; lwz W(s),(s)*4(%r4); \\\n-add RE(t),RE(t),%r0\n-\n-/*\n- * This is likewise awkward, 13 instructions. However, it can also\n- * execute starting with 2 out of 3 possible moduli, so it does 2 rounds\n- * in 9 cycles, 4.5 cycles/round.\n- */\n-#define STEPD0_UPDATE(t,s,loadk...) \\\n-add RE(t),RE(t),W(t); andc %r0,RD(t),RB(t); xor W(s),W((s)-16),W((s)-3); \\\n-add RE(t),RE(t),%r0; and %r0,RC(t),RB(t); xor W(s),W(s),W((s)-8); \\\n-add RE(t),RE(t),%r0; rotlwi %r0,RA(t),5; xor W(s),W(s),W((s)-14); \\\n-add RE(t),RE(t),%r5; loadk; rotlwi RB(t),RB(t),30; rotlwi W(s),W(s),1; \\\n-add RE(t),RE(t),%r0\n-\n-/* Nicely optimal. Conveniently, also the most common. */\n-#define STEPD1_UPDATE(t,s,loadk...) \\\n-add RE(t),RE(t),W(t); xor %r0,RD(t),RB(t); xor W(s),W((s)-16),W((s)-3); \\\n-add RE(t),RE(t),%r5; loadk; xor %r0,%r0,RC(t); xor W(s),W(s),W((s)-8); \\\n-add RE(t),RE(t),%r0; rotlwi %r0,RA(t),5; xor W(s),W(s),W((s)-14); \\\n-add RE(t),RE(t),%r0; rotlwi RB(t),RB(t),30; rotlwi W(s),W(s),1\n-\n-/*\n- * The naked version, no UPDATE, for the last 4 rounds. 3 cycles per.\n- * We could use W(s) as a temp register, but we don't need it.\n- */\n-#define STEPD1(t) \\\n- add RE(t),RE(t),W(t); xor %r0,RD(t),RB(t); \\\n-rotlwi RB(t),RB(t),30; add RE(t),RE(t),%r5; xor %r0,%r0,RC(t); \\\n-add RE(t),RE(t),%r0; rotlwi %r0,RA(t),5; /* spare slot */ \\\n-add RE(t),RE(t),%r0\n-\n-/*\n- * 14 instructions, 5 cycles per. The majority function is a bit\n- * awkward to compute. This can execute with a 1-instruction delay,\n- * but it causes a 2-instruction delay, which triggers a stall.\n- */\n-#define STEPD2_UPDATE(t,s,loadk...) \\\n-add RE(t),RE(t),W(t); and %r0,RD(t),RB(t); xor W(s),W((s)-16),W((s)-3); \\\n-add RE(t),RE(t),%r0; xor %r0,RD(t),RB(t); xor W(s),W(s),W((s)-8); \\\n-add RE(t),RE(t),%r5; loadk; and %r0,%r0,RC(t); xor W(s),W(s),W((s)-14); \\\n-add RE(t),RE(t),%r0; rotlwi %r0,RA(t),5; rotlwi W(s),W(s),1; \\\n-add RE(t),RE(t),%r0; rotlwi RB(t),RB(t),30\n-\n-#define STEP0_LOAD4(t,s)\t\t\\\n-\tSTEPD0_LOAD(t,s);\t\t\\\n-\tSTEPD0_LOAD((t+1),(s)+1);\t\\\n-\tSTEPD0_LOAD((t)+2,(s)+2);\t\\\n-\tSTEPD0_LOAD((t)+3,(s)+3)\n-\n-#define STEPUP4(fn, t, s, loadk...)\t\t\\\n-\tSTEP##fn##_UPDATE(t,s,);\t\t\\\n-\tSTEP##fn##_UPDATE((t)+1,(s)+1,);\t\\\n-\tSTEP##fn##_UPDATE((t)+2,(s)+2,);\t\\\n-\tSTEP##fn##_UPDATE((t)+3,(s)+3,loadk)\n-\n-#define STEPUP20(fn, t, s, loadk...)\t\\\n-\tSTEPUP4(fn, t, s,);\t\t\\\n-\tSTEPUP4(fn, (t)+4, (s)+4,);\t\\\n-\tSTEPUP4(fn, (t)+8, (s)+8,);\t\\\n-\tSTEPUP4(fn, (t)+12, (s)+12,);\t\\\n-\tSTEPUP4(fn, (t)+16, (s)+16, loadk)\n-\n-\t.globl\tppc_sha1_core\n-ppc_sha1_core:\n-\tstwu\t%r1,-80(%r1)\n-\tstmw\t%r13,4(%r1)\n-\n-\t/* Load up A - E */\n-\tlmw\t%r27,0(%r3)\n-\n-\tmtctr\t%r5\n-\n-1:\n-\tLOADW(0)\n-\tlis\t%r5,0x5a82\n-\tmr\tRE(0),%r31\n-\tLOADW(1)\n-\tmr\tRD(0),%r30\n-\tmr\tRC(0),%r29\n-\tLOADW(2)\n-\tori\t%r5,%r5,0x7999\t/* K0-19 */\n-\tmr\tRB(0),%r28\n-\tLOADW(3)\n-\tmr\tRA(0),%r27\n-\n-\tSTEP0_LOAD4(0, 4)\n-\tSTEP0_LOAD4(4, 8)\n-\tSTEP0_LOAD4(8, 12)\n-\tSTEPUP4(D0, 12, 16,)\n-\tSTEPUP4(D0, 16, 20, lis %r5,0x6ed9)\n-\n-\tori\t%r5,%r5,0xeba1\t/* K20-39 */\n-\tSTEPUP20(D1, 20, 24, lis %r5,0x8f1b)\n-\n-\tori\t%r5,%r5,0xbcdc\t/* K40-59 */\n-\tSTEPUP20(D2, 40, 44, lis %r5,0xca62)\n-\n-\tori\t%r5,%r5,0xc1d6\t/* K60-79 */\n-\tSTEPUP4(D1, 60, 64,)\n-\tSTEPUP4(D1, 64, 68,)\n-\tSTEPUP4(D1, 68, 72,)\n-\tSTEPUP4(D1, 72, 76,)\n-\taddi\t%r4,%r4,64\n-\tSTEPD1(76)\n-\tSTEPD1(77)\n-\tSTEPD1(78)\n-\tSTEPD1(79)\n-\n-\t/* Add results to original values */\n-\tadd\t%r31,%r31,RE(0)\n-\tadd\t%r30,%r30,RD(0)\n-\tadd\t%r29,%r29,RC(0)\n-\tadd\t%r28,%r28,RB(0)\n-\tadd\t%r27,%r27,RA(0)\n-\n-\tbdnz\t1b\n-\n-\t/* Save final hash, restore registers, and return */\n-\tstmw\t%r27,0(%r3)\n-\tlmw\t%r13,4(%r1)\n-\taddi\t%r1,%r1,80\n-\tblr\n-- \n2.36.0.879.g56a83971f3f\n\n"},{"id":"454272","messageId":"xmqqv8v17xrl.fsf@gitster.g","threadId":"57779","inReplyTo":"cover-0.5-00000000000-20220422T094624Z-avarab@gmail.com","subject":"Re: [PATCH 0/5] core: update our SHA-1 docs, use sha1collisiondetection on OSX too","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-04-22T18:56:14Z","receivedAt":"2022-04-22T19:02:44Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Ævar Arnfjörð Bjarmason writes:\n\n> I wasn't able to find any on-list references to it being intentional,\n> but it appears that while we made the sha1collisiondetection variant\n> of SHA-1 the default in early 2017 we've never updated the OSX builds\n> to do likewise.\n>\n> I don't know what various git packages for OSX to, but our vanilla OSX\n> distribution definitely uses Apple Common Crypto, and won't detect the\n> https://shattered.io attack.\n>\n> This series changes that, and while doing so in 2/5 updates our\n> documentation and Makefile interface for the SHA-1 selection. Our\n> INSTALL file was still claiming we used OpenSSL's SHA-1 by default.\n>\n> Then since we'd made sha1collisiondetection the default we hadn't\n> changed the code's default fallback to be that, it was still\n> block-sha1. Now our fallback behavior is \"error\" instead, which makes\n> it less likely that we'll get some foot-gun like the \"OSX not using\n> sha1collisiondetection\" again.\n>\n> The 4/5 and 5/5 then remove the PPC_SHA1 implementation. I submitted\n> this before as [1], and the range-diff is to that submission (it\n> wasn't picked up). I think it makes sense as part of this general\n> SHA-1 cleanup.\n\nThanks for this effort.\n\nI'd like to see somebody with \"building Git for distributing to\nmacOS\" background to comment (I am assuming that the mailing list\ngit-packagers@googlegroups.com is the way to reach them).\n\n"},{"id":"455538","messageId":"220519.86k0ah2ryh.gmgdl@evledraar.gmail.com","threadId":"57779","inReplyTo":"xmqqv8v17xrl.fsf@gitster.g","subject":"Re: [PATCH 0/5] core: update our SHA-1 docs, use sha1collisiondetection on OSX too","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-05-19T20:14:32Z","receivedAt":"2022-05-19T20:22:21Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Fri, Apr 22 2022, Junio C Hamano wrote:\n\n> Ævar Arnfjörð Bjarmason writes:\n>\n>> I wasn't able to find any on-list references to it being intentional,\n>> but it appears that while we made the sha1collisiondetection variant\n>> of SHA-1 the default in early 2017 we've never updated the OSX builds\n>> to do likewise.\n>>\n>> I don't know what various git packages for OSX to, but our vanilla OSX\n>> distribution definitely uses Apple Common Crypto, and won't detect the\n>> https://shattered.io attack.\n>>\n>> This series changes that, and while doing so in 2/5 updates our\n>> documentation and Makefile interface for the SHA-1 selection. Our\n>> INSTALL file was still claiming we used OpenSSL's SHA-1 by default.\n>>\n>> Then since we'd made sha1collisiondetection the default we hadn't\n>> changed the code's default fallback to be that, it was still\n>> block-sha1. Now our fallback behavior is \"error\" instead, which makes\n>> it less likely that we'll get some foot-gun like the \"OSX not using\n>> sha1collisiondetection\" again.\n>>\n>> The 4/5 and 5/5 then remove the PPC_SHA1 implementation. I submitted\n>> this before as [1], and the range-diff is to that submission (it\n>> wasn't picked up). I think it makes sense as part of this general\n>> SHA-1 cleanup.\n>\n> Thanks for this effort.\n>\n> I'd like to see somebody with \"building Git for distributing to\n> macOS\" background to comment (I am assuming that the mailing list\n> git-packagers@googlegroups.com is the way to reach them).\n\n*Bump* in case anyone there would like to chime on this bit of OSX\nportability.\n\nI also sent an off-list E-Mail to Tim Harper today, and addresses which\nI gather (from some old git-security@ traffic) are involved in packaging\nthe Apple Git shipped with OSX itself.\n\nIn the meantime do you mind if this topic were queued up? If there are\nany lingering portability concerns getting it into CI and exposed to\nanyone else building on OSX would be a good thing.\n\nI don't see any reason for why we'd decide that OSX out of all our\nsupported platforms should be the only one where we're not mitigating\nthe SHAttered attack (and similar future attacks) by default.\n\nSo aside from any finer details of OSX portability the direction here of\nbuilding with sha1collisiondetection by default on OSX by default seems\nlike a safe bet, just as we do on the rest of our (checks out\nconfig.mak.uname) 20+ supported platforms (per `uname -s`).\n"},{"id":"456217","messageId":"220526.86ilps3yk4.gmgdl@evledraar.gmail.com","threadId":"57779","inReplyTo":"220519.86k0ah2ryh.gmgdl@evledraar.gmail.com","subject":"Re: [PATCH 0/5] core: update our SHA-1 docs, use sha1collisiondetection on OSX too","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-05-26T19:02:54Z","receivedAt":"2022-05-26T19:04:50Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Thu, May 19 2022, Ævar Arnfjörð Bjarmason wrote:\n\n> On Fri, Apr 22 2022, Junio C Hamano wrote:\n>\n>> Ævar Arnfjörð Bjarmason writes:\n>>\n>>> I wasn't able to find any on-list references to it being intentional,\n>>> but it appears that while we made the sha1collisiondetection variant\n>>> of SHA-1 the default in early 2017 we've never updated the OSX builds\n>>> to do likewise.\n>>>\n>>> I don't know what various git packages for OSX to, but our vanilla OSX\n>>> distribution definitely uses Apple Common Crypto, and won't detect the\n>>> https://shattered.io attack.\n>>>\n>>> This series changes that, and while doing so in 2/5 updates our\n>>> documentation and Makefile interface for the SHA-1 selection. Our\n>>> INSTALL file was still claiming we used OpenSSL's SHA-1 by default.\n>>>\n>>> Then since we'd made sha1collisiondetection the default we hadn't\n>>> changed the code's default fallback to be that, it was still\n>>> block-sha1. Now our fallback behavior is \"error\" instead, which makes\n>>> it less likely that we'll get some foot-gun like the \"OSX not using\n>>> sha1collisiondetection\" again.\n>>>\n>>> The 4/5 and 5/5 then remove the PPC_SHA1 implementation. I submitted\n>>> this before as [1], and the range-diff is to that submission (it\n>>> wasn't picked up). I think it makes sense as part of this general\n>>> SHA-1 cleanup.\n>>\n>> Thanks for this effort.\n>>\n>> I'd like to see somebody with \"building Git for distributing to\n>> macOS\" background to comment (I am assuming that the mailing list\n>> git-packagers@googlegroups.com is the way to reach them).\n>\n> *Bump* in case anyone there would like to chime on this bit of OSX\n> portability.\n>\n> I also sent an off-list E-Mail to Tim Harper today, and addresses which\n> I gather (from some old git-security@ traffic) are involved in packaging\n> the Apple Git shipped with OSX itself.\n>\n> In the meantime do you mind if this topic were queued up? If there are\n> any lingering portability concerns getting it into CI and exposed to\n> anyone else building on OSX would be a good thing.\n>\n> I don't see any reason for why we'd decide that OSX out of all our\n> supported platforms should be the only one where we're not mitigating\n> the SHAttered attack (and similar future attacks) by default.\n>\n> So aside from any finer details of OSX portability the direction here of\n> building with sha1collisiondetection by default on OSX by default seems\n> like a safe bet, just as we do on the rest of our (checks out\n> config.mak.uname) 20+ supported platforms (per `uname -s`).\n\nUpdate: I didn't get a reply from Tim or the people I E-Mailed\n@apple.com a week ago.\n\nJunio: I still think it makes sense to just queue this anyway & finally\nget the OSX build to use the SHAttered mitigation sha1collisiondetection\ngives us by default.\n"},{"id":"465230","messageId":"patch-v2-1.4-392fabdb456-20221019T010222Z-avarab@gmail.com","threadId":"57779","inReplyTo":"cover-v2-0.4-00000000000-20221019T010222Z-avarab@gmail.com","subject":"[PATCH v2 1/4] fsmonitor OSX: compile with DC_SHA1=YesPlease","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-10-19T01:03:19Z","receivedAt":"2022-10-19T01:03:36Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"As we'll address in subsequent commits the \"DC_SHA1=YesPlease\" is not\non by default on OSX, instead we use Apple Common Crypto's SHA-1\nimplementation.\n\nIn 6beb2688d33 (fsmonitor: relocate socket file if .git directory is\nremote, 2022-10-04) the build was broken with \"DC_SHA1=YesPlease\" (and\nprobably other non-\"APPLE_COMMON_CRYPTO\" SHA-1 backends).\n\nSo let's extract the fix for this from [1] to get the build working\nagain with \"DC_SHA1=YesPlease\". In addition to the fix in [1] we also\nneed to replace \"SHA_DIGEST_LENGTH\" with \"GIT_MAX_RAWSZ\".\n\n1. https://lore.kernel.org/git/c085fc15b314abcb5e5ca6b4ee5ac54a28327cab.1665326258.git.gitgitgadget@gmail.com/\n\nSigned-off-by: Eric DeCosta \nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n compat/fsmonitor/fsm-ipc-darwin.c | 10 +++++-----\n 1 file changed, 5 insertions(+), 5 deletions(-)\n\ndiff --git a/compat/fsmonitor/fsm-ipc-darwin.c b/compat/fsmonitor/fsm-ipc-darwin.c\nindex ce843d63348..d67b0ee50d3 100644\n--- a/compat/fsmonitor/fsm-ipc-darwin.c\n+++ b/compat/fsmonitor/fsm-ipc-darwin.c\n@@ -10,10 +10,10 @@ static GIT_PATH_FUNC(fsmonitor_ipc__get_default_path, \"fsmonitor--daemon.ipc\")\n const char *fsmonitor_ipc__get_path(struct repository *r)\n {\n \tstatic const char *ipc_path = NULL;\n-\tSHA_CTX sha1ctx;\n+\tgit_SHA_CTX sha1ctx;\n \tchar *sock_dir = NULL;\n \tstruct strbuf ipc_file = STRBUF_INIT;\n-\tunsigned char hash[SHA_DIGEST_LENGTH];\n+\tunsigned char hash[GIT_MAX_RAWSZ];\n \n \tif (!r)\n \t\tBUG(\"No repository passed into fsmonitor_ipc__get_path\");\n@@ -28,9 +28,9 @@ const char *fsmonitor_ipc__get_path(struct repository *r)\n \t\treturn ipc_path;\n \t}\n \n-\tSHA1_Init(&sha1ctx);\n-\tSHA1_Update(&sha1ctx, r->worktree, strlen(r->worktree));\n-\tSHA1_Final(hash, &sha1ctx);\n+\tgit_SHA1_Init(&sha1ctx);\n+\tgit_SHA1_Update(&sha1ctx, r->worktree, strlen(r->worktree));\n+\tgit_SHA1_Final(hash, &sha1ctx);\n \n \trepo_config_get_string(r, \"fsmonitor.socketdir\", &sock_dir);\n \n-- \n2.38.0.1093.gcd4a685f0b1\n\n"},{"id":"465231","messageId":"cover-v2-0.4-00000000000-20221019T010222Z-avarab@gmail.com","threadId":"57779","inReplyTo":"cover-0.5-00000000000-20220422T094624Z-avarab@gmail.com","subject":"[PATCH v2 0/4] core: update our SHA-1 docs, use sha1collisiondetection on OSX too","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-10-19T01:03:18Z","receivedAt":"2022-10-19T01:03:39Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"The sha1collisiondetection alternative to SHA-1 has been our default\neverywhere for a while, except on OSX. As 3/4 notes this seems to have\nbeen a mistake.\n\nFurthermore our documentation (INSTALL) was still claiming that we\nused OpenSSL for SHA-1 by default, etc. See 3/4 for all the details.\n\nThis series was previously submitted in April[1], at the time it\nincluded a change to remove the SHA-1 implementation in PPC\nassembly. That part has since landed on master in fd1ec82547d (Merge\nbranch 'ab/retire-ppc-sha1', 2022-09-09).\n\nI meant to re-send this anyway, but was prompted by the failure\nreport(s) in [2]. The 1/4 here un-breaks \"master\", and could be peeled\noff.\n\nFor a branch for this & passing CI see:\nhttps://github.com/avar/git/tree/avar/sha1dc-really-everywhere-and-no-ppc-sha1-2\n\n1. https://lore.kernel.org/git/cover-0.5-00000000000-20220422T094624Z-avarab@gmail.com/\n2. https://lore.kernel.org/git/xmqqo7u9wyt7.fsf@gitster.g/\n\nÆvar Arnfjörð Bjarmason (4):\n fsmonitor OSX: compile with DC_SHA1=YesPlease\n Makefile: create and use sections for \"define\" flag listing\n Makefile: really use and document sha1collisiondetection by default\n Makefile: rephrase the discussion of *_SHA1 knobs\n\n INSTALL | 10 +-\n Makefile | 288 +++++++++++++++++-----------\n ci/lib.sh | 3 +\n compat/fsmonitor/fsm-ipc-darwin.c | 10 +-\n contrib/buildsystems/CMakeLists.txt | 3 +-\n hash.h | 10 +-\n t/t0013-sha1dc.sh | 4 +-\n 7 files changed, 198 insertions(+), 130 deletions(-)\n\nRange-diff against v1:\n-: ----------- > 1: 392fabdb456 fsmonitor OSX: compile with DC_SHA1=YesPlease\n1: 3a80fcb6784 ! 2: 7ae22276aa7 Makefile: create and use sections for \"define\" flag listing\n @@ Makefile: include shared.mak\n -# Define BLK_SHA1 environment variable to make use of the bundled\n -# optimized C SHA1 routine.\n -#\n --# Define PPC_SHA1 environment variable when running make to make use of\n --# a bundled SHA1 routine optimized for PowerPC.\n --#\n -# Define DC_SHA1 to unconditionally enable the collision-detecting sha1\n -# algorithm. This is slower, but may detect attempted collision attacks.\n -# Takes priority over other *_SHA1 knobs.\n @@ Makefile: include shared.mak\n -#\n -# Define BLK_SHA256 to use the built-in SHA-256 routines.\n -#\n +-# Define NETTLE_SHA256 to use the SHA-256 routines in libnettle.\n +-#\n -# Define GCRYPT_SHA256 to use the SHA-256 routines in libgcrypt.\n -#\n -# Define OPENSSL_SHA256 to use the SHA-256 routines in OpenSSL.\n @@ Makefile: include shared.mak\n #\n # Define NEEDS_SSL_WITH_CRYPTO if you need -lssl when using -lcrypto (Darwin).\n @@ Makefile: include shared.mak\n - # `compat/fsmonitor/fsm-listen-.c` that implements the\n - # `fsm_listen__*()` routines.\n + # to the \"\" of the corresponding `compat/fsmonitor/fsm-settings-.c`\n + # that implements the `fsm_os_settings__*()` routines.\n #\n +# === Optional library: libintl ===\n +#\n @@ Makefile: include shared.mak\n +# Define BLK_SHA1 environment variable to make use of the bundled\n +# optimized C SHA1 routine.\n +#\n -+# Define PPC_SHA1 environment variable when running make to make use of\n -+# a bundled SHA1 routine optimized for PowerPC.\n -+#\n +# Define DC_SHA1 to unconditionally enable the collision-detecting sha1\n +# algorithm. This is slower, but may detect attempted collision attacks.\n +# Takes priority over other *_SHA1 knobs.\n @@ Makefile: include shared.mak\n +#\n +# Define BLK_SHA256 to use the built-in SHA-256 routines.\n +#\n ++# Define NETTLE_SHA256 to use the SHA-256 routines in libnettle.\n ++#\n +# Define GCRYPT_SHA256 to use the SHA-256 routines in libgcrypt.\n +#\n +# Define OPENSSL_SHA256 to use the SHA-256 routines in OpenSSL.\n2: 3e250dc9d09 ! 3: 78ef8636c57 Makefile: really use and document sha1collisiondetection by default\n @@ Commit message\n * Have a rewritten discussion of SHA-1 and SHA-256 in the Makefile\n which covers all of this.\n \n + Let's also change the CI for \"osx-clang\" to test with the new\n + APPLE_SHA1 knob (\"osx-gcc\" uses the new sha1collisiondetection\n + default).\n +\n + In practice this will spot issues like the one noted in [7], as\n + testing with just two backends should be enough to spot unportable\n + code. Ideally we'd have other CI jobs to test the various SHA-1\n + combinations, but for now we have better CI coverage than before.\n +\n 1. 48b3693d3ce (Merge branch 'jk/sha1dc', 2017-03-24)\n 2. 8325e43b82d (Makefile: add DC_SHA1 knob, 2017-03-16)\n 3. e6b07da2780 (Makefile: make DC_SHA1 the default, 2017-03-17)\n @@ Commit message\n 2013-05-19)\n 6. 3ef2bcad02e (imap-send: use Apple's Security framework for base64\n encoding, 2013-07-29)\n + 7. https://lore.kernel.org/git/kl6l7d0yyu6r.fsf@chooglen-macbookpro.roam.corp.google.com/\n \n Signed-off-by: Ævar Arnfjörð Bjarmason \n \n @@ INSTALL: Issues of note:\n \n -\t By default, git uses OpenSSL for SHA1 but it will use its own\n -\t library (inspired by Mozilla's) with either NO_OPENSSL or\n --\t BLK_SHA1. Also included is a version optimized for PowerPC\n --\t (PPC_SHA1).\n +-\t BLK_SHA1.\n +\t- Git uses an altered version of SHA-1 by default which\n -+ detects the SHAttered attack via the sha1collisiondetection\n -+ counter-cryptanalysis library. For SHA-256 we'll select a\n -+ working implementation (and ship a fallback\n -+ implementation). See the \"SHA-1 and SHA-256 defines\" section\n -+ in the Makefile for details. You should not need to tweak\n -+ those settings.\n ++\t detects the SHAttered attack via the sha1collisiondetection\n ++\t counter-cryptanalysis library. For SHA-256 we'll select a\n ++\t working implementation (and ship a fallback\n ++\t implementation). See the \"SHA-1 and SHA-256 defines\" section\n ++\t in the Makefile for details. You should not need to tweak\n ++\t those settings.\n \n \t- \"libcurl\" library is used for fetching and pushing\n \t repositories over http:// or https://, as well as by\n @@ Makefile: include shared.mak\n -# Define BLK_SHA1 environment variable to make use of the bundled\n -# optimized C SHA1 routine.\n +# === SHA-1 backend ===\n - #\n --# Define PPC_SHA1 environment variable when running make to make use of\n --# a bundled SHA1 routine optimized for PowerPC.\n ++#\n +# Due to the SHAttered (https://shattered.io) attack vector on SHA-1\n +# Git uses the sha1collisiondetection counter-cryptanalysis library\n +# for SHA-1 hashing.\n +#\n +# You're strongly advised not to override this for any usage of Git\n +# where you don't 100% trust the repository content.\n -+#\n -+# ==== Options common to all SHA-1 implementations ====\n -+#\n -+# Define SHA1_MAX_BLOCK_SIZE to limit the amount of data that will be hashed\n -+# in one call to the platform's SHA1_Update(). e.g. APPLE_COMMON_CRYPTO\n -+# wants 'SHA1_MAX_BLOCK_SIZE=1024L*1024L*1024L' defined.\n #\n -# Define DC_SHA1 to unconditionally enable the collision-detecting sha1\n -# algorithm. This is slower, but may detect attempted collision attacks.\n -# Takes priority over other *_SHA1 knobs.\n -+# ===== Options for the default sha1collisiondetection implementations =====\n ++# ==== Options common to all SHA-1 implementations ====\n #\n -# Define DC_SHA1_EXTERNAL in addition to DC_SHA1 if you want to build / link\n ++# Define SHA1_MAX_BLOCK_SIZE to limit the amount of data that will be hashed\n ++# in one call to the platform's SHA1_Update(). e.g. APPLE_COMMON_CRYPTO\n ++# wants 'SHA1_MAX_BLOCK_SIZE=1024L*1024L*1024L' defined.\n ++#\n ++# ===== Options for the default sha1collisiondetection implementations =====\n ++#\n +# Define DC_SHA1_EXTERNAL if you want to build / link\n # git with the external SHA1 collision-detect library.\n # Without this option, i.e. the default behavior is to build git with its\n @@ Makefile: include shared.mak\n -# Define SHA1_MAX_BLOCK_SIZE to limit the amount of data that will be hashed\n -# in one call to the platform's SHA1_Update(). e.g. APPLE_COMMON_CRYPTO\n -# wants 'SHA1_MAX_BLOCK_SIZE=1024L*1024L*1024L' defined.\n -+# Define PPC_SHA1 environment variable when running make to make use of\n -+# a bundled SHA1 routine optimized for PowerPC.\n -+#\n +# Define APPLE_SHA1 to use Apple's CommonCrypto SHA-1 routines on\n +# Darwin/Mac OS X.\n +#\n @@ Makefile: ifeq ($(uname_S),Darwin)\n -\t\tAPPLE_COMMON_CRYPTO = YesPlease\n \t\tCOMPAT_CFLAGS += -DAPPLE_COMMON_CRYPTO\n \tendif\n - \tNO_REGEX = YesPlease\n + \tPTHREAD_LIBS =\n @@ Makefile: ifdef NO_POSIX_GOODIES\n \tBASIC_CFLAGS += -DNO_POSIX_GOODIES\n endif\n @@ Makefile: ifdef NO_POSIX_GOODIES\n -ifdef APPLE_COMMON_CRYPTO\n -\t# Apple CommonCrypto requires chunking\n -\tSHA1_MAX_BLOCK_SIZE = 1024L*1024L*1024L\n -+ifdef DC_SHA1\n -+$(error the DC_SHA1 flag is no longer used, and has become the default. Adjust your build scripts accordingly)\n - endif\n +-endif\n -\n + ifdef PPC_SHA1\n + $(error the PPC_SHA1 flag has been removed along with the PowerPC-specific SHA-1 implementation.)\n + endif\n + \n -ifdef OPENSSL_SHA1\n -\tEXTLIBS += $(LIB_4_CRYPTO)\n -\tBASIC_CFLAGS += -DSHA1_OPENSSL\n @@ Makefile: ifdef NO_POSIX_GOODIES\n -\tLIB_OBJS += block-sha1/sha1.o\n -\tBASIC_CFLAGS += -DSHA1_BLK\n -else\n --ifdef PPC_SHA1\n --\tLIB_OBJS += ppc/sha1.o ppc/sha1ppc.o\n --\tBASIC_CFLAGS += -DSHA1_PPC\n --else\n -ifdef APPLE_COMMON_CRYPTO\n -\tCOMPAT_CFLAGS += -DCOMMON_DIGEST_FOR_OPENSSL\n -\tBASIC_CFLAGS += -DSHA1_APPLE\n -else\n -\tDC_SHA1 := YesPlease\n -\tBASIC_CFLAGS += -DSHA1_DC\n ++ifdef DC_SHA1\n ++$(error the DC_SHA1 flag is no longer used, and has become the default. Adjust your build scripts accordingly)\n ++endif\n +ifndef NO_DC_SHA1\n -+\tifneq ($(OPENSSL_SHA1)$(BLK_SHA1)$(PPC_SHA1)$(APPLE_SHA1),)\n ++\tifneq ($(OPENSSL_SHA1)$(BLK_SHA1)$(APPLE_SHA1),)\n +$(error no other *_SHA1 option can be defined unless NO_DC_SHA1 is defined)\n +\tendif\n \tLIB_OBJS += sha1dc_git.o\n @@ Makefile: endif\n +\tLIB_OBJS += block-sha1/sha1.o\n +\tBASIC_CFLAGS += -DSHA1_BLK\n +else\n -+ifdef PPC_SHA1\n -+\tLIB_OBJS += ppc/sha1.o ppc/sha1ppc.o\n -+\tBASIC_CFLAGS += -DSHA1_PPC\n -+else\n +ifdef APPLE_SHA1\n +\tCOMPAT_CFLAGS += -DCOMMON_DIGEST_FOR_OPENSSL\n +\tBASIC_CFLAGS += -DSHA1_APPLE\n @@ Makefile: endif\n endif\n endif\n @@ Makefile: GIT-BUILD-OPTIONS: FORCE\n - \t@echo NO_PYTHON=\\''$(subst ','\\'',$(subst ','\\'',$(NO_PYTHON)))'\\' >>$@+\n + \t@echo NO_REGEX=\\''$(subst ','\\'',$(subst ','\\'',$(NO_REGEX)))'\\' >>$@+\n \t@echo NO_UNIX_SOCKETS=\\''$(subst ','\\'',$(subst ','\\'',$(NO_UNIX_SOCKETS)))'\\' >>$@+\n \t@echo PAGER_ENV=\\''$(subst ','\\'',$(subst ','\\'',$(PAGER_ENV)))'\\' >>$@+\n -\t@echo DC_SHA1=\\''$(subst ','\\'',$(subst ','\\'',$(DC_SHA1)))'\\' >>$@+\n @@ Makefile: GIT-BUILD-OPTIONS: FORCE\n \t@echo SANITIZE_ADDRESS=\\''$(subst ','\\'',$(subst ','\\'',$(SANITIZE_ADDRESS)))'\\' >>$@+\n \t@echo X=\\'$(X)\\' >>$@+\n \n + ## ci/lib.sh ##\n +@@ ci/lib.sh: macos-latest)\n + esac\n + \n + case \"$jobname\" in\n ++osx-clang)\n ++\tMAKEFLAGS=\"$MAKEFLAGS NO_DC_SHA1=Yes APPLE_SHA1=Yes\"\n ++\t;;\n + linux32)\n + \tCC=gcc\n + \t;;\n +\n ## contrib/buildsystems/CMakeLists.txt ##\n @@ contrib/buildsystems/CMakeLists.txt: set(NO_PERL )\n set(NO_PTHREADS )\n @@ hash.h\n #include \"git-compat-util.h\"\n #include \"repository.h\"\n \n --#if defined(SHA1_PPC)\n +-#if defined(SHA1_APPLE)\n +#if !defined(NO_SHA1_DC)\n +#include \"sha1dc_git.h\"\n -+#elif defined(SHA1_PPC)\n - #include \"ppc/sha1.h\"\n - #elif defined(SHA1_APPLE)\n ++#elif defined(SHA1_APPLE)\n #include \n #elif defined(SHA1_OPENSSL)\n #include \n @@ hash.h\n +#error \"need a SHA1_* implementation defined\"\n #endif\n \n - #if defined(SHA256_GCRYPT)\n + #if defined(SHA256_NETTLE)\n \n ## t/t0013-sha1dc.sh ##\n @@ t/t0013-sha1dc.sh: TEST_PASSES_SANITIZE_LEAK=true\n3: 1422c6fd497 ! 4: f1fb9775b33 Makefile: rephrase the discussion of *_SHA1 knobs\n @@ Makefile: include shared.mak\n +# Define OPENSSL_SHA1 to link to the the SHA-1 routines from\n +# the OpenSSL library.\n #\n --# Define PPC_SHA1 environment variable when running make to make use of\n --# a bundled SHA1 routine optimized for PowerPC.\n -+# Define PPC_SHA1 to make use of optimized (in assembly)\n -+# PowerPC SHA-1 routines.\n - #\n # Define APPLE_SHA1 to use Apple's CommonCrypto SHA-1 routines on\n # Darwin/Mac OS X.\n4: ae464dbd228 < -: ----------- Makefile + hash.h: remove PPC_SHA1 implementation\n5: 730d08da6fe < -: ----------- Makefile: use $(OBJECTS) instead of $(C_OBJ)\n-- \n2.38.0.1093.gcd4a685f0b1\n\n"},{"id":"465232","messageId":"patch-v2-3.4-78ef8636c57-20221019T010222Z-avarab@gmail.com","threadId":"57779","inReplyTo":"cover-v2-0.4-00000000000-20221019T010222Z-avarab@gmail.com","subject":"[PATCH v2 3/4] Makefile: really use and document sha1collisiondetection by default","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-10-19T01:03:21Z","receivedAt":"2022-10-19T01:03:41Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"When the sha1collisiondetection library was added and made the default\nin [1] we never updated the documentation added in [2] early in that\nseries once the default was flipped to DC_SHA1 in [3]. Furthermore the\nINSTALL file has been claiming that we use OpenSSL by default since\n[4], and hadn't been updated for the sha1collisiondetection switch.\n\nThe interaction between NO_APPLE_COMMON_CRYPTO and DC_SHA1 seems to\nhave been missed in [3], so ever since DC_SHA1 was made the default\nwe've still used Apple's CommonCrypto instead of\nsha1collisiondetection on Darwin and Mac OS X.\n\nInstead off all of this we now:\n\n* Don't have a DC_SHA1 know anymore (using it is an error), you need\n to set NO_DC_SHA1 instead to use any optional *_SHA1 implementation.\n* Re-arranged the algorithm inclusion in hash.h to correspond to\n NO_DC_SHA1, and \"#error\" if we have no defined SHA_*, rather than\n silently picking block-sha1/sha1.h as a fallback.\n* Have an INSTALL that reflects reality. We were still claiming to use\n OpenSSL's SHA-1 hashing by default.\n* Have Darwin and Mac OS X use sha1collisiondetection, like everywhere\n else. There is still a NO_APPLE_COMMON_CRYPTO knob, but it's used for\n things unrelated to SHA-1 (see [6]).\n* Have a rewritten discussion of SHA-1 and SHA-256 in the Makefile\n which covers all of this.\n\nLet's also change the CI for \"osx-clang\" to test with the new\nAPPLE_SHA1 knob (\"osx-gcc\" uses the new sha1collisiondetection\ndefault).\n\nIn practice this will spot issues like the one noted in [7], as\ntesting with just two backends should be enough to spot unportable\ncode. Ideally we'd have other CI jobs to test the various SHA-1\ncombinations, but for now we have better CI coverage than before.\n\n1. 48b3693d3ce (Merge branch 'jk/sha1dc', 2017-03-24)\n2. 8325e43b82d (Makefile: add DC_SHA1 knob, 2017-03-16)\n3. e6b07da2780 (Makefile: make DC_SHA1 the default, 2017-03-17)\n4. 5beb577db8c (INSTALL: Describe dependency knobs from Makefile,\n 2009-09-10)\n5. 4dcd7732db0 (Makefile: add support for Apple CommonCrypto facility,\n 2013-05-19)\n6. 3ef2bcad02e (imap-send: use Apple's Security framework for base64\n encoding, 2013-07-29)\n7. https://lore.kernel.org/git/kl6l7d0yyu6r.fsf@chooglen-macbookpro.roam.corp.google.com/\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n INSTALL | 10 ++-\n Makefile | 98 ++++++++++++++++++++---------\n ci/lib.sh | 3 +\n contrib/buildsystems/CMakeLists.txt | 3 +-\n hash.h | 10 +--\n t/t0013-sha1dc.sh | 4 +-\n 6 files changed, 86 insertions(+), 42 deletions(-)\n\ndiff --git a/INSTALL b/INSTALL\nindex 89b15d71df5..065ed81bd54 100644\n--- a/INSTALL\n+++ b/INSTALL\n@@ -133,9 +133,13 @@ Issues of note:\n \t you are using libcurl older than 7.34.0. Otherwise you can use\n \t NO_OPENSSL without losing git-imap-send.\n \n-\t By default, git uses OpenSSL for SHA1 but it will use its own\n-\t library (inspired by Mozilla's) with either NO_OPENSSL or\n-\t BLK_SHA1.\n+\t- Git uses an altered version of SHA-1 by default which\n+\t detects the SHAttered attack via the sha1collisiondetection\n+\t counter-cryptanalysis library. For SHA-256 we'll select a\n+\t working implementation (and ship a fallback\n+\t implementation). See the \"SHA-1 and SHA-256 defines\" section\n+\t in the Makefile for details. You should not need to tweak\n+\t those settings.\n \n \t- \"libcurl\" library is used for fetching and pushing\n \t repositories over http:// or https://, as well as by\ndiff --git a/Makefile b/Makefile\nindex 18ad487274e..7a7411df8c3 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -479,30 +479,62 @@ include shared.mak\n #\n # == SHA-1 and SHA-256 defines ==\n #\n-# Define BLK_SHA1 environment variable to make use of the bundled\n-# optimized C SHA1 routine.\n+# === SHA-1 backend ===\n+#\n+# Due to the SHAttered (https://shattered.io) attack vector on SHA-1\n+# Git uses the sha1collisiondetection counter-cryptanalysis library\n+# for SHA-1 hashing.\n+#\n+# You're strongly advised not to override this for any usage of Git\n+# where you don't 100% trust the repository content.\n #\n-# Define DC_SHA1 to unconditionally enable the collision-detecting sha1\n-# algorithm. This is slower, but may detect attempted collision attacks.\n-# Takes priority over other *_SHA1 knobs.\n+# ==== Options common to all SHA-1 implementations ====\n #\n-# Define DC_SHA1_EXTERNAL in addition to DC_SHA1 if you want to build / link\n+# Define SHA1_MAX_BLOCK_SIZE to limit the amount of data that will be hashed\n+# in one call to the platform's SHA1_Update(). e.g. APPLE_COMMON_CRYPTO\n+# wants 'SHA1_MAX_BLOCK_SIZE=1024L*1024L*1024L' defined.\n+#\n+# ===== Options for the default sha1collisiondetection implementations =====\n+#\n+# Define DC_SHA1_EXTERNAL if you want to build / link\n # git with the external SHA1 collision-detect library.\n # Without this option, i.e. the default behavior is to build git with its\n # own built-in code (or submodule).\n #\n-# Define DC_SHA1_SUBMODULE in addition to DC_SHA1 to use the\n+# Define DC_SHA1_SUBMODULE to use the\n # sha1collisiondetection shipped as a submodule instead of the\n # non-submodule copy in sha1dc/. This is an experimental option used\n # by the git project to migrate to using sha1collisiondetection as a\n # submodule.\n #\n+# ==== Alternate implementations ====\n+#\n+# Git still ships with alternate SHA-1 implementations. These are\n+# faster than the default, which is useful when hashing speed\n+# is imperative, consider using them if you're confident that you\n+# won't need to worry about SHA-1 collision attacks.\n+#\n+# To use them you must define NO_DC_SHA1 and one of the *_SHA1\n+# variables below:\n+#\n+# Define BLK_SHA1 environment variable to make use of the bundled\n+# optimized C SHA1 routine.\n+#\n # Define OPENSSL_SHA1 environment variable when running make to link\n # with the SHA1 routine from openssl library.\n #\n-# Define SHA1_MAX_BLOCK_SIZE to limit the amount of data that will be hashed\n-# in one call to the platform's SHA1_Update(). e.g. APPLE_COMMON_CRYPTO\n-# wants 'SHA1_MAX_BLOCK_SIZE=1024L*1024L*1024L' defined.\n+# Define APPLE_SHA1 to use Apple's CommonCrypto SHA-1 routines on\n+# Darwin/Mac OS X.\n+#\n+# The APPLE_SHA1 option is unrelated to the NO_APPLE_COMMON_CRYPTO\n+# flag, which determines if Apple's crypto libraries are used for\n+# things that aren't SHA-1.\n+#\n+# === SHA-256 backend ===\n+#\n+# Unlike SHA-1 the SHA-256 algorithm does not suffer from any known\n+# vulnerabilities, so any implementation will do. BLK_SHA256 is\n+# currently the default implementation (but that may change).\n #\n # Define BLK_SHA256 to use the built-in SHA-256 routines.\n #\n@@ -1464,7 +1496,6 @@ ifeq ($(uname_S),Darwin)\n \tendif\n \tifndef NO_APPLE_COMMON_CRYPTO\n \t\tNO_OPENSSL = YesPlease\n-\t\tAPPLE_COMMON_CRYPTO = YesPlease\n \t\tCOMPAT_CFLAGS += -DAPPLE_COMMON_CRYPTO\n \tendif\n \tPTHREAD_LIBS =\n@@ -1825,30 +1856,19 @@ ifdef NO_POSIX_GOODIES\n \tBASIC_CFLAGS += -DNO_POSIX_GOODIES\n endif\n \n-ifdef APPLE_COMMON_CRYPTO\n-\t# Apple CommonCrypto requires chunking\n-\tSHA1_MAX_BLOCK_SIZE = 1024L*1024L*1024L\n-endif\n-\n ifdef PPC_SHA1\n $(error the PPC_SHA1 flag has been removed along with the PowerPC-specific SHA-1 implementation.)\n endif\n \n-ifdef OPENSSL_SHA1\n-\tEXTLIBS += $(LIB_4_CRYPTO)\n-\tBASIC_CFLAGS += -DSHA1_OPENSSL\n-else\n-ifdef BLK_SHA1\n-\tLIB_OBJS += block-sha1/sha1.o\n-\tBASIC_CFLAGS += -DSHA1_BLK\n-else\n-ifdef APPLE_COMMON_CRYPTO\n-\tCOMPAT_CFLAGS += -DCOMMON_DIGEST_FOR_OPENSSL\n-\tBASIC_CFLAGS += -DSHA1_APPLE\n-else\n-\tDC_SHA1 := YesPlease\n-\tBASIC_CFLAGS += -DSHA1_DC\n+ifdef DC_SHA1\n+$(error the DC_SHA1 flag is no longer used, and has become the default. Adjust your build scripts accordingly)\n+endif\n+ifndef NO_DC_SHA1\n+\tifneq ($(OPENSSL_SHA1)$(BLK_SHA1)$(APPLE_SHA1),)\n+$(error no other *_SHA1 option can be defined unless NO_DC_SHA1 is defined)\n+\tendif\n \tLIB_OBJS += sha1dc_git.o\n+\n ifdef DC_SHA1_EXTERNAL\n \tifdef DC_SHA1_SUBMODULE\n \t\tifneq ($(DC_SHA1_SUBMODULE),auto)\n@@ -1872,6 +1892,22 @@ endif\n \t\t-DSHA1DC_CUSTOM_INCLUDE_SHA1_C=\"\\\"cache.h\\\"\" \\\n \t\t-DSHA1DC_CUSTOM_INCLUDE_UBC_CHECK_C=\"\\\"git-compat-util.h\\\"\"\n endif\n+else # !NO_DC_SHA1\n+BASIC_CFLAGS += -DNO_SHA1_DC\n+ifdef OPENSSL_SHA1\n+\tEXTLIBS += $(LIB_4_CRYPTO)\n+\tBASIC_CFLAGS += -DSHA1_OPENSSL\n+else\n+ifdef BLK_SHA1\n+\tLIB_OBJS += block-sha1/sha1.o\n+\tBASIC_CFLAGS += -DSHA1_BLK\n+else\n+ifdef APPLE_SHA1\n+\tCOMPAT_CFLAGS += -DCOMMON_DIGEST_FOR_OPENSSL\n+\tBASIC_CFLAGS += -DSHA1_APPLE\n+else\n+$(error when defining NO_DC_SHA1 another valid *_SHA1 variable must be defined!)\n+endif\n endif\n endif\n endif\n@@ -3009,7 +3045,7 @@ GIT-BUILD-OPTIONS: FORCE\n \t@echo NO_REGEX=\\''$(subst ','\\'',$(subst ','\\'',$(NO_REGEX)))'\\' >>$@+\n \t@echo NO_UNIX_SOCKETS=\\''$(subst ','\\'',$(subst ','\\'',$(NO_UNIX_SOCKETS)))'\\' >>$@+\n \t@echo PAGER_ENV=\\''$(subst ','\\'',$(subst ','\\'',$(PAGER_ENV)))'\\' >>$@+\n-\t@echo DC_SHA1=\\''$(subst ','\\'',$(subst ','\\'',$(DC_SHA1)))'\\' >>$@+\n+\t@echo NO_DC_SHA1=\\''$(subst ','\\'',$(subst ','\\'',$(NO_DC_SHA1)))'\\' >>$@+\n \t@echo SANITIZE_LEAK=\\''$(subst ','\\'',$(subst ','\\'',$(SANITIZE_LEAK)))'\\' >>$@+\n \t@echo SANITIZE_ADDRESS=\\''$(subst ','\\'',$(subst ','\\'',$(SANITIZE_ADDRESS)))'\\' >>$@+\n \t@echo X=\\'$(X)\\' >>$@+\ndiff --git a/ci/lib.sh b/ci/lib.sh\nindex 1b0cc2b57db..320f992680a 100755\n--- a/ci/lib.sh\n+++ b/ci/lib.sh\n@@ -264,6 +264,9 @@ macos-latest)\n esac\n \n case \"$jobname\" in\n+osx-clang)\n+\tMAKEFLAGS=\"$MAKEFLAGS NO_DC_SHA1=Yes APPLE_SHA1=Yes\"\n+\t;;\n linux32)\n \tCC=gcc\n \t;;\ndiff --git a/contrib/buildsystems/CMakeLists.txt b/contrib/buildsystems/CMakeLists.txt\nindex 787738e6fa3..14ac3d49849 100644\n--- a/contrib/buildsystems/CMakeLists.txt\n+++ b/contrib/buildsystems/CMakeLists.txt\n@@ -1025,7 +1025,6 @@ set(NO_PERL )\n set(NO_PTHREADS )\n set(NO_PYTHON )\n set(PAGER_ENV \"LESS=FRX LV=-c\")\n-set(DC_SHA1 YesPlease)\n set(RUNTIME_PREFIX true)\n set(NO_GETTEXT )\n \n@@ -1061,7 +1060,7 @@ file(APPEND ${CMAKE_BINARY_DIR}/GIT-BUILD-OPTIONS \"NO_PERL='${NO_PERL}'\\n\")\n file(APPEND ${CMAKE_BINARY_DIR}/GIT-BUILD-OPTIONS \"NO_PTHREADS='${NO_PTHREADS}'\\n\")\n file(APPEND ${CMAKE_BINARY_DIR}/GIT-BUILD-OPTIONS \"NO_UNIX_SOCKETS='${NO_UNIX_SOCKETS}'\\n\")\n file(APPEND ${CMAKE_BINARY_DIR}/GIT-BUILD-OPTIONS \"PAGER_ENV='${PAGER_ENV}'\\n\")\n-file(APPEND ${CMAKE_BINARY_DIR}/GIT-BUILD-OPTIONS \"DC_SHA1='${DC_SHA1}'\\n\")\n+file(APPEND ${CMAKE_BINARY_DIR}/GIT-BUILD-OPTIONS \"NO_DC_SHA1=''\\n\")\n file(APPEND ${CMAKE_BINARY_DIR}/GIT-BUILD-OPTIONS \"X='${EXE_EXTENSION}'\\n\")\n file(APPEND ${CMAKE_BINARY_DIR}/GIT-BUILD-OPTIONS \"NO_GETTEXT='${NO_GETTEXT}'\\n\")\n file(APPEND ${CMAKE_BINARY_DIR}/GIT-BUILD-OPTIONS \"RUNTIME_PREFIX='${RUNTIME_PREFIX}'\\n\")\ndiff --git a/hash.h b/hash.h\nindex 36b64165fc9..a7337779949 100644\n--- a/hash.h\n+++ b/hash.h\n@@ -4,14 +4,16 @@\n #include \"git-compat-util.h\"\n #include \"repository.h\"\n \n-#if defined(SHA1_APPLE)\n+#if !defined(NO_SHA1_DC)\n+#include \"sha1dc_git.h\"\n+#elif defined(SHA1_APPLE)\n #include \n #elif defined(SHA1_OPENSSL)\n #include \n-#elif defined(SHA1_DC)\n-#include \"sha1dc_git.h\"\n-#else /* SHA1_BLK */\n+#elif defined(SHA1_BLK)\n #include \"block-sha1/sha1.h\"\n+#else\n+#error \"need a SHA1_* implementation defined\"\n #endif\n \n #if defined(SHA256_NETTLE)\ndiff --git a/t/t0013-sha1dc.sh b/t/t0013-sha1dc.sh\nindex 9ad76080aa4..539270a2665 100755\n--- a/t/t0013-sha1dc.sh\n+++ b/t/t0013-sha1dc.sh\n@@ -6,9 +6,9 @@ TEST_PASSES_SANITIZE_LEAK=true\n . ./test-lib.sh\n TEST_DATA=\"$TEST_DIRECTORY/t0013\"\n \n-if test -z \"$DC_SHA1\"\n+if test -n \"$NO_DC_SHA1\"\n then\n-\tskip_all='skipping sha1 collision tests, DC_SHA1 not set'\n+\tskip_all='skipping sha1 collision tests, NO_DC_SHA1 set'\n \ttest_done\n fi\n \n-- \n2.38.0.1093.gcd4a685f0b1\n\n"},{"id":"465233","messageId":"patch-v2-4.4-f1fb9775b33-20221019T010222Z-avarab@gmail.com","threadId":"57779","inReplyTo":"cover-v2-0.4-00000000000-20221019T010222Z-avarab@gmail.com","subject":"[PATCH v2 4/4] Makefile: rephrase the discussion of *_SHA1 knobs","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-10-19T01:03:22Z","receivedAt":"2022-10-19T01:03:45Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"In the preceding commit the discussion of the *_SHA1 knobs was left\nas-is to benefit from a smaller diff, but since we're changing these\nlet's use the same phrasing we use for most other knobs. E.g. \"define\nX\", not \"define X environment variable\", and get rid of the \"when\nrunning make to link with\" entirely.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n Makefile | 8 ++++----\n 1 file changed, 4 insertions(+), 4 deletions(-)\n\ndiff --git a/Makefile b/Makefile\nindex 7a7411df8c3..16367c72ba8 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -517,11 +517,11 @@ include shared.mak\n # To use them you must define NO_DC_SHA1 and one of the *_SHA1\n # variables below:\n #\n-# Define BLK_SHA1 environment variable to make use of the bundled\n-# optimized C SHA1 routine.\n+# Define BLK_SHA1 to make use of optimized C SHA-1 routines bundled\n+# with git (in the block-sha1/ directory).\n #\n-# Define OPENSSL_SHA1 environment variable when running make to link\n-# with the SHA1 routine from openssl library.\n+# Define OPENSSL_SHA1 to link to the the SHA-1 routines from\n+# the OpenSSL library.\n #\n # Define APPLE_SHA1 to use Apple's CommonCrypto SHA-1 routines on\n # Darwin/Mac OS X.\n-- \n2.38.0.1093.gcd4a685f0b1\n\n"},{"id":"465234","messageId":"patch-v2-2.4-7ae22276aa7-20221019T010222Z-avarab@gmail.com","threadId":"57779","inReplyTo":"cover-v2-0.4-00000000000-20221019T010222Z-avarab@gmail.com","subject":"[PATCH v2 2/4] Makefile: create and use sections for \"define\" flag listing","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-10-19T01:03:20Z","receivedAt":"2022-10-19T01:03:47Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Since the \"Define ...\" template of comments at the top of the Makefile\nwas started in 5bdac8b3269 ([PATCH] Improve the compilation-time\nsettings interface, 2005-07-29) we've had a lot more flags added,\nincluding flags that come in \"groups\". Not having any obvious\nstructure to the >500 line comment at the top of the Makefile has made\nit hard to follow.\n\nThis change is almost entirely a move-only change, the two paragraphs\nat the start of the first two sections are new, and so are the added\nsections themselves, but other than that no lines are changed, only\nmoved.\n\nWe now list Makefile-only flags at the start, followed by stand-alone\nflags, and then cover \"optional library\" flags in their respective\ngroups, followed by SHA-1 and SHA-256 flags, and finally\nDEVELOPER-specific flags.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n Makefile | 210 +++++++++++++++++++++++++++++++------------------------\n 1 file changed, 117 insertions(+), 93 deletions(-)\n\ndiff --git a/Makefile b/Makefile\nindex d93ad956e58..18ad487274e 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -4,8 +4,20 @@ all::\n # Import tree-wide shared Makefile behavior and libraries\n include shared.mak\n \n+# == Makefile defines ==\n+#\n+# These defines change the behavior of the Makefile itself, but have\n+# no impact on what it builds:\n+#\n # Define V=1 to have a more verbose compile.\n #\n+# == Portability and optional library defines ==\n+#\n+# These defines indicate what Git can expect from the OS, what\n+# libraries are available etc. Much of this is auto-detected in\n+# config.mak.uname, or in configure.ac when using the optional \"make\n+# configure && ./configure\" (see INSTALL).\n+#\n # Define SHELL_PATH to a POSIX shell if your /bin/sh is broken.\n #\n # Define SANE_TOOL_PATH to a colon-separated list of paths to prepend\n@@ -30,68 +42,8 @@ include shared.mak\n #\n # Define NO_OPENSSL environment variable if you do not have OpenSSL.\n #\n-# Define USE_LIBPCRE if you have and want to use libpcre. Various\n-# commands such as log and grep offer runtime options to use\n-# Perl-compatible regular expressions instead of standard or extended\n-# POSIX regular expressions.\n-#\n-# Only libpcre version 2 is supported. USE_LIBPCRE2 is a synonym for\n-# USE_LIBPCRE, support for the old USE_LIBPCRE1 has been removed.\n-#\n-# Define LIBPCREDIR=/foo/bar if your PCRE header and library files are\n-# in /foo/bar/include and /foo/bar/lib directories.\n-#\n # Define HAVE_ALLOCA_H if you have working alloca(3) defined in that header.\n #\n-# Define NO_CURL if you do not have libcurl installed. git-http-fetch and\n-# git-http-push are not built, and you cannot use http:// and https://\n-# transports (neither smart nor dumb).\n-#\n-# Define CURLDIR=/foo/bar if your curl header and library files are in\n-# /foo/bar/include and /foo/bar/lib directories.\n-#\n-# Define CURL_CONFIG to curl's configuration program that prints information\n-# about the library (e.g., its version number). The default is 'curl-config'.\n-#\n-# Define CURL_LDFLAGS to specify flags that you need to link when using libcurl,\n-# if you do not want to rely on the libraries provided by CURL_CONFIG. The\n-# default value is a result of `curl-config --libs`. An example value for\n-# CURL_LDFLAGS is as follows:\n-#\n-# CURL_LDFLAGS=-lcurl\n-#\n-# Define NO_EXPAT if you do not have expat installed. git-http-push is\n-# not built, and you cannot push using http:// and https:// transports (dumb).\n-#\n-# Define EXPATDIR=/foo/bar if your expat header and library files are in\n-# /foo/bar/include and /foo/bar/lib directories.\n-#\n-# Define EXPAT_NEEDS_XMLPARSE_H if you have an old version of expat (e.g.,\n-# 1.1 or 1.2) that provides xmlparse.h instead of expat.h.\n-#\n-# Define NO_GETTEXT if you don't want Git output to be translated.\n-# A translated Git requires GNU libintl or another gettext implementation,\n-# plus libintl-perl at runtime.\n-#\n-# Define USE_GETTEXT_SCHEME and set it to 'fallthrough', if you don't trust\n-# the installed gettext translation of the shell scripts output.\n-#\n-# Define HAVE_LIBCHARSET_H if you haven't set NO_GETTEXT and you can't\n-# trust the langinfo.h's nl_langinfo(CODESET) function to return the\n-# current character set. GNU and Solaris have a nl_langinfo(CODESET),\n-# FreeBSD can use either, but MinGW and some others need to use\n-# libcharset.h's locale_charset() instead.\n-#\n-# Define CHARSET_LIB to the library you need to link with in order to\n-# use locale_charset() function. On some platforms this needs to set to\n-# -lcharset, on others to -liconv .\n-#\n-# Define LIBC_CONTAINS_LIBINTL if your gettext implementation doesn't\n-# need -lintl when linking.\n-#\n-# Define NO_MSGFMT_EXTENDED_OPTIONS if your implementation of msgfmt\n-# doesn't support GNU extensions like --check and --statistics\n-#\n # Define HAVE_PATHS_H if you have paths.h and want to use the default PATH\n # it specifies.\n #\n@@ -152,39 +104,6 @@ include shared.mak\n # and do not want to use Apple's CommonCrypto library. This allows you\n # to provide your own OpenSSL library, for example from MacPorts.\n #\n-# Define BLK_SHA1 environment variable to make use of the bundled\n-# optimized C SHA1 routine.\n-#\n-# Define DC_SHA1 to unconditionally enable the collision-detecting sha1\n-# algorithm. This is slower, but may detect attempted collision attacks.\n-# Takes priority over other *_SHA1 knobs.\n-#\n-# Define DC_SHA1_EXTERNAL in addition to DC_SHA1 if you want to build / link\n-# git with the external SHA1 collision-detect library.\n-# Without this option, i.e. the default behavior is to build git with its\n-# own built-in code (or submodule).\n-#\n-# Define DC_SHA1_SUBMODULE in addition to DC_SHA1 to use the\n-# sha1collisiondetection shipped as a submodule instead of the\n-# non-submodule copy in sha1dc/. This is an experimental option used\n-# by the git project to migrate to using sha1collisiondetection as a\n-# submodule.\n-#\n-# Define OPENSSL_SHA1 environment variable when running make to link\n-# with the SHA1 routine from openssl library.\n-#\n-# Define SHA1_MAX_BLOCK_SIZE to limit the amount of data that will be hashed\n-# in one call to the platform's SHA1_Update(). e.g. APPLE_COMMON_CRYPTO\n-# wants 'SHA1_MAX_BLOCK_SIZE=1024L*1024L*1024L' defined.\n-#\n-# Define BLK_SHA256 to use the built-in SHA-256 routines.\n-#\n-# Define NETTLE_SHA256 to use the SHA-256 routines in libnettle.\n-#\n-# Define GCRYPT_SHA256 to use the SHA-256 routines in libgcrypt.\n-#\n-# Define OPENSSL_SHA256 to use the SHA-256 routines in OpenSSL.\n-#\n # Define NEEDS_CRYPTO_WITH_SSL if you need -lcrypto when using -lssl (Darwin).\n #\n # Define NEEDS_SSL_WITH_CRYPTO if you need -lssl when using -lcrypto (Darwin).\n@@ -490,6 +409,111 @@ include shared.mak\n # to the \"\" of the corresponding `compat/fsmonitor/fsm-settings-.c`\n # that implements the `fsm_os_settings__*()` routines.\n #\n+# === Optional library: libintl ===\n+#\n+# Define NO_GETTEXT if you don't want Git output to be translated.\n+# A translated Git requires GNU libintl or another gettext implementation,\n+# plus libintl-perl at runtime.\n+#\n+# Define USE_GETTEXT_SCHEME and set it to 'fallthrough', if you don't trust\n+# the installed gettext translation of the shell scripts output.\n+#\n+# Define HAVE_LIBCHARSET_H if you haven't set NO_GETTEXT and you can't\n+# trust the langinfo.h's nl_langinfo(CODESET) function to return the\n+# current character set. GNU and Solaris have a nl_langinfo(CODESET),\n+# FreeBSD can use either, but MinGW and some others need to use\n+# libcharset.h's locale_charset() instead.\n+#\n+# Define CHARSET_LIB to the library you need to link with in order to\n+# use locale_charset() function. On some platforms this needs to set to\n+# -lcharset, on others to -liconv .\n+#\n+# Define LIBC_CONTAINS_LIBINTL if your gettext implementation doesn't\n+# need -lintl when linking.\n+#\n+# Define NO_MSGFMT_EXTENDED_OPTIONS if your implementation of msgfmt\n+# doesn't support GNU extensions like --check and --statistics\n+#\n+# === Optional library: libexpat ===\n+#\n+# Define NO_EXPAT if you do not have expat installed. git-http-push is\n+# not built, and you cannot push using http:// and https:// transports (dumb).\n+#\n+# Define EXPATDIR=/foo/bar if your expat header and library files are in\n+# /foo/bar/include and /foo/bar/lib directories.\n+#\n+# Define EXPAT_NEEDS_XMLPARSE_H if you have an old version of expat (e.g.,\n+# 1.1 or 1.2) that provides xmlparse.h instead of expat.h.\n+\n+# === Optional library: libcurl ===\n+#\n+# Define NO_CURL if you do not have libcurl installed. git-http-fetch and\n+# git-http-push are not built, and you cannot use http:// and https://\n+# transports (neither smart nor dumb).\n+#\n+# Define CURLDIR=/foo/bar if your curl header and library files are in\n+# /foo/bar/include and /foo/bar/lib directories.\n+#\n+# Define CURL_CONFIG to curl's configuration program that prints information\n+# about the library (e.g., its version number). The default is 'curl-config'.\n+#\n+# Define CURL_LDFLAGS to specify flags that you need to link when using libcurl,\n+# if you do not want to rely on the libraries provided by CURL_CONFIG. The\n+# default value is a result of `curl-config --libs`. An example value for\n+# CURL_LDFLAGS is as follows:\n+#\n+# CURL_LDFLAGS=-lcurl\n+#\n+# === Optional library: libpcre2 ===\n+#\n+# Define USE_LIBPCRE if you have and want to use libpcre. Various\n+# commands such as log and grep offer runtime options to use\n+# Perl-compatible regular expressions instead of standard or extended\n+# POSIX regular expressions.\n+#\n+# Only libpcre version 2 is supported. USE_LIBPCRE2 is a synonym for\n+# USE_LIBPCRE, support for the old USE_LIBPCRE1 has been removed.\n+#\n+# Define LIBPCREDIR=/foo/bar if your PCRE header and library files are\n+# in /foo/bar/include and /foo/bar/lib directories.\n+#\n+# == SHA-1 and SHA-256 defines ==\n+#\n+# Define BLK_SHA1 environment variable to make use of the bundled\n+# optimized C SHA1 routine.\n+#\n+# Define DC_SHA1 to unconditionally enable the collision-detecting sha1\n+# algorithm. This is slower, but may detect attempted collision attacks.\n+# Takes priority over other *_SHA1 knobs.\n+#\n+# Define DC_SHA1_EXTERNAL in addition to DC_SHA1 if you want to build / link\n+# git with the external SHA1 collision-detect library.\n+# Without this option, i.e. the default behavior is to build git with its\n+# own built-in code (or submodule).\n+#\n+# Define DC_SHA1_SUBMODULE in addition to DC_SHA1 to use the\n+# sha1collisiondetection shipped as a submodule instead of the\n+# non-submodule copy in sha1dc/. This is an experimental option used\n+# by the git project to migrate to using sha1collisiondetection as a\n+# submodule.\n+#\n+# Define OPENSSL_SHA1 environment variable when running make to link\n+# with the SHA1 routine from openssl library.\n+#\n+# Define SHA1_MAX_BLOCK_SIZE to limit the amount of data that will be hashed\n+# in one call to the platform's SHA1_Update(). e.g. APPLE_COMMON_CRYPTO\n+# wants 'SHA1_MAX_BLOCK_SIZE=1024L*1024L*1024L' defined.\n+#\n+# Define BLK_SHA256 to use the built-in SHA-256 routines.\n+#\n+# Define NETTLE_SHA256 to use the SHA-256 routines in libnettle.\n+#\n+# Define GCRYPT_SHA256 to use the SHA-256 routines in libgcrypt.\n+#\n+# Define OPENSSL_SHA256 to use the SHA-256 routines in OpenSSL.\n+#\n+# == DEVELOPER defines ==\n+#\n # Define DEVELOPER to enable more compiler warnings. Compiler version\n # and family are auto detected, but could be overridden by defining\n # COMPILER_FEATURES (see config.mak.dev). You can still set\n-- \n2.38.0.1093.gcd4a685f0b1\n\n"},{"id":"465239","messageId":"CAPig+cS8jvUn9bR=1ywWgCC3gPosgycdcdbm+aASo59mXz6rfw@mail.gmail.com","threadId":"57779","inReplyTo":"patch-v2-3.4-78ef8636c57-20221019T010222Z-avarab@gmail.com","subject":"Re: [PATCH v2 3/4] Makefile: really use and document sha1collisiondetection by default","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2022-10-19T02:59:23Z","receivedAt":"2022-10-19T02:59:47Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Tue, Oct 18, 2022 at 9:03 PM Ævar Arnfjörð Bjarmason\n wrote:\n> When the sha1collisiondetection library was added and made the default\n> in [1] we never updated the documentation added in [2] early in that\n> series once the default was flipped to DC_SHA1 in [3]. Furthermore the\n> INSTALL file has been claiming that we use OpenSSL by default since\n> [4], and hadn't been updated for the sha1collisiondetection switch.\n>\n> The interaction between NO_APPLE_COMMON_CRYPTO and DC_SHA1 seems to\n> have been missed in [3], so ever since DC_SHA1 was made the default\n> we've still used Apple's CommonCrypto instead of\n> sha1collisiondetection on Darwin and Mac OS X.\n>\n> Instead off all of this we now:\n>\n> * Don't have a DC_SHA1 know anymore (using it is an error), you need\n\ns/know/knob/\n\n> to set NO_DC_SHA1 instead to use any optional *_SHA1 implementation.\n> * Re-arranged the algorithm inclusion in hash.h to correspond to\n> NO_DC_SHA1, and \"#error\" if we have no defined SHA_*, rather than\n> silently picking block-sha1/sha1.h as a fallback.\n> * Have an INSTALL that reflects reality. We were still claiming to use\n> OpenSSL's SHA-1 hashing by default.\n> * Have Darwin and Mac OS X use sha1collisiondetection, like everywhere\n> else. There is still a NO_APPLE_COMMON_CRYPTO knob, but it's used for\n> things unrelated to SHA-1 (see [6]).\n> * Have a rewritten discussion of SHA-1 and SHA-256 in the Makefile\n> which covers all of this.\n>\n> Let's also change the CI for \"osx-clang\" to test with the new\n> APPLE_SHA1 knob (\"osx-gcc\" uses the new sha1collisiondetection\n> default).\n>\n> In practice this will spot issues like the one noted in [7], as\n> testing with just two backends should be enough to spot unportable\n> code. Ideally we'd have other CI jobs to test the various SHA-1\n> combinations, but for now we have better CI coverage than before.\n>\n> Signed-off-by: Ævar Arnfjörð Bjarmason \n> ---\n> diff --git a/Makefile b/Makefile\n> +ifdef DC_SHA1\n> +$(error the DC_SHA1 flag is no longer used, and has become the default. Adjust your build scripts accordingly)\n> +endif\n\nbikeshedding: Do we really need to penalize (abuse) people merely for\nasking us to do what we're already doing anyhow?\n\nOr do you have some future plan which wasn't stated in the commit log\nbut which explains this sort of hash stance?\n"},{"id":"465256","messageId":"xmqqo7u7srp4.fsf@gitster.g","threadId":"57779","inReplyTo":"CAPig+cS8jvUn9bR=1ywWgCC3gPosgycdcdbm+aASo59mXz6rfw@mail.gmail.com","subject":"Re: [PATCH v2 3/4] Makefile: really use and document sha1collisiondetection by default","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-10-19T16:28:55Z","receivedAt":"2022-10-19T16:29:09Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Eric Sunshine writes:\n\n>> diff --git a/Makefile b/Makefile\n>> +ifdef DC_SHA1\n>> +$(error the DC_SHA1 flag is no longer used, and has become the default. Adjust your build scripts accordingly)\n>> +endif\n>\n> bikeshedding: Do we really need to penalize (abuse) people merely for\n> asking us to do what we're already doing anyhow?\n\nA valid question.\n\nI can understand and very much appreciate [1/4] as a very focused\nfix to the problem. Very small part of this step, namely, make the\nDC_SHA1 the default everywhere, is also very much welcome.\n\nEverything else I see in these patches are extra \"while we are at\nit\" that should not exist. These \"while at it\" changes tend to\nsomehow implement more subjective choices that will cause more\ndiscussion and take more review resources. Not all \"white at it\"\nmay be more subjective, but at least in this series, they appear\nto be.\n\nThey distract us from the core changes and slows us down. It is OK\nto do them as totally unrelated clean-up changes long after the dust\nsettles, but not entangled with the other more important changes\nlike these patches.\n\nThanks.\n"},{"id":"465263","messageId":"221019.867d0vhbsz.gmgdl@evledraar.gmail.com","threadId":"57779","inReplyTo":"xmqqo7u7srp4.fsf@gitster.g","subject":"Re: [PATCH v2 3/4] Makefile: really use and document sha1collisiondetection by default","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-10-19T18:54:29Z","receivedAt":"2022-10-19T19:07:47Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Wed, Oct 19 2022, Junio C Hamano wrote:\n\n> Eric Sunshine writes:\n>\n>>> diff --git a/Makefile b/Makefile\n>>> +ifdef DC_SHA1\n>>> +$(error the DC_SHA1 flag is no longer used, and has become the default. Adjust your build scripts accordingly)\n>>> +endif\n>>\n>> bikeshedding: Do we really need to penalize (abuse) people merely for\n>> asking us to do what we're already doing anyhow?\n>\n> A valid question.\n>\n> I can understand and very much appreciate [1/4] as a very focused\n> fix to the problem. Very small part of this step, namely, make the\n> DC_SHA1 the default everywhere, is also very much welcome.\n>\n> Everything else I see in these patches are extra \"while we are at\n> it\" that should not exist. These \"while at it\" changes tend to\n> somehow implement more subjective choices that will cause more\n> discussion and take more review resources. Not all \"white at it\"\n> may be more subjective, but at least in this series, they appear\n> to be.\n>\n> They distract us from the core changes and slows us down. It is OK\n> to do them as totally unrelated clean-up changes long after the dust\n> settles, but not entangled with the other more important changes\n> like these patches.\n\nThere's things I can eject from this series, but I don't really find it\nto be \"while at it\" changes, I suspect what you're thinking of is\none/some of:\n\n - Re-arranging the Makefile commentary into sections: I did that\n because now the structure is very much one-paragraph-per-flag. \n\n So before 2/4 there's no good place to put that \"Alternate\n implementations\" in a way that clearly refers to the surrounding\n SHA{1,256} discussion. But yeah, I could try harder to keep that diff\n size down, or just not update the docs.\n\n - We're still claiming that we use OpenSSL by default, so I fixed the\n docs in general (not just the Makefile). Would you like this to be\n just a \"we forgot OSX?\" series?\n\n - Ditto asking to provide NO_DC_SHA1=Y now in addition to\n e.g. BLK_SHA1=Y if you *really* want to use that\n non-collision-detecting SHA-1 implementation.\n\n E.g. BLK_SHA1=Y was added in 2009. It's a small one-time bother to\n add NO_DC_SHA1=Y to your scripts if you *really* mean \"I want the\n less safe SHA-1\".\n\n But I think it's more likely that someone running into that error\n will be happy that the default has changed in a strong way.\n\n We've been *strongly* recommending sha1collisiondetection for a while\n now, but the structure of the build system has been the exact\n opposite, if you asked for anything else we'd avoid using it, and\n only default to it if nothing else was picked.\n\n - In particular the current flag on OSX is \"APPLE_COMMON_CRYPTO\", which\n is now split up into that & \"APPLE_SHA1\".\n\n Maybe changing our default flags is sufficient, and we don't need to\n worry about 3rd party build scripts having the previous \"Yeah, I have\n that OS library\" effectively meaning \"...and use it for everything\n possible, including SHA-1\".\n\nOr maybe you're OK with topic(s) at large, i.e. \"switch the default,\nupdate the docs, make sure noone's left behind\", but would just like it\ndone in more incremental steps?\n"},{"id":"465266","messageId":"xmqq1qr3r44k.fsf@gitster.g","threadId":"57779","inReplyTo":"221019.867d0vhbsz.gmgdl@evledraar.gmail.com","subject":"Re: [PATCH v2 3/4] Makefile: really use and document sha1collisiondetection by default","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-10-19T19:43:23Z","receivedAt":"2022-10-19T19:43:31Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Ævar Arnfjörð Bjarmason writes:\n\n>> They distract us from the core changes and slows us down. It is OK\n>> to do them as totally unrelated clean-up changes long after the dust\n>> settles, but not entangled with the other more important changes\n>> like these patches.\n>\n> There's things I can eject from this series, but I don't really find it\n> to be \"while at it\" changes, I suspect what you're thinking of is\n> one/some of:\n> ...\n> Or maybe you're OK with topic(s) at large, i.e. \"switch the default,\n> update the docs, make sure noone's left behind\", but would just like it\n> done in more incremental steps?\n\nI said something on this in a separate thread. Let me look it up.\n\n * Updating the build procedure so that sha1dc is used by default\n everywhere is a good idea, but that is less urgent and should be\n done separately, preferrably long after the dust settles from the\n above.\n\n"},{"id":"465273","messageId":"xmqqy1tbo3yz.fsf@gitster.g","threadId":"57779","inReplyTo":"221019.867d0vhbsz.gmgdl@evledraar.gmail.com","subject":"Re: [PATCH v2 3/4] Makefile: really use and document sha1collisiondetection by default","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-10-19T22:15:00Z","receivedAt":"2022-10-19T22:15:13Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Ævar Arnfjörð Bjarmason writes:\n\n> There's things I can eject from this series, but I don't really find it\n> to be \"while at it\" changes, I suspect what you're thinking of is\n> one/some of:\n>\n> - Re-arranging the Makefile commentary into sections: I did that\n> because now the structure is very much one-paragraph-per-flag. \n\nAnd it does not have anything to do with \"why doesn't macOS use\nsha1dc as the default like everybody else?\" fix.\n\n> - We're still claiming that we use OpenSSL by default, so I fixed the\n> docs in general (not just the Makefile). Would you like this to be\n> just a \"we forgot OSX?\" series?\n\nA focused change s/We use OpenSSL as default/We use sha1dc as default/ \ngoes well with \"why doesn't macOS use sha1dc as the default?\" fix.\n\n> - Ditto asking to provide NO_DC_SHA1=Y now in addition to\n\nThose who explicitly opt into DC_SHA1 do not need any new $(error).\nThose who have explicitly chosen something else should not be forced\nto say NO_DC_SHA1.\n\nLet's make it a focused change that does one thing very well without\nwasting reviewer's time on discussion on non-essential things.\n\nI do not mean by \"non-essential\" that reorganizing Makefile comments\nto make it easier to find and write relevant documents, or making it\nharder to turn conflicting knobs. They may by themselves be\nworthwhile thing to do. But in the context of \"why isn't sha1dc the\ndefault everywhere\", they are non-essential distractions.\n\n\"why isn't sha1dc the default on macOS?\" is also non-essential\ndistraction in the context of fixing \"oops, we no longer build on\nmacOS unless Apple Crypto is used\".\n\nI've queued [1/4] with an intention to fast-track it down to\n'master' (after cooking it in 'next' for a few days).\n\nThanks.\n"},{"id":"465274","messageId":"xmqqr0z3o3e0.fsf@gitster.g","threadId":"57779","inReplyTo":"xmqqy1tbo3yz.fsf@gitster.g","subject":"Re: [PATCH v2 3/4] Makefile: really use and document sha1collisiondetection by default","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-10-19T22:27:35Z","receivedAt":"2022-10-19T22:29:16Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Junio C Hamano writes:\n\n> Let's make it a focused change that does one thing very well without\n> wasting reviewer's time on discussion on non-essential things.\n>\n> I do not mean by \"non-essential\" that reorganizing Makefile comments\n> to make it easier to find and write relevant documents, or making it\n> harder to turn conflicting knobs. They may by themselves be\n\nSorry, I forgot to finish the sentence. I do not mean these other\nchanges I called \"non-essential\" are useless. They are not\nessential in the context of making sha1dc the default everywhere.\n\nThanks.\n"},{"id":"465350","messageId":"Y1G6ZEQbTBS1zqdL@tapette.crustytoothpaste.net","threadId":"57779","inReplyTo":"xmqqo7u7srp4.fsf@gitster.g","subject":"Re: [PATCH v2 3/4] Makefile: really use and document sha1collisiondetection by default","fromName":"brian m. carlson","fromEmail":"sandals@crustytoothpaste.net","sentAt":"2022-10-20T21:15:16Z","receivedAt":"2022-10-20T21:15:26Z","isPatch":true,"sender":{"key":"sandals@crustytoothpaste.net","avatar":"https://avatars.githubusercontent.com/u/497054?v=4"},"body":"On 2022-10-19 at 16:28:55, Junio C Hamano wrote:\n> Eric Sunshine writes:\n> \n> >> diff --git a/Makefile b/Makefile\n> >> +ifdef DC_SHA1\n> >> +$(error the DC_SHA1 flag is no longer used, and has become the default. Adjust your build scripts accordingly)\n> >> +endif\n> >\n> > bikeshedding: Do we really need to penalize (abuse) people merely for\n> > asking us to do what we're already doing anyhow?\n> \n> A valid question.\n> \n> I can understand and very much appreciate [1/4] as a very focused\n> fix to the problem. Very small part of this step, namely, make the\n> DC_SHA1 the default everywhere, is also very much welcome.\n> \n> Everything else I see in these patches are extra \"while we are at\n> it\" that should not exist. These \"while at it\" changes tend to\n> somehow implement more subjective choices that will cause more\n> discussion and take more review resources. Not all \"white at it\"\n> may be more subjective, but at least in this series, they appear\n> to be.\n\nI will definitely say that we should be using SHA-1-DC by default\neverywhere and I'm fine with that change. I don't think we should fail\nif someone sets the former knob, since I don't recall us doing that\nelsewhere. It also seems bad to bother folks if we're already doing\nwhat they asked for (and what we want them to do anyway).\n\nI don't have a strong attachment to the remainder of the series and\nagree that maybe a different series would be a better location for those\npatches.\n-- \nbrian m. carlson (he/him or they/them)\nToronto, Ontario, CA\n"},{"id":"465357","messageId":"cover-v3-0.9-00000000000-20221020T223946Z-avarab@gmail.com","threadId":"57779","inReplyTo":"cover-v2-0.4-00000000000-20221019T010222Z-avarab@gmail.com","subject":"[PATCH v3 0/9] Makefile & docs: document SHA-{1,256} behavior, fix bug","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-10-20T22:43:01Z","receivedAt":"2022-10-20T22:43:32Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"This v3 is a rewrite of the larger-in-scope v2[2] to only fix the\ncurrent Makefile & INSTALL documentation do to do with our SHA-1 and\nSHA-256 knobs.\n\nThe only behavior change here is in 1/8, where I fix what's obviously\na bug in the current behavior (so that we don't need to document that\nedge case). I'll submit a patch to change the behavior on OSX to use\nsha1collisiondetection by default on top of this series.\n\n1. https://lore.kernel.org/git/cover-v2-0.4-00000000000-20221019T010222Z-avarab@gmail.com/\n\nÆvar Arnfjörð Bjarmason (9):\n Makefile: always (re)set DC_SHA1 on fallback\n INSTALL: remove discussion of SHA-1 backends\n Makefile: correct DC_SHA1 documentation\n Makefile: create and use sections for \"define\" flag listing\n Makefile: rephrase the discussion of *_SHA1 knobs\n Makefile: document default SHA-256 backend\n Makefile: document SHA-1 and SHA-256 default and selection order\n Makefile: document default SHA-1 backend on OSX\n Makefile: discuss SHAttered in *_SHA{1,256} discussion\n\n INSTALL | 4 -\n Makefile | 260 +++++++++++++++++++++++++++++++++++--------------------\n 2 files changed, 166 insertions(+), 98 deletions(-)\n\nRange-diff against v2:\n 1: 392fabdb456 < -: ----------- fsmonitor OSX: compile with DC_SHA1=YesPlease\n -: ----------- > 1: ef3c5be11e0 Makefile: always (re)set DC_SHA1 on fallback\n -: ----------- > 2: 017a0a9791c INSTALL: remove discussion of SHA-1 backends\n -: ----------- > 3: 62dd2d5708d Makefile: correct DC_SHA1 documentation\n 2: 7ae22276aa7 ! 4: 933bef576b3 Makefile: create and use sections for \"define\" flag listing\n @@ Makefile: include shared.mak\n -# Define BLK_SHA1 environment variable to make use of the bundled\n -# optimized C SHA1 routine.\n -#\n --# Define DC_SHA1 to unconditionally enable the collision-detecting sha1\n +-# Define DC_SHA1 to enable the collision-detecting sha1\n -# algorithm. This is slower, but may detect attempted collision attacks.\n --# Takes priority over other *_SHA1 knobs.\n -#\n -# Define DC_SHA1_EXTERNAL in addition to DC_SHA1 if you want to build / link\n -# git with the external SHA1 collision-detect library.\n @@ Makefile: include shared.mak\n +#\n +# == SHA-1 and SHA-256 defines ==\n +#\n -+# Define BLK_SHA1 environment variable to make use of the bundled\n -+# optimized C SHA1 routine.\n ++# === SHA-1 backend ===\n ++#\n ++# ==== Options common to all SHA-1 implementations ====\n ++#\n ++# Define SHA1_MAX_BLOCK_SIZE to limit the amount of data that will be hashed\n ++# in one call to the platform's SHA1_Update(). e.g. APPLE_COMMON_CRYPTO\n ++# wants 'SHA1_MAX_BLOCK_SIZE=1024L*1024L*1024L' defined.\n ++#\n ++# ==== Options for the sha1collisiondetection implementation ====\n +#\n -+# Define DC_SHA1 to unconditionally enable the collision-detecting sha1\n ++# Define DC_SHA1 to enable the collision-detecting sha1\n +# algorithm. This is slower, but may detect attempted collision attacks.\n -+# Takes priority over other *_SHA1 knobs.\n +#\n +# Define DC_SHA1_EXTERNAL in addition to DC_SHA1 if you want to build / link\n +# git with the external SHA1 collision-detect library.\n @@ Makefile: include shared.mak\n +# by the git project to migrate to using sha1collisiondetection as a\n +# submodule.\n +#\n ++# ==== Other SHA-1 implementations ====\n ++#\n ++# Define BLK_SHA1 environment variable to make use of the bundled\n ++# optimized C SHA1 routine.\n ++#\n +# Define OPENSSL_SHA1 environment variable when running make to link\n +# with the SHA1 routine from openssl library.\n +#\n -+# Define SHA1_MAX_BLOCK_SIZE to limit the amount of data that will be hashed\n -+# in one call to the platform's SHA1_Update(). e.g. APPLE_COMMON_CRYPTO\n -+# wants 'SHA1_MAX_BLOCK_SIZE=1024L*1024L*1024L' defined.\n ++# === SHA-256 backend ===\n +#\n +# Define BLK_SHA256 to use the built-in SHA-256 routines.\n +#\n 3: 78ef8636c57 < -: ----------- Makefile: really use and document sha1collisiondetection by default\n 4: f1fb9775b33 < -: ----------- Makefile: rephrase the discussion of *_SHA1 knobs\n -: ----------- > 5: 5b18198c477 Makefile: rephrase the discussion of *_SHA1 knobs\n -: ----------- > 6: 73685592aba Makefile: document default SHA-256 backend\n -: ----------- > 7: 05edcfb9cd9 Makefile: document SHA-1 and SHA-256 default and selection order\n -: ----------- > 8: 859e69fbe9f Makefile: document default SHA-1 backend on OSX\n -: ----------- > 9: c1f27255d3e Makefile: discuss SHAttered in *_SHA{1,256} discussion\n-- \n2.38.0.1178.g509f5fa8ce0\n\n"},{"id":"465358","messageId":"patch-v3-1.9-ef3c5be11e0-20221020T223946Z-avarab@gmail.com","threadId":"57779","inReplyTo":"cover-v3-0.9-00000000000-20221020T223946Z-avarab@gmail.com","subject":"[PATCH v3 1/9] Makefile: always (re)set DC_SHA1 on fallback","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-10-20T22:43:02Z","receivedAt":"2022-10-20T22:43:35Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Fix an edge case introduced in in e6b07da2780 (Makefile: make DC_SHA1\nthe default, 2017-03-17), when DC_SHA1 was made the default fallback\nwe started unconditionally adding to BASIC_CFLAGS and LIB_OBJS, so\nwe'd use the sha1collisiondetection by default.\n\nBut the \"DC_SHA1\" variable remained unset, so e.g.:\n\n\tmake test DC_SHA1= T=t0013*.sh\n\nWould skip the sha1collisiondetection tests, as we'd write\n\"DC_SHA1=''\" to \"GIT-BUILD-OPTIONS\", but if we manually removed that\ntest prerequisite we'd pass the test (which we couldn't if we weren't\nusing sha1collisiondetection).\n\nSo let's have the fallback assignment use the 'override' directive\ninstead of the \":=\" simply expanded variable introduced in\ne6b07da2780. In this case we explicitly want to override the user's\nchoice.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n Makefile | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)\n\ndiff --git a/Makefile b/Makefile\nindex d93ad956e58..5440089db3d 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -1822,7 +1822,7 @@ ifdef APPLE_COMMON_CRYPTO\n \tCOMPAT_CFLAGS += -DCOMMON_DIGEST_FOR_OPENSSL\n \tBASIC_CFLAGS += -DSHA1_APPLE\n else\n-\tDC_SHA1 := YesPlease\n+\toverride DC_SHA1 = YesPlease\n \tBASIC_CFLAGS += -DSHA1_DC\n \tLIB_OBJS += sha1dc_git.o\n ifdef DC_SHA1_EXTERNAL\n-- \n2.38.0.1178.g509f5fa8ce0\n\n"},{"id":"465359","messageId":"patch-v3-2.9-017a0a9791c-20221020T223946Z-avarab@gmail.com","threadId":"57779","inReplyTo":"cover-v3-0.9-00000000000-20221020T223946Z-avarab@gmail.com","subject":"[PATCH v3 2/9] INSTALL: remove discussion of SHA-1 backends","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-10-20T22:43:03Z","receivedAt":"2022-10-20T22:43:39Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"The claim that OpenSSL is the default SHA-1 backend hasn't been true\nsince e6b07da2780 (Makefile: make DC_SHA1 the default, 2017-03-17),\nbut more importantly tweaking the SHA-1 backend isn't something that's\ncommon enough to warrant discussing in the INSTALL document, so let's\nremove this paragraph.\n\nThis discussion was originally added in c538d2d34ab (Add some\ninstallation notes in INSTALL, 2005-06-17) when tweaking the default\nbackend was more common. The current wording was added in\n5beb577db8c (INSTALL: Describe dependency knobs from Makefile,\n2009-09-10).\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n INSTALL | 4 ----\n 1 file changed, 4 deletions(-)\n\ndiff --git a/INSTALL b/INSTALL\nindex 89b15d71df5..33447883974 100644\n--- a/INSTALL\n+++ b/INSTALL\n@@ -133,10 +133,6 @@ Issues of note:\n \t you are using libcurl older than 7.34.0. Otherwise you can use\n \t NO_OPENSSL without losing git-imap-send.\n \n-\t By default, git uses OpenSSL for SHA1 but it will use its own\n-\t library (inspired by Mozilla's) with either NO_OPENSSL or\n-\t BLK_SHA1.\n-\n \t- \"libcurl\" library is used for fetching and pushing\n \t repositories over http:// or https://, as well as by\n \t git-imap-send if the curl version is >= 7.34.0. If you do\n-- \n2.38.0.1178.g509f5fa8ce0\n\n"},{"id":"465360","messageId":"patch-v3-3.9-62dd2d5708d-20221020T223946Z-avarab@gmail.com","threadId":"57779","inReplyTo":"cover-v3-0.9-00000000000-20221020T223946Z-avarab@gmail.com","subject":"[PATCH v3 3/9] Makefile: correct DC_SHA1 documentation","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-10-20T22:43:04Z","receivedAt":"2022-10-20T22:43:41Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"The claim that DC_SHA1 takes priority over other *_SHA1 knobs was true\nwhen it was added in [1], But that hasn't been the case since it was\nmade the fallback default in [2].\n\nWe should be making it not only the default, but something that takes\npriority over other *_SHA1 knobs, but that's outside the scope of this\nchange. For now let's correct the documentation to match reality.\n\nLet's also remove the \"unconditionally enable\" wording, per the above\nthe enabling of \"DC_SHA1\" is conditional on these other flags.\n\n1. 8325e43b82d (Makefile: add DC_SHA1 knob, 2017-03-16)\n2. e6b07da2780 (Makefile: make DC_SHA1 the default, 2017-03-17)\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n Makefile | 3 +--\n 1 file changed, 1 insertion(+), 2 deletions(-)\n\ndiff --git a/Makefile b/Makefile\nindex 5440089db3d..e55dc870c5d 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -155,9 +155,8 @@ include shared.mak\n # Define BLK_SHA1 environment variable to make use of the bundled\n # optimized C SHA1 routine.\n #\n-# Define DC_SHA1 to unconditionally enable the collision-detecting sha1\n+# Define DC_SHA1 to enable the collision-detecting sha1\n # algorithm. This is slower, but may detect attempted collision attacks.\n-# Takes priority over other *_SHA1 knobs.\n #\n # Define DC_SHA1_EXTERNAL in addition to DC_SHA1 if you want to build / link\n # git with the external SHA1 collision-detect library.\n-- \n2.38.0.1178.g509f5fa8ce0\n\n"},{"id":"465361","messageId":"patch-v3-5.9-5b18198c477-20221020T223946Z-avarab@gmail.com","threadId":"57779","inReplyTo":"cover-v3-0.9-00000000000-20221020T223946Z-avarab@gmail.com","subject":"[PATCH v3 5/9] Makefile: rephrase the discussion of *_SHA1 knobs","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-10-20T22:43:06Z","receivedAt":"2022-10-20T22:43:46Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"In the preceding commit the discussion of the *_SHA1 knobs was left\nas-is to benefit from a smaller diff, but since we're changing these\nlet's use the same phrasing we use for most other knobs. E.g. \"define\nX\", not \"define X environment variable\", and get rid of the \"when\nrunning make to link with\" entirely.\n\nFurthermore the discussion of DC_SHA1* options is now under a \"Options\nfor the sha1collisiondetection implementation\" heading, so we don't\nneed to clarify that these options go along with DC_SHA1=Y, so let's\nrephrase them accordingly.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n Makefile | 12 ++++++------\n 1 file changed, 6 insertions(+), 6 deletions(-)\n\ndiff --git a/Makefile b/Makefile\nindex b3a717792cf..61358f16acc 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -492,12 +492,12 @@ include shared.mak\n # Define DC_SHA1 to enable the collision-detecting sha1\n # algorithm. This is slower, but may detect attempted collision attacks.\n #\n-# Define DC_SHA1_EXTERNAL in addition to DC_SHA1 if you want to build / link\n+# Define DC_SHA1_EXTERNAL if you want to build / link\n # git with the external SHA1 collision-detect library.\n # Without this option, i.e. the default behavior is to build git with its\n # own built-in code (or submodule).\n #\n-# Define DC_SHA1_SUBMODULE in addition to DC_SHA1 to use the\n+# Define DC_SHA1_SUBMODULE to use the\n # sha1collisiondetection shipped as a submodule instead of the\n # non-submodule copy in sha1dc/. This is an experimental option used\n # by the git project to migrate to using sha1collisiondetection as a\n@@ -505,11 +505,11 @@ include shared.mak\n #\n # ==== Other SHA-1 implementations ====\n #\n-# Define BLK_SHA1 environment variable to make use of the bundled\n-# optimized C SHA1 routine.\n+# Define BLK_SHA1 to make use of optimized C SHA-1 routines bundled\n+# with git (in the block-sha1/ directory).\n #\n-# Define OPENSSL_SHA1 environment variable when running make to link\n-# with the SHA1 routine from openssl library.\n+# Define OPENSSL_SHA1 to link to the the SHA-1 routines from\n+# the OpenSSL library.\n #\n # === SHA-256 backend ===\n #\n-- \n2.38.0.1178.g509f5fa8ce0\n\n"},{"id":"465362","messageId":"patch-v3-6.9-73685592aba-20221020T223946Z-avarab@gmail.com","threadId":"57779","inReplyTo":"cover-v3-0.9-00000000000-20221020T223946Z-avarab@gmail.com","subject":"[PATCH v3 6/9] Makefile: document default SHA-256 backend","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-10-20T22:43:07Z","receivedAt":"2022-10-20T22:43:53Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Since 27dc04c5450 (sha256: add an SHA-256 implementation using\nlibgcrypt, 2018-11-14) we've claimed to support a BLK_SHA256 flag, but\nthere's no such SHA-256 backend.\n\nInstead we fall back on adding \"sha256/block/sha256.o\" to \"LIB_OBJS\"\nand adding \"-DSHA256_BLK\" to BASIC_CFLAGS.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n Makefile | 10 +++++++++-\n 1 file changed, 9 insertions(+), 1 deletion(-)\n\ndiff --git a/Makefile b/Makefile\nindex 61358f16acc..0bc8eaa4cf9 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -513,7 +513,15 @@ include shared.mak\n #\n # === SHA-256 backend ===\n #\n-# Define BLK_SHA256 to use the built-in SHA-256 routines.\n+# ==== Default SHA-256 backend ====\n+#\n+# If no *_SHA256 backend is picked we'll fall fall back on using the\n+# default.\n+#\n+# The default SHA-256 backend is shipped with Git. No flag is required\n+# to enable it. To select it don't define any other *_SHA256 flag.\n+#\n+# ==== Other SHA-256 implementations ====\n #\n # Define NETTLE_SHA256 to use the SHA-256 routines in libnettle.\n #\n-- \n2.38.0.1178.g509f5fa8ce0\n\n"},{"id":"465363","messageId":"patch-v3-7.9-05edcfb9cd9-20221020T223946Z-avarab@gmail.com","threadId":"57779","inReplyTo":"cover-v3-0.9-00000000000-20221020T223946Z-avarab@gmail.com","subject":"[PATCH v3 7/9] Makefile: document SHA-1 and SHA-256 default and selection order","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-10-20T22:43:08Z","receivedAt":"2022-10-20T22:43:56Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"For the *_SHA1 and *_SHA256 flags we've discussed the various flags,\nbut not the fact that when you define multiple flags we'll pick one.\n\nWhich one we pick depends on the order they're listed in the Makefile,\nwhich differed from the order we discussed them in this documentation.\n\nLet's be explicit about how we select these, and re-arrange the\nlistings so that they're listed in the priority order we've picked.\n\nI'd personally prefer that the selection was more explicit, and that\nwe'd error out if conflicting flags were provided, but per the\ndiscussion downhtread of[1] the consensus was to keep theses semantics.\n\nThis behavior make it easier to e.g. integrate with autoconf-like\nsystems, where the configuration can provide everything it can\nsupport, and Git is tasked with picking the first one it prefers.\n\n1. https://lore.kernel.org/git/220710.86mtdh81ty.gmgdl@evledraar.gmail.com/\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n Makefile | 21 ++++++++++++++++-----\n 1 file changed, 16 insertions(+), 5 deletions(-)\n\ndiff --git a/Makefile b/Makefile\nindex 0bc8eaa4cf9..8a9f3e561f8 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -481,6 +481,14 @@ include shared.mak\n #\n # === SHA-1 backend ===\n #\n+# ==== Default SHA-1 backend ====\n+#\n+# If no *_SHA1 backend is picked we'll fall fall back on using the\n+# default.\n+#\n+# Multiple *_SHA1 backends can be selected, the first supported one\n+# listed in \"Other SHA-1 implementations\" will be picked.\n+#\n # ==== Options common to all SHA-1 implementations ====\n #\n # Define SHA1_MAX_BLOCK_SIZE to limit the amount of data that will be hashed\n@@ -505,12 +513,12 @@ include shared.mak\n #\n # ==== Other SHA-1 implementations ====\n #\n-# Define BLK_SHA1 to make use of optimized C SHA-1 routines bundled\n-# with git (in the block-sha1/ directory).\n-#\n # Define OPENSSL_SHA1 to link to the the SHA-1 routines from\n # the OpenSSL library.\n #\n+# Define BLK_SHA1 to make use of optimized C SHA-1 routines bundled\n+# with git (in the block-sha1/ directory).\n+#\n # === SHA-256 backend ===\n #\n # ==== Default SHA-256 backend ====\n@@ -521,14 +529,17 @@ include shared.mak\n # The default SHA-256 backend is shipped with Git. No flag is required\n # to enable it. To select it don't define any other *_SHA256 flag.\n #\n+# Multiple *_SHA256 backends can be selected, the first supported one\n+# listed in \"Other SHA-256 implementations\" below will be picked.\n+#\n # ==== Other SHA-256 implementations ====\n #\n+# Define OPENSSL_SHA256 to use the SHA-256 routines in OpenSSL.\n+#\n # Define NETTLE_SHA256 to use the SHA-256 routines in libnettle.\n #\n # Define GCRYPT_SHA256 to use the SHA-256 routines in libgcrypt.\n #\n-# Define OPENSSL_SHA256 to use the SHA-256 routines in OpenSSL.\n-#\n # == DEVELOPER defines ==\n #\n # Define DEVELOPER to enable more compiler warnings. Compiler version\n-- \n2.38.0.1178.g509f5fa8ce0\n\n"},{"id":"465364","messageId":"patch-v3-4.9-933bef576b3-20221020T223946Z-avarab@gmail.com","threadId":"57779","inReplyTo":"cover-v3-0.9-00000000000-20221020T223946Z-avarab@gmail.com","subject":"[PATCH v3 4/9] Makefile: create and use sections for \"define\" flag listing","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-10-20T22:43:05Z","receivedAt":"2022-10-20T22:43:57Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Since the \"Define ...\" template of comments at the top of the Makefile\nwas started in 5bdac8b3269 ([PATCH] Improve the compilation-time\nsettings interface, 2005-07-29) we've had a lot more flags added,\nincluding flags that come in \"groups\". Not having any obvious\nstructure to the >500 line comment at the top of the Makefile has made\nit hard to follow.\n\nThis change is almost entirely a move-only change, the two paragraphs\nat the start of the first two sections are new, and so are the added\nsections themselves, but other than that no lines are changed, only\nmoved.\n\nWe now list Makefile-only flags at the start, followed by stand-alone\nflags, and then cover \"optional library\" flags in their respective\ngroups, followed by SHA-1 and SHA-256 flags, and finally\nDEVELOPER-specific flags.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n Makefile | 218 ++++++++++++++++++++++++++++++++-----------------------\n 1 file changed, 126 insertions(+), 92 deletions(-)\n\ndiff --git a/Makefile b/Makefile\nindex e55dc870c5d..b3a717792cf 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -4,8 +4,20 @@ all::\n # Import tree-wide shared Makefile behavior and libraries\n include shared.mak\n \n+# == Makefile defines ==\n+#\n+# These defines change the behavior of the Makefile itself, but have\n+# no impact on what it builds:\n+#\n # Define V=1 to have a more verbose compile.\n #\n+# == Portability and optional library defines ==\n+#\n+# These defines indicate what Git can expect from the OS, what\n+# libraries are available etc. Much of this is auto-detected in\n+# config.mak.uname, or in configure.ac when using the optional \"make\n+# configure && ./configure\" (see INSTALL).\n+#\n # Define SHELL_PATH to a POSIX shell if your /bin/sh is broken.\n #\n # Define SANE_TOOL_PATH to a colon-separated list of paths to prepend\n@@ -30,68 +42,8 @@ include shared.mak\n #\n # Define NO_OPENSSL environment variable if you do not have OpenSSL.\n #\n-# Define USE_LIBPCRE if you have and want to use libpcre. Various\n-# commands such as log and grep offer runtime options to use\n-# Perl-compatible regular expressions instead of standard or extended\n-# POSIX regular expressions.\n-#\n-# Only libpcre version 2 is supported. USE_LIBPCRE2 is a synonym for\n-# USE_LIBPCRE, support for the old USE_LIBPCRE1 has been removed.\n-#\n-# Define LIBPCREDIR=/foo/bar if your PCRE header and library files are\n-# in /foo/bar/include and /foo/bar/lib directories.\n-#\n # Define HAVE_ALLOCA_H if you have working alloca(3) defined in that header.\n #\n-# Define NO_CURL if you do not have libcurl installed. git-http-fetch and\n-# git-http-push are not built, and you cannot use http:// and https://\n-# transports (neither smart nor dumb).\n-#\n-# Define CURLDIR=/foo/bar if your curl header and library files are in\n-# /foo/bar/include and /foo/bar/lib directories.\n-#\n-# Define CURL_CONFIG to curl's configuration program that prints information\n-# about the library (e.g., its version number). The default is 'curl-config'.\n-#\n-# Define CURL_LDFLAGS to specify flags that you need to link when using libcurl,\n-# if you do not want to rely on the libraries provided by CURL_CONFIG. The\n-# default value is a result of `curl-config --libs`. An example value for\n-# CURL_LDFLAGS is as follows:\n-#\n-# CURL_LDFLAGS=-lcurl\n-#\n-# Define NO_EXPAT if you do not have expat installed. git-http-push is\n-# not built, and you cannot push using http:// and https:// transports (dumb).\n-#\n-# Define EXPATDIR=/foo/bar if your expat header and library files are in\n-# /foo/bar/include and /foo/bar/lib directories.\n-#\n-# Define EXPAT_NEEDS_XMLPARSE_H if you have an old version of expat (e.g.,\n-# 1.1 or 1.2) that provides xmlparse.h instead of expat.h.\n-#\n-# Define NO_GETTEXT if you don't want Git output to be translated.\n-# A translated Git requires GNU libintl or another gettext implementation,\n-# plus libintl-perl at runtime.\n-#\n-# Define USE_GETTEXT_SCHEME and set it to 'fallthrough', if you don't trust\n-# the installed gettext translation of the shell scripts output.\n-#\n-# Define HAVE_LIBCHARSET_H if you haven't set NO_GETTEXT and you can't\n-# trust the langinfo.h's nl_langinfo(CODESET) function to return the\n-# current character set. GNU and Solaris have a nl_langinfo(CODESET),\n-# FreeBSD can use either, but MinGW and some others need to use\n-# libcharset.h's locale_charset() instead.\n-#\n-# Define CHARSET_LIB to the library you need to link with in order to\n-# use locale_charset() function. On some platforms this needs to set to\n-# -lcharset, on others to -liconv .\n-#\n-# Define LIBC_CONTAINS_LIBINTL if your gettext implementation doesn't\n-# need -lintl when linking.\n-#\n-# Define NO_MSGFMT_EXTENDED_OPTIONS if your implementation of msgfmt\n-# doesn't support GNU extensions like --check and --statistics\n-#\n # Define HAVE_PATHS_H if you have paths.h and want to use the default PATH\n # it specifies.\n #\n@@ -152,38 +104,6 @@ include shared.mak\n # and do not want to use Apple's CommonCrypto library. This allows you\n # to provide your own OpenSSL library, for example from MacPorts.\n #\n-# Define BLK_SHA1 environment variable to make use of the bundled\n-# optimized C SHA1 routine.\n-#\n-# Define DC_SHA1 to enable the collision-detecting sha1\n-# algorithm. This is slower, but may detect attempted collision attacks.\n-#\n-# Define DC_SHA1_EXTERNAL in addition to DC_SHA1 if you want to build / link\n-# git with the external SHA1 collision-detect library.\n-# Without this option, i.e. the default behavior is to build git with its\n-# own built-in code (or submodule).\n-#\n-# Define DC_SHA1_SUBMODULE in addition to DC_SHA1 to use the\n-# sha1collisiondetection shipped as a submodule instead of the\n-# non-submodule copy in sha1dc/. This is an experimental option used\n-# by the git project to migrate to using sha1collisiondetection as a\n-# submodule.\n-#\n-# Define OPENSSL_SHA1 environment variable when running make to link\n-# with the SHA1 routine from openssl library.\n-#\n-# Define SHA1_MAX_BLOCK_SIZE to limit the amount of data that will be hashed\n-# in one call to the platform's SHA1_Update(). e.g. APPLE_COMMON_CRYPTO\n-# wants 'SHA1_MAX_BLOCK_SIZE=1024L*1024L*1024L' defined.\n-#\n-# Define BLK_SHA256 to use the built-in SHA-256 routines.\n-#\n-# Define NETTLE_SHA256 to use the SHA-256 routines in libnettle.\n-#\n-# Define GCRYPT_SHA256 to use the SHA-256 routines in libgcrypt.\n-#\n-# Define OPENSSL_SHA256 to use the SHA-256 routines in OpenSSL.\n-#\n # Define NEEDS_CRYPTO_WITH_SSL if you need -lcrypto when using -lssl (Darwin).\n #\n # Define NEEDS_SSL_WITH_CRYPTO if you need -lssl when using -lcrypto (Darwin).\n@@ -489,6 +409,120 @@ include shared.mak\n # to the \"\" of the corresponding `compat/fsmonitor/fsm-settings-.c`\n # that implements the `fsm_os_settings__*()` routines.\n #\n+# === Optional library: libintl ===\n+#\n+# Define NO_GETTEXT if you don't want Git output to be translated.\n+# A translated Git requires GNU libintl or another gettext implementation,\n+# plus libintl-perl at runtime.\n+#\n+# Define USE_GETTEXT_SCHEME and set it to 'fallthrough', if you don't trust\n+# the installed gettext translation of the shell scripts output.\n+#\n+# Define HAVE_LIBCHARSET_H if you haven't set NO_GETTEXT and you can't\n+# trust the langinfo.h's nl_langinfo(CODESET) function to return the\n+# current character set. GNU and Solaris have a nl_langinfo(CODESET),\n+# FreeBSD can use either, but MinGW and some others need to use\n+# libcharset.h's locale_charset() instead.\n+#\n+# Define CHARSET_LIB to the library you need to link with in order to\n+# use locale_charset() function. On some platforms this needs to set to\n+# -lcharset, on others to -liconv .\n+#\n+# Define LIBC_CONTAINS_LIBINTL if your gettext implementation doesn't\n+# need -lintl when linking.\n+#\n+# Define NO_MSGFMT_EXTENDED_OPTIONS if your implementation of msgfmt\n+# doesn't support GNU extensions like --check and --statistics\n+#\n+# === Optional library: libexpat ===\n+#\n+# Define NO_EXPAT if you do not have expat installed. git-http-push is\n+# not built, and you cannot push using http:// and https:// transports (dumb).\n+#\n+# Define EXPATDIR=/foo/bar if your expat header and library files are in\n+# /foo/bar/include and /foo/bar/lib directories.\n+#\n+# Define EXPAT_NEEDS_XMLPARSE_H if you have an old version of expat (e.g.,\n+# 1.1 or 1.2) that provides xmlparse.h instead of expat.h.\n+\n+# === Optional library: libcurl ===\n+#\n+# Define NO_CURL if you do not have libcurl installed. git-http-fetch and\n+# git-http-push are not built, and you cannot use http:// and https://\n+# transports (neither smart nor dumb).\n+#\n+# Define CURLDIR=/foo/bar if your curl header and library files are in\n+# /foo/bar/include and /foo/bar/lib directories.\n+#\n+# Define CURL_CONFIG to curl's configuration program that prints information\n+# about the library (e.g., its version number). The default is 'curl-config'.\n+#\n+# Define CURL_LDFLAGS to specify flags that you need to link when using libcurl,\n+# if you do not want to rely on the libraries provided by CURL_CONFIG. The\n+# default value is a result of `curl-config --libs`. An example value for\n+# CURL_LDFLAGS is as follows:\n+#\n+# CURL_LDFLAGS=-lcurl\n+#\n+# === Optional library: libpcre2 ===\n+#\n+# Define USE_LIBPCRE if you have and want to use libpcre. Various\n+# commands such as log and grep offer runtime options to use\n+# Perl-compatible regular expressions instead of standard or extended\n+# POSIX regular expressions.\n+#\n+# Only libpcre version 2 is supported. USE_LIBPCRE2 is a synonym for\n+# USE_LIBPCRE, support for the old USE_LIBPCRE1 has been removed.\n+#\n+# Define LIBPCREDIR=/foo/bar if your PCRE header and library files are\n+# in /foo/bar/include and /foo/bar/lib directories.\n+#\n+# == SHA-1 and SHA-256 defines ==\n+#\n+# === SHA-1 backend ===\n+#\n+# ==== Options common to all SHA-1 implementations ====\n+#\n+# Define SHA1_MAX_BLOCK_SIZE to limit the amount of data that will be hashed\n+# in one call to the platform's SHA1_Update(). e.g. APPLE_COMMON_CRYPTO\n+# wants 'SHA1_MAX_BLOCK_SIZE=1024L*1024L*1024L' defined.\n+#\n+# ==== Options for the sha1collisiondetection implementation ====\n+#\n+# Define DC_SHA1 to enable the collision-detecting sha1\n+# algorithm. This is slower, but may detect attempted collision attacks.\n+#\n+# Define DC_SHA1_EXTERNAL in addition to DC_SHA1 if you want to build / link\n+# git with the external SHA1 collision-detect library.\n+# Without this option, i.e. the default behavior is to build git with its\n+# own built-in code (or submodule).\n+#\n+# Define DC_SHA1_SUBMODULE in addition to DC_SHA1 to use the\n+# sha1collisiondetection shipped as a submodule instead of the\n+# non-submodule copy in sha1dc/. This is an experimental option used\n+# by the git project to migrate to using sha1collisiondetection as a\n+# submodule.\n+#\n+# ==== Other SHA-1 implementations ====\n+#\n+# Define BLK_SHA1 environment variable to make use of the bundled\n+# optimized C SHA1 routine.\n+#\n+# Define OPENSSL_SHA1 environment variable when running make to link\n+# with the SHA1 routine from openssl library.\n+#\n+# === SHA-256 backend ===\n+#\n+# Define BLK_SHA256 to use the built-in SHA-256 routines.\n+#\n+# Define NETTLE_SHA256 to use the SHA-256 routines in libnettle.\n+#\n+# Define GCRYPT_SHA256 to use the SHA-256 routines in libgcrypt.\n+#\n+# Define OPENSSL_SHA256 to use the SHA-256 routines in OpenSSL.\n+#\n+# == DEVELOPER defines ==\n+#\n # Define DEVELOPER to enable more compiler warnings. Compiler version\n # and family are auto detected, but could be overridden by defining\n # COMPILER_FEATURES (see config.mak.dev). You can still set\n-- \n2.38.0.1178.g509f5fa8ce0\n\n"},{"id":"465365","messageId":"patch-v3-8.9-859e69fbe9f-20221020T223946Z-avarab@gmail.com","threadId":"57779","inReplyTo":"cover-v3-0.9-00000000000-20221020T223946Z-avarab@gmail.com","subject":"[PATCH v3 8/9] Makefile: document default SHA-1 backend on OSX","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-10-20T22:43:09Z","receivedAt":"2022-10-20T22:44:09Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Since [1] the default SHA-1 backend on OSX has been\nAPPLE_COMMON_CRYPTO. Per [2] we'll skip using it on anything older\nthan Mac OS X 10.4 \"Tiger\"[3].\n\nWhen \"DC_SHA1\" was made the default in [4] this interaction between it\nand APPLE_COMMON_CRYPTO seems to have been missed in. Ever since\nDC_SHA1 was \"made the default\" we've still used Apple's CommonCrypto\ninstead of sha1collisiondetection on modern versions of Darwin and\nOSX.\n\n1. 61067954ce1 (cache.h: eliminate SHA-1 deprecation warnings on Mac\n OS X, 2013-05-19)\n2. 9c7a0beee09 (config.mak.uname: set NO_APPLE_COMMON_CRYPTO on older\n systems, 2014-08-15)\n3. We could probably drop \"NO_APPLE_COMMON_CRYPTO\", as nobody's likely\n to care about such on old version of OSX anymore. But let's leave that\n for now.\n4. e6b07da2780 (Makefile: make DC_SHA1 the default, 2017-03-17)\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n Makefile | 5 +++++\n 1 file changed, 5 insertions(+)\n\ndiff --git a/Makefile b/Makefile\nindex 8a9f3e561f8..0ca8a9eb318 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -519,6 +519,11 @@ include shared.mak\n # Define BLK_SHA1 to make use of optimized C SHA-1 routines bundled\n # with git (in the block-sha1/ directory).\n #\n+# Define NO_APPLE_COMMON_CRYPTO on OSX to opt-out of using the\n+# \"APPLE_COMMON_CRYPTO\" backend for SHA-1, which is currently the\n+# default on that OS. We'll define NO_APPLE_COMMON_CRYPTO on Mac OS\n+# 10.4 or older (\"Tiger\", released in early 2005).\n+#\n # === SHA-256 backend ===\n #\n # ==== Default SHA-256 backend ====\n-- \n2.38.0.1178.g509f5fa8ce0\n\n"},{"id":"465366","messageId":"patch-v3-9.9-c1f27255d3e-20221020T223946Z-avarab@gmail.com","threadId":"57779","inReplyTo":"cover-v3-0.9-00000000000-20221020T223946Z-avarab@gmail.com","subject":"[PATCH v3 9/9] Makefile: discuss SHAttered in *_SHA{1,256} discussion","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-10-20T22:43:10Z","receivedAt":"2022-10-20T22:44:09Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Let's mention the SHAttered attack and more generally why we use the\nsha1collisiondetection backend by default, and note that for SHA-256\nthe user should feel free to pick any of the supported backends as far\nas hashing security is concerned.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n Makefile | 15 +++++++++++++++\n 1 file changed, 15 insertions(+)\n\ndiff --git a/Makefile b/Makefile\nindex 0ca8a9eb318..36d6bffd1f1 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -481,6 +481,16 @@ include shared.mak\n #\n # === SHA-1 backend ===\n #\n+# ==== Security ====\n+#\n+# Due to the SHAttered (https://shattered.io) attack vector on SHA-1\n+# it's strongly recommended to use the sha1collisiondetection\n+# counter-cryptanalysis library for SHA-1 hashing (DC_SHA1).\n+#\n+# If you know that you can trust the repository contents, or where\n+# potential SHA-1 attacks are otherwise mitigated the backends listed\n+# in \"Other SHA-1 implementations\" are faster than DC_SHA1.\n+#\n # ==== Default SHA-1 backend ====\n #\n # If no *_SHA1 backend is picked we'll fall fall back on using the\n@@ -526,6 +536,11 @@ include shared.mak\n #\n # === SHA-256 backend ===\n #\n+# ==== Security ====\n+#\n+# Unlike SHA-1 the SHA-256 algorithm does not suffer from any known\n+# vulnerabilities, so any implementation will do.\n+#\n # ==== Default SHA-256 backend ====\n #\n # If no *_SHA256 backend is picked we'll fall fall back on using the\n-- \n2.38.0.1178.g509f5fa8ce0\n\n"},{"id":"465368","messageId":"CAPig+cQEugkHFQivkeRCXdn-_6pYuyRffchz0Aej3FSxra0Xwg@mail.gmail.com","threadId":"57779","inReplyTo":"patch-v3-7.9-05edcfb9cd9-20221020T223946Z-avarab@gmail.com","subject":"Re: [PATCH v3 7/9] Makefile: document SHA-1 and SHA-256 default and selection order","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2022-10-20T22:58:20Z","receivedAt":"2022-10-20T22:58:36Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Thu, Oct 20, 2022 at 6:43 PM Ævar Arnfjörð Bjarmason\n wrote:\n> For the *_SHA1 and *_SHA256 flags we've discussed the various flags,\n> but not the fact that when you define multiple flags we'll pick one.\n>\n> Which one we pick depends on the order they're listed in the Makefile,\n> which differed from the order we discussed them in this documentation.\n>\n> Let's be explicit about how we select these, and re-arrange the\n> listings so that they're listed in the priority order we've picked.\n>\n> I'd personally prefer that the selection was more explicit, and that\n> we'd error out if conflicting flags were provided, but per the\n> discussion downhtread of[1] the consensus was to keep theses semantics.\n>\n> This behavior make it easier to e.g. integrate with autoconf-like\n\ns/make/makes/\n\n> systems, where the configuration can provide everything it can\n> support, and Git is tasked with picking the first one it prefers.\n>\n> Signed-off-by: Ævar Arnfjörð Bjarmason \n> ---\n> diff --git a/Makefile b/Makefile\n> @@ -481,6 +481,14 @@ include shared.mak\n> +# ==== Default SHA-1 backend ====\n> +#\n> +# If no *_SHA1 backend is picked we'll fall fall back on using the\n> +# default.\n\ns/fall fall/fall/\n\n> # Define OPENSSL_SHA1 to link to the the SHA-1 routines from\n> # the OpenSSL library.\n\nNot a problem introduced by this patch, but this could use a: s/the the/the/\n"},{"id":"465370","messageId":"CAPig+cQHgc-4kfaRHJm9VKxbSE_Y0k_f_Fa=NFGs7wEq09nkYw@mail.gmail.com","threadId":"57779","inReplyTo":"patch-v3-8.9-859e69fbe9f-20221020T223946Z-avarab@gmail.com","subject":"Re: [PATCH v3 8/9] Makefile: document default SHA-1 backend on OSX","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2022-10-20T23:01:59Z","receivedAt":"2022-10-20T23:03:04Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Thu, Oct 20, 2022 at 6:43 PM Ævar Arnfjörð Bjarmason\n wrote:\n> Since [1] the default SHA-1 backend on OSX has been\n> APPLE_COMMON_CRYPTO. Per [2] we'll skip using it on anything older\n> than Mac OS X 10.4 \"Tiger\"[3].\n>\n> When \"DC_SHA1\" was made the default in [4] this interaction between it\n> and APPLE_COMMON_CRYPTO seems to have been missed in. Ever since\n> DC_SHA1 was \"made the default\" we've still used Apple's CommonCrypto\n> instead of sha1collisiondetection on modern versions of Darwin and\n> OSX.\n>\n> Signed-off-by: Ævar Arnfjörð Bjarmason \n> ---\n> diff --git a/Makefile b/Makefile\n> @@ -519,6 +519,11 @@ include shared.mak\n> +# Define NO_APPLE_COMMON_CRYPTO on OSX to opt-out of using the\n> +# \"APPLE_COMMON_CRYPTO\" backend for SHA-1, which is currently the\n> +# default on that OS. We'll define NO_APPLE_COMMON_CRYPTO on Mac OS\n> +# 10.4 or older (\"Tiger\", released in early 2005).\n\nI found the \"we'll define...\" part somewhat confusing. I might have\nwritten it as:\n\n On macOS 01.4 (Tiger) or older, NO_APPLE_COMMON_CRYPTO\n is defined by default.\n\nor:\n\n ...is defined automatically.\n\nProbably not worth a reroll.\n"},{"id":"465783","messageId":"cover-v4-0.9-00000000000-20221026T145255Z-avarab@gmail.com","threadId":"57779","inReplyTo":"cover-v3-0.9-00000000000-20221020T223946Z-avarab@gmail.com","subject":"[PATCH v4 0/9] Makefile & docs: document SHA-{1,256} behavior, fix bug","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-10-26T14:56:38Z","receivedAt":"2022-10-26T14:56:58Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"This is a documentation update to get to the eventual goal of making\nDC_SHA1 the defaul on OSX. First we need to stop claiming that OpenSSL\nis still our default everywhere, etc.\n\nFor v3 see:\nhttps://lore.kernel.org/git/cover-v3-0.9-00000000000-20221020T223946Z-avarab@gmail.com/;\nThe updates here are all typos/grammar etc. issues spotted by Eric\nSunshine, thanks!\n\nJunio: This v4 introduces a minor conflict with the subsequent DC_SHA1\ntopic[1], but you haven't picked that one up either, so I'm assuming\nif you're interested in this at all it's better to cook this first.\n\nIn case you want both It's easily solved (just keep the other topic's\npost-image). I.e. Eric pointed out a better wording for a paragraph,\nbut it's the one we're mostly removing/rewriting in [1].\n\n1. https://lore.kernel.org/git/cover-v2-0.4-00000000000-20221019T010222Z-avarab@gmail.com/\n\nÆvar Arnfjörð Bjarmason (9):\n Makefile: always (re)set DC_SHA1 on fallback\n INSTALL: remove discussion of SHA-1 backends\n Makefile: correct DC_SHA1 documentation\n Makefile: create and use sections for \"define\" flag listing\n Makefile: rephrase the discussion of *_SHA1 knobs\n Makefile: document default SHA-256 backend\n Makefile: document SHA-1 and SHA-256 default and selection order\n Makefile: document default SHA-1 backend on OSX\n Makefile: discuss SHAttered in *_SHA{1,256} discussion\n\n INSTALL | 4 -\n Makefile | 259 +++++++++++++++++++++++++++++++++++--------------------\n 2 files changed, 165 insertions(+), 98 deletions(-)\n\nRange-diff against v3:\n 1: ef3c5be11e0 = 1: 11e92d15616 Makefile: always (re)set DC_SHA1 on fallback\n 2: 017a0a9791c = 2: abbe25f56b3 INSTALL: remove discussion of SHA-1 backends\n 3: 62dd2d5708d = 3: b0bd35987c0 Makefile: correct DC_SHA1 documentation\n 4: 933bef576b3 = 4: d0451d6c3a3 Makefile: create and use sections for \"define\" flag listing\n 5: 5b18198c477 ! 5: b956d1c2640 Makefile: rephrase the discussion of *_SHA1 knobs\n @@ Makefile: include shared.mak\n #\n -# Define OPENSSL_SHA1 environment variable when running make to link\n -# with the SHA1 routine from openssl library.\n -+# Define OPENSSL_SHA1 to link to the the SHA-1 routines from\n -+# the OpenSSL library.\n ++# Define OPENSSL_SHA1 to link to the SHA-1 routines from the OpenSSL\n ++# library.\n #\n # === SHA-256 backend ===\n #\n 6: 73685592aba = 6: 1e4695d0ba0 Makefile: document default SHA-256 backend\n 7: 05edcfb9cd9 ! 7: 847be3d32e2 Makefile: document SHA-1 and SHA-256 default and selection order\n @@ Commit message\n we'd error out if conflicting flags were provided, but per the\n discussion downhtread of[1] the consensus was to keep theses semantics.\n \n - This behavior make it easier to e.g. integrate with autoconf-like\n + This behavior makes it easier to e.g. integrate with autoconf-like\n systems, where the configuration can provide everything it can\n support, and Git is tasked with picking the first one it prefers.\n \n @@ Makefile: include shared.mak\n #\n +# ==== Default SHA-1 backend ====\n +#\n -+# If no *_SHA1 backend is picked we'll fall fall back on using the\n -+# default.\n ++# If no *_SHA1 backend is picked we'll fall back on using the default.\n +#\n +# Multiple *_SHA1 backends can be selected, the first supported one\n +# listed in \"Other SHA-1 implementations\" will be picked.\n @@ Makefile: include shared.mak\n -# Define BLK_SHA1 to make use of optimized C SHA-1 routines bundled\n -# with git (in the block-sha1/ directory).\n -#\n - # Define OPENSSL_SHA1 to link to the the SHA-1 routines from\n - # the OpenSSL library.\n + # Define OPENSSL_SHA1 to link to the SHA-1 routines from the OpenSSL\n + # library.\n #\n +# Define BLK_SHA1 to make use of optimized C SHA-1 routines bundled\n +# with git (in the block-sha1/ directory).\n 8: 859e69fbe9f ! 8: 0af3ea78eaf Makefile: document default SHA-1 backend on OSX\n @@ Makefile: include shared.mak\n #\n +# Define NO_APPLE_COMMON_CRYPTO on OSX to opt-out of using the\n +# \"APPLE_COMMON_CRYPTO\" backend for SHA-1, which is currently the\n -+# default on that OS. We'll define NO_APPLE_COMMON_CRYPTO on Mac OS\n -+# 10.4 or older (\"Tiger\", released in early 2005).\n ++# default on that OS. On macOS 01.4 (Tiger) or older,\n ++# NO_APPLE_COMMON_CRYPTO is defined by default.\n +#\n # === SHA-256 backend ===\n #\n 9: c1f27255d3e ! 9: 9045ff9c4ed Makefile: discuss SHAttered in *_SHA{1,256} discussion\n @@ Makefile: include shared.mak\n +#\n # ==== Default SHA-1 backend ====\n #\n - # If no *_SHA1 backend is picked we'll fall fall back on using the\n + # If no *_SHA1 backend is picked we'll fall back on using the default.\n @@ Makefile: include shared.mak\n #\n # === SHA-256 backend ===\n-- \n2.38.0.1251.g3eefdfb5e7a\n\n"},{"id":"465784","messageId":"patch-v4-1.9-11e92d15616-20221026T145255Z-avarab@gmail.com","threadId":"57779","inReplyTo":"cover-v4-0.9-00000000000-20221026T145255Z-avarab@gmail.com","subject":"[PATCH v4 1/9] Makefile: always (re)set DC_SHA1 on fallback","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-10-26T14:56:39Z","receivedAt":"2022-10-26T14:57:01Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Fix an edge case introduced in in e6b07da2780 (Makefile: make DC_SHA1\nthe default, 2017-03-17), when DC_SHA1 was made the default fallback\nwe started unconditionally adding to BASIC_CFLAGS and LIB_OBJS, so\nwe'd use the sha1collisiondetection by default.\n\nBut the \"DC_SHA1\" variable remained unset, so e.g.:\n\n\tmake test DC_SHA1= T=t0013*.sh\n\nWould skip the sha1collisiondetection tests, as we'd write\n\"DC_SHA1=''\" to \"GIT-BUILD-OPTIONS\", but if we manually removed that\ntest prerequisite we'd pass the test (which we couldn't if we weren't\nusing sha1collisiondetection).\n\nSo let's have the fallback assignment use the 'override' directive\ninstead of the \":=\" simply expanded variable introduced in\ne6b07da2780. In this case we explicitly want to override the user's\nchoice.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n Makefile | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)\n\ndiff --git a/Makefile b/Makefile\nindex 85f03c6aed1..744bd4344f4 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -1823,7 +1823,7 @@ ifdef APPLE_COMMON_CRYPTO\n \tCOMPAT_CFLAGS += -DCOMMON_DIGEST_FOR_OPENSSL\n \tBASIC_CFLAGS += -DSHA1_APPLE\n else\n-\tDC_SHA1 := YesPlease\n+\toverride DC_SHA1 = YesPlease\n \tBASIC_CFLAGS += -DSHA1_DC\n \tLIB_OBJS += sha1dc_git.o\n ifdef DC_SHA1_EXTERNAL\n-- \n2.38.0.1251.g3eefdfb5e7a\n\n"},{"id":"465785","messageId":"patch-v4-2.9-abbe25f56b3-20221026T145255Z-avarab@gmail.com","threadId":"57779","inReplyTo":"cover-v4-0.9-00000000000-20221026T145255Z-avarab@gmail.com","subject":"[PATCH v4 2/9] INSTALL: remove discussion of SHA-1 backends","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-10-26T14:56:40Z","receivedAt":"2022-10-26T14:57:03Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"The claim that OpenSSL is the default SHA-1 backend hasn't been true\nsince e6b07da2780 (Makefile: make DC_SHA1 the default, 2017-03-17),\nbut more importantly tweaking the SHA-1 backend isn't something that's\ncommon enough to warrant discussing in the INSTALL document, so let's\nremove this paragraph.\n\nThis discussion was originally added in c538d2d34ab (Add some\ninstallation notes in INSTALL, 2005-06-17) when tweaking the default\nbackend was more common. The current wording was added in\n5beb577db8c (INSTALL: Describe dependency knobs from Makefile,\n2009-09-10).\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n INSTALL | 4 ----\n 1 file changed, 4 deletions(-)\n\ndiff --git a/INSTALL b/INSTALL\nindex 89b15d71df5..33447883974 100644\n--- a/INSTALL\n+++ b/INSTALL\n@@ -133,10 +133,6 @@ Issues of note:\n \t you are using libcurl older than 7.34.0. Otherwise you can use\n \t NO_OPENSSL without losing git-imap-send.\n \n-\t By default, git uses OpenSSL for SHA1 but it will use its own\n-\t library (inspired by Mozilla's) with either NO_OPENSSL or\n-\t BLK_SHA1.\n-\n \t- \"libcurl\" library is used for fetching and pushing\n \t repositories over http:// or https://, as well as by\n \t git-imap-send if the curl version is >= 7.34.0. If you do\n-- \n2.38.0.1251.g3eefdfb5e7a\n\n"},{"id":"465786","messageId":"patch-v4-3.9-b0bd35987c0-20221026T145255Z-avarab@gmail.com","threadId":"57779","inReplyTo":"cover-v4-0.9-00000000000-20221026T145255Z-avarab@gmail.com","subject":"[PATCH v4 3/9] Makefile: correct DC_SHA1 documentation","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-10-26T14:56:41Z","receivedAt":"2022-10-26T14:57:04Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"The claim that DC_SHA1 takes priority over other *_SHA1 knobs was true\nwhen it was added in [1], But that hasn't been the case since it was\nmade the fallback default in [2].\n\nWe should be making it not only the default, but something that takes\npriority over other *_SHA1 knobs, but that's outside the scope of this\nchange. For now let's correct the documentation to match reality.\n\nLet's also remove the \"unconditionally enable\" wording, per the above\nthe enabling of \"DC_SHA1\" is conditional on these other flags.\n\n1. 8325e43b82d (Makefile: add DC_SHA1 knob, 2017-03-16)\n2. e6b07da2780 (Makefile: make DC_SHA1 the default, 2017-03-17)\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n Makefile | 3 +--\n 1 file changed, 1 insertion(+), 2 deletions(-)\n\ndiff --git a/Makefile b/Makefile\nindex 744bd4344f4..a2762e583a1 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -155,9 +155,8 @@ include shared.mak\n # Define BLK_SHA1 environment variable to make use of the bundled\n # optimized C SHA1 routine.\n #\n-# Define DC_SHA1 to unconditionally enable the collision-detecting sha1\n+# Define DC_SHA1 to enable the collision-detecting sha1\n # algorithm. This is slower, but may detect attempted collision attacks.\n-# Takes priority over other *_SHA1 knobs.\n #\n # Define DC_SHA1_EXTERNAL in addition to DC_SHA1 if you want to build / link\n # git with the external SHA1 collision-detect library.\n-- \n2.38.0.1251.g3eefdfb5e7a\n\n"},{"id":"465787","messageId":"patch-v4-5.9-b956d1c2640-20221026T145255Z-avarab@gmail.com","threadId":"57779","inReplyTo":"cover-v4-0.9-00000000000-20221026T145255Z-avarab@gmail.com","subject":"[PATCH v4 5/9] Makefile: rephrase the discussion of *_SHA1 knobs","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-10-26T14:56:43Z","receivedAt":"2022-10-26T14:57:18Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"In the preceding commit the discussion of the *_SHA1 knobs was left\nas-is to benefit from a smaller diff, but since we're changing these\nlet's use the same phrasing we use for most other knobs. E.g. \"define\nX\", not \"define X environment variable\", and get rid of the \"when\nrunning make to link with\" entirely.\n\nFurthermore the discussion of DC_SHA1* options is now under a \"Options\nfor the sha1collisiondetection implementation\" heading, so we don't\nneed to clarify that these options go along with DC_SHA1=Y, so let's\nrephrase them accordingly.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n Makefile | 12 ++++++------\n 1 file changed, 6 insertions(+), 6 deletions(-)\n\ndiff --git a/Makefile b/Makefile\nindex a983a57fb51..870ecdb0d85 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -492,12 +492,12 @@ include shared.mak\n # Define DC_SHA1 to enable the collision-detecting sha1\n # algorithm. This is slower, but may detect attempted collision attacks.\n #\n-# Define DC_SHA1_EXTERNAL in addition to DC_SHA1 if you want to build / link\n+# Define DC_SHA1_EXTERNAL if you want to build / link\n # git with the external SHA1 collision-detect library.\n # Without this option, i.e. the default behavior is to build git with its\n # own built-in code (or submodule).\n #\n-# Define DC_SHA1_SUBMODULE in addition to DC_SHA1 to use the\n+# Define DC_SHA1_SUBMODULE to use the\n # sha1collisiondetection shipped as a submodule instead of the\n # non-submodule copy in sha1dc/. This is an experimental option used\n # by the git project to migrate to using sha1collisiondetection as a\n@@ -505,11 +505,11 @@ include shared.mak\n #\n # ==== Other SHA-1 implementations ====\n #\n-# Define BLK_SHA1 environment variable to make use of the bundled\n-# optimized C SHA1 routine.\n+# Define BLK_SHA1 to make use of optimized C SHA-1 routines bundled\n+# with git (in the block-sha1/ directory).\n #\n-# Define OPENSSL_SHA1 environment variable when running make to link\n-# with the SHA1 routine from openssl library.\n+# Define OPENSSL_SHA1 to link to the SHA-1 routines from the OpenSSL\n+# library.\n #\n # === SHA-256 backend ===\n #\n-- \n2.38.0.1251.g3eefdfb5e7a\n\n"},{"id":"465788","messageId":"patch-v4-6.9-1e4695d0ba0-20221026T145255Z-avarab@gmail.com","threadId":"57779","inReplyTo":"cover-v4-0.9-00000000000-20221026T145255Z-avarab@gmail.com","subject":"[PATCH v4 6/9] Makefile: document default SHA-256 backend","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-10-26T14:56:44Z","receivedAt":"2022-10-26T14:57:20Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Since 27dc04c5450 (sha256: add an SHA-256 implementation using\nlibgcrypt, 2018-11-14) we've claimed to support a BLK_SHA256 flag, but\nthere's no such SHA-256 backend.\n\nInstead we fall back on adding \"sha256/block/sha256.o\" to \"LIB_OBJS\"\nand adding \"-DSHA256_BLK\" to BASIC_CFLAGS.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n Makefile | 10 +++++++++-\n 1 file changed, 9 insertions(+), 1 deletion(-)\n\ndiff --git a/Makefile b/Makefile\nindex 870ecdb0d85..992eba9e0c8 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -513,7 +513,15 @@ include shared.mak\n #\n # === SHA-256 backend ===\n #\n-# Define BLK_SHA256 to use the built-in SHA-256 routines.\n+# ==== Default SHA-256 backend ====\n+#\n+# If no *_SHA256 backend is picked we'll fall fall back on using the\n+# default.\n+#\n+# The default SHA-256 backend is shipped with Git. No flag is required\n+# to enable it. To select it don't define any other *_SHA256 flag.\n+#\n+# ==== Other SHA-256 implementations ====\n #\n # Define NETTLE_SHA256 to use the SHA-256 routines in libnettle.\n #\n-- \n2.38.0.1251.g3eefdfb5e7a\n\n"},{"id":"465789","messageId":"patch-v4-7.9-847be3d32e2-20221026T145255Z-avarab@gmail.com","threadId":"57779","inReplyTo":"cover-v4-0.9-00000000000-20221026T145255Z-avarab@gmail.com","subject":"[PATCH v4 7/9] Makefile: document SHA-1 and SHA-256 default and selection order","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-10-26T14:56:45Z","receivedAt":"2022-10-26T14:57:22Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"For the *_SHA1 and *_SHA256 flags we've discussed the various flags,\nbut not the fact that when you define multiple flags we'll pick one.\n\nWhich one we pick depends on the order they're listed in the Makefile,\nwhich differed from the order we discussed them in this documentation.\n\nLet's be explicit about how we select these, and re-arrange the\nlistings so that they're listed in the priority order we've picked.\n\nI'd personally prefer that the selection was more explicit, and that\nwe'd error out if conflicting flags were provided, but per the\ndiscussion downhtread of[1] the consensus was to keep theses semantics.\n\nThis behavior makes it easier to e.g. integrate with autoconf-like\nsystems, where the configuration can provide everything it can\nsupport, and Git is tasked with picking the first one it prefers.\n\n1. https://lore.kernel.org/git/220710.86mtdh81ty.gmgdl@evledraar.gmail.com/\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n Makefile | 20 +++++++++++++++-----\n 1 file changed, 15 insertions(+), 5 deletions(-)\n\ndiff --git a/Makefile b/Makefile\nindex 992eba9e0c8..9f7cf1f22d4 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -481,6 +481,13 @@ include shared.mak\n #\n # === SHA-1 backend ===\n #\n+# ==== Default SHA-1 backend ====\n+#\n+# If no *_SHA1 backend is picked we'll fall back on using the default.\n+#\n+# Multiple *_SHA1 backends can be selected, the first supported one\n+# listed in \"Other SHA-1 implementations\" will be picked.\n+#\n # ==== Options common to all SHA-1 implementations ====\n #\n # Define SHA1_MAX_BLOCK_SIZE to limit the amount of data that will be hashed\n@@ -505,12 +512,12 @@ include shared.mak\n #\n # ==== Other SHA-1 implementations ====\n #\n-# Define BLK_SHA1 to make use of optimized C SHA-1 routines bundled\n-# with git (in the block-sha1/ directory).\n-#\n # Define OPENSSL_SHA1 to link to the SHA-1 routines from the OpenSSL\n # library.\n #\n+# Define BLK_SHA1 to make use of optimized C SHA-1 routines bundled\n+# with git (in the block-sha1/ directory).\n+#\n # === SHA-256 backend ===\n #\n # ==== Default SHA-256 backend ====\n@@ -521,14 +528,17 @@ include shared.mak\n # The default SHA-256 backend is shipped with Git. No flag is required\n # to enable it. To select it don't define any other *_SHA256 flag.\n #\n+# Multiple *_SHA256 backends can be selected, the first supported one\n+# listed in \"Other SHA-256 implementations\" below will be picked.\n+#\n # ==== Other SHA-256 implementations ====\n #\n+# Define OPENSSL_SHA256 to use the SHA-256 routines in OpenSSL.\n+#\n # Define NETTLE_SHA256 to use the SHA-256 routines in libnettle.\n #\n # Define GCRYPT_SHA256 to use the SHA-256 routines in libgcrypt.\n #\n-# Define OPENSSL_SHA256 to use the SHA-256 routines in OpenSSL.\n-#\n # == DEVELOPER defines ==\n #\n # Define DEVELOPER to enable more compiler warnings. Compiler version\n-- \n2.38.0.1251.g3eefdfb5e7a\n\n"},{"id":"465790","messageId":"patch-v4-4.9-d0451d6c3a3-20221026T145255Z-avarab@gmail.com","threadId":"57779","inReplyTo":"cover-v4-0.9-00000000000-20221026T145255Z-avarab@gmail.com","subject":"[PATCH v4 4/9] Makefile: create and use sections for \"define\" flag listing","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-10-26T14:56:42Z","receivedAt":"2022-10-26T14:57:23Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Since the \"Define ...\" template of comments at the top of the Makefile\nwas started in 5bdac8b3269 ([PATCH] Improve the compilation-time\nsettings interface, 2005-07-29) we've had a lot more flags added,\nincluding flags that come in \"groups\". Not having any obvious\nstructure to the >500 line comment at the top of the Makefile has made\nit hard to follow.\n\nThis change is almost entirely a move-only change, the two paragraphs\nat the start of the first two sections are new, and so are the added\nsections themselves, but other than that no lines are changed, only\nmoved.\n\nWe now list Makefile-only flags at the start, followed by stand-alone\nflags, and then cover \"optional library\" flags in their respective\ngroups, followed by SHA-1 and SHA-256 flags, and finally\nDEVELOPER-specific flags.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n Makefile | 218 ++++++++++++++++++++++++++++++++-----------------------\n 1 file changed, 126 insertions(+), 92 deletions(-)\n\ndiff --git a/Makefile b/Makefile\nindex a2762e583a1..a983a57fb51 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -4,8 +4,20 @@ all::\n # Import tree-wide shared Makefile behavior and libraries\n include shared.mak\n \n+# == Makefile defines ==\n+#\n+# These defines change the behavior of the Makefile itself, but have\n+# no impact on what it builds:\n+#\n # Define V=1 to have a more verbose compile.\n #\n+# == Portability and optional library defines ==\n+#\n+# These defines indicate what Git can expect from the OS, what\n+# libraries are available etc. Much of this is auto-detected in\n+# config.mak.uname, or in configure.ac when using the optional \"make\n+# configure && ./configure\" (see INSTALL).\n+#\n # Define SHELL_PATH to a POSIX shell if your /bin/sh is broken.\n #\n # Define SANE_TOOL_PATH to a colon-separated list of paths to prepend\n@@ -30,68 +42,8 @@ include shared.mak\n #\n # Define NO_OPENSSL environment variable if you do not have OpenSSL.\n #\n-# Define USE_LIBPCRE if you have and want to use libpcre. Various\n-# commands such as log and grep offer runtime options to use\n-# Perl-compatible regular expressions instead of standard or extended\n-# POSIX regular expressions.\n-#\n-# Only libpcre version 2 is supported. USE_LIBPCRE2 is a synonym for\n-# USE_LIBPCRE, support for the old USE_LIBPCRE1 has been removed.\n-#\n-# Define LIBPCREDIR=/foo/bar if your PCRE header and library files are\n-# in /foo/bar/include and /foo/bar/lib directories.\n-#\n # Define HAVE_ALLOCA_H if you have working alloca(3) defined in that header.\n #\n-# Define NO_CURL if you do not have libcurl installed. git-http-fetch and\n-# git-http-push are not built, and you cannot use http:// and https://\n-# transports (neither smart nor dumb).\n-#\n-# Define CURLDIR=/foo/bar if your curl header and library files are in\n-# /foo/bar/include and /foo/bar/lib directories.\n-#\n-# Define CURL_CONFIG to curl's configuration program that prints information\n-# about the library (e.g., its version number). The default is 'curl-config'.\n-#\n-# Define CURL_LDFLAGS to specify flags that you need to link when using libcurl,\n-# if you do not want to rely on the libraries provided by CURL_CONFIG. The\n-# default value is a result of `curl-config --libs`. An example value for\n-# CURL_LDFLAGS is as follows:\n-#\n-# CURL_LDFLAGS=-lcurl\n-#\n-# Define NO_EXPAT if you do not have expat installed. git-http-push is\n-# not built, and you cannot push using http:// and https:// transports (dumb).\n-#\n-# Define EXPATDIR=/foo/bar if your expat header and library files are in\n-# /foo/bar/include and /foo/bar/lib directories.\n-#\n-# Define EXPAT_NEEDS_XMLPARSE_H if you have an old version of expat (e.g.,\n-# 1.1 or 1.2) that provides xmlparse.h instead of expat.h.\n-#\n-# Define NO_GETTEXT if you don't want Git output to be translated.\n-# A translated Git requires GNU libintl or another gettext implementation,\n-# plus libintl-perl at runtime.\n-#\n-# Define USE_GETTEXT_SCHEME and set it to 'fallthrough', if you don't trust\n-# the installed gettext translation of the shell scripts output.\n-#\n-# Define HAVE_LIBCHARSET_H if you haven't set NO_GETTEXT and you can't\n-# trust the langinfo.h's nl_langinfo(CODESET) function to return the\n-# current character set. GNU and Solaris have a nl_langinfo(CODESET),\n-# FreeBSD can use either, but MinGW and some others need to use\n-# libcharset.h's locale_charset() instead.\n-#\n-# Define CHARSET_LIB to the library you need to link with in order to\n-# use locale_charset() function. On some platforms this needs to set to\n-# -lcharset, on others to -liconv .\n-#\n-# Define LIBC_CONTAINS_LIBINTL if your gettext implementation doesn't\n-# need -lintl when linking.\n-#\n-# Define NO_MSGFMT_EXTENDED_OPTIONS if your implementation of msgfmt\n-# doesn't support GNU extensions like --check and --statistics\n-#\n # Define HAVE_PATHS_H if you have paths.h and want to use the default PATH\n # it specifies.\n #\n@@ -152,38 +104,6 @@ include shared.mak\n # and do not want to use Apple's CommonCrypto library. This allows you\n # to provide your own OpenSSL library, for example from MacPorts.\n #\n-# Define BLK_SHA1 environment variable to make use of the bundled\n-# optimized C SHA1 routine.\n-#\n-# Define DC_SHA1 to enable the collision-detecting sha1\n-# algorithm. This is slower, but may detect attempted collision attacks.\n-#\n-# Define DC_SHA1_EXTERNAL in addition to DC_SHA1 if you want to build / link\n-# git with the external SHA1 collision-detect library.\n-# Without this option, i.e. the default behavior is to build git with its\n-# own built-in code (or submodule).\n-#\n-# Define DC_SHA1_SUBMODULE in addition to DC_SHA1 to use the\n-# sha1collisiondetection shipped as a submodule instead of the\n-# non-submodule copy in sha1dc/. This is an experimental option used\n-# by the git project to migrate to using sha1collisiondetection as a\n-# submodule.\n-#\n-# Define OPENSSL_SHA1 environment variable when running make to link\n-# with the SHA1 routine from openssl library.\n-#\n-# Define SHA1_MAX_BLOCK_SIZE to limit the amount of data that will be hashed\n-# in one call to the platform's SHA1_Update(). e.g. APPLE_COMMON_CRYPTO\n-# wants 'SHA1_MAX_BLOCK_SIZE=1024L*1024L*1024L' defined.\n-#\n-# Define BLK_SHA256 to use the built-in SHA-256 routines.\n-#\n-# Define NETTLE_SHA256 to use the SHA-256 routines in libnettle.\n-#\n-# Define GCRYPT_SHA256 to use the SHA-256 routines in libgcrypt.\n-#\n-# Define OPENSSL_SHA256 to use the SHA-256 routines in OpenSSL.\n-#\n # Define NEEDS_CRYPTO_WITH_SSL if you need -lcrypto when using -lssl (Darwin).\n #\n # Define NEEDS_SSL_WITH_CRYPTO if you need -lssl when using -lcrypto (Darwin).\n@@ -489,6 +409,120 @@ include shared.mak\n # to the \"\" of the corresponding `compat/fsmonitor/fsm-settings-.c`\n # that implements the `fsm_os_settings__*()` routines.\n #\n+# === Optional library: libintl ===\n+#\n+# Define NO_GETTEXT if you don't want Git output to be translated.\n+# A translated Git requires GNU libintl or another gettext implementation,\n+# plus libintl-perl at runtime.\n+#\n+# Define USE_GETTEXT_SCHEME and set it to 'fallthrough', if you don't trust\n+# the installed gettext translation of the shell scripts output.\n+#\n+# Define HAVE_LIBCHARSET_H if you haven't set NO_GETTEXT and you can't\n+# trust the langinfo.h's nl_langinfo(CODESET) function to return the\n+# current character set. GNU and Solaris have a nl_langinfo(CODESET),\n+# FreeBSD can use either, but MinGW and some others need to use\n+# libcharset.h's locale_charset() instead.\n+#\n+# Define CHARSET_LIB to the library you need to link with in order to\n+# use locale_charset() function. On some platforms this needs to set to\n+# -lcharset, on others to -liconv .\n+#\n+# Define LIBC_CONTAINS_LIBINTL if your gettext implementation doesn't\n+# need -lintl when linking.\n+#\n+# Define NO_MSGFMT_EXTENDED_OPTIONS if your implementation of msgfmt\n+# doesn't support GNU extensions like --check and --statistics\n+#\n+# === Optional library: libexpat ===\n+#\n+# Define NO_EXPAT if you do not have expat installed. git-http-push is\n+# not built, and you cannot push using http:// and https:// transports (dumb).\n+#\n+# Define EXPATDIR=/foo/bar if your expat header and library files are in\n+# /foo/bar/include and /foo/bar/lib directories.\n+#\n+# Define EXPAT_NEEDS_XMLPARSE_H if you have an old version of expat (e.g.,\n+# 1.1 or 1.2) that provides xmlparse.h instead of expat.h.\n+\n+# === Optional library: libcurl ===\n+#\n+# Define NO_CURL if you do not have libcurl installed. git-http-fetch and\n+# git-http-push are not built, and you cannot use http:// and https://\n+# transports (neither smart nor dumb).\n+#\n+# Define CURLDIR=/foo/bar if your curl header and library files are in\n+# /foo/bar/include and /foo/bar/lib directories.\n+#\n+# Define CURL_CONFIG to curl's configuration program that prints information\n+# about the library (e.g., its version number). The default is 'curl-config'.\n+#\n+# Define CURL_LDFLAGS to specify flags that you need to link when using libcurl,\n+# if you do not want to rely on the libraries provided by CURL_CONFIG. The\n+# default value is a result of `curl-config --libs`. An example value for\n+# CURL_LDFLAGS is as follows:\n+#\n+# CURL_LDFLAGS=-lcurl\n+#\n+# === Optional library: libpcre2 ===\n+#\n+# Define USE_LIBPCRE if you have and want to use libpcre. Various\n+# commands such as log and grep offer runtime options to use\n+# Perl-compatible regular expressions instead of standard or extended\n+# POSIX regular expressions.\n+#\n+# Only libpcre version 2 is supported. USE_LIBPCRE2 is a synonym for\n+# USE_LIBPCRE, support for the old USE_LIBPCRE1 has been removed.\n+#\n+# Define LIBPCREDIR=/foo/bar if your PCRE header and library files are\n+# in /foo/bar/include and /foo/bar/lib directories.\n+#\n+# == SHA-1 and SHA-256 defines ==\n+#\n+# === SHA-1 backend ===\n+#\n+# ==== Options common to all SHA-1 implementations ====\n+#\n+# Define SHA1_MAX_BLOCK_SIZE to limit the amount of data that will be hashed\n+# in one call to the platform's SHA1_Update(). e.g. APPLE_COMMON_CRYPTO\n+# wants 'SHA1_MAX_BLOCK_SIZE=1024L*1024L*1024L' defined.\n+#\n+# ==== Options for the sha1collisiondetection implementation ====\n+#\n+# Define DC_SHA1 to enable the collision-detecting sha1\n+# algorithm. This is slower, but may detect attempted collision attacks.\n+#\n+# Define DC_SHA1_EXTERNAL in addition to DC_SHA1 if you want to build / link\n+# git with the external SHA1 collision-detect library.\n+# Without this option, i.e. the default behavior is to build git with its\n+# own built-in code (or submodule).\n+#\n+# Define DC_SHA1_SUBMODULE in addition to DC_SHA1 to use the\n+# sha1collisiondetection shipped as a submodule instead of the\n+# non-submodule copy in sha1dc/. This is an experimental option used\n+# by the git project to migrate to using sha1collisiondetection as a\n+# submodule.\n+#\n+# ==== Other SHA-1 implementations ====\n+#\n+# Define BLK_SHA1 environment variable to make use of the bundled\n+# optimized C SHA1 routine.\n+#\n+# Define OPENSSL_SHA1 environment variable when running make to link\n+# with the SHA1 routine from openssl library.\n+#\n+# === SHA-256 backend ===\n+#\n+# Define BLK_SHA256 to use the built-in SHA-256 routines.\n+#\n+# Define NETTLE_SHA256 to use the SHA-256 routines in libnettle.\n+#\n+# Define GCRYPT_SHA256 to use the SHA-256 routines in libgcrypt.\n+#\n+# Define OPENSSL_SHA256 to use the SHA-256 routines in OpenSSL.\n+#\n+# == DEVELOPER defines ==\n+#\n # Define DEVELOPER to enable more compiler warnings. Compiler version\n # and family are auto detected, but could be overridden by defining\n # COMPILER_FEATURES (see config.mak.dev). You can still set\n-- \n2.38.0.1251.g3eefdfb5e7a\n\n"},{"id":"465791","messageId":"patch-v4-8.9-0af3ea78eaf-20221026T145255Z-avarab@gmail.com","threadId":"57779","inReplyTo":"cover-v4-0.9-00000000000-20221026T145255Z-avarab@gmail.com","subject":"[PATCH v4 8/9] Makefile: document default SHA-1 backend on OSX","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-10-26T14:56:46Z","receivedAt":"2022-10-26T14:57:37Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Since [1] the default SHA-1 backend on OSX has been\nAPPLE_COMMON_CRYPTO. Per [2] we'll skip using it on anything older\nthan Mac OS X 10.4 \"Tiger\"[3].\n\nWhen \"DC_SHA1\" was made the default in [4] this interaction between it\nand APPLE_COMMON_CRYPTO seems to have been missed in. Ever since\nDC_SHA1 was \"made the default\" we've still used Apple's CommonCrypto\ninstead of sha1collisiondetection on modern versions of Darwin and\nOSX.\n\n1. 61067954ce1 (cache.h: eliminate SHA-1 deprecation warnings on Mac\n OS X, 2013-05-19)\n2. 9c7a0beee09 (config.mak.uname: set NO_APPLE_COMMON_CRYPTO on older\n systems, 2014-08-15)\n3. We could probably drop \"NO_APPLE_COMMON_CRYPTO\", as nobody's likely\n to care about such on old version of OSX anymore. But let's leave that\n for now.\n4. e6b07da2780 (Makefile: make DC_SHA1 the default, 2017-03-17)\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n Makefile | 5 +++++\n 1 file changed, 5 insertions(+)\n\ndiff --git a/Makefile b/Makefile\nindex 9f7cf1f22d4..a0ca6456b85 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -518,6 +518,11 @@ include shared.mak\n # Define BLK_SHA1 to make use of optimized C SHA-1 routines bundled\n # with git (in the block-sha1/ directory).\n #\n+# Define NO_APPLE_COMMON_CRYPTO on OSX to opt-out of using the\n+# \"APPLE_COMMON_CRYPTO\" backend for SHA-1, which is currently the\n+# default on that OS. On macOS 01.4 (Tiger) or older,\n+# NO_APPLE_COMMON_CRYPTO is defined by default.\n+#\n # === SHA-256 backend ===\n #\n # ==== Default SHA-256 backend ====\n-- \n2.38.0.1251.g3eefdfb5e7a\n\n"},{"id":"465792","messageId":"patch-v4-9.9-9045ff9c4ed-20221026T145255Z-avarab@gmail.com","threadId":"57779","inReplyTo":"cover-v4-0.9-00000000000-20221026T145255Z-avarab@gmail.com","subject":"[PATCH v4 9/9] Makefile: discuss SHAttered in *_SHA{1,256} discussion","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-10-26T14:56:47Z","receivedAt":"2022-10-26T14:57:39Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Let's mention the SHAttered attack and more generally why we use the\nsha1collisiondetection backend by default, and note that for SHA-256\nthe user should feel free to pick any of the supported backends as far\nas hashing security is concerned.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n Makefile | 15 +++++++++++++++\n 1 file changed, 15 insertions(+)\n\ndiff --git a/Makefile b/Makefile\nindex a0ca6456b85..805e88ed5fd 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -481,6 +481,16 @@ include shared.mak\n #\n # === SHA-1 backend ===\n #\n+# ==== Security ====\n+#\n+# Due to the SHAttered (https://shattered.io) attack vector on SHA-1\n+# it's strongly recommended to use the sha1collisiondetection\n+# counter-cryptanalysis library for SHA-1 hashing (DC_SHA1).\n+#\n+# If you know that you can trust the repository contents, or where\n+# potential SHA-1 attacks are otherwise mitigated the backends listed\n+# in \"Other SHA-1 implementations\" are faster than DC_SHA1.\n+#\n # ==== Default SHA-1 backend ====\n #\n # If no *_SHA1 backend is picked we'll fall back on using the default.\n@@ -525,6 +535,11 @@ include shared.mak\n #\n # === SHA-256 backend ===\n #\n+# ==== Security ====\n+#\n+# Unlike SHA-1 the SHA-256 algorithm does not suffer from any known\n+# vulnerabilities, so any implementation will do.\n+#\n # ==== Default SHA-256 backend ====\n #\n # If no *_SHA256 backend is picked we'll fall fall back on using the\n-- \n2.38.0.1251.g3eefdfb5e7a\n\n"},{"id":"465845","messageId":"xmqq7d0mfcan.fsf@gitster.g","threadId":"57779","inReplyTo":"patch-v4-7.9-847be3d32e2-20221026T145255Z-avarab@gmail.com","subject":"Re: [PATCH v4 7/9] Makefile: document SHA-1 and SHA-256 default and selection order","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2022-10-26T22:30:24Z","receivedAt":"2022-10-26T22:30:31Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Ævar Arnfjörð Bjarmason writes:\n\n> For the *_SHA1 and *_SHA256 flags we've discussed the various flags,\n> but not the fact that when you define multiple flags we'll pick one.\n>\n> Which one we pick depends on the order they're listed in the Makefile,\n> which differed from the order we discussed them in this documentation.\n>\n> Let's be explicit about how we select these, and re-arrange the\n> listings so that they're listed in the priority order we've picked.\n\nThat makes sense. What is not explicit enough, at least at this\nstage, is that, even though we say \"if you do not choose one, the\ndefault is used\" (which by the way is an awkward thing to see---that\nis the definition of being the default, so the statement itself has\nzero bit of useful information), we do not say what implementation\nexactly is used as the default.\n\n> This behavior makes it easier to e.g. integrate with autoconf-like\n> systems, where the configuration can provide everything it can\n> support, and Git is tasked with picking the first one it prefers.\n\nWell explained.\n\n> +# ==== Default SHA-1 backend ====\n> +#\n> +# If no *_SHA1 backend is picked we'll fall back on using the default.\n> +#\n> +# Multiple *_SHA1 backends can be selected, the first supported one\n> +# listed in \"Other SHA-1 implementations\" will be picked.\n"},{"id":"466741","messageId":"patch-v5-01.10-24d503e5a2b-20221107T211736Z-avarab@gmail.com","threadId":"57779","inReplyTo":"cover-v5-00.10-00000000000-20221107T211736Z-avarab@gmail.com","subject":"[PATCH v5 01/10] Makefile: always (re)set DC_SHA1 on fallback","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-11-07T21:23:03Z","receivedAt":"2022-11-07T21:23:27Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Fix an edge case introduced in in e6b07da2780 (Makefile: make DC_SHA1\nthe default, 2017-03-17), when DC_SHA1 was made the default fallback\nwe started unconditionally adding to BASIC_CFLAGS and LIB_OBJS, so\nwe'd use the sha1collisiondetection by default.\n\nBut the \"DC_SHA1\" variable remained unset, so e.g.:\n\n\tmake test DC_SHA1= T=t0013*.sh\n\nWould skip the sha1collisiondetection tests, as we'd write\n\"DC_SHA1=''\" to \"GIT-BUILD-OPTIONS\", but if we manually removed that\ntest prerequisite we'd pass the test (which we couldn't if we weren't\nusing sha1collisiondetection).\n\nSo let's have the fallback assignment use the 'override' directive\ninstead of the \":=\" simply expanded variable introduced in\ne6b07da2780. In this case we explicitly want to override the user's\nchoice.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n Makefile | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)\n\ndiff --git a/Makefile b/Makefile\nindex 4927379184c..0ad9a6c5bc1 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -1826,7 +1826,7 @@ ifdef APPLE_COMMON_CRYPTO\n \tCOMPAT_CFLAGS += -DCOMMON_DIGEST_FOR_OPENSSL\n \tBASIC_CFLAGS += -DSHA1_APPLE\n else\n-\tDC_SHA1 := YesPlease\n+\toverride DC_SHA1 = YesPlease\n \tBASIC_CFLAGS += -DSHA1_DC\n \tLIB_OBJS += sha1dc_git.o\n ifdef DC_SHA1_EXTERNAL\n-- \n2.38.0.1464.gea6794aacbc\n\n"},{"id":"466742","messageId":"cover-v5-00.10-00000000000-20221107T211736Z-avarab@gmail.com","threadId":"57779","inReplyTo":"cover-v4-0.9-00000000000-20221026T145255Z-avarab@gmail.com","subject":"[PATCH v5 00/10] Makefile, docs & code: document & fix SHA-{1,256} selection behavior","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-11-07T21:23:02Z","receivedAt":"2022-11-07T21:23:29Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"This is mostly a documentation update to get to the eventual goal of\nmaking DC_SHA1 the default on OSX. First we need to stop claiming that\nOpenSSL is still our default everywhere, etc.\n\nFor v4, see:\nhttps://lore.kernel.org/git/cover-v4-0.9-00000000000-20221026T145255Z-avarab@gmail.com/\n\nFor v4 Junio pointed out that it wasn't very clear what the \"default\"\nis. The root cause of that (which I was hoping to avoid, but bit the\nbullet here) is that DC_SHA1=Y on master doesn't actually do anything\nat all.\n\nInstead, whether we use DC_SHA1 or not depends on whether or not we\nasked for any *other* backend, it's just our default fallback.\n\nThe documentation now reflects this, and to reinforce that a new 8/10\nhere gets rid of the Makefile flag entirely.\n\nThe only thing it was actually used for was to tell the test suite\nthat it should be running the SHA1DC tests. Instead we just have a\ntrivial test helper that we can ask whether that's the case. It picks\nup a macro that we define in sha1dc_git.h.\n\nÆvar Arnfjörð Bjarmason (10):\n Makefile: always (re)set DC_SHA1 on fallback\n INSTALL: remove discussion of SHA-1 backends\n Makefile: correct DC_SHA1 documentation\n Makefile: create and use sections for \"define\" flag listing\n Makefile: rephrase the discussion of *_SHA1 knobs\n Makefile: document default SHA-256 backend\n Makefile: document SHA-1 and SHA-256 default and selection order\n Makefile & test-tool: replace \"DC_SHA1\" variable with a \"define\"\n Makefile: document default SHA-1 backend on OSX\n Makefile: discuss SHAttered in *_SHA{1,256} discussion\n\n INSTALL | 4 -\n Makefile | 252 +++++++++++++++++-----------\n ci/lib.sh | 2 +-\n contrib/buildsystems/CMakeLists.txt | 2 -\n sha1dc_git.h | 1 +\n t/helper/test-sha1.c | 8 +\n t/helper/test-tool.c | 1 +\n t/helper/test-tool.h | 1 +\n t/t0013-sha1dc.sh | 6 +-\n 9 files changed, 173 insertions(+), 104 deletions(-)\n\nRange-diff against v4:\n 1: 11e92d15616 = 1: 24d503e5a2b Makefile: always (re)set DC_SHA1 on fallback\n 2: abbe25f56b3 = 2: 2f6c37b90cd INSTALL: remove discussion of SHA-1 backends\n 3: b0bd35987c0 ! 3: d05fbf8c6d8 Makefile: correct DC_SHA1 documentation\n @@ Commit message\n Let's also remove the \"unconditionally enable\" wording, per the above\n the enabling of \"DC_SHA1\" is conditional on these other flags.\n \n + The \"Define DC_SHA1\" here is also a lie, actually it's \"we don't care\n + if you define DC_SHA1, just don't define anything else\", but that's a\n + more general issue that'll be addressed in a subsequent commit. Let's\n + first stop pretending that this setting (which we actually don't even\n + use) takes priority over anything else.\n +\n 1. 8325e43b82d (Makefile: add DC_SHA1 knob, 2017-03-16)\n 2. e6b07da2780 (Makefile: make DC_SHA1 the default, 2017-03-17)\n \n 4: d0451d6c3a3 ! 4: 9ab55314296 Makefile: create and use sections for \"define\" flag listing\n @@ Makefile: include shared.mak\n +# in one call to the platform's SHA1_Update(). e.g. APPLE_COMMON_CRYPTO\n +# wants 'SHA1_MAX_BLOCK_SIZE=1024L*1024L*1024L' defined.\n +#\n -+# ==== Options for the sha1collisiondetection implementation ====\n ++# ==== SHA-1 implementations ====\n +#\n +# Define DC_SHA1 to enable the collision-detecting sha1\n +# algorithm. This is slower, but may detect attempted collision attacks.\n +#\n ++# Define BLK_SHA1 environment variable to make use of the bundled\n ++# optimized C SHA1 routine.\n ++#\n ++# Define OPENSSL_SHA1 environment variable when running make to link\n ++# with the SHA1 routine from openssl library.\n ++#\n ++# ==== Options for the sha1collisiondetection library ====\n ++#\n +# Define DC_SHA1_EXTERNAL in addition to DC_SHA1 if you want to build / link\n +# git with the external SHA1 collision-detect library.\n +# Without this option, i.e. the default behavior is to build git with its\n @@ Makefile: include shared.mak\n +# by the git project to migrate to using sha1collisiondetection as a\n +# submodule.\n +#\n -+# ==== Other SHA-1 implementations ====\n -+#\n -+# Define BLK_SHA1 environment variable to make use of the bundled\n -+# optimized C SHA1 routine.\n -+#\n -+# Define OPENSSL_SHA1 environment variable when running make to link\n -+# with the SHA1 routine from openssl library.\n -+#\n +# === SHA-256 backend ===\n +#\n ++# ==== SHA-256 implementations ====\n ++#\n +# Define BLK_SHA256 to use the built-in SHA-256 routines.\n +#\n +# Define NETTLE_SHA256 to use the SHA-256 routines in libnettle.\n 5: b956d1c2640 ! 5: 5f1da5d7dc4 Makefile: rephrase the discussion of *_SHA1 knobs\n @@ Makefile: include shared.mak\n # Define DC_SHA1 to enable the collision-detecting sha1\n # algorithm. This is slower, but may detect attempted collision attacks.\n #\n --# Define DC_SHA1_EXTERNAL in addition to DC_SHA1 if you want to build / link\n -+# Define DC_SHA1_EXTERNAL if you want to build / link\n - # git with the external SHA1 collision-detect library.\n - # Without this option, i.e. the default behavior is to build git with its\n - # own built-in code (or submodule).\n - #\n --# Define DC_SHA1_SUBMODULE in addition to DC_SHA1 to use the\n -+# Define DC_SHA1_SUBMODULE to use the\n - # sha1collisiondetection shipped as a submodule instead of the\n - # non-submodule copy in sha1dc/. This is an experimental option used\n - # by the git project to migrate to using sha1collisiondetection as a\n -@@ Makefile: include shared.mak\n - #\n - # ==== Other SHA-1 implementations ====\n - #\n -# Define BLK_SHA1 environment variable to make use of the bundled\n -# optimized C SHA1 routine.\n +# Define BLK_SHA1 to make use of optimized C SHA-1 routines bundled\n @@ Makefile: include shared.mak\n +# Define OPENSSL_SHA1 to link to the SHA-1 routines from the OpenSSL\n +# library.\n #\n - # === SHA-256 backend ===\n + # ==== Options for the sha1collisiondetection library ====\n + #\n +-# Define DC_SHA1_EXTERNAL in addition to DC_SHA1 if you want to build / link\n ++# Define DC_SHA1_EXTERNAL if you want to build / link\n + # git with the external SHA1 collision-detect library.\n + # Without this option, i.e. the default behavior is to build git with its\n + # own built-in code (or submodule).\n #\n +-# Define DC_SHA1_SUBMODULE in addition to DC_SHA1 to use the\n ++# Define DC_SHA1_SUBMODULE to use the\n + # sha1collisiondetection shipped as a submodule instead of the\n + # non-submodule copy in sha1dc/. This is an experimental option used\n + # by the git project to migrate to using sha1collisiondetection as a\n 6: 1e4695d0ba0 < -: ----------- Makefile: document default SHA-256 backend\n -: ----------- > 6: 2cadbddcc04 Makefile: document default SHA-256 backend\n 7: 847be3d32e2 ! 7: 0d62359649f Makefile: document SHA-1 and SHA-256 default and selection order\n @@ Makefile: include shared.mak\n #\n +# ==== Default SHA-1 backend ====\n +#\n -+# If no *_SHA1 backend is picked we'll fall back on using the default.\n -+#\n -+# Multiple *_SHA1 backends can be selected, the first supported one\n -+# listed in \"Other SHA-1 implementations\" will be picked.\n ++# If no *_SHA1 backend is picked, the first supported one listed in\n ++# \"SHA-1 implementations\" will be picked.\n +#\n # ==== Options common to all SHA-1 implementations ====\n #\n # Define SHA1_MAX_BLOCK_SIZE to limit the amount of data that will be hashed\n @@ Makefile: include shared.mak\n #\n - # ==== Other SHA-1 implementations ====\n + # ==== SHA-1 implementations ====\n #\n --# Define BLK_SHA1 to make use of optimized C SHA-1 routines bundled\n --# with git (in the block-sha1/ directory).\n --#\n - # Define OPENSSL_SHA1 to link to the SHA-1 routines from the OpenSSL\n - # library.\n +-# Define DC_SHA1 to enable the collision-detecting sha1\n +-# algorithm. This is slower, but may detect attempted collision attacks.\n ++# Define OPENSSL_SHA1 to link to the SHA-1 routines from the OpenSSL\n ++# library.\n #\n -+# Define BLK_SHA1 to make use of optimized C SHA-1 routines bundled\n -+# with git (in the block-sha1/ directory).\n -+#\n - # === SHA-256 backend ===\n + # Define BLK_SHA1 to make use of optimized C SHA-1 routines bundled\n + # with git (in the block-sha1/ directory).\n + #\n +-# Define OPENSSL_SHA1 to link to the SHA-1 routines from the OpenSSL\n +-# library.\n ++# Define DC_SHA1 to enable the collision-detecting sha1\n ++# algorithm. This is slower, but may detect attempted collision attacks.\n + #\n + # ==== Options for the sha1collisiondetection library ====\n #\n - # ==== Default SHA-256 backend ====\n @@ Makefile: include shared.mak\n - # The default SHA-256 backend is shipped with Git. No flag is required\n - # to enable it. To select it don't define any other *_SHA256 flag.\n #\n -+# Multiple *_SHA256 backends can be selected, the first supported one\n -+# listed in \"Other SHA-256 implementations\" below will be picked.\n -+#\n - # ==== Other SHA-256 implementations ====\n + # ==== SHA-256 implementations ====\n #\n +# Define OPENSSL_SHA256 to use the SHA-256 routines in OpenSSL.\n +#\n @@ Makefile: include shared.mak\n #\n -# Define OPENSSL_SHA256 to use the SHA-256 routines in OpenSSL.\n -#\n - # == DEVELOPER defines ==\n + # If don't enable any of the *_SHA256 settings in this section, Git\n + # will default to its built-in sha256 implementation.\n #\n - # Define DEVELOPER to enable more compiler warnings. Compiler version\n -: ----------- > 8: 6daac578fd0 Makefile & test-tool: replace \"DC_SHA1\" variable with a \"define\"\n 8: 0af3ea78eaf ! 9: 8d3dddf88a4 Makefile: document default SHA-1 backend on OSX\n @@ Makefile: include shared.mak\n +# default on that OS. On macOS 01.4 (Tiger) or older,\n +# NO_APPLE_COMMON_CRYPTO is defined by default.\n +#\n - # === SHA-256 backend ===\n - #\n - # ==== Default SHA-256 backend ====\n + # If don't enable any of the *_SHA1 settings in this section, Git will\n + # default to its built-in sha1collisiondetection library, which is a\n + # collision-detecting sha1 This is slower, but may detect attempted\n 9: 9045ff9c4ed ! 10: 55d3751faea Makefile: discuss SHAttered in *_SHA{1,256} discussion\n @@ Makefile: include shared.mak\n +#\n +# Due to the SHAttered (https://shattered.io) attack vector on SHA-1\n +# it's strongly recommended to use the sha1collisiondetection\n -+# counter-cryptanalysis library for SHA-1 hashing (DC_SHA1).\n ++# counter-cryptanalysis library for SHA-1 hashing.\n +#\n +# If you know that you can trust the repository contents, or where\n -+# potential SHA-1 attacks are otherwise mitigated the backends listed\n -+# in \"Other SHA-1 implementations\" are faster than DC_SHA1.\n ++# potential SHA-1 attacks are otherwise mitigated the other backends\n ++# listed in \"SHA-1 implementations\" are faster than\n ++# sha1collisiondetection.\n +#\n # ==== Default SHA-1 backend ====\n #\n - # If no *_SHA1 backend is picked we'll fall back on using the default.\n + # If no *_SHA1 backend is picked, the first supported one listed in\n @@ Makefile: include shared.mak\n #\n # === SHA-256 backend ===\n @@ Makefile: include shared.mak\n +# Unlike SHA-1 the SHA-256 algorithm does not suffer from any known\n +# vulnerabilities, so any implementation will do.\n +#\n - # ==== Default SHA-256 backend ====\n + # ==== SHA-256 implementations ====\n #\n - # If no *_SHA256 backend is picked we'll fall fall back on using the\n + # Define OPENSSL_SHA256 to use the SHA-256 routines in OpenSSL.\n-- \n2.38.0.1464.gea6794aacbc\n\n"},{"id":"466743","messageId":"patch-v5-02.10-2f6c37b90cd-20221107T211736Z-avarab@gmail.com","threadId":"57779","inReplyTo":"cover-v5-00.10-00000000000-20221107T211736Z-avarab@gmail.com","subject":"[PATCH v5 02/10] INSTALL: remove discussion of SHA-1 backends","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-11-07T21:23:04Z","receivedAt":"2022-11-07T21:23:31Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"The claim that OpenSSL is the default SHA-1 backend hasn't been true\nsince e6b07da2780 (Makefile: make DC_SHA1 the default, 2017-03-17),\nbut more importantly tweaking the SHA-1 backend isn't something that's\ncommon enough to warrant discussing in the INSTALL document, so let's\nremove this paragraph.\n\nThis discussion was originally added in c538d2d34ab (Add some\ninstallation notes in INSTALL, 2005-06-17) when tweaking the default\nbackend was more common. The current wording was added in\n5beb577db8c (INSTALL: Describe dependency knobs from Makefile,\n2009-09-10).\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n INSTALL | 4 ----\n 1 file changed, 4 deletions(-)\n\ndiff --git a/INSTALL b/INSTALL\nindex 89b15d71df5..33447883974 100644\n--- a/INSTALL\n+++ b/INSTALL\n@@ -133,10 +133,6 @@ Issues of note:\n \t you are using libcurl older than 7.34.0. Otherwise you can use\n \t NO_OPENSSL without losing git-imap-send.\n \n-\t By default, git uses OpenSSL for SHA1 but it will use its own\n-\t library (inspired by Mozilla's) with either NO_OPENSSL or\n-\t BLK_SHA1.\n-\n \t- \"libcurl\" library is used for fetching and pushing\n \t repositories over http:// or https://, as well as by\n \t git-imap-send if the curl version is >= 7.34.0. If you do\n-- \n2.38.0.1464.gea6794aacbc\n\n"},{"id":"466744","messageId":"patch-v5-03.10-d05fbf8c6d8-20221107T211736Z-avarab@gmail.com","threadId":"57779","inReplyTo":"cover-v5-00.10-00000000000-20221107T211736Z-avarab@gmail.com","subject":"[PATCH v5 03/10] Makefile: correct DC_SHA1 documentation","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-11-07T21:23:05Z","receivedAt":"2022-11-07T21:23:33Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"The claim that DC_SHA1 takes priority over other *_SHA1 knobs was true\nwhen it was added in [1], But that hasn't been the case since it was\nmade the fallback default in [2].\n\nWe should be making it not only the default, but something that takes\npriority over other *_SHA1 knobs, but that's outside the scope of this\nchange. For now let's correct the documentation to match reality.\n\nLet's also remove the \"unconditionally enable\" wording, per the above\nthe enabling of \"DC_SHA1\" is conditional on these other flags.\n\nThe \"Define DC_SHA1\" here is also a lie, actually it's \"we don't care\nif you define DC_SHA1, just don't define anything else\", but that's a\nmore general issue that'll be addressed in a subsequent commit. Let's\nfirst stop pretending that this setting (which we actually don't even\nuse) takes priority over anything else.\n\n1. 8325e43b82d (Makefile: add DC_SHA1 knob, 2017-03-16)\n2. e6b07da2780 (Makefile: make DC_SHA1 the default, 2017-03-17)\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n Makefile | 3 +--\n 1 file changed, 1 insertion(+), 2 deletions(-)\n\ndiff --git a/Makefile b/Makefile\nindex 0ad9a6c5bc1..489327ecd9b 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -155,9 +155,8 @@ include shared.mak\n # Define BLK_SHA1 environment variable to make use of the bundled\n # optimized C SHA1 routine.\n #\n-# Define DC_SHA1 to unconditionally enable the collision-detecting sha1\n+# Define DC_SHA1 to enable the collision-detecting sha1\n # algorithm. This is slower, but may detect attempted collision attacks.\n-# Takes priority over other *_SHA1 knobs.\n #\n # Define DC_SHA1_EXTERNAL in addition to DC_SHA1 if you want to build / link\n # git with the external SHA1 collision-detect library.\n-- \n2.38.0.1464.gea6794aacbc\n\n"},{"id":"466745","messageId":"patch-v5-06.10-2cadbddcc04-20221107T211736Z-avarab@gmail.com","threadId":"57779","inReplyTo":"cover-v5-00.10-00000000000-20221107T211736Z-avarab@gmail.com","subject":"[PATCH v5 06/10] Makefile: document default SHA-256 backend","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-11-07T21:23:08Z","receivedAt":"2022-11-07T21:23:48Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Since 27dc04c5450 (sha256: add an SHA-256 implementation using\nlibgcrypt, 2018-11-14) we've claimed to support a BLK_SHA256 flag, but\nthere's no such SHA-256 backend.\n\nInstead we fall back on adding \"sha256/block/sha256.o\" to \"LIB_OBJS\"\nand adding \"-DSHA256_BLK\" to BASIC_CFLAGS.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n Makefile | 5 +++--\n 1 file changed, 3 insertions(+), 2 deletions(-)\n\ndiff --git a/Makefile b/Makefile\nindex 251368b315d..646fbe5b7dd 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -515,14 +515,15 @@ include shared.mak\n #\n # ==== SHA-256 implementations ====\n #\n-# Define BLK_SHA256 to use the built-in SHA-256 routines.\n-#\n # Define NETTLE_SHA256 to use the SHA-256 routines in libnettle.\n #\n # Define GCRYPT_SHA256 to use the SHA-256 routines in libgcrypt.\n #\n # Define OPENSSL_SHA256 to use the SHA-256 routines in OpenSSL.\n #\n+# If don't enable any of the *_SHA256 settings in this section, Git\n+# will default to its built-in sha256 implementation.\n+#\n # == DEVELOPER defines ==\n #\n # Define DEVELOPER to enable more compiler warnings. Compiler version\n-- \n2.38.0.1464.gea6794aacbc\n\n"},{"id":"466746","messageId":"patch-v5-05.10-5f1da5d7dc4-20221107T211736Z-avarab@gmail.com","threadId":"57779","inReplyTo":"cover-v5-00.10-00000000000-20221107T211736Z-avarab@gmail.com","subject":"[PATCH v5 05/10] Makefile: rephrase the discussion of *_SHA1 knobs","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-11-07T21:23:07Z","receivedAt":"2022-11-07T21:23:50Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"In the preceding commit the discussion of the *_SHA1 knobs was left\nas-is to benefit from a smaller diff, but since we're changing these\nlet's use the same phrasing we use for most other knobs. E.g. \"define\nX\", not \"define X environment variable\", and get rid of the \"when\nrunning make to link with\" entirely.\n\nFurthermore the discussion of DC_SHA1* options is now under a \"Options\nfor the sha1collisiondetection implementation\" heading, so we don't\nneed to clarify that these options go along with DC_SHA1=Y, so let's\nrephrase them accordingly.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n Makefile | 12 ++++++------\n 1 file changed, 6 insertions(+), 6 deletions(-)\n\ndiff --git a/Makefile b/Makefile\nindex 19235624684..251368b315d 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -492,20 +492,20 @@ include shared.mak\n # Define DC_SHA1 to enable the collision-detecting sha1\n # algorithm. This is slower, but may detect attempted collision attacks.\n #\n-# Define BLK_SHA1 environment variable to make use of the bundled\n-# optimized C SHA1 routine.\n+# Define BLK_SHA1 to make use of optimized C SHA-1 routines bundled\n+# with git (in the block-sha1/ directory).\n #\n-# Define OPENSSL_SHA1 environment variable when running make to link\n-# with the SHA1 routine from openssl library.\n+# Define OPENSSL_SHA1 to link to the SHA-1 routines from the OpenSSL\n+# library.\n #\n # ==== Options for the sha1collisiondetection library ====\n #\n-# Define DC_SHA1_EXTERNAL in addition to DC_SHA1 if you want to build / link\n+# Define DC_SHA1_EXTERNAL if you want to build / link\n # git with the external SHA1 collision-detect library.\n # Without this option, i.e. the default behavior is to build git with its\n # own built-in code (or submodule).\n #\n-# Define DC_SHA1_SUBMODULE in addition to DC_SHA1 to use the\n+# Define DC_SHA1_SUBMODULE to use the\n # sha1collisiondetection shipped as a submodule instead of the\n # non-submodule copy in sha1dc/. This is an experimental option used\n # by the git project to migrate to using sha1collisiondetection as a\n-- \n2.38.0.1464.gea6794aacbc\n\n"},{"id":"466747","messageId":"patch-v5-07.10-0d62359649f-20221107T211736Z-avarab@gmail.com","threadId":"57779","inReplyTo":"cover-v5-00.10-00000000000-20221107T211736Z-avarab@gmail.com","subject":"[PATCH v5 07/10] Makefile: document SHA-1 and SHA-256 default and selection order","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-11-07T21:23:09Z","receivedAt":"2022-11-07T21:23:53Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"For the *_SHA1 and *_SHA256 flags we've discussed the various flags,\nbut not the fact that when you define multiple flags we'll pick one.\n\nWhich one we pick depends on the order they're listed in the Makefile,\nwhich differed from the order we discussed them in this documentation.\n\nLet's be explicit about how we select these, and re-arrange the\nlistings so that they're listed in the priority order we've picked.\n\nI'd personally prefer that the selection was more explicit, and that\nwe'd error out if conflicting flags were provided, but per the\ndiscussion downhtread of[1] the consensus was to keep theses semantics.\n\nThis behavior makes it easier to e.g. integrate with autoconf-like\nsystems, where the configuration can provide everything it can\nsupport, and Git is tasked with picking the first one it prefers.\n\n1. https://lore.kernel.org/git/220710.86mtdh81ty.gmgdl@evledraar.gmail.com/\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n Makefile | 17 +++++++++++------\n 1 file changed, 11 insertions(+), 6 deletions(-)\n\ndiff --git a/Makefile b/Makefile\nindex 646fbe5b7dd..9b5f872d107 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -481,6 +481,11 @@ include shared.mak\n #\n # === SHA-1 backend ===\n #\n+# ==== Default SHA-1 backend ====\n+#\n+# If no *_SHA1 backend is picked, the first supported one listed in\n+# \"SHA-1 implementations\" will be picked.\n+#\n # ==== Options common to all SHA-1 implementations ====\n #\n # Define SHA1_MAX_BLOCK_SIZE to limit the amount of data that will be hashed\n@@ -489,14 +494,14 @@ include shared.mak\n #\n # ==== SHA-1 implementations ====\n #\n-# Define DC_SHA1 to enable the collision-detecting sha1\n-# algorithm. This is slower, but may detect attempted collision attacks.\n+# Define OPENSSL_SHA1 to link to the SHA-1 routines from the OpenSSL\n+# library.\n #\n # Define BLK_SHA1 to make use of optimized C SHA-1 routines bundled\n # with git (in the block-sha1/ directory).\n #\n-# Define OPENSSL_SHA1 to link to the SHA-1 routines from the OpenSSL\n-# library.\n+# Define DC_SHA1 to enable the collision-detecting sha1\n+# algorithm. This is slower, but may detect attempted collision attacks.\n #\n # ==== Options for the sha1collisiondetection library ====\n #\n@@ -515,12 +520,12 @@ include shared.mak\n #\n # ==== SHA-256 implementations ====\n #\n+# Define OPENSSL_SHA256 to use the SHA-256 routines in OpenSSL.\n+#\n # Define NETTLE_SHA256 to use the SHA-256 routines in libnettle.\n #\n # Define GCRYPT_SHA256 to use the SHA-256 routines in libgcrypt.\n #\n-# Define OPENSSL_SHA256 to use the SHA-256 routines in OpenSSL.\n-#\n # If don't enable any of the *_SHA256 settings in this section, Git\n # will default to its built-in sha256 implementation.\n #\n-- \n2.38.0.1464.gea6794aacbc\n\n"},{"id":"466748","messageId":"patch-v5-04.10-9ab55314296-20221107T211736Z-avarab@gmail.com","threadId":"57779","inReplyTo":"cover-v5-00.10-00000000000-20221107T211736Z-avarab@gmail.com","subject":"[PATCH v5 04/10] Makefile: create and use sections for \"define\" flag listing","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-11-07T21:23:06Z","receivedAt":"2022-11-07T21:23:55Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Since the \"Define ...\" template of comments at the top of the Makefile\nwas started in 5bdac8b3269 ([PATCH] Improve the compilation-time\nsettings interface, 2005-07-29) we've had a lot more flags added,\nincluding flags that come in \"groups\". Not having any obvious\nstructure to the >500 line comment at the top of the Makefile has made\nit hard to follow.\n\nThis change is almost entirely a move-only change, the two paragraphs\nat the start of the first two sections are new, and so are the added\nsections themselves, but other than that no lines are changed, only\nmoved.\n\nWe now list Makefile-only flags at the start, followed by stand-alone\nflags, and then cover \"optional library\" flags in their respective\ngroups, followed by SHA-1 and SHA-256 flags, and finally\nDEVELOPER-specific flags.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n Makefile | 220 ++++++++++++++++++++++++++++++++-----------------------\n 1 file changed, 128 insertions(+), 92 deletions(-)\n\ndiff --git a/Makefile b/Makefile\nindex 489327ecd9b..19235624684 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -4,8 +4,20 @@ all::\n # Import tree-wide shared Makefile behavior and libraries\n include shared.mak\n \n+# == Makefile defines ==\n+#\n+# These defines change the behavior of the Makefile itself, but have\n+# no impact on what it builds:\n+#\n # Define V=1 to have a more verbose compile.\n #\n+# == Portability and optional library defines ==\n+#\n+# These defines indicate what Git can expect from the OS, what\n+# libraries are available etc. Much of this is auto-detected in\n+# config.mak.uname, or in configure.ac when using the optional \"make\n+# configure && ./configure\" (see INSTALL).\n+#\n # Define SHELL_PATH to a POSIX shell if your /bin/sh is broken.\n #\n # Define SANE_TOOL_PATH to a colon-separated list of paths to prepend\n@@ -30,68 +42,8 @@ include shared.mak\n #\n # Define NO_OPENSSL environment variable if you do not have OpenSSL.\n #\n-# Define USE_LIBPCRE if you have and want to use libpcre. Various\n-# commands such as log and grep offer runtime options to use\n-# Perl-compatible regular expressions instead of standard or extended\n-# POSIX regular expressions.\n-#\n-# Only libpcre version 2 is supported. USE_LIBPCRE2 is a synonym for\n-# USE_LIBPCRE, support for the old USE_LIBPCRE1 has been removed.\n-#\n-# Define LIBPCREDIR=/foo/bar if your PCRE header and library files are\n-# in /foo/bar/include and /foo/bar/lib directories.\n-#\n # Define HAVE_ALLOCA_H if you have working alloca(3) defined in that header.\n #\n-# Define NO_CURL if you do not have libcurl installed. git-http-fetch and\n-# git-http-push are not built, and you cannot use http:// and https://\n-# transports (neither smart nor dumb).\n-#\n-# Define CURLDIR=/foo/bar if your curl header and library files are in\n-# /foo/bar/include and /foo/bar/lib directories.\n-#\n-# Define CURL_CONFIG to curl's configuration program that prints information\n-# about the library (e.g., its version number). The default is 'curl-config'.\n-#\n-# Define CURL_LDFLAGS to specify flags that you need to link when using libcurl,\n-# if you do not want to rely on the libraries provided by CURL_CONFIG. The\n-# default value is a result of `curl-config --libs`. An example value for\n-# CURL_LDFLAGS is as follows:\n-#\n-# CURL_LDFLAGS=-lcurl\n-#\n-# Define NO_EXPAT if you do not have expat installed. git-http-push is\n-# not built, and you cannot push using http:// and https:// transports (dumb).\n-#\n-# Define EXPATDIR=/foo/bar if your expat header and library files are in\n-# /foo/bar/include and /foo/bar/lib directories.\n-#\n-# Define EXPAT_NEEDS_XMLPARSE_H if you have an old version of expat (e.g.,\n-# 1.1 or 1.2) that provides xmlparse.h instead of expat.h.\n-#\n-# Define NO_GETTEXT if you don't want Git output to be translated.\n-# A translated Git requires GNU libintl or another gettext implementation,\n-# plus libintl-perl at runtime.\n-#\n-# Define USE_GETTEXT_SCHEME and set it to 'fallthrough', if you don't trust\n-# the installed gettext translation of the shell scripts output.\n-#\n-# Define HAVE_LIBCHARSET_H if you haven't set NO_GETTEXT and you can't\n-# trust the langinfo.h's nl_langinfo(CODESET) function to return the\n-# current character set. GNU and Solaris have a nl_langinfo(CODESET),\n-# FreeBSD can use either, but MinGW and some others need to use\n-# libcharset.h's locale_charset() instead.\n-#\n-# Define CHARSET_LIB to the library you need to link with in order to\n-# use locale_charset() function. On some platforms this needs to set to\n-# -lcharset, on others to -liconv .\n-#\n-# Define LIBC_CONTAINS_LIBINTL if your gettext implementation doesn't\n-# need -lintl when linking.\n-#\n-# Define NO_MSGFMT_EXTENDED_OPTIONS if your implementation of msgfmt\n-# doesn't support GNU extensions like --check and --statistics\n-#\n # Define HAVE_PATHS_H if you have paths.h and want to use the default PATH\n # it specifies.\n #\n@@ -152,38 +104,6 @@ include shared.mak\n # and do not want to use Apple's CommonCrypto library. This allows you\n # to provide your own OpenSSL library, for example from MacPorts.\n #\n-# Define BLK_SHA1 environment variable to make use of the bundled\n-# optimized C SHA1 routine.\n-#\n-# Define DC_SHA1 to enable the collision-detecting sha1\n-# algorithm. This is slower, but may detect attempted collision attacks.\n-#\n-# Define DC_SHA1_EXTERNAL in addition to DC_SHA1 if you want to build / link\n-# git with the external SHA1 collision-detect library.\n-# Without this option, i.e. the default behavior is to build git with its\n-# own built-in code (or submodule).\n-#\n-# Define DC_SHA1_SUBMODULE in addition to DC_SHA1 to use the\n-# sha1collisiondetection shipped as a submodule instead of the\n-# non-submodule copy in sha1dc/. This is an experimental option used\n-# by the git project to migrate to using sha1collisiondetection as a\n-# submodule.\n-#\n-# Define OPENSSL_SHA1 environment variable when running make to link\n-# with the SHA1 routine from openssl library.\n-#\n-# Define SHA1_MAX_BLOCK_SIZE to limit the amount of data that will be hashed\n-# in one call to the platform's SHA1_Update(). e.g. APPLE_COMMON_CRYPTO\n-# wants 'SHA1_MAX_BLOCK_SIZE=1024L*1024L*1024L' defined.\n-#\n-# Define BLK_SHA256 to use the built-in SHA-256 routines.\n-#\n-# Define NETTLE_SHA256 to use the SHA-256 routines in libnettle.\n-#\n-# Define GCRYPT_SHA256 to use the SHA-256 routines in libgcrypt.\n-#\n-# Define OPENSSL_SHA256 to use the SHA-256 routines in OpenSSL.\n-#\n # Define NEEDS_CRYPTO_WITH_SSL if you need -lcrypto when using -lssl (Darwin).\n #\n # Define NEEDS_SSL_WITH_CRYPTO if you need -lssl when using -lcrypto (Darwin).\n@@ -489,6 +409,122 @@ include shared.mak\n # to the \"\" of the corresponding `compat/fsmonitor/fsm-settings-.c`\n # that implements the `fsm_os_settings__*()` routines.\n #\n+# === Optional library: libintl ===\n+#\n+# Define NO_GETTEXT if you don't want Git output to be translated.\n+# A translated Git requires GNU libintl or another gettext implementation,\n+# plus libintl-perl at runtime.\n+#\n+# Define USE_GETTEXT_SCHEME and set it to 'fallthrough', if you don't trust\n+# the installed gettext translation of the shell scripts output.\n+#\n+# Define HAVE_LIBCHARSET_H if you haven't set NO_GETTEXT and you can't\n+# trust the langinfo.h's nl_langinfo(CODESET) function to return the\n+# current character set. GNU and Solaris have a nl_langinfo(CODESET),\n+# FreeBSD can use either, but MinGW and some others need to use\n+# libcharset.h's locale_charset() instead.\n+#\n+# Define CHARSET_LIB to the library you need to link with in order to\n+# use locale_charset() function. On some platforms this needs to set to\n+# -lcharset, on others to -liconv .\n+#\n+# Define LIBC_CONTAINS_LIBINTL if your gettext implementation doesn't\n+# need -lintl when linking.\n+#\n+# Define NO_MSGFMT_EXTENDED_OPTIONS if your implementation of msgfmt\n+# doesn't support GNU extensions like --check and --statistics\n+#\n+# === Optional library: libexpat ===\n+#\n+# Define NO_EXPAT if you do not have expat installed. git-http-push is\n+# not built, and you cannot push using http:// and https:// transports (dumb).\n+#\n+# Define EXPATDIR=/foo/bar if your expat header and library files are in\n+# /foo/bar/include and /foo/bar/lib directories.\n+#\n+# Define EXPAT_NEEDS_XMLPARSE_H if you have an old version of expat (e.g.,\n+# 1.1 or 1.2) that provides xmlparse.h instead of expat.h.\n+\n+# === Optional library: libcurl ===\n+#\n+# Define NO_CURL if you do not have libcurl installed. git-http-fetch and\n+# git-http-push are not built, and you cannot use http:// and https://\n+# transports (neither smart nor dumb).\n+#\n+# Define CURLDIR=/foo/bar if your curl header and library files are in\n+# /foo/bar/include and /foo/bar/lib directories.\n+#\n+# Define CURL_CONFIG to curl's configuration program that prints information\n+# about the library (e.g., its version number). The default is 'curl-config'.\n+#\n+# Define CURL_LDFLAGS to specify flags that you need to link when using libcurl,\n+# if you do not want to rely on the libraries provided by CURL_CONFIG. The\n+# default value is a result of `curl-config --libs`. An example value for\n+# CURL_LDFLAGS is as follows:\n+#\n+# CURL_LDFLAGS=-lcurl\n+#\n+# === Optional library: libpcre2 ===\n+#\n+# Define USE_LIBPCRE if you have and want to use libpcre. Various\n+# commands such as log and grep offer runtime options to use\n+# Perl-compatible regular expressions instead of standard or extended\n+# POSIX regular expressions.\n+#\n+# Only libpcre version 2 is supported. USE_LIBPCRE2 is a synonym for\n+# USE_LIBPCRE, support for the old USE_LIBPCRE1 has been removed.\n+#\n+# Define LIBPCREDIR=/foo/bar if your PCRE header and library files are\n+# in /foo/bar/include and /foo/bar/lib directories.\n+#\n+# == SHA-1 and SHA-256 defines ==\n+#\n+# === SHA-1 backend ===\n+#\n+# ==== Options common to all SHA-1 implementations ====\n+#\n+# Define SHA1_MAX_BLOCK_SIZE to limit the amount of data that will be hashed\n+# in one call to the platform's SHA1_Update(). e.g. APPLE_COMMON_CRYPTO\n+# wants 'SHA1_MAX_BLOCK_SIZE=1024L*1024L*1024L' defined.\n+#\n+# ==== SHA-1 implementations ====\n+#\n+# Define DC_SHA1 to enable the collision-detecting sha1\n+# algorithm. This is slower, but may detect attempted collision attacks.\n+#\n+# Define BLK_SHA1 environment variable to make use of the bundled\n+# optimized C SHA1 routine.\n+#\n+# Define OPENSSL_SHA1 environment variable when running make to link\n+# with the SHA1 routine from openssl library.\n+#\n+# ==== Options for the sha1collisiondetection library ====\n+#\n+# Define DC_SHA1_EXTERNAL in addition to DC_SHA1 if you want to build / link\n+# git with the external SHA1 collision-detect library.\n+# Without this option, i.e. the default behavior is to build git with its\n+# own built-in code (or submodule).\n+#\n+# Define DC_SHA1_SUBMODULE in addition to DC_SHA1 to use the\n+# sha1collisiondetection shipped as a submodule instead of the\n+# non-submodule copy in sha1dc/. This is an experimental option used\n+# by the git project to migrate to using sha1collisiondetection as a\n+# submodule.\n+#\n+# === SHA-256 backend ===\n+#\n+# ==== SHA-256 implementations ====\n+#\n+# Define BLK_SHA256 to use the built-in SHA-256 routines.\n+#\n+# Define NETTLE_SHA256 to use the SHA-256 routines in libnettle.\n+#\n+# Define GCRYPT_SHA256 to use the SHA-256 routines in libgcrypt.\n+#\n+# Define OPENSSL_SHA256 to use the SHA-256 routines in OpenSSL.\n+#\n+# == DEVELOPER defines ==\n+#\n # Define DEVELOPER to enable more compiler warnings. Compiler version\n # and family are auto detected, but could be overridden by defining\n # COMPILER_FEATURES (see config.mak.dev). You can still set\n-- \n2.38.0.1464.gea6794aacbc\n\n"},{"id":"466749","messageId":"patch-v5-08.10-6daac578fd0-20221107T211736Z-avarab@gmail.com","threadId":"57779","inReplyTo":"cover-v5-00.10-00000000000-20221107T211736Z-avarab@gmail.com","subject":"[PATCH v5 08/10] Makefile & test-tool: replace \"DC_SHA1\" variable with a \"define\"","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-11-07T21:23:10Z","receivedAt":"2022-11-07T21:23:59Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Address the root cause of technical debt we've been carrying since\nsha1collisiondetection was made the default in [1]. In a preceding\ncommit we narrowly fixed a bug where the \"DC_SHA1\" variable would be\nunset (in combination with \"NO_APPLE_COMMON_CRYPTO=\" on OSX), even\nthough we had the sha1collisiondetection library enabled.\n\nBut the only reason we needed to have such a user-exposed knob went\naway with [1], and it's been doing nothing useful since then. We don't\ncare if you define DC_SHA1=*, we only care that you don't ask for any\nother SHA-1 implementation. If it turns out that you didn't, we'll use\nsha1collisiondetection, whether you had \"DC_SHA1\" set or not.\n\nAs a result of this being confusing we had e.g. [2] for cmake and the\nrecent [3] for ci/lib.sh setting \"DC_SHA1\" explicitly, even though\nthis was always a NOOP.\n\nA much simpler way to do this is to stop having the Makefile and\nCMakeLists.txt set \"DC_SHA1\" to be picked up by the test-lib.sh, let's\ninstead add a trivial \"test-tool sha1-is-sha1dc\". It returns zero if\nwe're using sha1collisiondetection, non-zero otherwise.\n\n1. e6b07da2780 (Makefile: make DC_SHA1 the default, 2017-03-17)\n2. c4b2f41b5f5 (cmake: support for testing git with ctest, 2020-06-26)\n3. 1ad5c3df35a (ci: use DC_SHA1=YesPlease on osx-clang job for CI,\n 2022-10-20)\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n Makefile | 8 ++++----\n ci/lib.sh | 2 +-\n contrib/buildsystems/CMakeLists.txt | 2 --\n sha1dc_git.h | 1 +\n t/helper/test-sha1.c | 8 ++++++++\n t/helper/test-tool.c | 1 +\n t/helper/test-tool.h | 1 +\n t/t0013-sha1dc.sh | 6 ++++--\n 8 files changed, 20 insertions(+), 9 deletions(-)\n\ndiff --git a/Makefile b/Makefile\nindex 9b5f872d107..0c79546712f 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -500,8 +500,10 @@ include shared.mak\n # Define BLK_SHA1 to make use of optimized C SHA-1 routines bundled\n # with git (in the block-sha1/ directory).\n #\n-# Define DC_SHA1 to enable the collision-detecting sha1\n-# algorithm. This is slower, but may detect attempted collision attacks.\n+# If don't enable any of the *_SHA1 settings in this section, Git will\n+# default to its built-in sha1collisiondetection library, which is a\n+# collision-detecting sha1 This is slower, but may detect attempted\n+# collision attacks.\n #\n # ==== Options for the sha1collisiondetection library ====\n #\n@@ -1867,7 +1869,6 @@ ifdef APPLE_COMMON_CRYPTO\n \tCOMPAT_CFLAGS += -DCOMMON_DIGEST_FOR_OPENSSL\n \tBASIC_CFLAGS += -DSHA1_APPLE\n else\n-\toverride DC_SHA1 = YesPlease\n \tBASIC_CFLAGS += -DSHA1_DC\n \tLIB_OBJS += sha1dc_git.o\n ifdef DC_SHA1_EXTERNAL\n@@ -3030,7 +3031,6 @@ GIT-BUILD-OPTIONS: FORCE\n \t@echo NO_REGEX=\\''$(subst ','\\'',$(subst ','\\'',$(NO_REGEX)))'\\' >>$@+\n \t@echo NO_UNIX_SOCKETS=\\''$(subst ','\\'',$(subst ','\\'',$(NO_UNIX_SOCKETS)))'\\' >>$@+\n \t@echo PAGER_ENV=\\''$(subst ','\\'',$(subst ','\\'',$(PAGER_ENV)))'\\' >>$@+\n-\t@echo DC_SHA1=\\''$(subst ','\\'',$(subst ','\\'',$(DC_SHA1)))'\\' >>$@+\n \t@echo SANITIZE_LEAK=\\''$(subst ','\\'',$(subst ','\\'',$(SANITIZE_LEAK)))'\\' >>$@+\n \t@echo SANITIZE_ADDRESS=\\''$(subst ','\\'',$(subst ','\\'',$(SANITIZE_ADDRESS)))'\\' >>$@+\n \t@echo X=\\'$(X)\\' >>$@+\ndiff --git a/ci/lib.sh b/ci/lib.sh\nindex 1808e3b1ce1..24d20a5d648 100755\n--- a/ci/lib.sh\n+++ b/ci/lib.sh\n@@ -260,7 +260,7 @@ macos-latest)\n \telse\n \t\tMAKEFLAGS=\"$MAKEFLAGS PYTHON_PATH=$(which python2)\"\n \t\tMAKEFLAGS=\"$MAKEFLAGS NO_APPLE_COMMON_CRYPTO=NoThanks\"\n-\t\tMAKEFLAGS=\"$MAKEFLAGS DC_SHA1=YesPlease NO_OPENSSL=NoThanks\"\n+\t\tMAKEFLAGS=\"$MAKEFLAGS NO_OPENSSL=NoThanks\"\n \tfi\n \t;;\n esac\ndiff --git a/contrib/buildsystems/CMakeLists.txt b/contrib/buildsystems/CMakeLists.txt\nindex 3957e4cf8cd..2f6e0197ffa 100644\n--- a/contrib/buildsystems/CMakeLists.txt\n+++ b/contrib/buildsystems/CMakeLists.txt\n@@ -1025,7 +1025,6 @@ set(NO_PERL )\n set(NO_PTHREADS )\n set(NO_PYTHON )\n set(PAGER_ENV \"LESS=FRX LV=-c\")\n-set(DC_SHA1 YesPlease)\n set(RUNTIME_PREFIX true)\n set(NO_GETTEXT )\n \n@@ -1061,7 +1060,6 @@ file(APPEND ${CMAKE_BINARY_DIR}/GIT-BUILD-OPTIONS \"NO_PERL='${NO_PERL}'\\n\")\n file(APPEND ${CMAKE_BINARY_DIR}/GIT-BUILD-OPTIONS \"NO_PTHREADS='${NO_PTHREADS}'\\n\")\n file(APPEND ${CMAKE_BINARY_DIR}/GIT-BUILD-OPTIONS \"NO_UNIX_SOCKETS='${NO_UNIX_SOCKETS}'\\n\")\n file(APPEND ${CMAKE_BINARY_DIR}/GIT-BUILD-OPTIONS \"PAGER_ENV='${PAGER_ENV}'\\n\")\n-file(APPEND ${CMAKE_BINARY_DIR}/GIT-BUILD-OPTIONS \"DC_SHA1='${DC_SHA1}'\\n\")\n file(APPEND ${CMAKE_BINARY_DIR}/GIT-BUILD-OPTIONS \"X='${EXE_EXTENSION}'\\n\")\n file(APPEND ${CMAKE_BINARY_DIR}/GIT-BUILD-OPTIONS \"NO_GETTEXT='${NO_GETTEXT}'\\n\")\n file(APPEND ${CMAKE_BINARY_DIR}/GIT-BUILD-OPTIONS \"RUNTIME_PREFIX='${RUNTIME_PREFIX}'\\n\")\ndiff --git a/sha1dc_git.h b/sha1dc_git.h\nindex 41e1c3fd3f7..60e3ce84395 100644\n--- a/sha1dc_git.h\n+++ b/sha1dc_git.h\n@@ -17,6 +17,7 @@ void git_SHA1DCInit(SHA1_CTX *);\n void git_SHA1DCFinal(unsigned char [20], SHA1_CTX *);\n void git_SHA1DCUpdate(SHA1_CTX *ctx, const void *data, unsigned long len);\n \n+#define platform_SHA_IS_SHA1DC /* used by \"test-tool sha1-is-sha1dc\" */\n #define platform_SHA_CTX SHA1_CTX\n #define platform_SHA1_Init git_SHA1DCInit\n #define platform_SHA1_Update git_SHA1DCUpdate\ndiff --git a/t/helper/test-sha1.c b/t/helper/test-sha1.c\nindex d860c387c38..71fe5c61455 100644\n--- a/t/helper/test-sha1.c\n+++ b/t/helper/test-sha1.c\n@@ -5,3 +5,11 @@ int cmd__sha1(int ac, const char **av)\n {\n \treturn cmd_hash_impl(ac, av, GIT_HASH_SHA1);\n }\n+\n+int cmd__sha1_is_sha1dc(int argc UNUSED, const char **argv UNUSED)\n+{\n+#ifdef platform_SHA_IS_SHA1DC\n+\treturn 0;\n+#endif\n+\treturn 1;\n+}\ndiff --git a/t/helper/test-tool.c b/t/helper/test-tool.c\nindex 01cda9358df..775854c9f96 100644\n--- a/t/helper/test-tool.c\n+++ b/t/helper/test-tool.c\n@@ -73,6 +73,7 @@ static struct test_cmd cmds[] = {\n \t{ \"scrap-cache-tree\", cmd__scrap_cache_tree },\n \t{ \"serve-v2\", cmd__serve_v2 },\n \t{ \"sha1\", cmd__sha1 },\n+\t{ \"sha1-is-sha1dc\", cmd__sha1_is_sha1dc },\n \t{ \"sha256\", cmd__sha256 },\n \t{ \"sigchain\", cmd__sigchain },\n \t{ \"simple-ipc\", cmd__simple_ipc },\ndiff --git a/t/helper/test-tool.h b/t/helper/test-tool.h\nindex ca2948066fd..7cf84eca12e 100644\n--- a/t/helper/test-tool.h\n+++ b/t/helper/test-tool.h\n@@ -66,6 +66,7 @@ int cmd__run_command(int argc, const char **argv);\n int cmd__scrap_cache_tree(int argc, const char **argv);\n int cmd__serve_v2(int argc, const char **argv);\n int cmd__sha1(int argc, const char **argv);\n+int cmd__sha1_is_sha1dc(int argc, const char **argv);\n int cmd__oid_array(int argc, const char **argv);\n int cmd__sha256(int argc, const char **argv);\n int cmd__sigchain(int argc, const char **argv);\ndiff --git a/t/t0013-sha1dc.sh b/t/t0013-sha1dc.sh\nindex 9ad76080aa4..53240476896 100755\n--- a/t/t0013-sha1dc.sh\n+++ b/t/t0013-sha1dc.sh\n@@ -6,9 +6,11 @@ TEST_PASSES_SANITIZE_LEAK=true\n . ./test-lib.sh\n TEST_DATA=\"$TEST_DIRECTORY/t0013\"\n \n-if test -z \"$DC_SHA1\"\n+test_lazy_prereq SHA1_IS_SHA1DC 'test-tool sha1-is-sha1dc'\n+\n+if ! test_have_prereq SHA1_IS_SHA1DC\n then\n-\tskip_all='skipping sha1 collision tests, DC_SHA1 not set'\n+\tskip_all='skipping sha1 collision tests, not using sha1collisiondetection'\n \ttest_done\n fi\n \n-- \n2.38.0.1464.gea6794aacbc\n\n"},{"id":"466750","messageId":"patch-v5-09.10-8d3dddf88a4-20221107T211736Z-avarab@gmail.com","threadId":"57779","inReplyTo":"cover-v5-00.10-00000000000-20221107T211736Z-avarab@gmail.com","subject":"[PATCH v5 09/10] Makefile: document default SHA-1 backend on OSX","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-11-07T21:23:11Z","receivedAt":"2022-11-07T21:24:01Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Since [1] the default SHA-1 backend on OSX has been\nAPPLE_COMMON_CRYPTO. Per [2] we'll skip using it on anything older\nthan Mac OS X 10.4 \"Tiger\"[3].\n\nWhen \"DC_SHA1\" was made the default in [4] this interaction between it\nand APPLE_COMMON_CRYPTO seems to have been missed in. Ever since\nDC_SHA1 was \"made the default\" we've still used Apple's CommonCrypto\ninstead of sha1collisiondetection on modern versions of Darwin and\nOSX.\n\n1. 61067954ce1 (cache.h: eliminate SHA-1 deprecation warnings on Mac\n OS X, 2013-05-19)\n2. 9c7a0beee09 (config.mak.uname: set NO_APPLE_COMMON_CRYPTO on older\n systems, 2014-08-15)\n3. We could probably drop \"NO_APPLE_COMMON_CRYPTO\", as nobody's likely\n to care about such on old version of OSX anymore. But let's leave that\n for now.\n4. e6b07da2780 (Makefile: make DC_SHA1 the default, 2017-03-17)\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n Makefile | 5 +++++\n 1 file changed, 5 insertions(+)\n\ndiff --git a/Makefile b/Makefile\nindex 0c79546712f..7d0fa7adb61 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -500,6 +500,11 @@ include shared.mak\n # Define BLK_SHA1 to make use of optimized C SHA-1 routines bundled\n # with git (in the block-sha1/ directory).\n #\n+# Define NO_APPLE_COMMON_CRYPTO on OSX to opt-out of using the\n+# \"APPLE_COMMON_CRYPTO\" backend for SHA-1, which is currently the\n+# default on that OS. On macOS 01.4 (Tiger) or older,\n+# NO_APPLE_COMMON_CRYPTO is defined by default.\n+#\n # If don't enable any of the *_SHA1 settings in this section, Git will\n # default to its built-in sha1collisiondetection library, which is a\n # collision-detecting sha1 This is slower, but may detect attempted\n-- \n2.38.0.1464.gea6794aacbc\n\n"},{"id":"466751","messageId":"patch-v5-10.10-55d3751faea-20221107T211736Z-avarab@gmail.com","threadId":"57779","inReplyTo":"cover-v5-00.10-00000000000-20221107T211736Z-avarab@gmail.com","subject":"[PATCH v5 10/10] Makefile: discuss SHAttered in *_SHA{1,256} discussion","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2022-11-07T21:23:12Z","receivedAt":"2022-11-07T21:24:02Z","isPatch":true,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"Let's mention the SHAttered attack and more generally why we use the\nsha1collisiondetection backend by default, and note that for SHA-256\nthe user should feel free to pick any of the supported backends as far\nas hashing security is concerned.\n\nSigned-off-by: Ævar Arnfjörð Bjarmason \n---\n Makefile | 16 ++++++++++++++++\n 1 file changed, 16 insertions(+)\n\ndiff --git a/Makefile b/Makefile\nindex 7d0fa7adb61..91596bac4c0 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -481,6 +481,17 @@ include shared.mak\n #\n # === SHA-1 backend ===\n #\n+# ==== Security ====\n+#\n+# Due to the SHAttered (https://shattered.io) attack vector on SHA-1\n+# it's strongly recommended to use the sha1collisiondetection\n+# counter-cryptanalysis library for SHA-1 hashing.\n+#\n+# If you know that you can trust the repository contents, or where\n+# potential SHA-1 attacks are otherwise mitigated the other backends\n+# listed in \"SHA-1 implementations\" are faster than\n+# sha1collisiondetection.\n+#\n # ==== Default SHA-1 backend ====\n #\n # If no *_SHA1 backend is picked, the first supported one listed in\n@@ -525,6 +536,11 @@ include shared.mak\n #\n # === SHA-256 backend ===\n #\n+# ==== Security ====\n+#\n+# Unlike SHA-1 the SHA-256 algorithm does not suffer from any known\n+# vulnerabilities, so any implementation will do.\n+#\n # ==== SHA-256 implementations ====\n #\n # Define OPENSSL_SHA256 to use the SHA-256 routines in OpenSSL.\n-- \n2.38.0.1464.gea6794aacbc\n\n"}]}