{"thread":{"id":"40885","subject":"Combining APPLE_COMMON_CRYPTO=1 and NO_OPENSSL=1 produces unexpected result","startedAt":"2015-11-25T15:10:44Z","lastAt":"2016-01-15T20:28:06Z","messageCount":6,"participants":["Jack Nagel","Eric Sunshine","Junio C Hamano","David Aguilar"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"273737","messageId":"CAMYxyaVQyVRQb-b0nVv412tMZ3rEnOfUPRakg2dEREg5_Ba5Ag@mail.gmail.com","threadId":"40885","inReplyTo":null,"subject":"Combining APPLE_COMMON_CRYPTO=1 and NO_OPENSSL=1 produces unexpected result","fromName":"Jack Nagel","fromEmail":"jacknagel@gmail.com","sentAt":"2015-11-25T15:10:44Z","receivedAt":"2015-11-25T15:10:44Z","isPatch":false,"sender":{"key":"jacknagel@gmail.com","avatar":"https://avatars.githubusercontent.com/u/568243?v=4"},"body":"When compiling git on OS X (where APPLE_COMMON_CRYPTO=1 is the\ndefault) and specifying NO_OPENSSL=1, the resulting git uses the\nBLK_SHA1 implementation rather than the functions available in\nCommonCrypto.\n\n$ uname -a\nDarwin broadwell.local 15.0.0 Darwin Kernel Version 15.0.0: Sat Sep 19\n15:53:46 PDT 2015; root:xnu-3247.10.11~1/RELEASE_X86_64 x86_64\n\n$ make NO_OPENSSL=1\n[...]\n$ nm git | grep _SHA1_\n0000000100173f00 t _blk_SHA1_Block\n0000000100174e80 T _blk_SHA1_Final\n000000010018fbb0 s _blk_SHA1_Final.pad\n0000000100173de0 T _blk_SHA1_Init\n0000000100173e10 T _blk_SHA1_Update\n\nHowever, with OpenSSL available, it does use the CommonCrypto functions:\n\n$ make\n[...]\n$ nm git | grep _SHA1_\n U _CC_SHA1_Final\n U _CC_SHA1_Init\n U _CC_SHA1_Update\n\nI would not expect the presence of NO_OPENSSL=1 to change the behavior\nhere, since neither case actually makes use of the OpenSSL SHA1\nfunctions.\n"},{"id":"274936","messageId":"CAPig+cS2+NP=-XEYA6e=doQu=+Qn-Lzut-7OCeYJRFZFchNepg@mail.gmail.com","threadId":"40885","inReplyTo":"CAMYxyaVQyVRQb-b0nVv412tMZ3rEnOfUPRakg2dEREg5_Ba5Ag@mail.gmail.com","subject":"Re: Combining APPLE_COMMON_CRYPTO=1 and NO_OPENSSL=1 produces unexpected result","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2015-12-23T08:51:56Z","receivedAt":"2015-12-23T08:51:56Z","isPatch":false,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Wed, Nov 25, 2015 at 10:10 AM, Jack Nagel wrote:\n> When compiling git on OS X (where APPLE_COMMON_CRYPTO=1 is the\n> default) and specifying NO_OPENSSL=1, the resulting git uses the\n> BLK_SHA1 implementation rather than the functions available in\n> CommonCrypto.\n>\n> $ make NO_OPENSSL=1\n> [...]\n> $ nm git | grep _SHA1_\n> 0000000100173f00 t _blk_SHA1_Block\n> 0000000100174e80 T _blk_SHA1_Final\n> 000000010018fbb0 s _blk_SHA1_Final.pad\n> 0000000100173de0 T _blk_SHA1_Init\n> 0000000100173e10 T _blk_SHA1_Update\n\nNO_OPENSSL disables all SSL-related functionality in Git, not just\nSHA1 computation, however, Git still needs to compute SHA1 hashes for\nother reasons, so it uses its own BLK_SHA1 versions when OpenSSL is\nunavailable.\n\n> However, with OpenSSL available, it does use the CommonCrypto functions:\n>\n> $ make\n> [...]\n> $ nm git | grep _SHA1_\n> U _CC_SHA1_Final\n> U _CC_SHA1_Init\n> U _CC_SHA1_Update\n>\n> I would not expect the presence of NO_OPENSSL=1 to change the behavior\n> here, since neither case actually makes use of the OpenSSL SHA1\n> functions.\n\nApple's CommonCrypto headers present OpenSSL-compatible API. When\nCommonCrypto is enabled in the Git makefile, aside from pulling in the\nCommonCrypto header and linking against the CommonCrypto framework,\nfrom Git's perspective, it still thinks it's using OpenSSL, and all\nSSL-related functionality is enabled (including the SHA1 functions).\n\nSo, it might be easier to think of NO_OPENSSL as really meaning NO_SSL\n(that is, \"disable all SSL-related functionality\"). Since the only SSL\nimplementation Git knows how to use is OpenSSL, perhaps one can\nconsider the name NO_OPENSSL a historic anomaly.\n"},{"id":"275032","messageId":"xmqqy4cf9ugm.fsf@gitster.mtv.corp.google.com","threadId":"40885","inReplyTo":"CAPig+cS2+NP=-XEYA6e=doQu=+Qn-Lzut-7OCeYJRFZFchNepg@mail.gmail.com","subject":"Re: Combining APPLE_COMMON_CRYPTO=1 and NO_OPENSSL=1 produces unexpected result","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2015-12-28T02:29:29Z","receivedAt":"2015-12-28T02:29:29Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Eric Sunshine writes:\n\n> So, it might be easier to think of NO_OPENSSL as really meaning NO_SSL\n> (that is, \"disable all SSL-related functionality\"). Since the only SSL\n> implementation Git knows how to use is OpenSSL, perhaps one can\n> consider the name NO_OPENSSL a historic anomaly.\n\nThat is a good explanation of what is observed. I am not sure if it\nis a good justification, though. If you tell somebody who needs to\nlink an implementation of SHA-1 in that you (1) do not want to use\nOpenSSL (or do not want to have SSL at all), and (2) do not mind\nusing Apple's CommonCrypto, and if you _know_ that CommonCrypto is\na possible source of the SHA-1 implementation, then I would think it\nis reasonable to expect that CommonCrypto SHA-1 to be used.\n\n\tNote. To further explain the situation, the only reason we\n\tadded CommonCrypto knob in the build system was to allow\n\tpeople to use OpenSSL as the SSL implementation. Those who\n\tadded the knob weren't making a conscious decision on which\n\tSHA-1 implementation to use in that scenario---they may not\n\teven have been aware of the fact that SHA-1 was offered by\n\tCommonCrypto for that matter.\n\nA few questions we should be asking Apple users are:\n\n - Is there a strong-enough reason why those who do not want to use\n SSL should be able to choose the SHA-1 implementation available\n from CommonCrypto over block-sha1?\n\n - Is CommonCrypto SHA-1 a better implementation than block-sha1?\n\nDepending on the answers to these questions, we might want to:\n\n - add a knob to allow choosing between two available\n implementations (i.e. when NO_APPLE_COMMON_CRYPTO is unset) of\n SHA-1, regardless of the setting of NO_OPENSSL.\n\n - decide which one between CommonCrypto and block-sha1 should be\n the default.\n\nIf we end up deciding that we use block-sha1 as the default, we\nshould do so even when both NO_OPENSSL and NO_APPLE_COMMON_CRYPTO\nare left unset. If we decide that block-sha1 should merely be a\nfallback when no other SHA-1 implementation is availble, on the\nother hand, we should be using CommonCrypto SHA-1 as long as the\nuser did not set NO_APPLE_COMMON_CRYPTO explicitly, even when we are\nbuilding with NO_OPENSSL.\n\nIf people do not care, we can leave things as they are. It would\nseem mysterious to use block-sha1 when we are not using CommonCrypto\nfor SSL (i.e. NO_OPENSSL), and otherwise CommonCrypto SHA-1, and\nwould invite a puzzlement we saw in this thread, though.\n"},{"id":"275271","messageId":"20160102234923.GA14424@gmail.com","threadId":"40885","inReplyTo":"xmqqy4cf9ugm.fsf@gitster.mtv.corp.google.com","subject":"Re: Combining APPLE_COMMON_CRYPTO=1 and NO_OPENSSL=1 produces unexpected result","fromName":"David Aguilar","fromEmail":"davvid@gmail.com","sentAt":"2016-01-02T23:49:23Z","receivedAt":"2016-01-02T23:49:23Z","isPatch":false,"sender":{"key":"davvid@gmail.com","avatar":"https://avatars.githubusercontent.com/u/13196?v=4"},"body":"On Sun, Dec 27, 2015 at 06:29:29PM -0800, Junio C Hamano wrote:\n> Eric Sunshine writes:\n> \n> > So, it might be easier to think of NO_OPENSSL as really meaning NO_SSL\n> > (that is, \"disable all SSL-related functionality\"). Since the only SSL\n> > implementation Git knows how to use is OpenSSL, perhaps one can\n> > consider the name NO_OPENSSL a historic anomaly.\n> \n> That is a good explanation of what is observed. I am not sure if it\n> is a good justification, though. If you tell somebody who needs to\n> link an implementation of SHA-1 in that you (1) do not want to use\n> OpenSSL (or do not want to have SSL at all), and (2) do not mind\n> using Apple's CommonCrypto, and if you _know_ that CommonCrypto is\n> a possible source of the SHA-1 implementation, then I would think it\n> is reasonable to expect that CommonCrypto SHA-1 to be used.\n> \n> \tNote. To further explain the situation, the only reason we\n> \tadded CommonCrypto knob in the build system was to allow\n> \tpeople to use OpenSSL as the SSL implementation. Those who\n> \tadded the knob weren't making a conscious decision on which\n> \tSHA-1 implementation to use in that scenario---they may not\n> \teven have been aware of the fact that SHA-1 was offered by\n> \tCommonCrypto for that matter.\n\nMy original motivation for going down the CommonCrypto route was,\n\"bummer, git is not compiling.. let me try to fix that, somehow\".\n\nI think the best long-term solution would be to abandon the\nCommonCrypto backend, if possible. There's not a strong reason\nfor its existence. It always seemed kinda hacky, and bolted-on.\n\n> A few questions we should be asking Apple users are:\n> \n> - Is there a strong-enough reason why those who do not want to use\n> SSL should be able to choose the SHA-1 implementation available\n> from CommonCrypto over block-sha1?\n\nIMO, no.\n\n> - Is CommonCrypto SHA-1 a better implementation than block-sha1?\n\nI do not believe this to be true.\n\nMy gut feeling is that we cannot rely on the long-term stability\nand availability of Apple's APIs. Block-sha1 works fine on\nthe current Apple hardware and I suspect that it (or openssl)\nwill continue to work fine in the future.\n\n> Depending on the answers to these questions, we might want to:\n> \n> - add a knob to allow choosing between two available\n> implementations (i.e. when NO_APPLE_COMMON_CRYPTO is unset) of\n> SHA-1, regardless of the setting of NO_OPENSSL.\n> \n> - decide which one between CommonCrypto and block-sha1 should be\n> the default.\n\nHow about, drop support for CommonCrypto so that there's no need\nfor the end-user to choose? That means we would get block-sha1\nby default.\n\n> If we end up deciding that we use block-sha1 as the default, we\n> should do so even when both NO_OPENSSL and NO_APPLE_COMMON_CRYPTO\n> are left unset. If we decide that block-sha1 should merely be a\n> fallback when no other SHA-1 implementation is availble, on the\n> other hand, we should be using CommonCrypto SHA-1 as long as the\n> user did not set NO_APPLE_COMMON_CRYPTO explicitly, even when we are\n> building with NO_OPENSSL.\n> \n> If people do not care, we can leave things as they are. It would\n> seem mysterious to use block-sha1 when we are not using CommonCrypto\n> for SSL (i.e. NO_OPENSSL), and otherwise CommonCrypto SHA-1, and\n> would invite a puzzlement we saw in this thread, though.\n\nI'm curious to see what others think about dropping CommonCrypto.\nIt seems like a good choice from a maintenance POV.\n-- \nDavid\n"},{"id":"276185","messageId":"xmqqfuxyisjp.fsf@gitster.mtv.corp.google.com","threadId":"40885","inReplyTo":"20160102234923.GA14424@gmail.com","subject":"Re: Combining APPLE_COMMON_CRYPTO=1 and NO_OPENSSL=1 produces unexpected result","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-01-15T18:52:42Z","receivedAt":"2016-01-15T18:52:42Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"David Aguilar writes:\n\n> I think the best long-term solution would be to abandon the\n> CommonCrypto backend, if possible. There's not a strong reason\n> for its existence. It always seemed kinda hacky, and bolted-on.\n> ...\n>> A few questions we should be asking Apple users are:\n>> \n>> - Is there a strong-enough reason why those who do not want to use\n>> SSL should be able to choose the SHA-1 implementation available\n>> from CommonCrypto over block-sha1?\n>\n> IMO, no.\n>\n>> - Is CommonCrypto SHA-1 a better implementation than block-sha1?\n>\n> I do not believe this to be true.\n>\n> My gut feeling is that we cannot rely on the long-term stability\n> and availability of Apple's APIs. Block-sha1 works fine on\n> the current Apple hardware and I suspect that it (or openssl)\n> will continue to work fine in the future.\n> ...\n>> If people do not care, we can leave things as they are. It would\n>> seem mysterious to use block-sha1 when we are not using CommonCrypto\n>> for SSL (i.e. NO_OPENSSL), and otherwise CommonCrypto SHA-1, and\n>> would invite a puzzlement we saw in this thread, though.\n>\n> I'm curious to see what others think about dropping CommonCrypto.\n> It seems like a good choice from a maintenance POV.\n\nJudging by a week-long silence, it seems nobody seems to have much\nto say on this issue. Let me summon the git_osx_installer\nmaintainer to hear from somebody who know a lot better than me about\nthings around OS X.\n\nThanks.\n"},{"id":"276198","messageId":"CAPig+cQ5kKAt2_RQnqT7Rn=uGmHV9VvxpQ+UgDPOj=D=pq6arg@mail.gmail.com","threadId":"40885","inReplyTo":"xmqqfuxyisjp.fsf@gitster.mtv.corp.google.com","subject":"Re: Combining APPLE_COMMON_CRYPTO=1 and NO_OPENSSL=1 produces unexpected result","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2016-01-15T20:28:06Z","receivedAt":"2016-01-15T20:28:06Z","isPatch":false,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Fri, Jan 15, 2016 at 1:52 PM, Junio C Hamano wrote:\n> David Aguilar writes:\n>> I'm curious to see what others think about dropping CommonCrypto.\n>> It seems like a good choice from a maintenance POV.\n>\n> Judging by a week-long silence, it seems nobody seems to have much\n> to say on this issue. Let me summon the git_osx_installer\n> maintainer to hear from somebody who know a lot better than me about\n> things around OS X.\n\nDropping CommonCrypto seems reasonable to me. Since Apple's\nCommonCrypto provides only a subset of the OpenSSL functionality Git\nuses, we still end up bringing in OpenSSL anyhow in addition to\nCommonCrypto, so CommonCrypto isn't buying us much (if anything).\nUnfortunately, we'd still want to keep the ugly hack to suppress the\nunavoidable \"noise\" Apple thrust upon us by populating the OpenSSL\nheaders with #warnings about unstable ABI.\n"}]}