{"thread":{"id":"45095","subject":"[PATCH] mingw: use OpenSSL's SHA-1 routines","startedAt":"2017-02-09T22:28:05Z","lastAt":"2017-03-02T17:44:26Z","messageCount":18,"participants":["Johannes Schindelin","Junio C Hamano","Jeff King","Johannes Sixt","Linus Torvalds"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"311232","messageId":"6a29f8c60d315a24292c1fa9f5e84df4dfdbf813.1486679254.git.johannes.schindelin@gmx.de","threadId":"45095","inReplyTo":null,"subject":"[PATCH] mingw: use OpenSSL's SHA-1 routines","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2017-02-09T22:27:49Z","receivedAt":"2017-02-09T22:28:05Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"From: Jeff Hostetler \n\nUse OpenSSL's SHA-1 routines rather than builtin block-sha1 routines.\nThis improves performance on SHA1 operations on Intel processors.\n\nOpenSSL 1.0.2 has made considerable performance improvements and\nsupport the Intel hardware acceleration features. See:\nhttps://software.intel.com/en-us/articles/improving-openssl-performance\nhttps://software.intel.com/en-us/articles/intel-sha-extensions\n\nTo test this I added/staged a single file in a gigantic\nrepository having a 450MB index file. The code in read-cache.c\nverifies the header SHA as it reads the index and computes a new\nheader SHA as it writes out the new index. Therefore, in this test\nthe SHA code must process 900MB of data. Testing was done on an\nIntel I7-4770 CPU @ 3.40GHz (Intel64, Family 6, Model 60) CPU.\n\nThe block-sha1 version averaged 5.27 seconds.\nThe OpenSSL version averaged 4.50 seconds.\n\n================================================================\n\n$ echo xxx >> project.mk\n$ time /e/blk_sha/bin/git.exe add project.mk\n\nreal 0m5.207s\nuser 0m0.000s\nsys 0m0.250s\n\n$ echo xxx >> project.mk\n$ time /e/blk_sha/bin/git.exe add project.mk\n\nreal 0m5.362s\nuser 0m0.015s\nsys 0m0.234s\n\n$ echo xxx >> project.mk\n$ time /e/blk_sha/bin/git.exe add project.mk\n\nreal 0m5.300s\nuser 0m0.016s\nsys 0m0.250s\n\n$ echo xxx >> project.mk\n$ time /e/blk_sha/bin/git.exe add project.mk\n\nreal 0m5.216s\nuser 0m0.000s\nsys 0m0.250s\n\n================================================================\n$ echo xxx >> project.mk\n$ time /e/openssl/bin/git.exe add project.mk\n\nreal 0m4.431s\nuser 0m0.000s\nsys 0m0.250s\n\n$ echo xxx >> project.mk\n$ time /e/openssl/bin/git.exe add project.mk\n\nreal 0m4.478s\nuser 0m0.000s\nsys 0m0.265s\n\n$ echo xxx >> project.mk\n$ time /e/openssl/bin/git.exe add project.mk\n\nreal 0m4.690s\nuser 0m0.000s\nsys 0m0.250s\n\n$ echo xxx >> project.mk\n$ time /e/openssl/bin/git.exe add project.mk\n\nreal 0m4.420s\nuser 0m0.000s\nsys 0m0.234s\n\n================================================================\n\nSigned-off-by: Jeff Hostetler \nSigned-off-by: Johannes Schindelin \n---\nPublished-As: https://github.com/dscho/git/releases/tag/mingw-openssl-sha1-v1\nFetch-It-Via: git fetch https://github.com/dscho/git mingw-openssl-sha1-v1\n\n config.mak.uname | 1 -\n 1 file changed, 1 deletion(-)\n\ndiff --git a/config.mak.uname b/config.mak.uname\nindex 447f36ac2e..a07936da8b 100644\n--- a/config.mak.uname\n+++ b/config.mak.uname\n@@ -515,7 +515,6 @@ ifneq (,$(findstring MINGW,$(uname_S)))\n \tOBJECT_CREATION_USES_RENAMES = UnfortunatelyNeedsTo\n \tNO_REGEX = YesPlease\n \tNO_PYTHON = YesPlease\n-\tBLK_SHA1 = YesPlease\n \tETAGS_TARGET = ETAGS\n \tNO_INET_PTON = YesPlease\n \tNO_INET_NTOP = YesPlease\n\nbase-commit: 6e3a7b3398559305c7a239a42e447c21a8f39ff8\n-- \n2.11.1.windows.1\n"},{"id":"311238","messageId":"xmqqbmublyo0.fsf@gitster.mtv.corp.google.com","threadId":"45095","inReplyTo":"6a29f8c60d315a24292c1fa9f5e84df4dfdbf813.1486679254.git.johannes.schindelin@gmx.de","subject":"Re: [PATCH] mingw: use OpenSSL's SHA-1 routines","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-02-09T23:41:51Z","receivedAt":"2017-02-09T23:43:45Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Johannes Schindelin writes:\n\n> From: Jeff Hostetler \n>\n> Use OpenSSL's SHA-1 routines rather than builtin block-sha1 routines.\n> This improves performance on SHA1 operations on Intel processors.\n> ...\n>\n> Signed-off-by: Jeff Hostetler \n> Signed-off-by: Johannes Schindelin \n> ---\n\nNice. Will queue as jh/mingw-openssl-sha1 topic; it is a bit too\nlate for today's integration cycle to be merged to 'next', but let's\nhave this by the end of the week in 'master'.\n\nThanks.\n"},{"id":"311248","messageId":"20170210050237.gajicliueuvk6s5d@sigill.intra.peff.net","threadId":"45095","inReplyTo":"6a29f8c60d315a24292c1fa9f5e84df4dfdbf813.1486679254.git.johannes.schindelin@gmx.de","subject":"Re: [PATCH] mingw: use OpenSSL's SHA-1 routines","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2017-02-10T05:02:37Z","receivedAt":"2017-02-10T05:30:28Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Thu, Feb 09, 2017 at 11:27:49PM +0100, Johannes Schindelin wrote:\n\n> From: Jeff Hostetler \n> \n> Use OpenSSL's SHA-1 routines rather than builtin block-sha1 routines.\n> This improves performance on SHA1 operations on Intel processors.\n> \n> OpenSSL 1.0.2 has made considerable performance improvements and\n> support the Intel hardware acceleration features. See:\n> https://software.intel.com/en-us/articles/improving-openssl-performance\n> https://software.intel.com/en-us/articles/intel-sha-extensions\n> \n> To test this I added/staged a single file in a gigantic\n> repository having a 450MB index file. The code in read-cache.c\n> verifies the header SHA as it reads the index and computes a new\n> header SHA as it writes out the new index. Therefore, in this test\n> the SHA code must process 900MB of data. Testing was done on an\n> Intel I7-4770 CPU @ 3.40GHz (Intel64, Family 6, Model 60) CPU.\n\nI think this is only half the story. A heavy-sha1 workload is faster,\nwhich is good. But one of the original reasons to prefer blk-sha1 (at\nleast on Linux) is that resolving libcrypto.so symbols takes a\nnon-trivial amount of time. I just timed it again, and it seems to be\nconsistently 1ms slower to run \"git rev-parse --git-dir\" on my machine\n(from the top-level of a repo).\n\n1ms is mostly irrelevant, but it adds up on scripted workloads that\nstart a lot of git processes. Whether it's a net win or not depends on\nhow much sha1 computation you do in your workload versus how many\nprocesses you start.\n\nI don't know what that means for Windows, though. My impression is that\nprocess startup is so painfully slow there that the link time may just\nbe lost in the noise. It may just always be a win there. So not really\nan objection to your patch, but something you may want to consider.\n\n(Of course, it would in theory be possible to have the best of both\nworlds either by static-linking openssl, or by teaching block-sha1 the\nsame optimizations, but both of those are obviously much more complex).\n\n-Peff\n"},{"id":"311271","messageId":"alpine.DEB.2.20.1702101647340.3496@virtualbox","threadId":"45095","inReplyTo":"20170210050237.gajicliueuvk6s5d@sigill.intra.peff.net","subject":"Re: [PATCH] mingw: use OpenSSL's SHA-1 routines","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2017-02-10T15:49:02Z","receivedAt":"2017-02-10T15:51:24Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Peff,\n\nOn Fri, 10 Feb 2017, Jeff King wrote:\n\n> On Thu, Feb 09, 2017 at 11:27:49PM +0100, Johannes Schindelin wrote:\n> \n> > From: Jeff Hostetler \n> > \n> > Use OpenSSL's SHA-1 routines rather than builtin block-sha1 routines.\n> > This improves performance on SHA1 operations on Intel processors.\n> > \n> > OpenSSL 1.0.2 has made considerable performance improvements and\n> > support the Intel hardware acceleration features. See:\n> > https://software.intel.com/en-us/articles/improving-openssl-performance\n> > https://software.intel.com/en-us/articles/intel-sha-extensions\n> > \n> > To test this I added/staged a single file in a gigantic repository\n> > having a 450MB index file. The code in read-cache.c verifies the\n> > header SHA as it reads the index and computes a new header SHA as it\n> > writes out the new index. Therefore, in this test the SHA code must\n> > process 900MB of data. Testing was done on an Intel I7-4770 CPU @\n> > 3.40GHz (Intel64, Family 6, Model 60) CPU.\n> \n> I think this is only half the story. A heavy-sha1 workload is faster,\n> which is good. But one of the original reasons to prefer blk-sha1 (at\n> least on Linux) is that resolving libcrypto.so symbols takes a\n> non-trivial amount of time. I just timed it again, and it seems to be\n> consistently 1ms slower to run \"git rev-parse --git-dir\" on my machine\n> (from the top-level of a repo).\n> \n> 1ms is mostly irrelevant, but it adds up on scripted workloads that\n> start a lot of git processes.\n\nYou know my answer to that. If scripting slows things down, we should\navoid it in production code. As it is, scripting slows us down. Therefore\nI work slowly but steadily to get rid of scripting where it hurts most.\n\nCiao,\nDscho\n"},{"id":"311278","messageId":"20170210160458.pcp7mupdz24m6cms@sigill.intra.peff.net","threadId":"45095","inReplyTo":"alpine.DEB.2.20.1702101647340.3496@virtualbox","subject":"Re: [PATCH] mingw: use OpenSSL's SHA-1 routines","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2017-02-10T16:04:58Z","receivedAt":"2017-02-10T16:31:46Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Fri, Feb 10, 2017 at 04:49:02PM +0100, Johannes Schindelin wrote:\n\n> > I think this is only half the story. A heavy-sha1 workload is faster,\n> > which is good. But one of the original reasons to prefer blk-sha1 (at\n> > least on Linux) is that resolving libcrypto.so symbols takes a\n> > non-trivial amount of time. I just timed it again, and it seems to be\n> > consistently 1ms slower to run \"git rev-parse --git-dir\" on my machine\n> > (from the top-level of a repo).\n> > \n> > 1ms is mostly irrelevant, but it adds up on scripted workloads that\n> > start a lot of git processes.\n> \n> You know my answer to that. If scripting slows things down, we should\n> avoid it in production code. As it is, scripting slows us down. Therefore\n> I work slowly but steadily to get rid of scripting where it hurts most.\n\nWell, yes. My question is more \"what does it look like on normal Git\nworkloads?\". Are you trading off an optimization for your giant 450MB\nindex workload (which _also_ could be fixed by trying do the slow\noperation less, rather than micro-optimizing it) in a way that hurts\npeople working with more normal sized repos?\n\nFor instance, \"make BLK_SHA1=Yes test\" is measurably faster for me than\n\"make BLK_SHA1= test\".\n\nI'm open to the argument that it doesn't matter in practice for normal\ngit users. But it's not _just_ scripting. It depends on the user's\npattern of invoking git commands (and how expensive the symbol\nresolution is on their system).\n\n-Peff\n"},{"id":"311334","messageId":"31bb0b9f-d498-24b3-57d5-9f34cb8e3914@kdbg.org","threadId":"45095","inReplyTo":"xmqqbmublyo0.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH] mingw: use OpenSSL's SHA-1 routines","fromName":"Johannes Sixt","fromEmail":"j6t@kdbg.org","sentAt":"2017-02-11T08:01:23Z","receivedAt":"2017-02-11T08:01:35Z","isPatch":true,"sender":{"key":"j6t@kdbg.org","avatar":"https://avatars.githubusercontent.com/u/14810926?v=4"},"body":"Am 10.02.2017 um 00:41 schrieb Junio C Hamano:\n> Johannes Schindelin writes:\n>\n>> From: Jeff Hostetler \n>>\n>> Use OpenSSL's SHA-1 routines rather than builtin block-sha1 routines.\n>> This improves performance on SHA1 operations on Intel processors.\n>> ...\n>>\n>> Signed-off-by: Jeff Hostetler \n>> Signed-off-by: Johannes Schindelin \n>> ---\n>\n> Nice. Will queue as jh/mingw-openssl-sha1 topic; it is a bit too\n> late for today's integration cycle to be merged to 'next', but let's\n> have this by the end of the week in 'master'.\n\nPlease don't rush this through. I didn't have a chance to cross-check \nthe patch; it will have to wait for Monday. I would like to address \nPeff's concerns about additional runtime dependencies.\n\n-- Hannes\n\n"},{"id":"311345","messageId":"xmqqzihsh873.fsf@gitster.mtv.corp.google.com","threadId":"45095","inReplyTo":"31bb0b9f-d498-24b3-57d5-9f34cb8e3914@kdbg.org","subject":"Re: [PATCH] mingw: use OpenSSL's SHA-1 routines","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-02-11T18:51:44Z","receivedAt":"2017-02-11T18:51:54Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Johannes Sixt writes:\n\n> Am 10.02.2017 um 00:41 schrieb Junio C Hamano:\n>> ...\n>> Nice. Will queue as jh/mingw-openssl-sha1 topic; it is a bit too\n>> late for today's integration cycle to be merged to 'next', but let's\n>> have this by the end of the week in 'master'.\n>\n> Please don't rush this through. I didn't have a chance to cross-check\n> the patch; it will have to wait for Monday. I would like to address\n> Peff's concerns about additional runtime dependencies.\n\nOK. Will mark it as \"Will cook in 'next'\" for now.\n"},{"id":"311411","messageId":"alpine.DEB.2.20.1702131815351.3496@virtualbox","threadId":"45095","inReplyTo":"31bb0b9f-d498-24b3-57d5-9f34cb8e3914@kdbg.org","subject":"Re: [PATCH] mingw: use OpenSSL's SHA-1 routines","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2017-02-13T17:16:15Z","receivedAt":"2017-02-13T17:16:56Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Hannes,\n\nOn Sat, 11 Feb 2017, Johannes Sixt wrote:\n\n> Am 10.02.2017 um 00:41 schrieb Junio C Hamano:\n> > Johannes Schindelin writes:\n> >\n> > > From: Jeff Hostetler \n> > >\n> > > Use OpenSSL's SHA-1 routines rather than builtin block-sha1\n> > > routines. This improves performance on SHA1 operations on Intel\n> > > processors.\n> > > ...\n> > >\n> > > Signed-off-by: Jeff Hostetler \n> > > Signed-off-by: Johannes Schindelin \n> > > ---\n> >\n> > Nice. Will queue as jh/mingw-openssl-sha1 topic; it is a bit too\n> > late for today's integration cycle to be merged to 'next', but let's\n> > have this by the end of the week in 'master'.\n> \n> Please don't rush this through. I didn't have a chance to cross-check the\n> patch; it will have to wait for Monday. I would like to address Peff's\n> concerns about additional runtime dependencies.\n\nI never meant this to be fast-tracked into git.git. We have all the time\nin our lives to get this in, as Git for Windows already carries this patch\nfor a while, and shall continue to do so.\n\nCiao,\nDscho\n"},{"id":"311412","messageId":"9913e513-553e-eba6-e81a-9c21030dd767@kdbg.org","threadId":"45095","inReplyTo":"20170210160458.pcp7mupdz24m6cms@sigill.intra.peff.net","subject":"Re: [PATCH] mingw: use OpenSSL's SHA-1 routines","fromName":"Johannes Sixt","fromEmail":"j6t@kdbg.org","sentAt":"2017-02-13T17:46:35Z","receivedAt":"2017-02-13T17:46:44Z","isPatch":true,"sender":{"key":"j6t@kdbg.org","avatar":"https://avatars.githubusercontent.com/u/14810926?v=4"},"body":"Am 10.02.2017 um 17:04 schrieb Jeff King:\n> On Fri, Feb 10, 2017 at 04:49:02PM +0100, Johannes Schindelin wrote:\n>\n>>> I think this is only half the story. A heavy-sha1 workload is faster,\n>>> which is good. But one of the original reasons to prefer blk-sha1 (at\n>>> least on Linux) is that resolving libcrypto.so symbols takes a\n>>> non-trivial amount of time. I just timed it again, and it seems to be\n>>> consistently 1ms slower to run \"git rev-parse --git-dir\" on my machine\n>>> (from the top-level of a repo).\n>>>\n>>> 1ms is mostly irrelevant, but it adds up on scripted workloads that\n>>> start a lot of git processes.\n>>\n>> You know my answer to that. If scripting slows things down, we should\n>> avoid it in production code. As it is, scripting slows us down. Therefore\n>> I work slowly but steadily to get rid of scripting where it hurts most.\n>\n> Well, yes. My question is more \"what does it look like on normal Git\n> workloads?\". Are you trading off an optimization for your giant 450MB\n> index workload (which _also_ could be fixed by trying do the slow\n> operation less, rather than micro-optimizing it) in a way that hurts\n> people working with more normal sized repos?\n>\n> For instance, \"make BLK_SHA1=Yes test\" is measurably faster for me than\n> \"make BLK_SHA1= test\".\n\nThe patch does add a new runtime dependency on libcrypto.dll in my \nenvironment. I would be surprised if it does not also with your modern \nbuild tools.\n\nI haven't had time to compare test suite runtimes.\n\n> I'm open to the argument that it doesn't matter in practice for normal\n> git users. But it's not _just_ scripting. It depends on the user's\n> pattern of invoking git commands (and how expensive the symbol\n> resolution is on their system).\n\nIt can be argued that in normal interactive use, it is hard to notice \nthat another DLL is loaded. Don't forget, though, that on Windows it is \nnot only the pure time to resolve the entry points, but also that \ntypically virus scanners inspect every executable file that is loaded, \nwhich adds another share of time.\n\nI'll use the patch for daily work for a while to see whether it hurts.\n\n-- Hannes\n\n"},{"id":"311425","messageId":"xmqq60kdev2r.fsf@gitster.mtv.corp.google.com","threadId":"45095","inReplyTo":"9913e513-553e-eba6-e81a-9c21030dd767@kdbg.org","subject":"Re: [PATCH] mingw: use OpenSSL's SHA-1 routines","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-02-13T19:42:36Z","receivedAt":"2017-02-13T19:42:49Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Johannes Sixt writes:\n\n> The patch does add a new runtime dependency on libcrypto.dll in my\n> environment. I would be surprised if it does not also with your modern\n> build tools.\n>\n> I haven't had time to compare test suite runtimes.\n>\n>> I'm open to the argument that it doesn't matter in practice for normal\n>> git users. But it's not _just_ scripting. It depends on the user's\n>> pattern of invoking git commands (and how expensive the symbol\n>> resolution is on their system).\n>\n> It can be argued that in normal interactive use, it is hard to notice\n> that another DLL is loaded. Don't forget, though, that on Windows it\n> is not only the pure time to resolve the entry points, but also that\n> typically virus scanners inspect every executable file that is loaded,\n> which adds another share of time.\n>\n> I'll use the patch for daily work for a while to see whether it hurts.\n\nThanks.\n\nI need to ask an unrelated question at a bit higher level, though.\n\nI have been operating under the assumption that everybody on Windows\nwho builds Git works off of Dscho's Git for Windows tree, and\npatches that are specific to Windows from Dscho's are sent to me via\nthe list only after they have been in Git for Windows and proven to\nhelp Windows users in the wild. \n\nThe consequence of these two assumptions is that I would feel safe\nto treat Windows specific changes that do not touch generic part of\nthe codebase from Dscho just like updates from any other subsystem\nmaintainers (any git-svn thing from Eric, any gitk thing from Paul,\nany p4 thing Luke and Lars are both happy with, etc.).\n\nYou seem to be saying that the first of the two assumptions does not\nhold. Should I change my expectations while queuing Windows specific\npatches from Dscho?\n\n"},{"id":"311444","messageId":"b530c820-9956-4396-d853-c7d70ccaf11d@kdbg.org","threadId":"45095","inReplyTo":"xmqq60kdev2r.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH] mingw: use OpenSSL's SHA-1 routines","fromName":"Johannes Sixt","fromEmail":"j6t@kdbg.org","sentAt":"2017-02-13T21:07:16Z","receivedAt":"2017-02-13T21:07:33Z","isPatch":true,"sender":{"key":"j6t@kdbg.org","avatar":"https://avatars.githubusercontent.com/u/14810926?v=4"},"body":"Am 13.02.2017 um 20:42 schrieb Junio C Hamano:\n> I have been operating under the assumption that everybody on Windows\n> who builds Git works off of Dscho's Git for Windows tree, and\n> patches that are specific to Windows from Dscho's are sent to me via\n> the list only after they have been in Git for Windows and proven to\n> help Windows users in the wild.\n>\n> The consequence of these two assumptions is that I would feel safe\n> to treat Windows specific changes that do not touch generic part of\n> the codebase from Dscho just like updates from any other subsystem\n> maintainers (any git-svn thing from Eric, any gitk thing from Paul,\n> any p4 thing Luke and Lars are both happy with, etc.).\n>\n> You seem to be saying that the first of the two assumptions does not\n> hold. Should I change my expectations while queuing Windows specific\n> patches from Dscho?\n\nYour first assumption is incorrect as far as I am concerned. I build \nfrom your tree plus some topics. During -rc period, I build off of \nmaster; after a release, I build off of next. I merge some of the topics \nthat you carry in pu when I find them interesting or when I suspect them \nto regress on Windows. Then I carry around a few additional patches that \nthe public has never seen, and these days I also merge Dscho's rebase-i \ntopic.\n\n-- Hannes\n\n"},{"id":"311463","messageId":"alpine.DEB.2.20.1702132337470.3496@virtualbox","threadId":"45095","inReplyTo":"b530c820-9956-4396-d853-c7d70ccaf11d@kdbg.org","subject":"Re: [PATCH] mingw: use OpenSSL's SHA-1 routines","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2017-02-13T22:38:17Z","receivedAt":"2017-02-13T22:38:51Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Mon, 13 Feb 2017, Johannes Sixt wrote:\n\n> Am 13.02.2017 um 20:42 schrieb Junio C Hamano:\n> > I have been operating under the assumption that everybody on Windows\n> > who builds Git works off of Dscho's Git for Windows tree, and patches\n> > that are specific to Windows from Dscho's are sent to me via the list\n> > only after they have been in Git for Windows and proven to help\n> > Windows users in the wild.\n> >\n> > The consequence of these two assumptions is that I would feel safe to\n> > treat Windows specific changes that do not touch generic part of the\n> > codebase from Dscho just like updates from any other subsystem\n> > maintainers (any git-svn thing from Eric, any gitk thing from Paul,\n> > any p4 thing Luke and Lars are both happy with, etc.).\n> >\n> > You seem to be saying that the first of the two assumptions does not\n> > hold. Should I change my expectations while queuing Windows specific\n> > patches from Dscho?\n> \n> Your first assumption is incorrect as far as I am concerned. I build\n> from your tree plus some topics. During -rc period, I build off of\n> master; after a release, I build off of next. I merge some of the topics\n> that you carry in pu when I find them interesting or when I suspect them\n> to regress on Windows. Then I carry around a few additional patches\n> that the public has never seen, and these days I also merge Dscho's\n> rebase-i topic.\n\nIn addition, you build from a custom MINGW/MSys1 setup, correct?\n\nCiao,\nJohannes\n"},{"id":"311520","messageId":"6c2bbca7-7a8f-d3d8-04b6-31494a3e1b43@kdbg.org","threadId":"45095","inReplyTo":"alpine.DEB.2.20.1702132337470.3496@virtualbox","subject":"Re: [PATCH] mingw: use OpenSSL's SHA-1 routines","fromName":"Johannes Sixt","fromEmail":"j6t@kdbg.org","sentAt":"2017-02-14T06:14:00Z","receivedAt":"2017-02-14T06:14:08Z","isPatch":true,"sender":{"key":"j6t@kdbg.org","avatar":"https://avatars.githubusercontent.com/u/14810926?v=4"},"body":"Am 13.02.2017 um 23:38 schrieb Johannes Schindelin:\n> In addition, you build from a custom MINGW/MSys1 setup, correct?\n\nCorrect. Specifically, I use the build tools from \"msysgit\" times, but \nbuild outside the premanufactured build environement; i.e., the \n\"THIS_IS_MSYSGIT\" section in config.mak.uname is not used.\n\n-- Hannes\n\n"},{"id":"311787","messageId":"0080edd6-a515-2fe9-6266-b6f6bbedfdde@kdbg.org","threadId":"45095","inReplyTo":"alpine.DEB.2.20.1702131815351.3496@virtualbox","subject":"Re: [PATCH] mingw: use OpenSSL's SHA-1 routines","fromName":"Johannes Sixt","fromEmail":"j6t@kdbg.org","sentAt":"2017-02-16T18:12:17Z","receivedAt":"2017-02-16T18:12:26Z","isPatch":true,"sender":{"key":"j6t@kdbg.org","avatar":"https://avatars.githubusercontent.com/u/14810926?v=4"},"body":"Am 13.02.2017 um 18:16 schrieb Johannes Schindelin:\n> On Sat, 11 Feb 2017, Johannes Sixt wrote:\n>> Am 10.02.2017 um 00:41 schrieb Junio C Hamano:\n>>> Johannes Schindelin writes:\n>>>\n>>>> From: Jeff Hostetler \n>>>>\n>>>> Use OpenSSL's SHA-1 routines rather than builtin block-sha1\n>>>> routines. This improves performance on SHA1 operations on Intel\n>>>> processors.\n>>>> ...\n>>>>\n>>>> Signed-off-by: Jeff Hostetler \n>>>> Signed-off-by: Johannes Schindelin \n>>>> ---\n>>>\n>>> Nice. Will queue as jh/mingw-openssl-sha1 topic; it is a bit too\n>>> late for today's integration cycle to be merged to 'next', but let's\n>>> have this by the end of the week in 'master'.\n>>\n>> Please don't rush this through. I didn't have a chance to cross-check the\n>> patch; it will have to wait for Monday. I would like to address Peff's\n>> concerns about additional runtime dependencies.\n>\n> I never meant this to be fast-tracked into git.git. We have all the time\n> in our lives to get this in, as Git for Windows already carries this patch\n> for a while, and shall continue to do so.\n\nI've been working with this patch for the past few days, and I did not \nnotice any disadvantage during interactive work even though there is a \nnew dependency on libcrypto.dll.\n\nHere are some unscientific numbers collected during test suite runs:\n\nbash -c \"time make -j4 -k test\"\n\nwith this patch:\n\nreal 34m47.242s\nuser 9m55.827s\nsys 25m20.483s\n\nwithout this patch:\n\nreal 34m2.330s\nuser 9m56.556s\nsys 25m5.520s\n\nIt looks like BLK_SHA1 has some advantage, but I would not count on \nthese figures too much. (I certainly did not sit idly in front of the \nworkstation during these tests, for example. That may have skewed the \nnumbers somewhat.) (And, no, I'm not going to measure best-of-five \ntimings, not even best-of-two. ;)\n\nIn summary: Interactive response times do not decline noticably. I do \nnot object the patch.\n\n-- Hannes\n\n"},{"id":"312573","messageId":"xmqqmvdbz23d.fsf@gitster.mtv.corp.google.com","threadId":"45095","inReplyTo":"9913e513-553e-eba6-e81a-9c21030dd767@kdbg.org","subject":"Re: [PATCH] mingw: use OpenSSL's SHA-1 routines","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-02-24T21:54:46Z","receivedAt":"2017-02-24T21:54:53Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Johannes Sixt writes:\n\n> It can be argued that in normal interactive use, it is hard to notice\n> that another DLL is loaded. Don't forget, though, that on Windows it\n> is not only the pure time to resolve the entry points, but also that\n> typically virus scanners inspect every executable file that is loaded,\n> which adds another share of time.\n>\n> I'll use the patch for daily work for a while to see whether it hurts.\n\nPlease ping this thread again when you have something to add. For\nnow, I'll demote this patch from 'next' to 'pu' when we rewind and\nrebuild 'next' post 2.12 release.\n\nThanks.\n"},{"id":"312594","messageId":"CA+55aFxY2CMJxdDs-LGChJFxHmhjnifEoi-aDV3AmDu8goM+rg@mail.gmail.com","threadId":"45095","inReplyTo":"20170210050237.gajicliueuvk6s5d@sigill.intra.peff.net","subject":"Re: [PATCH] mingw: use OpenSSL's SHA-1 routines","fromName":"Linus Torvalds","fromEmail":"torvalds@linux-foundation.org","sentAt":"2017-02-25T00:44:09Z","receivedAt":"2017-02-25T00:45:31Z","isPatch":true,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"On Thu, Feb 9, 2017 at 9:02 PM, Jeff King wrote:\n>\n> I think this is only half the story. A heavy-sha1 workload is faster,\n> which is good. But one of the original reasons to prefer blk-sha1 (at\n> least on Linux) is that resolving libcrypto.so symbols takes a\n> non-trivial amount of time. I just timed it again, and it seems to be\n> consistently 1ms slower to run \"git rev-parse --git-dir\" on my machine\n> (from the top-level of a repo).\n\nYes. It's also a horrible plain to profile those things.\n\nAvoiding openssl was a great thing, because it avoided a lot of crazy overhead.\n\nI suspect that most of the openssl win comes from using the actual SHA\ninstructions on modern CPU's. Because last I looked, the hand-coded\nassembly simply wasn't that much faster.\n\nWe could easily have some x86-specific library that just does \"use SHA\ninstructions if you have them, use blk-sha1 otherwise\".\n\nOf course, if we end up using the collision checking SHA libraries\nthis is all moot anyway.\n\n Linus\n"},{"id":"313038","messageId":"fa38e22e-3114-fab8-38ed-5f52125e81bb@kdbg.org","threadId":"45095","inReplyTo":"xmqqmvdbz23d.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH] mingw: use OpenSSL's SHA-1 routines","fromName":"Johannes Sixt","fromEmail":"j6t@kdbg.org","sentAt":"2017-03-02T06:07:22Z","receivedAt":"2017-03-02T06:10:22Z","isPatch":true,"sender":{"key":"j6t@kdbg.org","avatar":"https://avatars.githubusercontent.com/u/14810926?v=4"},"body":"Am 24.02.2017 um 22:54 schrieb Junio C Hamano:\n> Johannes Sixt writes:\n>> I'll use the patch for daily work for a while to see whether it hurts.\n>\n> Please ping this thread again when you have something to add. For\n> now, I'll demote this patch from 'next' to 'pu' when we rewind and\n> rebuild 'next' post 2.12 release.\n\nAs I already said in [1], I have no objection. I've used the patch in \nproduction for some time now and did not notice any slowdowns.\n\n[1] \nhttps://public-inbox.org/git/0080edd6-a515-2fe9-6266-b6f6bbedfdde@kdbg.org/\n\n-- Hannes\n\n"},{"id":"313080","messageId":"xmqqshmvd2ty.fsf@gitster.mtv.corp.google.com","threadId":"45095","inReplyTo":"fa38e22e-3114-fab8-38ed-5f52125e81bb@kdbg.org","subject":"Re: [PATCH] mingw: use OpenSSL's SHA-1 routines","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2017-03-02T17:07:53Z","receivedAt":"2017-03-02T17:44:26Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Johannes Sixt writes:\n\n> Am 24.02.2017 um 22:54 schrieb Junio C Hamano:\n>> Johannes Sixt writes:\n>>> I'll use the patch for daily work for a while to see whether it hurts.\n>>\n>> Please ping this thread again when you have something to add. For\n>> now, I'll demote this patch from 'next' to 'pu' when we rewind and\n>> rebuild 'next' post 2.12 release.\n>\n> ... I've used the patch in\n> production for some time now and did not notice any slowdowns.\n\nThanks. Dscho obviously thinks this is the right thing for Windows,\nand you agree. That's more than sufficient votes to make me feel\nsafe ;-)\n\nThanks.\n"}]}