{"thread":{"id":"51186","subject":"GIT issue while cloning (fatal: pack is corrupted (SHA1 mismatch)) !!!","startedAt":"2019-05-28T09:10:17Z","lastAt":"2019-06-28T23:55:36Z","messageCount":14,"participants":["Vanak, Ibrahim","Jeff King","Ævar Arnfjörð Bjarmason","Johannes Schindelin","Jeff Hostetler"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"376279","messageId":"TU4PR8401MB121664A8A588D799803F1E84E11E0@TU4PR8401MB1216.NAMPRD84.PROD.OUTLOOK.COM","threadId":"51186","inReplyTo":null,"subject":"GIT issue while cloning (fatal: pack is corrupted (SHA1 mismatch)) !!!","fromName":"Vanak, Ibrahim","fromEmail":"ibrahim.vanak@hpe.com","sentAt":"2019-05-28T09:10:12Z","receivedAt":"2019-05-28T09:10:17Z","isPatch":false,"sender":{"key":"ibrahim.vanak@hpe.com","avatar":null},"body":"Hello,\n\nWe are seeing issue with GIT 2.14 version. When we try to clone the repos, it is taking HUGE amount of time on HPUX, whereas on the linux machine with same network configuration, it's getting cloned in less than mins. So we want to know has anyone reported this issue? What is the fix for this? Has the fix been released for this? Whom should we contact for this? \n\nThanks & Regards,\nIbrahim Vanak\n\nBelow is the HPUX system where we are seeing issue, it is taking 1 hr 45 mins and later it failed:\n\nroot@sstl002.in.rdlabs.hpecorp.net# uname -a\nHP-UX sstl002 B.11.31 U ia64 0158936019 unlimited-user license\nroot@sstl002.in.rdlabs.hpecorp.net# time git clone git@github.hpe.com:HPUX/SysFaultMgmt.git\nCloning into 'SysFaultMgmt'...\nremote: Enumerating objects: 63627, done.\nremote: Total 63627 (delta 0), reused 0 (delta 0), pack-reused 63627\nReceiving objects: 100% (63627/63627), 681.90 MiB | 111.00 KiB/s, done.\nfatal: pack is corrupted (SHA1 mismatch)\nfatal: index-pack failed\n\nreal    104m3.373s\nuser    0m36.220s\nsys     0m6.090s\nYou have mail in /var/mail/root\nroot@sstl002.in.rdlabs.hpecorp.net#\n\nBelow is the system where it has passed in less than 3 mins:\n\n[root@chandana ~]# uname -a\nLinux chandana.in.rdlabs.hpecorp.net 3.10.0-327.el7.x86_64 #1 SMP Thu Nov 19 22:10:57 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux \n[root@chandana ~]# time git clone git@github.hpe.com:HPUX/SysFaultMgmt.git\nCloning into 'SysFaultMgmt'...\nremote: Enumerating objects: 63627, done.\nremote: Total 63627 (delta 0), reused 0 (delta 0), pack-reused 63627\nReceiving objects: 100% (63627/63627), 681.90 MiB | 5.59 MiB/s, done.\nResolving deltas: 100% (35495/35495), done.\nChecking connectivity... done.\nChecking out files: 100% (10634/10634), done.\n\nreal    2m46.517s\nuser    0m36.601s\nsys     0m10.503s\n\n"},{"id":"376281","messageId":"20190528093728.GA15106@sigill.intra.peff.net","threadId":"51186","inReplyTo":"TU4PR8401MB121664A8A588D799803F1E84E11E0@TU4PR8401MB1216.NAMPRD84.PROD.OUTLOOK.COM","subject":"Re: GIT issue while cloning (fatal: pack is corrupted (SHA1 mismatch)) !!!","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2019-05-28T09:37:28Z","receivedAt":"2019-05-28T09:37:32Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Tue, May 28, 2019 at 09:10:12AM +0000, Vanak, Ibrahim wrote:\n\n> We are seeing issue with GIT 2.14 version. When we try to clone the\n> repos, it is taking HUGE amount of time on HPUX, whereas on the linux\n> machine with same network configuration, it's getting cloned in less\n> than mins. So we want to know has anyone reported this issue? What is\n> the fix for this? Has the fix been released for this? Whom should we\n> contact for this?\n\nI don't know about the slowness, but...\n\n> Below is the HPUX system where we are seeing issue, it is taking 1 hr 45 mins and later it failed:\n> \n> root@sstl002.in.rdlabs.hpecorp.net# uname -a\n> HP-UX sstl002 B.11.31 U ia64 0158936019 unlimited-user license\n> root@sstl002.in.rdlabs.hpecorp.net# time git clone git@github.hpe.com:HPUX/SysFaultMgmt.git\n> Cloning into 'SysFaultMgmt'...\n> remote: Enumerating objects: 63627, done.\n> remote: Total 63627 (delta 0), reused 0 (delta 0), pack-reused 63627\n> Receiving objects: 100% (63627/63627), 681.90 MiB | 111.00 KiB/s, done.\n> fatal: pack is corrupted (SHA1 mismatch)\n> fatal: index-pack failed\n\nGit v2.14 uses the sha1collision-detection implementation of sha1 by\ndefault. That has a bug that was fixed recently with:\n\n commit 4125f78222749cb8fc91115abec3ac83e5dfb194\n Author: Ævar Arnfjörð Bjarmason \n Date: Tue May 14 00:17:01 2019 +0200\n \n sha1dc: update from upstream\n \n Update sha1dc from the latest version by the upstream\n maintainer[1]. See 07a20f569b (\"Makefile: fix unaligned loads in\n sha1dc with UBSan\", 2019-03-12) for the last update.\n \n This fixes an issue where HP-UX IA64 was wrongly detected as a\n Little-endian instead of a Big-endian system, see [2] and [3].\n \n 1. https://github.com/cr-marcstevens/sha1collisiondetection/commit/855827c583bc30645ba427885caa40c5b81764d2\n 2. https://public-inbox.org/git/603989bd-f86d-c61d-c6f5-fb6748a65ba9@siemens.com/\n 3. https://github.com/cr-marcstevens/sha1collisiondetection/pull/50\n\nwhich looks like it would impact your system. You can either:\n\n 1. Try v2.22.0-rc1, which will be the first release with that fix.\n\n 2. Try cherry-picking the various fixes on top of v2.14.0:\n\n git checkout v2.14.0\n git cherry-pick 23e37f8e9d5961c0c8d52ac481693d3fca5309ce\n git cherry-pick 07a20f569b4b1690e717eaac0954007a8edfbfc2\n git cherry-pick 4125f78222749cb8fc91115abec3ac83e5dfb194\n\n 3. Compile with another sha1 implementation. E.g.:\n\n # if you have openssl available; otherwise,\n # try BLK_SHA1\n echo 'OPENSSL_SHA1 = Yes' >config.mak\n make\n\n Note that you won't be protected from collision attacks, but those\n are still impractically expensive to mount at this point. It may be\n a good tradeoff until you can upgrade to a more recent Git.\n\n-Peff\n"},{"id":"376282","messageId":"87ef4inbzb.fsf@evledraar.gmail.com","threadId":"51186","inReplyTo":"20190528093728.GA15106@sigill.intra.peff.net","subject":"Re: GIT issue while cloning (fatal: pack is corrupted (SHA1 mismatch)) !!!","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2019-05-28T09:51:20Z","receivedAt":"2019-05-28T09:51:26Z","isPatch":false,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"\nOn Tue, May 28 2019, Jeff King wrote:\n\n> On Tue, May 28, 2019 at 09:10:12AM +0000, Vanak, Ibrahim wrote:\n>\n>> We are seeing issue with GIT 2.14 version. When we try to clone the\n>> repos, it is taking HUGE amount of time on HPUX, whereas on the linux\n>> machine with same network configuration, it's getting cloned in less\n>> than mins. So we want to know has anyone reported this issue? What is\n>> the fix for this? Has the fix been released for this? Whom should we\n>> contact for this?\n>\n> I don't know about the slowness, but...\n>\n>> Below is the HPUX system where we are seeing issue, it is taking 1 hr 45 mins and later it failed:\n>>\n>> root@sstl002.in.rdlabs.hpecorp.net# uname -a\n>> HP-UX sstl002 B.11.31 U ia64 0158936019 unlimited-user license\n>> root@sstl002.in.rdlabs.hpecorp.net# time git clone git@github.hpe.com:HPUX/SysFaultMgmt.git\n>> Cloning into 'SysFaultMgmt'...\n>> remote: Enumerating objects: 63627, done.\n>> remote: Total 63627 (delta 0), reused 0 (delta 0), pack-reused 63627\n>> Receiving objects: 100% (63627/63627), 681.90 MiB | 111.00 KiB/s, done.\n>> fatal: pack is corrupted (SHA1 mismatch)\n>> fatal: index-pack failed\n>\n> Git v2.14 uses the sha1collision-detection implementation of sha1 by\n> default. That has a bug that was fixed recently with:\n>\n> commit 4125f78222749cb8fc91115abec3ac83e5dfb194\n> Author: Ævar Arnfjörð Bjarmason \n> Date: Tue May 14 00:17:01 2019 +0200\n>\n> sha1dc: update from upstream\n>\n> Update sha1dc from the latest version by the upstream\n> maintainer[1]. See 07a20f569b (\"Makefile: fix unaligned loads in\n> sha1dc with UBSan\", 2019-03-12) for the last update.\n>\n> This fixes an issue where HP-UX IA64 was wrongly detected as a\n> Little-endian instead of a Big-endian system, see [2] and [3].\n>\n> 1. https://github.com/cr-marcstevens/sha1collisiondetection/commit/855827c583bc30645ba427885caa40c5b81764d2\n> 2. https://public-inbox.org/git/603989bd-f86d-c61d-c6f5-fb6748a65ba9@siemens.com/\n> 3. https://github.com/cr-marcstevens/sha1collisiondetection/pull/50\n>\n> which looks like it would impact your system. You can either:\n>\n> 1. Try v2.22.0-rc1, which will be the first release with that fix.\n>\n> 2. Try cherry-picking the various fixes on top of v2.14.0:\n>\n> git checkout v2.14.0\n> git cherry-pick 23e37f8e9d5961c0c8d52ac481693d3fca5309ce\n> git cherry-pick 07a20f569b4b1690e717eaac0954007a8edfbfc2\n> git cherry-pick 4125f78222749cb8fc91115abec3ac83e5dfb194\n>\n> 3. Compile with another sha1 implementation. E.g.:\n>\n> # if you have openssl available; otherwise,\n> # try BLK_SHA1\n> echo 'OPENSSL_SHA1 = Yes' >config.mak\n> make\n>\n> Note that you won't be protected from collision attacks, but those\n> are still impractically expensive to mount at this point. It may be\n> a good tradeoff until you can upgrade to a more recent Git.\n>\n> -Peff\n\nAs a follow-up perhaps we should hash_object_file_literally() early in\nmain() (or maybe just clone & init) to detect this issue & exit with\nsome \"zomg broken!\".\n\nVanak: Also, this issue suggests whoever compiled the package you're\nusing (you?) installed it with a failing \"make test\", so a lot of other\nthings may be broken...\n"},{"id":"376289","messageId":"nycvar.QRO.7.76.6.1905281214380.44@tvgsbejvaqbjf.bet","threadId":"51186","inReplyTo":"TU4PR8401MB121664A8A588D799803F1E84E11E0@TU4PR8401MB1216.NAMPRD84.PROD.OUTLOOK.COM","subject":"Re: GIT issue while cloning (fatal: pack is corrupted (SHA1 mismatch)) !!!","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2019-05-28T10:16:33Z","receivedAt":"2019-05-28T10:16:55Z","isPatch":false,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn Tue, 28 May 2019, Vanak, Ibrahim wrote:\n\n> We are seeing issue with GIT 2.14 version.\n\nYou definitely want to upgrade ASAP. Not only is the issue that you\nreported fixed, but two distinct vulnerabilities have been fixed since\nv2.14.0. Your version is still vulnerable.\n\nCiao,\nJohannes\n"},{"id":"376337","messageId":"TU4PR8401MB1216A7B920D40B5063123A94E11E0@TU4PR8401MB1216.NAMPRD84.PROD.OUTLOOK.COM","threadId":"51186","inReplyTo":"20190528093728.GA15106@sigill.intra.peff.net","subject":"RE: GIT issue while cloning (fatal: pack is corrupted (SHA1 mismatch)) !!!","fromName":"Vanak, Ibrahim","fromEmail":"ibrahim.vanak@hpe.com","sentAt":"2019-05-28T18:45:18Z","receivedAt":"2019-05-28T18:45:30Z","isPatch":false,"sender":{"key":"ibrahim.vanak@hpe.com","avatar":null},"body":"Thanks everyone for the inputs !!! upgrade of GIT to 2.21 worked.\n\nBUT still I have significant slowness(50 times slower than clone on linux machine) while cloning. HPUX box is having very good H/W configuration and network is also stable.\n\nDo you have any inputs on this.?\n\n-Ibrahim\n\n-----Original Message-----\nFrom: Jeff King [mailto:peff@peff.net] \nSent: Tuesday, May 28, 2019 3:07 PM\nTo: Vanak, Ibrahim We are seeing issue with GIT 2.14 version. When we try to clone the \n> repos, it is taking HUGE amount of time on HPUX, whereas on the linux \n> machine with same network configuration, it's getting cloned in less \n> than mins. So we want to know has anyone reported this issue? What is \n> the fix for this? Has the fix been released for this? Whom should we \n> contact for this?\n\nI don't know about the slowness, but...\n\n> Below is the HPUX system where we are seeing issue, it is taking 1 hr 45 mins and later it failed:\n> \n> root@sstl002.in.rdlabs.hpecorp.net# uname -a HP-UX sstl002 B.11.31 U \n> ia64 0158936019 unlimited-user license \n> root@sstl002.in.rdlabs.hpecorp.net# time git clone \n> git@github.hpe.com:HPUX/SysFaultMgmt.git\n> Cloning into 'SysFaultMgmt'...\n> remote: Enumerating objects: 63627, done.\n> remote: Total 63627 (delta 0), reused 0 (delta 0), pack-reused 63627 \n> Receiving objects: 100% (63627/63627), 681.90 MiB | 111.00 KiB/s, done.\n> fatal: pack is corrupted (SHA1 mismatch)\n> fatal: index-pack failed\n\nGit v2.14 uses the sha1collision-detection implementation of sha1 by default. That has a bug that was fixed recently with:\n\n commit 4125f78222749cb8fc91115abec3ac83e5dfb194\n Author: Ævar Arnfjörð Bjarmason \n Date: Tue May 14 00:17:01 2019 +0200\n \n sha1dc: update from upstream\n \n Update sha1dc from the latest version by the upstream\n maintainer[1]. See 07a20f569b (\"Makefile: fix unaligned loads in\n sha1dc with UBSan\", 2019-03-12) for the last update.\n \n This fixes an issue where HP-UX IA64 was wrongly detected as a\n Little-endian instead of a Big-endian system, see [2] and [3].\n \n 1. https://github.com/cr-marcstevens/sha1collisiondetection/commit/855827c583bc30645ba427885caa40c5b81764d2\n 2. https://public-inbox.org/git/603989bd-f86d-c61d-c6f5-fb6748a65ba9@siemens.com/\n 3. https://github.com/cr-marcstevens/sha1collisiondetection/pull/50\n\nwhich looks like it would impact your system. You can either:\n\n 1. Try v2.22.0-rc1, which will be the first release with that fix.\n\n 2. Try cherry-picking the various fixes on top of v2.14.0:\n\n git checkout v2.14.0\n git cherry-pick 23e37f8e9d5961c0c8d52ac481693d3fca5309ce\n git cherry-pick 07a20f569b4b1690e717eaac0954007a8edfbfc2\n git cherry-pick 4125f78222749cb8fc91115abec3ac83e5dfb194\n\n 3. Compile with another sha1 implementation. E.g.:\n\n # if you have openssl available; otherwise,\n # try BLK_SHA1\n echo 'OPENSSL_SHA1 = Yes' >config.mak\n make\n\n Note that you won't be protected from collision attacks, but those\n are still impractically expensive to mount at this point. It may be\n a good tradeoff until you can upgrade to a more recent Git.\n\n-Peff\n"},{"id":"376361","messageId":"20190528213004.GF24650@sigill.intra.peff.net","threadId":"51186","inReplyTo":"TU4PR8401MB1216A7B920D40B5063123A94E11E0@TU4PR8401MB1216.NAMPRD84.PROD.OUTLOOK.COM","subject":"Re: GIT issue while cloning (fatal: pack is corrupted (SHA1 mismatch)) !!!","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2019-05-28T21:30:04Z","receivedAt":"2019-05-28T21:30:08Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Tue, May 28, 2019 at 06:45:18PM +0000, Vanak, Ibrahim wrote:\n\n> BUT still I have significant slowness(50 times slower than clone on\n> linux machine) while cloning. HPUX box is having very good H/W\n> configuration and network is also stable.\n\nFrom your output:\n\n> [hpux]\n> Receiving objects: 100% (63627/63627), 681.90 MiB | 111.00 KiB/s, done.\n> [linux]\n> Receiving objects: 100% (63627/63627), 681.90 MiB | 5.59 MiB/s, done.\n\nThe main things going on there are:\n\n 1. we're getting the bytes over the network\n\n 2. we're computing a sha1 on the stream we're getting\n\n 3. we're zlib inflating any non-delta objects we find and computing\n their sha1\n\n 4. we're checking the filesystem to see if we have other copies of any\n of those objects\n\nFor (2) and (3) it could be that the sha1 implementation is not quite as\nfast. But 50x is much larger than I'd expect. If you've built Git from\nsource, you could try running \"t/helper/test-sha1 \nCc: git@vger.kernel.org\nSubject: Re: GIT issue while cloning (fatal: pack is corrupted (SHA1 mismatch)) !!!\n\nOn Tue, May 28, 2019 at 06:45:18PM +0000, Vanak, Ibrahim wrote:\n\n> BUT still I have significant slowness(50 times slower than clone on \n> linux machine) while cloning. HPUX box is having very good H/W \n> configuration and network is also stable.\n\nFrom your output:\n\n> [hpux]\n> Receiving objects: 100% (63627/63627), 681.90 MiB | 111.00 KiB/s, done.\n> [linux]\n> Receiving objects: 100% (63627/63627), 681.90 MiB | 5.59 MiB/s, done.\n\nThe main things going on there are:\n\n 1. we're getting the bytes over the network\n\n 2. we're computing a sha1 on the stream we're getting\n\n 3. we're zlib inflating any non-delta objects we find and computing\n their sha1\n\n 4. we're checking the filesystem to see if we have other copies of any\n of those objects\n\nFor (2) and (3) it could be that the sha1 implementation is not quite as fast. But 50x is much larger than I'd expect. If you've built Git from source, you could try running \"t/helper/test-sha1 \n\nHEAD is now at 39b6110f Commiting to isvn for WEB1303_1123 SFM release\n\nreal    8m17.234s\nuser    0m7.070s\nsys     0m1.960s\n\n==============================================\nCheckout of the same branch on Linux as below:\n==============================================\n[root@chandana SysFaultMgmt]# time git checkout remotes/origin/11.23/sfm_1123_main\nChecking out files: 100% (12949/12949), done.\nNote: checking out 'remotes/origin/11.23/sfm_1123_main'.\n\nYou are in 'detached HEAD' state. You can look around, make experimental\nchanges and commit them, and you can discard any commits you make in this\nstate without impacting any branches by performing another checkout.\n\nIf you want to create a new branch to retain commits you create, you may\ndo so (now or later) by using -b with the checkout command again. Example:\n\n  git checkout -b \n\nHEAD is now at 39b6110... Commiting to isvn for WEB1303_1123 SFM release\n\nreal    0m4.709s\nuser    0m3.539s\nsys     0m1.098s\n[root@chandana SysFaultMgmt]#\n\n-----Original Message-----\nFrom: Vanak, Ibrahim \nSent: Wednesday, May 29, 2019 10:59 AM\nTo: Jeff King \nCc: git@vger.kernel.org\nSubject: RE: GIT issue while cloning (fatal: pack is corrupted (SHA1 mismatch)) !!!\n\nHello Jeff,\n\nThanks again for the response. \n\nI am not building GIT and also don't know either how to build. I am just pulling up the binaries from http://hpux.connect.org.uk/. \n\nBoth HPUX and linux machine are in same network/subnet so network shouldn't be a problem here. ssh to HPUX box is super quick and box is having very high configuration. There is no NFS involved here.\n\nI am surprised on slowness.... If you get a chance to look more into this, please.\n\nThanks & Regards,\nIbrahim Vanak\n\n-----Original Message-----\nFrom: Jeff King [mailto:peff@peff.net]\nSent: Wednesday, May 29, 2019 3:00 AM\nTo: Vanak, Ibrahim \nCc: git@vger.kernel.org\nSubject: Re: GIT issue while cloning (fatal: pack is corrupted (SHA1 mismatch)) !!!\n\nOn Tue, May 28, 2019 at 06:45:18PM +0000, Vanak, Ibrahim wrote:\n\n> BUT still I have significant slowness(50 times slower than clone on \n> linux machine) while cloning. HPUX box is having very good H/W \n> configuration and network is also stable.\n\nFrom your output:\n\n> [hpux]\n> Receiving objects: 100% (63627/63627), 681.90 MiB | 111.00 KiB/s, done.\n> [linux]\n> Receiving objects: 100% (63627/63627), 681.90 MiB | 5.59 MiB/s, done.\n\nThe main things going on there are:\n\n  1. we're getting the bytes over the network\n\n  2. we're computing a sha1 on the stream we're getting\n\n  3. we're zlib inflating any non-delta objects we find and computing\n     their sha1\n\n  4. we're checking the filesystem to see if we have other copies of any\n     of those objects\n\nFor (2) and (3) it could be that the sha1 implementation is not quite as fast. But 50x is much larger than I'd expect. If you've built Git from source, you could try running \"t/helper/test-sha1 After cloning when I tried to checkout a branch on HPUX and Linux, I\n> still significant time difference there as well even though network is\n> not involve here. Do you suspect anything with HPUX OS? Do you have\n> any mechanism to find out who is the causing gigantic delay ?\n\nIf it's happening for just a local checkout operation, I'd suspect very\npoor filesystem performance. On Linux, I'd run it under \"perf\" to see\nwhere the time is going. Or possibly \"strace -T\" to time system calls. I\ndon't know what tools are available under HP-UX for that kind of\nprofiling.\n\n-Peff\n"},{"id":"377013","messageId":"TU4PR8401MB12163C00148C39F0083C732DE1ED0@TU4PR8401MB1216.NAMPRD84.PROD.OUTLOOK.COM","threadId":"51186","inReplyTo":"20190530115742.GC31607@sigill.intra.peff.net","subject":"RE: GIT issue while cloning (fatal: pack is corrupted (SHA1 mismatch)) !!!","fromName":"Vanak, Ibrahim","fromEmail":"ibrahim.vanak@hpe.com","sentAt":"2019-06-11T16:39:24Z","receivedAt":"2019-06-11T16:39:35Z","isPatch":false,"sender":{"key":"ibrahim.vanak@hpe.com","avatar":null},"body":"Thank Peff. Unfortunately I am living with this issue, no one able to find out what is causing this significant delay. Re-initiating this thread if someone can help on this.\n\nThanks & Regards,\nIbrahim Vanak\n\n-----Original Message-----\nFrom: Jeff King [mailto:peff@peff.net] \nSent: Thursday, May 30, 2019 5:28 PM\nTo: Vanak, Ibrahim \nCc: git@vger.kernel.org\nSubject: Re: GIT issue while cloning (fatal: pack is corrupted (SHA1 mismatch)) !!!\n\nOn Wed, May 29, 2019 at 09:06:18AM +0000, Vanak, Ibrahim wrote:\n\n> After cloning when I tried to checkout a branch on HPUX and Linux, I \n> still significant time difference there as well even though network is \n> not involve here. Do you suspect anything with HPUX OS? Do you have \n> any mechanism to find out who is the causing gigantic delay ?\n\nIf it's happening for just a local checkout operation, I'd suspect very poor filesystem performance. On Linux, I'd run it under \"perf\" to see where the time is going. Or possibly \"strace -T\" to time system calls. I don't know what tools are available under HP-UX for that kind of profiling.\n\n-Peff\n"},{"id":"377409","messageId":"TU4PR8401MB1216962E3F135E850E81E407E1EA0@TU4PR8401MB1216.NAMPRD84.PROD.OUTLOOK.COM","threadId":"51186","inReplyTo":"TU4PR8401MB12163C00148C39F0083C732DE1ED0@TU4PR8401MB1216.NAMPRD84.PROD.OUTLOOK.COM","subject":"RE: GIT issue while cloning (fatal: pack is corrupted (SHA1 mismatch)) !!!","fromName":"Vanak, Ibrahim","fromEmail":"ibrahim.vanak@hpe.com","sentAt":"2019-06-18T14:31:47Z","receivedAt":"2019-06-18T14:31:52Z","isPatch":false,"sender":{"key":"ibrahim.vanak@hpe.com","avatar":null},"body":"Hello ALL again,\n\nHas anyone tested the performance of GIT on HP-UX platform? Can someone please look into the issue we are seeing.\n\nThanks,\nIbrahim\n\n-----Original Message-----\nFrom: Vanak, Ibrahim \nSent: Tuesday, June 11, 2019 10:09 PM\nTo: Jeff King \nCc: git@vger.kernel.org\nSubject: RE: GIT issue while cloning (fatal: pack is corrupted (SHA1 mismatch)) !!!\n\nThank Peff. Unfortunately I am living with this issue, no one able to find out what is causing this significant delay. Re-initiating this thread if someone can help on this.\n\nThanks & Regards,\nIbrahim Vanak\n\n-----Original Message-----\nFrom: Jeff King [mailto:peff@peff.net]\nSent: Thursday, May 30, 2019 5:28 PM\nTo: Vanak, Ibrahim \nCc: git@vger.kernel.org\nSubject: Re: GIT issue while cloning (fatal: pack is corrupted (SHA1 mismatch)) !!!\n\nOn Wed, May 29, 2019 at 09:06:18AM +0000, Vanak, Ibrahim wrote:\n\n> After cloning when I tried to checkout a branch on HPUX and Linux, I \n> still significant time difference there as well even though network is \n> not involve here. Do you suspect anything with HPUX OS? Do you have \n> any mechanism to find out who is the causing gigantic delay ?\n\nIf it's happening for just a local checkout operation, I'd suspect very poor filesystem performance. On Linux, I'd run it under \"perf\" to see where the time is going. Or possibly \"strace -T\" to time system calls. I don't know what tools are available under HP-UX for that kind of profiling.\n\n-Peff\n"},{"id":"377427","messageId":"6e78d3b7-07f2-f635-1dac-c4d98961b8eb@jeffhostetler.com","threadId":"51186","inReplyTo":"TU4PR8401MB1216962E3F135E850E81E407E1EA0@TU4PR8401MB1216.NAMPRD84.PROD.OUTLOOK.COM","subject":"Re: GIT issue while cloning (fatal: pack is corrupted (SHA1 mismatch)) !!!","fromName":"Jeff Hostetler","fromEmail":"git@jeffhostetler.com","sentAt":"2019-06-18T17:15:02Z","receivedAt":"2019-06-18T17:15:06Z","isPatch":false,"sender":{"key":"git@jeffhostetler.com","avatar":null},"body":"\n\nOn 6/18/2019 10:31 AM, Vanak, Ibrahim wrote:\n> Hello ALL again,\n> \n> Has anyone tested the performance of GIT on HP-UX platform? Can someone please look into the issue we are seeing.\n> \n> Thanks,\n> Ibrahim\n\n\nYou might try setting some of the GIT_TRACE* environment variables\nlisted in [1] on both your HP-UX and Linux machines and see if you\ncan isolate any performance differences between them.\n\n\n[1] https://git-scm.com/book/en/v2/Git-Internals-Environment-Variables\n"},{"id":"378275","messageId":"TU4PR8401MB1216FC788B6A421F8FD82B2EE1FC0@TU4PR8401MB1216.NAMPRD84.PROD.OUTLOOK.COM","threadId":"51186","inReplyTo":"6e78d3b7-07f2-f635-1dac-c4d98961b8eb@jeffhostetler.com","subject":"RE: GIT issue while cloning (fatal: pack is corrupted (SHA1 mismatch)) !!!","fromName":"Vanak, Ibrahim","fromEmail":"ibrahim.vanak@hpe.com","sentAt":"2019-06-28T16:18:37Z","receivedAt":"2019-06-28T16:18:50Z","isPatch":false,"sender":{"key":"ibrahim.vanak@hpe.com","avatar":null},"body":"Hello ALL,\n\nI did try GIT_TRACE* environment variables but couldn't isolate the problem. \n\nBut HPUX Dev team have seen one significance difference, when they were triaging this issue, the ways GIT operations handled by 2 OSs: On linux, around 40 processes are spawned for handling a GIT operation whereas on HPUX only one process manages the operation. Is there something special being written for linux on GIT code ?\n\nCan you please have a look into this.\n\nThanks & Regards,\nIbrahim Vanak \n\n-----Original Message-----\nFrom: Jeff Hostetler [mailto:git@jeffhostetler.com] \nSent: Tuesday, June 18, 2019 10:45 PM\nTo: Vanak, Ibrahim ; Jeff King \nCc: git@vger.kernel.org\nSubject: Re: GIT issue while cloning (fatal: pack is corrupted (SHA1 mismatch)) !!!\n\n\n\nOn 6/18/2019 10:31 AM, Vanak, Ibrahim wrote:\n> Hello ALL again,\n> \n> Has anyone tested the performance of GIT on HP-UX platform? Can someone please look into the issue we are seeing.\n> \n> Thanks,\n> Ibrahim\n\n\nYou might try setting some of the GIT_TRACE* environment variables listed in [1] on both your HP-UX and Linux machines and see if you can isolate any performance differences between them.\n\n\n[1] https://git-scm.com/book/en/v2/Git-Internals-Environment-Variables \n"},{"id":"378304","messageId":"20190628235532.GA2625@sigill.intra.peff.net","threadId":"51186","inReplyTo":"TU4PR8401MB1216FC788B6A421F8FD82B2EE1FC0@TU4PR8401MB1216.NAMPRD84.PROD.OUTLOOK.COM","subject":"Re: GIT issue while cloning (fatal: pack is corrupted (SHA1 mismatch)) !!!","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2019-06-28T23:55:33Z","receivedAt":"2019-06-28T23:55:36Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Fri, Jun 28, 2019 at 04:18:37PM +0000, Vanak, Ibrahim wrote:\n\n> But HPUX Dev team have seen one significance difference, when they\n> were triaging this issue, the ways GIT operations handled by 2 OSs: On\n> linux, around 40 processes are spawned for handling a GIT operation\n> whereas on HPUX only one process manages the operation. Is there\n> something special being written for linux on GIT code ?\n\nOn Linux (and any platform that supports pthreads), Git will do the\nlstat() operations to refresh the index in parallel. See the\ncore.preloadIndex option (which I think defaults to true).\n\nIf you have a very high-latency lstat() and your build does not have\npthread support, that could be a significant difference.\n\nThat might be worth exploring.\n\n-Peff\n"}]}