{"thread":{"id":"32089","subject":"[regression] Newer gits cannot clone any remote repos","startedAt":"2012-11-12T16:32:44Z","lastAt":"2012-11-21T19:20:40Z","messageCount":17,"participants":["Douglas Mencken","Kevin","Ramsay Jones","Torsten Bögershausen","Andreas Schwab","Nguyen Thai Ngoc Duy"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"202958","messageId":"CACYvZ7jPd0_XD6YVdfJ2AnKRnKewmzX4uu7w3zt+_gK+qU49gQ@mail.gmail.com","threadId":"32089","inReplyTo":null,"subject":"[regression] Newer gits cannot clone any remote repos","fromName":"Douglas Mencken","fromEmail":"dougmencken@gmail.com","sentAt":"2012-11-12T16:32:44Z","receivedAt":"2012-11-12T16:32:44Z","isPatch":false,"sender":{"key":"dougmencken@gmail.com","avatar":"https://gravatar.com/avatar/6e6892c4aed51719e6c271923f2ab9e3d9e3418dd138c4aae9dd9fee70d6965a?d=mp&s=160"},"body":"*Any* git clone fails with:\n\nfatal: premature end of pack file, 106 bytes missing\nfatal: index-pack failed\n\nAt first, I tried 1.8.0, and it failed. Then I tried to build 1.7.10.5\nthen, and it worked. Then I tried 1.7.12.2, but it fails the same way\nas 1.8.0.\nSo I decided to git bisect.\n\nb8a2486f1524947f232f657e9f2ebf44e3e7a243 is the first bad commit\n``index-pack: support multithreaded delta resolving''\n\n--------------------------------------------------------------------------------\n\nb8a2486f1524947f232f657e9f2ebf44e3e7a243 is the first bad commit\ncommit b8a2486f1524947f232f657e9f2ebf44e3e7a243\nAuthor: Nguyễn Thái Ngọc Duy \nDate: Sun May 6 19:31:55 2012 +0700\n\n index-pack: support multithreaded delta resolving\n\n This puts delta resolving on each base on a separate thread, one base\n cache per thread. Per-thread data is grouped in struct thread_local.\n When running with nr_threads == 1, no pthreads calls are made. The\n system essentially runs in non-thread mode.\n\n An experiment on a Xeon 24 core machine with git.git shows that\n performance does not increase proportional to the number of cores. So\n by default, we use maximum 3 cores. Some numbers with --threads from 1\n to 16:\n\n 1..4\n real 0m8.003s 0m5.307s 0m4.321s 0m3.830s\n user 0m7.720s 0m8.009s 0m8.133s 0m8.305s\n sys 0m0.224s 0m0.372s 0m0.360s 0m0.360s\n\n 5..8\n real 0m3.727s 0m3.604s 0m3.332s 0m3.369s\n user 0m9.361s 0m9.817s 0m9.525s 0m9.769s\n sys 0m0.584s 0m0.624s 0m0.540s 0m0.560s\n\n 9..12\n real 0m3.036s 0m3.139s 0m3.177s 0m2.961s\n user 0m8.977s 0m10.205s 0m9.737s 0m10.073s\n sys 0m0.596s 0m0.680s 0m0.684s 0m0.680s\n\n 13..16\n real 0m2.985s 0m2.894s 0m2.975s 0m2.971s\n user 0m9.825s 0m10.573s 0m10.833s 0m11.361s\n sys 0m0.788s 0m0.732s 0m0.904s 0m1.016s\n\n On an Intel dual core and linux-2.6.git\n\n 1..4\n real 2m37.789s 2m7.963s 2m0.920s 1m58.213s\n user 2m28.415s 2m52.325s 2m50.176s 2m41.187s\n sys 0m7.808s 0m11.181s 0m11.224s 0m10.731s\n\n Thanks Ramsay Jones for troubleshooting and support on MinGW platform.\n\n Signed-off-by: Nguyễn Thái Ngọc Duy \n Signed-off-by: Junio C Hamano \n\n:040000 040000 e5597fe041cd7985648030c8cf1cc35ea57786bb\n6e64f3e76b350c04c9d0c435e0a53b89e0821339 M Documentation\n:100644 100644 cf2c40b44f8383d002235400660d76ef7f6de33c\ne41955ff955693ddc78722c14fff2a6e98663a46 M Makefile\n:040000 040000 1e6b5ac5f11bf7ba3eff7b5be4410bad379feee6\n85bc79414f00b7fc3698c069161ea10039641082 M builtin\n:040000 040000 04916545d959c8af772cd3a2de5e6b470529bea1\n325c8c748dd41d469135db84c7621d8225cc1cdd M t\n\n$ git bisect log\n git bisect start\n # bad: [d8cf053dacb4f78920c112d10c7be21e4f5a5817] Git 1.7.12.2\n git bisect bad d8cf053dacb4f78920c112d10c7be21e4f5a5817\n # good: [785ee4960c3d334cbc2b17ab74d2cebdf1b4db64] Git 1.7.10.5\n git bisect good 785ee4960c3d334cbc2b17ab74d2cebdf1b4db64\n # bad: [d692d34653f74be6b16add3e993e957f33fe049b] Merge branch\n'rs/git-blame-mapcar-mapc'\n git bisect bad d692d34653f74be6b16add3e993e957f33fe049b\n # good: [73ff8cf784b6ee447072fad6c06fd0eef0e9c9f6] Merge branch\n'lp/diffstat-with-graph'\n git bisect good 73ff8cf784b6ee447072fad6c06fd0eef0e9c9f6\n # bad: [7903e66a3e10e98f29fc9524c13274376bae5303] Merge branch\n'mh/test-keep-prove-cache'\n git bisect bad 7903e66a3e10e98f29fc9524c13274376bae5303\n # good: [499e7b31509cfbb59dcb2a046f8e2fd1a3e73d6f] Merge branch\n'jc/install-no-hardlinks'\n git bisect good 499e7b31509cfbb59dcb2a046f8e2fd1a3e73d6f\n # bad: [2e464dd5b220d4d2d8a16d5c43efe7af3c9adda9] Merge branch\n'rs/xdiff-lose-emit-func'\n git bisect bad 2e464dd5b220d4d2d8a16d5c43efe7af3c9adda9\n # good: [8fbe0db4fce68ed8239742b14e9d77d45201870b] Merge branch\n'master' of git://github.com/git-l10n/git-po\n git bisect good 8fbe0db4fce68ed8239742b14e9d77d45201870b\n # bad: [2b26b65f9abc77c4af87626452005a73edda0c8f] git-svn: clarify\nthe referent of dcommit's optional argument\n git bisect bad 2b26b65f9abc77c4af87626452005a73edda0c8f\n # bad: [cc13431a49800a6a1d2b7ff0b94f67da0fecdbab] Merge branch\n'nd/threaded-index-pack'\n git bisect bad cc13431a49800a6a1d2b7ff0b94f67da0fecdbab\n # bad: [b8a2486f1524947f232f657e9f2ebf44e3e7a243] index-pack:\nsupport multithreaded delta resolving\n git bisect bad b8a2486f1524947f232f657e9f2ebf44e3e7a243\n # good: [5272f75587abb4cb396059ecbf1a6130bb2e69d3] index-pack:\nrestructure pack processing into three main functions\n git bisect good 5272f75587abb4cb396059ecbf1a6130bb2e69d3\n\n$ git bisect reset\nPrevious HEAD position was 5272f75... index-pack: restructure pack\nprocessing into three main functions\nHEAD is now at d8cf053... Git 1.7.12.2\n"},{"id":"202963","messageId":"CAO54GHC9ibTwxqLnk1vGSo9R011HE05JOL1mgpTGzeWX7cCcwQ@mail.gmail.com","threadId":"32089","inReplyTo":"CACYvZ7jPd0_XD6YVdfJ2AnKRnKewmzX4uu7w3zt+_gK+qU49gQ@mail.gmail.com","subject":"Re: [regression] Newer gits cannot clone any remote repos","fromName":"Kevin","fromEmail":"ikke@ikke.info","sentAt":"2012-11-12T17:12:16Z","receivedAt":"2012-11-12T17:12:16Z","isPatch":false,"sender":{"key":"ikke@ikke.info","avatar":"https://gravatar.com/avatar/60a0d08eeccb16914fdbfc7f3441548dc827dc436d5c6900ec6600e274e53c5b?d=mp&s=160"},"body":"Maybe handy to say that you're on a Powerpc platform.\n\nOn Mon, Nov 12, 2012 at 5:32 PM, Douglas Mencken wrote:\n> *Any* git clone fails with:\n>\n> fatal: premature end of pack file, 106 bytes missing\n> fatal: index-pack failed\n>\n> At first, I tried 1.8.0, and it failed. Then I tried to build 1.7.10.5\n> then, and it worked. Then I tried 1.7.12.2, but it fails the same way\n> as 1.8.0.\n> So I decided to git bisect.\n>\n"},{"id":"202964","messageId":"CACYvZ7hPAejfjqXQFEHO0DySPN9xmvGgvK7gqj=4qX-XKz0VHQ@mail.gmail.com","threadId":"32089","inReplyTo":"CAO54GHA=r8WUK3okm2imVkoB=SuZ=3o+wwdwLr_rmPXhdFzG_g@mail.gmail.com","subject":"Re: [regression] Newer gits cannot clone any remote repos","fromName":"Douglas Mencken","fromEmail":"dougmencken@gmail.com","sentAt":"2012-11-12T17:13:19Z","receivedAt":"2012-11-12T17:13:19Z","isPatch":false,"sender":{"key":"dougmencken@gmail.com","avatar":"https://gravatar.com/avatar/6e6892c4aed51719e6c271923f2ab9e3d9e3418dd138c4aae9dd9fee70d6965a?d=mp&s=160"},"body":"> Maybe handy to say that you're on a Powerpc platform.\n\nMy host, build and target is: powerpc-linux-gnu\n"},{"id":"202985","messageId":"CACYvZ7jVRcnR0WQmc3P+WNEterEHFkUkVGH_mDXdHK-rbCvGbA@mail.gmail.com","threadId":"32089","inReplyTo":"CAO54GHC9ibTwxqLnk1vGSo9R011HE05JOL1mgpTGzeWX7cCcwQ@mail.gmail.com","subject":"Re: [regression] Newer gits cannot clone any remote repos","fromName":"Douglas Mencken","fromEmail":"dougmencken@gmail.com","sentAt":"2012-11-12T19:35:22Z","receivedAt":"2012-11-12T19:35:22Z","isPatch":false,"sender":{"key":"dougmencken@gmail.com","avatar":"https://gravatar.com/avatar/6e6892c4aed51719e6c271923f2ab9e3d9e3418dd138c4aae9dd9fee70d6965a?d=mp&s=160"},"body":"> On Mon, Nov 12, 2012 at 12:12 PM, Kevin wrote:\n> > Maybe handy to say that you're on a Powerpc platform.\n\nOh, and yes, I'm on 2 x 2-core (\"4-core\") machine.\n"},{"id":"203129","messageId":"50A2978D.6080805@ramsay1.demon.co.uk","threadId":"32089","inReplyTo":"CACYvZ7jPd0_XD6YVdfJ2AnKRnKewmzX4uu7w3zt+_gK+qU49gQ@mail.gmail.com","subject":"Re: [regression] Newer gits cannot clone any remote repos","fromName":"Ramsay Jones","fromEmail":"ramsay@ramsay1.demon.co.uk","sentAt":"2012-11-13T18:55:09Z","receivedAt":"2012-11-13T18:55:09Z","isPatch":false,"sender":{"key":"ramsay@ramsayjones.plus.com","avatar":"https://avatars.githubusercontent.com/u/33702710?v=4"},"body":"Douglas Mencken wrote:\n> *Any* git clone fails with:\n> \n> fatal: premature end of pack file, 106 bytes missing\n> fatal: index-pack failed\n> \n> At first, I tried 1.8.0, and it failed. Then I tried to build 1.7.10.5\n> then, and it worked. Then I tried 1.7.12.2, but it fails the same way\n> as 1.8.0.\n> So I decided to git bisect.\n> \n> b8a2486f1524947f232f657e9f2ebf44e3e7a243 is the first bad commit\n> ``index-pack: support multithreaded delta resolving''\n\nThis looks like the same problem I had on cygwin, which lead to\ncommit c0f86547c (\"index-pack: Disable threading on cygwin\", 26-06-2012).\n\nI didn't notice which platform you are on, but maybe you also have a\nthread-unsafe pread()? Could you try re-building git with the\nNO_THREAD_SAFE_PREAD build variable set?\n\nHTH.\n\nATB,\nRamsay Jones\n"},{"id":"203138","messageId":"CACYvZ7jMC5xw4LxiuG5m+=grpQEg+wZb_7BaU4Xn-r7ix=S-bw@mail.gmail.com","threadId":"32089","inReplyTo":"50A2978D.6080805@ramsay1.demon.co.uk","subject":"Re: [regression] Newer gits cannot clone any remote repos","fromName":"Douglas Mencken","fromEmail":"dougmencken@gmail.com","sentAt":"2012-11-13T19:55:10Z","receivedAt":"2012-11-13T19:55:10Z","isPatch":false,"sender":{"key":"dougmencken@gmail.com","avatar":"https://gravatar.com/avatar/6e6892c4aed51719e6c271923f2ab9e3d9e3418dd138c4aae9dd9fee70d6965a?d=mp&s=160"},"body":"> Could you try re-building git with the\n> NO_THREAD_SAFE_PREAD build variable set?\n\nYeah! It works!!!\n\n--- evil/Makefile\n+++ good/Makefile\n@@ -957,6 +957,7 @@\n \tHAVE_PATHS_H = YesPlease\n \tLIBC_CONTAINS_LIBINTL = YesPlease\n \tHAVE_DEV_TTY = YesPlease\n+\tNO_THREAD_SAFE_PREAD = YesPlease\n endif\n ifeq ($(uname_S),GNU/kFreeBSD)\n \tNO_STRLCPY = YesPlease\n\nWith this, I do have correctly working git clone.\n"},{"id":"203143","messageId":"CACYvZ7h7eTM0RGchG9SWRoEjvzLN0hYq6ARiJhyXov1VVX+UKg@mail.gmail.com","threadId":"32089","inReplyTo":"CACYvZ7jMC5xw4LxiuG5m+=grpQEg+wZb_7BaU4Xn-r7ix=S-bw@mail.gmail.com","subject":"Re: [regression] Newer gits cannot clone any remote repos","fromName":"Douglas Mencken","fromEmail":"dougmencken@gmail.com","sentAt":"2012-11-13T20:33:43Z","receivedAt":"2012-11-13T20:33:43Z","isPatch":false,"sender":{"key":"dougmencken@gmail.com","avatar":"https://gravatar.com/avatar/6e6892c4aed51719e6c271923f2ab9e3d9e3418dd138c4aae9dd9fee70d6965a?d=mp&s=160"},"body":"By the way, it is the job of autotools to check such stuff, isn't it?\n"},{"id":"203148","messageId":"50A2B2DF.1080704@web.de","threadId":"32089","inReplyTo":"50A2978D.6080805@ramsay1.demon.co.uk","subject":"Re: [regression] Newer gits cannot clone any remote repos","fromName":"Torsten Bögershausen","fromEmail":"tboegi@web.de","sentAt":"2012-11-13T20:51:43Z","receivedAt":"2012-11-13T20:51:43Z","isPatch":false,"sender":{"key":"tboegi@web.de","avatar":"https://avatars.githubusercontent.com/u/7138363?v=4"},"body":"On 13.11.12 19:55, Ramsay Jones wrote:\n> Douglas Mencken wrote:\n>> *Any* git clone fails with:\n>>\n>> fatal: premature end of pack file, 106 bytes missing\n>> fatal: index-pack failed\n>>\n>> At first, I tried 1.8.0, and it failed. Then I tried to build 1.7.10.5\n>> then, and it worked. Then I tried 1.7.12.2, but it fails the same way\n>> as 1.8.0.\n>> So I decided to git bisect.\n>>\n>> b8a2486f1524947f232f657e9f2ebf44e3e7a243 is the first bad commit\n>> ``index-pack: support multithreaded delta resolving''\n> \n> This looks like the same problem I had on cygwin, which lead to\n> commit c0f86547c (\"index-pack: Disable threading on cygwin\", 26-06-2012).\n> \n> I didn't notice which platform you are on, but maybe you also have a\n> thread-unsafe pread()? Could you try re-building git with the\n> NO_THREAD_SAFE_PREAD build variable set?\n> \n> HTH.\n> \n> ATB,\n> Ramsay Jones\n\nThis is interesting.\nI had the same problem on a PowerPC \n(Old PowerBook G4 running Linux).\n\nUsing NO_THREAD_SAFE_PREAD helped, thanks for the hint.\n(After recompiling without NO_THREAD_SAFE_PREAD I could clone\nfrom this machine again, so the problem is not really reproducable)\n\nAre there more people running PowerPC (on the server side) ?\n/Torsten\n\n \n"},{"id":"203151","messageId":"m2pq3h9ll1.fsf@igel.home","threadId":"32089","inReplyTo":"50A2B2DF.1080704@web.de","subject":"Re: [regression] Newer gits cannot clone any remote repos","fromName":"Andreas Schwab","fromEmail":"schwab@linux-m68k.org","sentAt":"2012-11-13T22:36:42Z","receivedAt":"2012-11-13T22:36:42Z","isPatch":false,"sender":{"key":"schwab@linux-m68k.org","avatar":"https://avatars.githubusercontent.com/u/2175493?v=4"},"body":"Torsten Bögershausen writes:\n\n> Are there more people running PowerPC (on the server side) ?\n\nI cannot reproduce the problem (on openSUSE 12.2).\n\nAndreas.\n\n-- \nAndreas Schwab, schwab@linux-m68k.org\nGPG Key fingerprint = 58CA 54C7 6D53 942B 1756 01D3 44D5 214B 8276 4ED5\n\"And now for something completely different.\"\n"},{"id":"203194","messageId":"CACYvZ7jwjVsW4=QSbxFVL8N269DE4=tv8_WvZ0gVOw6B+WLP=w@mail.gmail.com","threadId":"32089","inReplyTo":"m2pq3h9ll1.fsf@igel.home","subject":"Re: [regression] Newer gits cannot clone any remote repos","fromName":"Douglas Mencken","fromEmail":"dougmencken@gmail.com","sentAt":"2012-11-14T16:19:16Z","receivedAt":"2012-11-14T16:19:16Z","isPatch":false,"sender":{"key":"dougmencken@gmail.com","avatar":"https://gravatar.com/avatar/6e6892c4aed51719e6c271923f2ab9e3d9e3418dd138c4aae9dd9fee70d6965a?d=mp&s=160"},"body":"> I cannot reproduce the problem (on openSUSE 12.2).\n\nYou do need multiple CPU/multi-core machine, as I got it.\n"},{"id":"203195","messageId":"mvmvcd8f8ml.fsf@g208.suse.de","threadId":"32089","inReplyTo":"CACYvZ7jwjVsW4=QSbxFVL8N269DE4=tv8_WvZ0gVOw6B+WLP=w@mail.gmail.com","subject":"Re: [regression] Newer gits cannot clone any remote repos","fromName":"Andreas Schwab","fromEmail":"schwab@linux-m68k.org","sentAt":"2012-11-14T16:32:18Z","receivedAt":"2012-11-14T16:32:18Z","isPatch":false,"sender":{"key":"schwab@linux-m68k.org","avatar":"https://avatars.githubusercontent.com/u/2175493?v=4"},"body":"Douglas Mencken writes:\n\n>> I cannot reproduce the problem (on openSUSE 12.2).\n>\n> You do need multiple CPU/multi-core machine, as I got it.\n\nWhich is what I have.\n\nAndreas.\n\n-- \nAndreas Schwab, schwab@linux-m68k.org\nGPG Key fingerprint = 58CA 54C7 6D53 942B 1756 01D3 44D5 214B 8276 4ED5\n\"And now for something completely different.\"\n"},{"id":"203205","messageId":"CACYvZ7g7nCgQmcnwJYxvx5hJ+Y8rCQOBQyWYLZOt8NT5BTwTrw@mail.gmail.com","threadId":"32089","inReplyTo":"mvmvcd8f8ml.fsf@g208.suse.de","subject":"Re: [regression] Newer gits cannot clone any remote repos","fromName":"Douglas Mencken","fromEmail":"dougmencken@gmail.com","sentAt":"2012-11-14T18:01:01Z","receivedAt":"2012-11-14T18:01:01Z","isPatch":false,"sender":{"key":"dougmencken@gmail.com","avatar":"https://gravatar.com/avatar/6e6892c4aed51719e6c271923f2ab9e3d9e3418dd138c4aae9dd9fee70d6965a?d=mp&s=160"},"body":">>> I cannot reproduce the problem (on openSUSE 12.2).\n>>\n>> You do need multiple CPU/multi-core machine, as I got it.\n>\n> Which is what I have.\n\nThen try to build *vanilla* git 1.8.0, not OpenSuSE's one (with a lot\nof patches inside srcrpm).\n"},{"id":"203227","messageId":"m28va3epzc.fsf@igel.home","threadId":"32089","inReplyTo":"CACYvZ7g7nCgQmcnwJYxvx5hJ+Y8rCQOBQyWYLZOt8NT5BTwTrw@mail.gmail.com","subject":"Re: [regression] Newer gits cannot clone any remote repos","fromName":"Andreas Schwab","fromEmail":"schwab@linux-m68k.org","sentAt":"2012-11-14T23:15:03Z","receivedAt":"2012-11-14T23:15:03Z","isPatch":false,"sender":{"key":"schwab@linux-m68k.org","avatar":"https://avatars.githubusercontent.com/u/2175493?v=4"},"body":"Douglas Mencken writes:\n\n>>>> I cannot reproduce the problem (on openSUSE 12.2).\n>>>\n>>> You do need multiple CPU/multi-core machine, as I got it.\n>>\n>> Which is what I have.\n>\n> Then try to build *vanilla* git 1.8.0,\n\nWhich is what I did.\n\n> not OpenSuSE's one (with a lot of patches inside srcrpm).\n\nWhich lot of patches?\n\nAndreas.\n\n-- \nAndreas Schwab, schwab@linux-m68k.org\nGPG Key fingerprint = 58CA 54C7 6D53 942B 1756 01D3 44D5 214B 8276 4ED5\n\"And now for something completely different.\"\n"},{"id":"203289","messageId":"CACsJy8Atj0WsFaYTZE9y9+tu1+L2LumVOk3sUj-t9iKbiqsBZw@mail.gmail.com","threadId":"32089","inReplyTo":"CACYvZ7jMC5xw4LxiuG5m+=grpQEg+wZb_7BaU4Xn-r7ix=S-bw@mail.gmail.com","subject":"Re: [regression] Newer gits cannot clone any remote repos","fromName":"Nguyen Thai Ngoc Duy","fromEmail":"pclouds@gmail.com","sentAt":"2012-11-15T12:51:29Z","receivedAt":"2012-11-15T12:51:29Z","isPatch":false,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Wed, Nov 14, 2012 at 2:55 AM, Douglas Mencken wrote:\n>> Could you try re-building git with the\n>> NO_THREAD_SAFE_PREAD build variable set?\n>\n> Yeah! It works!!!\n>\n> --- evil/Makefile\n> +++ good/Makefile\n> @@ -957,6 +957,7 @@\n> HAVE_PATHS_H = YesPlease\n> LIBC_CONTAINS_LIBINTL = YesPlease\n> HAVE_DEV_TTY = YesPlease\n> + NO_THREAD_SAFE_PREAD = YesPlease\n> endif\n> ifeq ($(uname_S),GNU/kFreeBSD)\n> NO_STRLCPY = YesPlease\n>\n> With this, I do have correctly working git clone.\n\nSorry you had to figure that out the hard way. Could you make it a\nproper patch? I'm surprised that Linux pread does not behave the same\nway across platforms though. Or maybe it only happens with certain\nLinux versions. What version are you using?\n-- \nDuy\n"},{"id":"203308","messageId":"50A53AAE.7080406@ramsay1.demon.co.uk","threadId":"32089","inReplyTo":"CACYvZ7jMC5xw4LxiuG5m+=grpQEg+wZb_7BaU4Xn-r7ix=S-bw@mail.gmail.com","subject":"Re: [regression] Newer gits cannot clone any remote repos","fromName":"Ramsay Jones","fromEmail":"ramsay@ramsay1.demon.co.uk","sentAt":"2012-11-15T18:55:42Z","receivedAt":"2012-11-15T18:55:42Z","isPatch":false,"sender":{"key":"ramsay@ramsayjones.plus.com","avatar":"https://avatars.githubusercontent.com/u/33702710?v=4"},"body":"Douglas Mencken wrote:\n>> Could you try re-building git with the\n>> NO_THREAD_SAFE_PREAD build variable set?\n> \n> Yeah! It works!!!\n> \n> --- evil/Makefile\n> +++ good/Makefile\n> @@ -957,6 +957,7 @@\n> \tHAVE_PATHS_H = YesPlease\n> \tLIBC_CONTAINS_LIBINTL = YesPlease\n> \tHAVE_DEV_TTY = YesPlease\n> +\tNO_THREAD_SAFE_PREAD = YesPlease\n> endif\n> ifeq ($(uname_S),GNU/kFreeBSD)\n> \tNO_STRLCPY = YesPlease\n> \n> With this, I do have correctly working git clone.\n\nOK, good.\n\nYou didn't mention which platform you are on; from the above Makefile\nhunk, however, I can deduce that you are on *some* version of Linux.\n\nHmm, it doesn't seem too likely that your pread() is thread-unsafe\n(possible, just unlikely), so it could be a more fundamental problem\nwith the threaded index-pack code (ie commit b8a2486f1 et. seq.).\n\nHowever, as a first step could you try running the test program\n(given below) on your system to determine if your pread() is thread-safe\nor not. (gcc -I. -o test-pread test-pread.c; ./test-pread)\n\nAlso, what is the output of \"uname -a\".\n\nATB,\nRamsay Jones\n\n-- >8 --\n#include \"git-compat-util.h\"\n#include \"thread-utils.h\"\n\n#define DATA_FILE \"junk.data\"\n#define MAX_DATA 256 * 1024\n#define NUM_THREADS 3\n#define TRIALS 500000\n\nstruct thread_data {\n\tpthread_t t;\n\tint fd;\n\tint cnt;\n\tint fails;\n\tunsigned long n;\n};\n\nstatic struct thread_data t[NUM_THREADS+1];\n\nint create_data_file(void)\n{\n\tint i, fd = open(DATA_FILE, O_CREAT | O_TRUNC | O_WRONLY, 0600);\n\tif (fd < 0)\n\t\treturn -1;\n\tfor (i = 0; i < MAX_DATA; i++)\n\t\tif (write(fd, &i, sizeof(int)) < 0) {\n\t\t\tclose(fd);\n\t\t\tunlink(DATA_FILE);\n\t\t\treturn -1;\n\t\t}\n\tclose(fd);\n\treturn 0;\n}\n\nvoid *read_thread(void *data)\n{\n\tstruct thread_data *d = (struct thread_data *)data;\n\tint i, j, rd;\n\tfor (i = 0; i < TRIALS; i += MAX_DATA) {\n\t\tfor (j = 0; j < MAX_DATA; j++) {\n\t\t\tssize_t sz = read(d->fd, &rd, sizeof(int));\n\t\t\tif (sz < 0 || rd != j)\n\t\t\t\td->fails++;\n\t\t\td->cnt++;\n\t\t}\n\t\tlseek(d->fd, 0, SEEK_SET);\n\t}\n\treturn NULL;\n}\n\nvoid *pread_thread(void *data)\n{\n\tstruct thread_data *d = (struct thread_data *)data;\n\tint i, j, rd;\n\tfor (i = 0; i < TRIALS; i++) {\n\t\tssize_t sz;\n\t\td->n = d->n * 1103515245 + 12345;\n\t\tj = d->n % MAX_DATA;\n\t\tsz = pread(d->fd, &rd, sizeof(int), j * sizeof(int));\n\t\tif (sz < 0 || rd != j)\n\t\t\td->fails++;\n\t\td->cnt++;\n\t}\n\treturn NULL;\n}\n\nint main(int argc, char *argv[])\n{\n\tint fd, i;\n\n\tif (create_data_file() < 0) {\n\t\tprintf(\"can't create data file\\n\");\n\t\treturn 1;\n\t}\n\n\tif ((fd = open(DATA_FILE, O_RDONLY)) < 0) {\n\t\tprintf(\"can't open data file\\n\");\n\t\tunlink(DATA_FILE);\n\t\treturn 1;\n\t}\n\n\tfor (i = 0; i < NUM_THREADS+1; i++) {\n\t\tint ret;\n\n\t\tt[i].fd = fd;\n\t\tt[i].cnt = 0;\n\t\tt[i].fails = 0;\n\t\tt[i].n = i * 16381;\n\t\tret = pthread_create(&t[i].t, NULL,\n\t\t\t\t(i == 0) ? read_thread : pread_thread,\n\t\t\t\t&t[i]);\n\t\tif (ret) {\n\t\t\tprintf(\"can't create thread %d (%s)\\n\", i, strerror(ret));\n\t\t\tunlink(DATA_FILE);\n\t\t\treturn 1;\n\t\t}\n\t}\n\n\tfor (i = 0; i < NUM_THREADS+1; i++)\n\t\tpthread_join(t[i].t, NULL);\n\tclose(fd);\n\n\tfor (i = 0; i < NUM_THREADS+1; i++)\n\t\tprintf(\"%2d: trials %d, failed %d\\n\", i, t[i].cnt, t[i].fails);\n\n\tunlink(DATA_FILE);\n\treturn 0;\n}\n"},{"id":"203310","messageId":"50A53FAC.8020401@ramsay1.demon.co.uk","threadId":"32089","inReplyTo":"50A2B2DF.1080704@web.de","subject":"Re: [regression] Newer gits cannot clone any remote repos","fromName":"Ramsay Jones","fromEmail":"ramsay@ramsay1.demon.co.uk","sentAt":"2012-11-15T19:17:00Z","receivedAt":"2012-11-15T19:17:00Z","isPatch":false,"sender":{"key":"ramsay@ramsayjones.plus.com","avatar":"https://avatars.githubusercontent.com/u/33702710?v=4"},"body":"Torsten Bögershausen wrote:\n> On 13.11.12 19:55, Ramsay Jones wrote:\n>> Douglas Mencken wrote:\n>>> *Any* git clone fails with:\n>>>\n>>> fatal: premature end of pack file, 106 bytes missing\n>>> fatal: index-pack failed\n>>>\n>>> At first, I tried 1.8.0, and it failed. Then I tried to build 1.7.10.5\n>>> then, and it worked. Then I tried 1.7.12.2, but it fails the same way\n>>> as 1.8.0.\n>>> So I decided to git bisect.\n>>>\n>>> b8a2486f1524947f232f657e9f2ebf44e3e7a243 is the first bad commit\n>>> ``index-pack: support multithreaded delta resolving''\n>>\n>> This looks like the same problem I had on cygwin, which lead to\n>> commit c0f86547c (\"index-pack: Disable threading on cygwin\", 26-06-2012).\n>>\n>> I didn't notice which platform you are on, but maybe you also have a\n>> thread-unsafe pread()? Could you try re-building git with the\n>> NO_THREAD_SAFE_PREAD build variable set?\n>>\n>> HTH.\n>>\n>> ATB,\n>> Ramsay Jones\n> \n> This is interesting.\n> I had the same problem on a PowerPC \n> (Old PowerBook G4 running Linux).\n> \n> Using NO_THREAD_SAFE_PREAD helped, thanks for the hint.\n> (After recompiling without NO_THREAD_SAFE_PREAD I could clone\n> from this machine again, so the problem is not really reproducable)\n\nYes, the failures would be intermittent (and often not easily\nreproducible). The threaded index-pack code did not fail for\nme on cygwin at all during development, including tests, but failed\nimmediately I installed v1.7.11. On real repositories, it failed\nintermittently. On some repos it always failed, on some it never\nfailed and on some others it would sometimes fail, sometimes not.\n\nATB,\nRamsay Jones\n"},{"id":"203615","messageId":"CACYvZ7hDu=r1=x_ofsVud6+0krZD5wPoE0Q9yjLg_e6y7MzPuQ@mail.gmail.com","threadId":"32089","inReplyTo":"50A53FAC.8020401@ramsay1.demon.co.uk","subject":"Re: [regression] Newer gits cannot clone any remote repos","fromName":"Douglas Mencken","fromEmail":"dougmencken@gmail.com","sentAt":"2012-11-21T19:20:40Z","receivedAt":"2012-11-21T19:20:40Z","isPatch":false,"sender":{"key":"dougmencken@gmail.com","avatar":"https://gravatar.com/avatar/6e6892c4aed51719e6c271923f2ab9e3d9e3418dd138c4aae9dd9fee70d6965a?d=mp&s=160"},"body":"> The threaded index-pack code did not fail for\n> me on cygwin at all during development, including tests, but failed\n> immediately I installed v1.7.11. On real repositories, it failed\n> intermittently. On some repos it always failed, on some it never\n> failed and on some others it would sometimes fail, sometimes not.\n\nThen why did you commit it? If it has so high random failure rate.\n"}]}