{"thread":{"id":"16279","subject":"hosting git on a nfs","startedAt":"2008-11-12T09:29:34Z","lastAt":"2008-12-03T02:18:01Z","messageCount":44,"participants":["Thomas Koch","Julian Phillips","David Brown","Linus Torvalds","Brandon Casey","J. Bruce Fields","James Pickens","Pieter de Bie","Michael J Gruber","Kyle Moffett","Junio C Hamano","Mike Ralphson","Johannes Sixt","dhruva"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"95544","messageId":"200811121029.34841.thomas@koch.ro","threadId":"16279","inReplyTo":null,"subject":"hosting git on a nfs","fromName":"Thomas Koch","fromEmail":"thomas@koch.ro","sentAt":"2008-11-12T09:29:34Z","receivedAt":"2008-11-12T09:29:34Z","isPatch":false,"sender":{"key":"thomas@koch.ro","avatar":null},"body":"Hi,\n\nfinally I managed to convince a critical mass of developers (our chief\ndev :-) in our company so that we are starting to migrate to GIT.\n\nThe final question is, whether GIT will life peacefully on our cluster\nfileservers. The GIT repository dir (/var/cache/git) should be mounted\nvia NFS via PAN on top of DRBD (so I was told).\n\nAre there any known problems with this setup? We're asking, because\nthere are problems with SVN on such a setup[1].\n\n[1] http://subversion.tigris.org/faq.html#nfs\n\nBest regards,\n-- \nThomas Koch, Software Developer\nhttp://www.koch.ro\n\nYoung Media Concepts GmbH\nSonnenstr. 4\nCH-8280 Kreuzlingen\nSwitzerland\n\nTel +41 (0)71 / 508 24 86\nFax +41 (0)71 / 560 53 89\nMobile +49 (0)170 / 753 89 16\nWeb www.ymc.ch\n"},{"id":"95546","messageId":"alpine.LNX.2.00.0811121006400.23345@reaper.quantumfyre.co.uk","threadId":"16279","inReplyTo":"200811121029.34841.thomas@koch.ro","subject":"Re: hosting git on a nfs","fromName":"Julian Phillips","fromEmail":"julian@quantumfyre.co.uk","sentAt":"2008-11-12T10:10:03Z","receivedAt":"2008-11-12T10:10:03Z","isPatch":false,"sender":{"key":"julian@quantumfyre.co.uk","avatar":"https://avatars.githubusercontent.com/u/948888?v=4"},"body":"On Wed, 12 Nov 2008, Thomas Koch wrote:\n\n> Hi,\n>\n> finally I managed to convince a critical mass of developers (our chief\n> dev :-) in our company so that we are starting to migrate to GIT.\n>\n> The final question is, whether GIT will life peacefully on our cluster\n> fileservers. The GIT repository dir (/var/cache/git) should be mounted\n> via NFS via PAN on top of DRBD (so I was told).\n>\n> Are there any known problems with this setup? We're asking, because\n> there are problems with SVN on such a setup[1].\n>\n> [1] http://subversion.tigris.org/faq.html#nfs\n\nI've been running git on NFS for years (though it's only NFS exported \nsoftware RAID), and the only issue I've encountered is that it's not quite \nas blisteringly fast as running git on a local disk.\n\n-- \nJulian\n\n ---\nThere's something different about us -- different from people of Europe,\nAfrica, Asia ... a deep and abiding belief in the Easter Bunny.\n \t\t-- G. Gordon Liddy\n"},{"id":"95591","messageId":"20081112173651.GA9127@linode.davidb.org","threadId":"16279","inReplyTo":"200811121029.34841.thomas@koch.ro","subject":"Re: hosting git on a nfs","fromName":"David Brown","fromEmail":"git@davidb.org","sentAt":"2008-11-12T17:36:52Z","receivedAt":"2008-11-12T17:36:52Z","isPatch":false,"sender":{"key":"git@davidb.org","avatar":"https://gravatar.com/avatar/94c86a2938470a74c2eac5e2b69afc0871f79a660295c02219597aba8cb101c1?d=mp&s=160"},"body":"On Wed, Nov 12, 2008 at 10:29:34AM +0100, Thomas Koch wrote:\n\n>finally I managed to convince a critical mass of developers (our chief\n>dev :-) in our company so that we are starting to migrate to GIT.\n>\n>The final question is, whether GIT will life peacefully on our cluster\n>fileservers. The GIT repository dir (/var/cache/git) should be mounted\n>via NFS via PAN on top of DRBD (so I was told).\n>\n>Are there any known problems with this setup? We're asking, because\n>there are problems with SVN on such a setup[1].\n\nWe had occasionally run into locking problems with 1.5.4.x with\nrenames between different directories. This should be fixed in\n1.6.0.3, but we have since migrated to a server model so I don't have\nany way of testing this.\n\nNone of these problems ever caused repository corruption, only errors\nduring fetch/clone that were resolved by repeating the operation.\n\nUsing ssh: or git: does seem to be a bit faster than NFS. The\nconfiguration we did find completely unworkable was using git with the\nwork tree on NFS.\n\nDavid\n"},{"id":"95609","messageId":"alpine.LFD.2.00.0811120959050.3468@nehalem.linux-foundation.org","threadId":"16279","inReplyTo":"20081112173651.GA9127@linode.davidb.org","subject":"Re: hosting git on a nfs","fromName":"Linus Torvalds","fromEmail":"torvalds@linux-foundation.org","sentAt":"2008-11-12T18:14:44Z","receivedAt":"2008-11-12T18:14:44Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Wed, 12 Nov 2008, David Brown wrote:\n> \n> We had occasionally run into locking problems with 1.5.4.x with\n> renames between different directories. This should be fixed in\n> 1.6.0.3, but we have since migrated to a server model so I don't have\n> any way of testing this.\n\nI suspect it also depends very much on the particular client/server \ncombination. Renaming across directories is one of those NFS things that \nsome servers don't mind at all.\n\n> The configuration we did find completely unworkable was using git with \n> the work tree on NFS.\n\nDoing an 'lstat()' on every single file in the tree would tend to do that \nto you, yes. Even with a fast network and a good NFS server, we're talking \nmillisecond-range latencies, and if your tree has tens of thousands of \nfiles, you're going to have each \"git diff\" take several seconds.\n\nNFS metadata caching can help, but not all clients do it, and even clients \nthat _do_ do it tend to have rather low timeouts or rather limited cache \nsizes, so doing \"git diff\" twice may speed up the second one only if it's \ndone really back-to-back - if even then.\n\nAnd once you get used to \"git diff\" being instantaneous, I don't think \nanybody is ever agan willing to go back to it taking \"a few seconds\" (and \ndepending on speed of network/server and size of project, the \"few\" can be \nquite many ;)\n\nSo putting the work-tree on NFS certainly _works_, but yes, from a \nperformance angle it is going to be really irritatingly slower. I don't \neven think the newer versions of NFS will help with directory and \nattribute caching - the delegations are per-file afaik, and there is no \ngood support for extending the caching to directories.\n\nThat said, I don't think git is any _worse_ than anybody else in the \n\"worktree on NFS\" model. A \"git diff\" will still be superior ot a CVS diff \nin every way. It's just that when people compare to their home machines \nwhere they have the work tree on local disk and aggressively cached, when \nthey then use a NFS work-tree, they'll likely be very very disappointed.\n\n\t\t\t\tLinus\n"},{"id":"95625","messageId":"9ww97aHlVPEurT6kkb9TAxOaH5OSLhFQTgI4LcQRGzCQgkut0VsbwQ@cipher.nrlssc.navy.mil","threadId":"16279","inReplyTo":"alpine.LNX.2.00.0811121006400.23345@reaper.quantumfyre.co.uk","subject":"Re: hosting git on a nfs","fromName":"Brandon Casey","fromEmail":"casey@nrlssc.navy.mil","sentAt":"2008-11-12T20:31:16Z","receivedAt":"2008-11-12T20:31:16Z","isPatch":false,"sender":{"key":"drafnel@gmail.com","avatar":"https://avatars.githubusercontent.com/u/921167?v=4"},"body":"Julian Phillips wrote:\n> On Wed, 12 Nov 2008, Thomas Koch wrote:\n> \n>> Hi,\n>>\n>> finally I managed to convince a critical mass of developers (our chief\n>> dev :-) in our company so that we are starting to migrate to GIT.\n>>\n>> The final question is, whether GIT will life peacefully on our cluster\n>> fileservers. The GIT repository dir (/var/cache/git) should be mounted\n>> via NFS via PAN on top of DRBD (so I was told).\n>>\n>> Are there any known problems with this setup? We're asking, because\n>> there are problems with SVN on such a setup[1].\n>>\n>> [1] http://subversion.tigris.org/faq.html#nfs\n> \n> I've been running git on NFS for years (though it's only NFS exported\n> software RAID), and the only issue I've encountered is that it's not\n> quite as blisteringly fast as running git on a local disk.\n\nditto. (except for the years part)\n\n-brandon\n"},{"id":"95706","messageId":"20081113181826.GA16741@fieldses.org","threadId":"16279","inReplyTo":"alpine.LFD.2.00.0811120959050.3468@nehalem.linux-foundation.org","subject":"Re: hosting git on a nfs","fromName":"J. Bruce Fields","fromEmail":"bfields@fieldses.org","sentAt":"2008-11-13T18:18:26Z","receivedAt":"2008-11-13T18:18:26Z","isPatch":false,"sender":{"key":"bfields@citi.umich.edu","avatar":null},"body":"On Wed, Nov 12, 2008 at 10:14:44AM -0800, Linus Torvalds wrote:\n> \n> \n> On Wed, 12 Nov 2008, David Brown wrote:\n> > \n> > We had occasionally run into locking problems with 1.5.4.x with\n> > renames between different directories. This should be fixed in\n> > 1.6.0.3, but we have since migrated to a server model so I don't have\n> > any way of testing this.\n> \n> I suspect it also depends very much on the particular client/server \n> combination. Renaming across directories is one of those NFS things that \n> some servers don't mind at all.\n\nOn the linux server you want to make sure you're exporting with\nno_subtree_check (see \"man exports\").\n\n> > The configuration we did find completely unworkable was using git with \n> > the work tree on NFS.\n> \n> Doing an 'lstat()' on every single file in the tree would tend to do that \n> to you, yes. Even with a fast network and a good NFS server, we're talking \n> millisecond-range latencies, and if your tree has tens of thousands of \n> files, you're going to have each \"git diff\" take several seconds.\n> \n> NFS metadata caching can help, but not all clients do it, and even clients \n> that _do_ do it tend to have rather low timeouts or rather limited cache \n> sizes, so doing \"git diff\" twice may speed up the second one only if it's \n> done really back-to-back - if even then.\n> \n> And once you get used to \"git diff\" being instantaneous, I don't think \n> anybody is ever agan willing to go back to it taking \"a few seconds\" (and \n> depending on speed of network/server and size of project, the \"few\" can be \n> quite many ;)\n\nYep.\n\n> So putting the work-tree on NFS certainly _works_, but yes, from a \n> performance angle it is going to be really irritatingly slower. I don't \n> even think the newer versions of NFS will help with directory and \n> attribute caching - the delegations are per-file afaik, and there is no \n> good support for extending the caching to directories.\n\nFile delegations do cover a file's attributes, so in theory they could\nhelp. But they're only given out on open. The upcoming 4.1 spec has a\nfew improvements here, and it might be worth looking at whether they're\nsufficient to make this work.\n\n--b.\n\n> That said, I don't think git is any _worse_ than anybody else in the \n> \"worktree on NFS\" model. A \"git diff\" will still be superior ot a CVS diff \n> in every way. It's just that when people compare to their home machines \n> where they have the work tree on local disk and aggressively cached, when \n> they then use a NFS work-tree, they'll likely be very very disappointed.\n> \n> \t\t\t\tLinus\n> --\n> To unsubscribe from this list: send the line \"unsubscribe git\" in\n> the body of a message to majordomo@vger.kernel.org\n> More majordomo info at http://vger.kernel.org/majordomo-info.html\n"},{"id":"95707","messageId":"loom.20081113T174625-994@post.gmane.org","threadId":"16279","inReplyTo":"alpine.LFD.2.00.0811120959050.3468@nehalem.linux-foundation.org","subject":"Re: hosting git on a nfs","fromName":"James Pickens","fromEmail":"jepicken@gmail.com","sentAt":"2008-11-13T18:32:53Z","receivedAt":"2008-11-13T18:32:53Z","isPatch":false,"sender":{"key":"jepicken@gmail.com","avatar":null},"body":"Linus Torvalds linux-foundation.org> writes:\n> Doing an 'lstat()' on every single file in the tree would tend to do that \n> to you, yes. Even with a fast network and a good NFS server, we're talking \n> millisecond-range latencies, and if your tree has tens of thousands of \n> files, you're going to have each \"git diff\" take several seconds.\n\nIs there any way to improve 'git status' performance on nfs? I know nothing\nabout how that code works, but if it's strictly serial, i.e. it waits for the\nresult of each lstat() before doing the next lstat(), then perhaps it could be\nsped up by overlapping the lstat() calls via multi threading.\n\nReason I ask is that at my work place, using only local disks would be\ndifficult. We run lots of long running tests in a server farm, and working on\nnfs allows the compute servers to access our data transparently.\n"},{"id":"95722","messageId":"alpine.LFD.2.00.0811131214020.3468@nehalem.linux-foundation.org","threadId":"16279","inReplyTo":"loom.20081113T174625-994@post.gmane.org","subject":"Re: hosting git on a nfs","fromName":"Linus Torvalds","fromEmail":"torvalds@linux-foundation.org","sentAt":"2008-11-13T20:18:24Z","receivedAt":"2008-11-13T20:18:24Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Thu, 13 Nov 2008, James Pickens wrote:\n> \n> Is there any way to improve 'git status' performance on nfs? I know nothing\n> about how that code works, but if it's strictly serial, i.e. it waits for the\n> result of each lstat() before doing the next lstat(), then perhaps it could be\n> sped up by overlapping the lstat() calls via multi threading.\n\nIt's fairly doable. \"git status\" to some degree is actually the worst \ncase, since it has to do readdir()'s etc to find new files, but I've \noccasionally considered trying to do a parallel version of the regular \nindex file checking where we have the list of files a priori.\n\nThat would speed up \"git diff\" by potentially a big amount (it would also \nspeed up git status, just not as much as also doing readdirs in parallel).\n\nHowever, every time I think about it, I end up looking at my own setup \nwhich has effectively no parallelism. Sure, in theory parallel reads can \nhelp even with a local disk, but in practice the potential seek advantage \nis very small, and so I've never really had it as a high priority.\n\nIf I were still using NFS (I gave up on it years ago exactly because it \nwas so painful for software development - and that was when I was using \nCVS) I'd surely have done it long since.\n\nBit I'll think about it again. \n\n\t\tLinus\n"},{"id":"95727","messageId":"alpine.LFD.2.00.0811131252040.3468@nehalem.linux-foundation.org","threadId":"16279","inReplyTo":"alpine.LFD.2.00.0811131214020.3468@nehalem.linux-foundation.org","subject":"Re: hosting git on a nfs","fromName":"Linus Torvalds","fromEmail":"torvalds@linux-foundation.org","sentAt":"2008-11-13T21:05:04Z","receivedAt":"2008-11-13T21:05:04Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Thu, 13 Nov 2008, Linus Torvalds wrote:\n> \n> If I were still using NFS (I gave up on it years ago exactly because it \n> was so painful for software development - and that was when I was using \n> CVS) I'd surely have done it long since.\n> \n> Bit I'll think about it again. \n\nTHIS IS TOTALLY UNTESTED! It's trivial, it's hacky, and it would need to \nbe cleaned up before being merged, but I'm not even going to bother \n_trying_ to make it anything cleaner unless somebody can test it on a nice \nNFS setup.\n\nBecause it's entirely possible that there are various directory inode \nlocks etc that means that doing parallel lookups in the same directory \nwon't actually be doing a whole lot. Whatever. It's kind of cute, and it \nreally isn't all that many lines of code, and even if it doesn't work due \nto some locking reason, maybe it's worth looking at.\n\nIt arbitrarily caps the number of threads to 10, and it has no way to turn \nit on or off. It actually _does_ seem to work in the sense than when \ncached, it can take advantage of the fact that I have a nice multi-core \nthing, but let's face it, it's not really worth it for that reason only. \n\nBefore:\n\n\t[torvalds@nehalem linux]$ /usr/bin/time git diff > /dev/null \n\t0.03user 0.04system 0:00.07elapsed 100%CPU (0avgtext+0avgdata 0maxresident)k\n\nAfter:\n\n\t0.02user 0.07system 0:00.04elapsed 243%CPU (0avgtext+0avgdata 0maxresident)k\n\t0inputs+0outputs (0major+2241minor)pagefaults 0swaps\n\nie it actually did cut elapsed time from 7 hundredths of a second to just \n4. And the CPU usage went from 100% to 243%. Ooooh. Magic.\n\nBut it's still hacky as hell. Who has NFS? Can you do the same thing over \nNFS and test it? I'm not going to set up NFS to test this, and as I \nsuspected, on a local disk, the cold-cache case makes no difference \nwhat-so-ever, because whatever seek optimizations can be done are still \ntotally irrelevant.\n\nAnd if there are per-directory locking etc that screws this up, we can \nlook at using different heuristics for the lstat() patterns. Right now I \ndivvy it up so that thread 0 gets entries 0, 10, 20, 30.. and thread 1 \ndoes entries 1, 11, 21, 31.., but we could easily split it up differently, \nand do 0,1,2,3.. and 1000,1001,1002,1003.. instead. That migth avoid some \nper-directory locks. Dunno.\n\nAnyway, it was kind of fun writing this. The reason it threads so well is \nthat all the lstat() code really works on private data already, so there \nare no global data structures that change that need to be worried about. \n\nSo no locking necessary - just fire it up in parallel, and wait for the \nresults. We already had everything else in place (ie the per-cache_entry \nflag to say \"I've checked this entry on disk\").\n\nOf course, if a lot of entries do _not_ match, then this will actually \ngenerate more work (because we'll do the parallel thing to verify that \nthe on-disk version matches, and if it doesn't match, then we'll end up \nre-doing it linearly later more carefully).\n\n\t\t\tLinus\n\n---\n diff-lib.c | 67 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++\n 1 files changed, 67 insertions(+), 0 deletions(-)\n\ndiff --git a/diff-lib.c b/diff-lib.c\nindex ae96c64..7d972c9 100644\n--- a/diff-lib.c\n+++ b/diff-lib.c\n@@ -1,6 +1,7 @@\n /*\n * Copyright (C) 2005 Junio C Hamano\n */\n+#include \n #include \"cache.h\"\n #include \"quote.h\"\n #include \"commit.h\"\n@@ -54,6 +55,70 @@ static int check_removed(const struct cache_entry *ce, struct stat *st)\n \treturn 0;\n }\n \n+/* Hacky hack-hack start */\n+#define MAX_PARALLEL (10)\n+static int parallel_lstats = 1;\n+\n+struct thread_data {\n+\tpthread_t pthread;\n+\tstruct index_state *index;\n+\tstruct rev_info *revs;\n+\tint i, step;\n+};\n+\n+static void *preload_thread(void *_data)\n+{\n+\tint i;\n+\tstruct thread_data *p = _data;\n+\tstruct index_state *index = p->index;\n+\tstruct rev_info *revs = p->revs;\n+\n+\tfor (i = p->i; i < index->cache_nr; i += p->step) {\n+\t\tstruct cache_entry *ce = index->cache[i];\n+\t\tstruct stat st;\n+\n+\t\tif (ce_stage(ce))\n+\t\t\tcontinue;\n+\t\tif (ce_uptodate(ce))\n+\t\t\tcontinue;\n+\t\tif (!ce_path_match(ce, revs->prune_data))\n+\t\t\tcontinue;\n+\t\tif (lstat(ce->name, &st))\n+\t\t\tcontinue;\n+\t\tif (ie_match_stat(index, ce, &st, 0))\n+\t\t\tcontinue;\n+\t\tce_mark_uptodate(ce);\n+\t}\n+\treturn NULL;\n+}\n+\n+static void preload_uptodate(struct rev_info *revs, struct index_state *index)\n+{\n+\tint i;\n+\tint threads = index->cache_nr / 100;\n+\tstruct thread_data data[MAX_PARALLEL];\n+\n+\tif (threads < 2)\n+\t\treturn;\n+\tif (threads > MAX_PARALLEL)\n+\t\tthreads = MAX_PARALLEL;\n+\tfor (i = 0; i < threads; i++) {\n+\t\tstruct thread_data *p = data+i;\n+\t\tp->index = index;\n+\t\tp->revs = revs;\n+\t\tp->i = i;\n+\t\tp->step = threads;\n+\t\tif (pthread_create(&p->pthread, NULL, preload_thread, p))\n+\t\t\tdie(\"unable to create threaded lstat\");\n+\t}\n+\tfor (i = 0; i < threads; i++) {\n+\t\tstruct thread_data *p = data+i;\n+\t\tif (pthread_join(p->pthread, NULL))\n+\t\t\tdie(\"unable to join threaded lstat\");\n+\t}\n+}\n+/* Hacky hack-hack mostly ends */\n+\n int run_diff_files(struct rev_info *revs, unsigned int option)\n {\n \tint entries, i;\n@@ -68,6 +133,8 @@ int run_diff_files(struct rev_info *revs, unsigned int option)\n \tif (diff_unmerged_stage < 0)\n \t\tdiff_unmerged_stage = 2;\n \tentries = active_nr;\n+\tif (parallel_lstats)\n+\t\tpreload_uptodate(revs, &the_index);\n \tsymcache[0] = '\\0';\n \tfor (i = 0; i < entries; i++) {\n \t\tstruct stat st;\n"},{"id":"95737","messageId":"885649360811131523h2e10dc44x8603c9793dae03b8@mail.gmail.com","threadId":"16279","inReplyTo":"alpine.LFD.2.00.0811131252040.3468@nehalem.linux-foundation.org","subject":"Re: hosting git on a nfs","fromName":"James Pickens","fromEmail":"jepicken@gmail.com","sentAt":"2008-11-13T23:23:16Z","receivedAt":"2008-11-13T23:23:16Z","isPatch":false,"sender":{"key":"jepicken@gmail.com","avatar":null},"body":"On Thu, Nov 13, 2008, Linus Torvalds wrote:\n> But it's still hacky as hell. Who has NFS? Can you do the same thing over\n> NFS and test it? I'm not going to set up NFS to test this, and as I\n> suspected, on a local disk, the cold-cache case makes no difference\n> what-so-ever, because whatever seek optimizations can be done are still\n> totally irrelevant.\n\nI'm trying to test this and so far I'm not seeing any effect. The\ntimings are all over the map, but the average time is about the same as\nbefore. If I run 'git status' over and over without pausing in between,\nit runs very fast, so I guess the client is caching some things for a\nshort time. Here are the timings I got by running git 1.6.0.4 and\n1.6.0.4 plus this patch, alternating between them and sleeping for 60\nseconds after each run:\n\nGit 1.6.0.4:\n4.83\n4.47\n4.54\n9.04\n5.28\n4.33\n6.40\n13.71\n4.51\n5.90\n\nGit 1.6.0.4 plus patch:\n7.82\n10.94\n4.61\n5.06\n5.59\n5.22\n4.65\n5.58\n6.70\n9.15\n\nI'll play around with the code and see if I figure anything out.\n\nIn case it matters, I'm using a 4 core machine with a fairly old kernel:\n\n$ uname -r\n2.6.5-7.287.3.PTF.363939.1-smp\n\nBTW thanks a lot for working on it. I was expecting a response along\nthe lines of \"it's possible but would require extensive code changes so\nit's not likely to happen\", and instead a patch was posted only a few\nhours later. Maybe I should ask for all those other upgrades I've had\nin mind... ;-)\n\nJames\n"},{"id":"95738","messageId":"alpine.LNX.2.00.0811132317100.3965@reaper.quantumfyre.co.uk","threadId":"16279","inReplyTo":"alpine.LFD.2.00.0811131252040.3468@nehalem.linux-foundation.org","subject":"Re: hosting git on a nfs","fromName":"Julian Phillips","fromEmail":"julian@quantumfyre.co.uk","sentAt":"2008-11-13T23:23:38Z","receivedAt":"2008-11-13T23:23:38Z","isPatch":false,"sender":{"key":"julian@quantumfyre.co.uk","avatar":"https://avatars.githubusercontent.com/u/948888?v=4"},"body":"On Thu, 13 Nov 2008, Linus Torvalds wrote:\n\n> Before:\n>\n> \t[torvalds@nehalem linux]$ /usr/bin/time git diff > /dev/null\n> \t0.03user 0.04system 0:00.07elapsed 100%CPU (0avgtext+0avgdata 0maxresident)k\n>\n> After:\n>\n> \t0.02user 0.07system 0:00.04elapsed 243%CPU (0avgtext+0avgdata 0maxresident)k\n> \t0inputs+0outputs (0major+2241minor)pagefaults 0swaps\n>\n> ie it actually did cut elapsed time from 7 hundredths of a second to just\n> 4. And the CPU usage went from 100% to 243%. Ooooh. Magic.\n>\n> But it's still hacky as hell. Who has NFS? Can you do the same thing over\n> NFS and test it? I'm not going to set up NFS to test this, and as I\n> suspected, on a local disk, the cold-cache case makes no difference\n> what-so-ever, because whatever seek optimizations can be done are still\n> totally irrelevant.\n\nThe timings seem to vary quite a bit (not really a surprise with a network \ninvolved ;), but the patch definately makes things faster:\n\nmaster:\n\njp3@kaos: linux-2.6(master)>/usr/bin/time ~/bin/git diff > /dev/null\n0.01user 0.19system 0:02.50elapsed 8%CPU (0avgtext+0avgdata 0maxresident)k\n0inputs+0outputs (0major+1766minor)pagefaults 0swaps\n\nmaster + patch:\n\njp3@kaos: linux-2.6(master)>/usr/bin/time ~/bin/git diff > /dev/null\n0.02user 0.88system 0:00.96elapsed 93%CPU (0avgtext+0avgdata \n0maxresident)k\n0inputs+0outputs (0major+1783minor)pagefaults 0swaps\n\nseems to be approximately twice as fast?\n\n-- \nJulian\n\n ---\n \"XML is like violence, if it doesn't solve the problem, just\n use more.\"\n* nelchael hides\n"},{"id":"95741","messageId":"alpine.LFD.2.00.0811131518070.3468@nehalem.linux-foundation.org","threadId":"16279","inReplyTo":"alpine.LFD.2.00.0811131252040.3468@nehalem.linux-foundation.org","subject":"Re: hosting git on a nfs","fromName":"Linus Torvalds","fromEmail":"torvalds@linux-foundation.org","sentAt":"2008-11-13T23:42:29Z","receivedAt":"2008-11-13T23:42:29Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Thu, 13 Nov 2008, Linus Torvalds wrote:\n>\n> And if there are per-directory locking etc that screws this up, we can \n> look at using different heuristics for the lstat() patterns. Right now I \n> divvy it up so that thread 0 gets entries 0, 10, 20, 30.. and thread 1 \n> does entries 1, 11, 21, 31.., but we could easily split it up differently, \n> and do 0,1,2,3.. and 1000,1001,1002,1003.. instead. That migth avoid some \n> per-directory locks. Dunno.\n\nYeah, I just checked. We really should to the lookups in clusters, because \nyes, we do have directory locks that limit parallel lookups in the same \ndirectory, so to get good parallelism you want to look up in different \nplaces.\n\nAdmittedly that's really a Linux kernel deficiency, and I'd love to fix it \n(in the long run), but we optimize file lookup for the common (cached) \ncase, and the uncommon case of parallel misses in the same directory has \nbeen written for simplicity and robustness.\n\nSo don't even bother trying the previous patch, even if it might work. At \nleast not on Linux. Try this one instead, which does the parallel things \nin batches, so that we hopefully hit different directories.\n\nParallelism is hard. \n\nThe good news is that I seem to actualluy see a bit of a win from this \neven on a disk, now that the kernel doesn't serialize things. So it may \nbe worth it. So I have some hope that it actually helps on NFS too. \nThe numbers for five runs (with clearing of the caches in between, of \ncourse) are:\n\nBefore:\n\n\t0.01user 0.23system 0:10.87elapsed 2%CPU\n\t0.04user 0.19system 0:10.86elapsed 2%CPU\n\t0.03user 0.26system 0:10.82elapsed 2%CPU\n\t0.02user 0.27system 0:12.67elapsed 2%CPU\n\t0.01user 0.22system 0:10.86elapsed 2%CPU\n\nAfter:\n\n\t0.03user 0.26system 0:07.88elapsed 3%CPU\n\t0.02user 0.25system 0:07.63elapsed 3%CPU\n\t0.01user 0.26system 0:08.62elapsed 3%CPU\n\t0.01user 0.26system 0:07.27elapsed 3%CPU\n\t0.05user 0.28system 0:08.61elapsed 3%CPU\n\nso it really does seem like it has possibly given a 30% improvement in \ncold-cache performance even on a disk. \n\nNOTE NOTE NOTE! This is still a total hack. If this was done properly, we \nwould do the \"preload_uptodate()\" thing when we load the cache for all the \ndifferent programs where it can matter, not just the one special case \nplace. So see this as a proof-of-concept, nothing more.\n\n\t\tLinus\n\n---\n diff-lib.c | 74 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++\n 1 files changed, 74 insertions(+), 0 deletions(-)\n\ndiff --git a/diff-lib.c b/diff-lib.c\nindex ae96c64..83c180d 100644\n--- a/diff-lib.c\n+++ b/diff-lib.c\n@@ -1,6 +1,7 @@\n /*\n * Copyright (C) 2005 Junio C Hamano\n */\n+#include \n #include \"cache.h\"\n #include \"quote.h\"\n #include \"commit.h\"\n@@ -54,6 +55,77 @@ static int check_removed(const struct cache_entry *ce, struct stat *st)\n \treturn 0;\n }\n \n+/* Hacky hack-hack start */\n+#define MAX_PARALLEL (10)\n+static int parallel_lstats = 1;\n+\n+struct thread_data {\n+\tpthread_t pthread;\n+\tstruct index_state *index;\n+\tconst char **pathspec;\n+\tint offset, nr;\n+};\n+\n+static void *preload_thread(void *_data)\n+{\n+\tint nr;\n+\tstruct thread_data *p = _data;\n+\tstruct index_state *index = p->index;\n+\tstruct cache_entry **cep = index->cache + p->offset;\n+\n+\tnr = p->nr;\n+\tif (nr + p->offset > index->cache_nr)\n+\t\tnr = index->cache_nr - p->offset;\n+\n+\tdo {\n+\t\tstruct cache_entry *ce = *cep++;\n+\t\tstruct stat st;\n+\n+\t\tif (ce_stage(ce))\n+\t\t\tcontinue;\n+\t\tif (ce_uptodate(ce))\n+\t\t\tcontinue;\n+\t\tif (!ce_path_match(ce, p->pathspec))\n+\t\t\tcontinue;\n+\t\tif (lstat(ce->name, &st))\n+\t\t\tcontinue;\n+\t\tif (ie_match_stat(index, ce, &st, 0))\n+\t\t\tcontinue;\n+\t\tce_mark_uptodate(ce);\n+\t} while (--nr > 0);\n+\treturn NULL;\n+}\n+\n+static void preload_uptodate(const char **pathspec, struct index_state *index)\n+{\n+\tint i, work, offset;\n+\tint threads = index->cache_nr / 100;\n+\tstruct thread_data data[MAX_PARALLEL];\n+\n+\tif (threads < 2)\n+\t\treturn;\n+\tif (threads > MAX_PARALLEL)\n+\t\tthreads = MAX_PARALLEL;\n+\toffset = 0;\n+\twork = (index->cache_nr + threads - 1) / threads;\n+\tfor (i = 0; i < threads; i++) {\n+\t\tstruct thread_data *p = data+i;\n+\t\tp->index = index;\n+\t\tp->pathspec = pathspec;\n+\t\tp->offset = offset;\n+\t\tp->nr = work;\n+\t\toffset += work;\n+\t\tif (pthread_create(&p->pthread, NULL, preload_thread, p))\n+\t\t\tdie(\"unable to create threaded lstat\");\n+\t}\n+\tfor (i = 0; i < threads; i++) {\n+\t\tstruct thread_data *p = data+i;\n+\t\tif (pthread_join(p->pthread, NULL))\n+\t\t\tdie(\"unable to join threaded lstat\");\n+\t}\n+}\n+/* Hacky hack-hack mostly ends */\n+\n int run_diff_files(struct rev_info *revs, unsigned int option)\n {\n \tint entries, i;\n@@ -68,6 +140,8 @@ int run_diff_files(struct rev_info *revs, unsigned int option)\n \tif (diff_unmerged_stage < 0)\n \t\tdiff_unmerged_stage = 2;\n \tentries = active_nr;\n+\tif (parallel_lstats)\n+\t\tpreload_uptodate(revs->prune_data, &the_index);\n \tsymcache[0] = '\\0';\n \tfor (i = 0; i < entries; i++) {\n \t\tstruct stat st;\n"},{"id":"95742","messageId":"alpine.LFD.2.00.0811131545071.3468@nehalem.linux-foundation.org","threadId":"16279","inReplyTo":"885649360811131523h2e10dc44x8603c9793dae03b8@mail.gmail.com","subject":"Re: hosting git on a nfs","fromName":"Linus Torvalds","fromEmail":"torvalds@linux-foundation.org","sentAt":"2008-11-13T23:48:30Z","receivedAt":"2008-11-13T23:48:30Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Thu, 13 Nov 2008, James Pickens wrote:\n> \n> I'm trying to test this and so far I'm not seeing any effect.\n\nOk, try my second version. The first version would get _some_ parallelism, \nbut very little due to all the threads often trying to just look up in the \nsame directory. The second version should hopefully have less of that \neffect.\n\nAlso, under Linux, try it with caches cleared in between runs, so that you \ncan avoid any issues of the Linux client caching the directory lookup data \n(which linux _will_ do - I think the default timeout is 30 seconds or \nsomething like that):\n\n\techo 3 > /proc/sys/vm/drop_caches ; time git diff\n\nwhich should hopefully get you more reliable timings.\n\n\t\t\tLinus\n"},{"id":"95746","messageId":"alpine.LNX.2.00.0811140001070.5938@reaper.quantumfyre.co.uk","threadId":"16279","inReplyTo":"alpine.LFD.2.00.0811131518070.3468@nehalem.linux-foundation.org","subject":"Re: hosting git on a nfs","fromName":"Julian Phillips","fromEmail":"julian@quantumfyre.co.uk","sentAt":"2008-11-14T00:04:35Z","receivedAt":"2008-11-14T00:04:35Z","isPatch":false,"sender":{"key":"julian@quantumfyre.co.uk","avatar":"https://avatars.githubusercontent.com/u/948888?v=4"},"body":"On Thu, 13 Nov 2008, Linus Torvalds wrote:\n\n> The good news is that I seem to actualluy see a bit of a win from this\n> even on a disk, now that the kernel doesn't serialize things. So it may\n> be worth it. So I have some hope that it actually helps on NFS too.\n> The numbers for five runs (with clearing of the caches in between, of\n> course) are:\n>\n> Before:\n>\n> \t0.01user 0.23system 0:10.87elapsed 2%CPU\n> \t0.04user 0.19system 0:10.86elapsed 2%CPU\n> \t0.03user 0.26system 0:10.82elapsed 2%CPU\n> \t0.02user 0.27system 0:12.67elapsed 2%CPU\n> \t0.01user 0.22system 0:10.86elapsed 2%CPU\n>\n> After:\n>\n> \t0.03user 0.26system 0:07.88elapsed 3%CPU\n> \t0.02user 0.25system 0:07.63elapsed 3%CPU\n> \t0.01user 0.26system 0:08.62elapsed 3%CPU\n> \t0.01user 0.26system 0:07.27elapsed 3%CPU\n> \t0.05user 0.28system 0:08.61elapsed 3%CPU\n>\n> so it really does seem like it has possibly given a 30% improvement in\n> cold-cache performance even on a disk.\n\nOn an NFS kernel checkout I get the following elapsed times:\n\nmaster:\n\n0:02.78\n0:02.70\n0:02.43\n0:02.28\n0:02.71\n0:02.80\n0:02.60\n0:02.06\n0:02.00\n\nmaster + new patch:\n\n0:00.77\n0:00.83\n0:01.02\n0:00.77\n0:00.91\n0:00.78\n0:00.78\n0:01.09\n0:01.00\n\n-- \nJulian\n\n ---\nThere are no accidents whatsoever in the universe.\n \t\t-- Baba Ram Dass\n"},{"id":"95747","messageId":"371xaQfxsMMQ-9LK24q-nhcS4loEggn8Cj3J1IzfMbzzYDGE6HKbQQ@cipher.nrlssc.navy.mil","threadId":"16279","inReplyTo":"alpine.LFD.2.00.0811131518070.3468@nehalem.linux-foundation.org","subject":"Re: hosting git on a nfs","fromName":"Brandon Casey","fromEmail":"casey@nrlssc.navy.mil","sentAt":"2008-11-14T00:14:13Z","receivedAt":"2008-11-14T00:14:13Z","isPatch":false,"sender":{"key":"drafnel@gmail.com","avatar":"https://avatars.githubusercontent.com/u/921167?v=4"},"body":"\n\nSecond version.\n\n\nHot cache (over NFS):\n\nBefore:\n\n0.01user 0.11system 0:00.13elapsed 95%CPU\n0.01user 0.11system 0:00.13elapsed 96%CPU\n0.01user 0.12system 0:00.13elapsed 97%CPU\n\nAfter:\n\n0.01user 0.09system 0:00.11elapsed 95%CPU\n0.01user 0.09system 0:00.11elapsed 95%CPU\n0.01user 0.09system 0:00.11elapsed 95%CPU\n\n\nCold cache* (over NFS):\n\nBefore:\n\n0.01user 0.31system 0:04.40elapsed 7%CPU\n0.01user 0.31system 0:06.47elapsed 5%CPU\n0.01user 0.26system 0:04.19elapsed 6%CPU\n0.01user 0.30system 0:04.99elapsed 6%CPU\n\nAfter:\n\n0.01user 0.46system 0:01.39elapsed 34%CPU\n0.01user 0.41system 0:00.88elapsed 47%CPU\n0.01user 0.45system 0:01.16elapsed 40%CPU\n0.01user 0.45system 0:00.99elapsed 47%CPU\n0.01user 0.45system 0:01.02elapsed 45%CPU\n\n\n* Note: I can't do 'echo 3 > /proc/sys/vm/drop_caches', so cold\n cache means 'sleep 60' between git diff (30 wasn't long enough).\n\n-brandon\n"},{"id":"95749","messageId":"alpine.LFD.2.00.0811131630470.3468@nehalem.linux-foundation.org","threadId":"16279","inReplyTo":"371xaQfxsMMQ-9LK24q-nhcS4loEggn8Cj3J1IzfMbzzYDGE6HKbQQ@cipher.nrlssc.navy.mil","subject":"Re: hosting git on a nfs","fromName":"Linus Torvalds","fromEmail":"torvalds@linux-foundation.org","sentAt":"2008-11-14T00:38:15Z","receivedAt":"2008-11-14T00:38:15Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Thu, 13 Nov 2008, Brandon Casey wrote:\n>\n> Cold cache* (over NFS):\n> \n> Before:\n> \n> 0.01user 0.31system 0:04.40elapsed 7%CPU\n> 0.01user 0.31system 0:06.47elapsed 5%CPU\n> 0.01user 0.26system 0:04.19elapsed 6%CPU\n> 0.01user 0.30system 0:04.99elapsed 6%CPU\n> \n> After:\n> \n> 0.01user 0.46system 0:01.39elapsed 34%CPU\n> 0.01user 0.41system 0:00.88elapsed 47%CPU\n> 0.01user 0.45system 0:01.16elapsed 40%CPU\n> 0.01user 0.45system 0:00.99elapsed 47%CPU\n> 0.01user 0.45system 0:01.02elapsed 45%CPU\n\nOk, both you and Julian do seem to be getting a nice speedup from this.\n\nI'll clean it up a bit and make a less hacky version. And I'll try to make \nit work for \"git status\" and friends too.\n\n\t\tLinus\n"},{"id":"95753","messageId":"D0569391-2DB2-46A1-9355-769FF1B1DE1B@frim.nl","threadId":"16279","inReplyTo":"alpine.LFD.2.00.0811131630470.3468@nehalem.linux-foundation.org","subject":"Re: hosting git on a nfs","fromName":"Pieter de Bie","fromEmail":"pieter@frim.nl","sentAt":"2008-11-14T00:59:15Z","receivedAt":"2008-11-14T00:59:15Z","isPatch":false,"sender":{"key":"pieter@frim.nl","avatar":null},"body":"\nOn Nov 14, 2008, at 1:38 AM, Linus Torvalds wrote:\n\n> Ok, both you and Julian do seem to be getting a nice speedup from \n> this.\n>\n> I'll clean it up a bit and make a less hacky version. And I'll try \n> to make\n> it work for \"git status\" and friends too.\n\nI have two more datapoints.\n\nThe first is OS X 10.5 on a local HFS repository. git.git goes down \nfrom ~70 to ~63ms and mozilla's repository (30000) files goes from \n~350ms to ~230ms (yeah, HFS sucks).\n\nUsing AFP for the same repositories, the first goes down from ~80 to \n~67ms, the second from ~70 seconds to ~50 seconds, which is a nice \nspeedup :)\n\n- Pieter\n"},{"id":"95754","messageId":"alpine.LFD.2.00.0811131707090.3468@nehalem.linux-foundation.org","threadId":"16279","inReplyTo":"alpine.LFD.2.00.0811131630470.3468@nehalem.linux-foundation.org","subject":"Re: hosting git on a nfs","fromName":"Linus Torvalds","fromEmail":"torvalds@linux-foundation.org","sentAt":"2008-11-14T01:15:12Z","receivedAt":"2008-11-14T01:15:12Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Thu, 13 Nov 2008, Linus Torvalds wrote:\n> \n> I'll clean it up a bit and make a less hacky version. And I'll try to make \n> it work for \"git status\" and friends too.\n\nOk, this is a no-longer-totally-hacky thing, which also adds support for \ndoing the same for \"git status\". I haven't actually done any timings, but \nthe preload algorithm is all the same. The interface is just a much more \nnatural one.\n\nNOTE NOTE NOTE! It may not be totally hacky, but it's still not \n\"finished\". There's a few more things to look at:\n\n - maybe there are other users of \"read_cache()\" that want to do the \n preloading. I just did \"git diff\" and \"git status\" (where the stuff \n that \"git commit\" does falls out of the status changes)\n\n - I do think the thing should be more configurable. The \"ten threads\" \n approach makes no sense for people who have single-core CPU's and tend \n to have their trees cached - it will just slow things down for that \n case. So there should probably be some config option to turn this on \n and off.\n\n - It would be really cool to find some way to automatically notice when \n the tree is hot-cached and we might as well just be linear. I don't \n know exactly what it might be, but one of the nice things is that the \n preloading is _entirely_ optimistic, and we could just stop it in the \n middle if we notice that there's no upside.\n\n - Somebody else should take a look at it in general. It may work, but \n maybe there's something stupid I'm doing, or maybe somebody else can \n come up with a clever thing.\n\nSo I think this is probably my last version for now, and let's see if \nothers can find improvements. I'm not ashamed of it any more. There's \nstill a \"Hacky hack start\" comment, it's about the whole config thing. \n\n\t\t\tLinus\n\n---\nFrom: Linus Torvalds \nDate: Thu, 13 Nov 2008 16:36:30 -0800\nSubject: [PATCH] Add cache preload facility\n\nThis can do the lstat() storm in parallel, giving potentially much\nimproved performance for cold-cache cases or things like NFS that have\nweak metadata caching.\n\nJust use \"read_cache_preload()\" instead of \"read_cache()\" to force an\noptimistic preload of the index stat data. The function takes a\npathspec as its argument, allowing us to preload only the relevant\nportion of the index.\n\nSigned-off-by: Linus Torvalds \n---\n Makefile | 1 +\n builtin-commit.c | 8 ++--\n builtin-diff-files.c | 4 +-\n builtin-diff.c | 8 ++--\n cache.h | 2 +\n preload-index.c | 83 ++++++++++++++++++++++++++++++++++++++++++++++++++\n 6 files changed, 96 insertions(+), 10 deletions(-)\n\ndiff --git a/Makefile b/Makefile\nindex 920cb42..8387005 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -495,6 +495,7 @@ LIB_OBJS += write_or_die.o\n LIB_OBJS += ws.o\n LIB_OBJS += wt-status.o\n LIB_OBJS += xdiff-interface.o\n+LIB_OBJS += preload-index.o\n \n BUILTIN_OBJS += builtin-add.o\n BUILTIN_OBJS += builtin-annotate.o\ndiff --git a/builtin-commit.c b/builtin-commit.c\nindex 93ca496..90b976e 100644\n--- a/builtin-commit.c\n+++ b/builtin-commit.c\n@@ -225,18 +225,18 @@ static char *prepare_index(int argc, const char **argv, const char *prefix)\n \n \tif (interactive) {\n \t\tinteractive_add(argc, argv, prefix);\n-\t\tif (read_cache() < 0)\n+\t\tif (read_cache_preload(NULL) < 0)\n \t\t\tdie(\"index file corrupt\");\n \t\tcommit_style = COMMIT_AS_IS;\n \t\treturn get_index_file();\n \t}\n \n-\tif (read_cache() < 0)\n-\t\tdie(\"index file corrupt\");\n-\n \tif (*argv)\n \t\tpathspec = get_pathspec(prefix, argv);\n \n+\tif (read_cache_preload(pathspec) < 0)\n+\t\tdie(\"index file corrupt\");\n+\n \t/*\n \t * Non partial, non as-is commit.\n \t *\ndiff --git a/builtin-diff-files.c b/builtin-diff-files.c\nindex 2b578c7..5b64011 100644\n--- a/builtin-diff-files.c\n+++ b/builtin-diff-files.c\n@@ -59,8 +59,8 @@ int cmd_diff_files(int argc, const char **argv, const char *prefix)\n \t (rev.diffopt.output_format & DIFF_FORMAT_PATCH))\n \t\trev.combine_merges = rev.dense_combined_merges = 1;\n \n-\tif (read_cache() < 0) {\n-\t\tperror(\"read_cache\");\n+\tif (read_cache_preload(rev.diffopt.paths) < 0) {\n+\t\tperror(\"read_cache_preload\");\n \t\treturn -1;\n \t}\n \tresult = run_diff_files(&rev, options);\ndiff --git a/builtin-diff.c b/builtin-diff.c\nindex 82d4dda..b9a2b37 100644\n--- a/builtin-diff.c\n+++ b/builtin-diff.c\n@@ -134,8 +134,8 @@ static int builtin_diff_index(struct rev_info *revs,\n \t revs->max_count != -1 || revs->min_age != -1 ||\n \t revs->max_age != -1)\n \t\tusage(builtin_diff_usage);\n-\tif (read_cache() < 0) {\n-\t\tperror(\"read_cache\");\n+\tif (read_cache_preload(revs->diffopt.paths) < 0) {\n+\t\tperror(\"read_cache_preload\");\n \t\treturn -1;\n \t}\n \treturn run_diff_index(revs, cached);\n@@ -234,8 +234,8 @@ static int builtin_diff_files(struct rev_info *revs, int argc, const char **argv\n \t\trevs->combine_merges = revs->dense_combined_merges = 1;\n \n \tsetup_work_tree();\n-\tif (read_cache() < 0) {\n-\t\tperror(\"read_cache\");\n+\tif (read_cache_preload(revs->diffopt.paths) < 0) {\n+\t\tperror(\"read_cache_preload\");\n \t\treturn -1;\n \t}\n \tresult = run_diff_files(revs, options);\ndiff --git a/cache.h b/cache.h\nindex 6be60ea..c7b69e8 100644\n--- a/cache.h\n+++ b/cache.h\n@@ -262,6 +262,7 @@ static inline void remove_name_hash(struct cache_entry *ce)\n \n #define read_cache() read_index(&the_index)\n #define read_cache_from(path) read_index_from(&the_index, (path))\n+#define read_cache_preload(pathspec) read_index_preload(&the_index, (pathspec))\n #define is_cache_unborn() is_index_unborn(&the_index)\n #define read_cache_unmerged() read_index_unmerged(&the_index)\n #define write_cache(newfd, cache, entries) write_index(&the_index, (newfd))\n@@ -368,6 +369,7 @@ extern int init_db(const char *template_dir, unsigned int flags);\n \n /* Initialize and use the cache information */\n extern int read_index(struct index_state *);\n+extern int read_index_preload(struct index_state *, const char **pathspec);\n extern int read_index_from(struct index_state *, const char *path);\n extern int is_index_unborn(struct index_state *);\n extern int read_index_unmerged(struct index_state *);\ndiff --git a/preload-index.c b/preload-index.c\nnew file mode 100644\nindex 0000000..e322c27\n--- /dev/null\n+++ b/preload-index.c\n@@ -0,0 +1,83 @@\n+/*\n+ * Copyright (C) 2008 Linus Torvalds\n+ */\n+#include \n+#include \"cache.h\"\n+\n+/* Hacky hack-hack start */\n+#define MAX_PARALLEL (100)\n+int parallel_lstats = 1;\n+\n+struct thread_data {\n+\tpthread_t pthread;\n+\tstruct index_state *index;\n+\tconst char **pathspec;\n+\tint offset, nr;\n+};\n+\n+static void *preload_thread(void *_data)\n+{\n+\tint nr;\n+\tstruct thread_data *p = _data;\n+\tstruct index_state *index = p->index;\n+\tstruct cache_entry **cep = index->cache + p->offset;\n+\n+\tnr = p->nr;\n+\tif (nr + p->offset > index->cache_nr)\n+\t\tnr = index->cache_nr - p->offset;\n+\n+\tdo {\n+\t\tstruct cache_entry *ce = *cep++;\n+\t\tstruct stat st;\n+\n+\t\tif (ce_stage(ce))\n+\t\t\tcontinue;\n+\t\tif (ce_uptodate(ce))\n+\t\t\tcontinue;\n+\t\tif (!ce_path_match(ce, p->pathspec))\n+\t\t\tcontinue;\n+\t\tif (lstat(ce->name, &st))\n+\t\t\tcontinue;\n+\t\tif (ie_match_stat(index, ce, &st, 0))\n+\t\t\tcontinue;\n+\t\tce_mark_uptodate(ce);\n+\t} while (--nr > 0);\n+\treturn NULL;\n+}\n+\n+static void preload_index(struct index_state *index, const char **pathspec)\n+{\n+\tint i, work, offset;\n+\tint threads = index->cache_nr / 100;\n+\tstruct thread_data data[MAX_PARALLEL];\n+\n+\tif (threads < 2)\n+\t\treturn;\n+\tif (threads > MAX_PARALLEL)\n+\t\tthreads = MAX_PARALLEL;\n+\toffset = 0;\n+\twork = (index->cache_nr + threads - 1) / threads;\n+\tfor (i = 0; i < threads; i++) {\n+\t\tstruct thread_data *p = data+i;\n+\t\tp->index = index;\n+\t\tp->pathspec = pathspec;\n+\t\tp->offset = offset;\n+\t\tp->nr = work;\n+\t\toffset += work;\n+\t\tif (pthread_create(&p->pthread, NULL, preload_thread, p))\n+\t\t\tdie(\"unable to create threaded lstat\");\n+\t}\n+\tfor (i = 0; i < threads; i++) {\n+\t\tstruct thread_data *p = data+i;\n+\t\tif (pthread_join(p->pthread, NULL))\n+\t\t\tdie(\"unable to join threaded lstat\");\n+\t}\n+}\n+\n+int read_index_preload(struct index_state *index, const char **pathspec)\n+{\n+\tint retval = read_index(index);\n+\n+\tpreload_index(index, pathspec);\n+\treturn retval;\n+}\n"},{"id":"95761","messageId":"885649360811131933webae91w134dce4c5c0ccf89@mail.gmail.com","threadId":"16279","inReplyTo":"alpine.LFD.2.00.0811131707090.3468@nehalem.linux-foundation.org","subject":"Re: hosting git on a nfs","fromName":"James Pickens","fromEmail":"jepicken@gmail.com","sentAt":"2008-11-14T03:33:15Z","receivedAt":"2008-11-14T03:33:15Z","isPatch":false,"sender":{"key":"jepicken@gmail.com","avatar":null},"body":"On Thu, Nov 13, 2008 at 6:15 PM, Linus Torvalds\n wrote:\n> - maybe there are other users of \"read_cache()\" that want to do the\n> preloading. I just did \"git diff\" and \"git status\" (where the stuff\n> that \"git commit\" does falls out of the status changes)\n\nI wonder if there are other completely different parts of git that could\nbenefit from multi threading when the work tree is on nfs? I'm thinking\nspecifically of 'git checkout', since while testing this patch I\nhappened to do a 'git pull' that resulted in several thousand new files\nbeing created, and the \"Checking out files\" part took *forever* to run.\n\nAnd FWIW, I timed 50 iterations of 'git diff', and the average runtime\ndropped from 11.7s to 2.8s after this patch. A nice improvement.\n\nJames\n"},{"id":"95762","messageId":"alpine.LFD.2.00.0811132044460.3468@nehalem.linux-foundation.org","threadId":"16279","inReplyTo":"885649360811131933webae91w134dce4c5c0ccf89@mail.gmail.com","subject":"Re: hosting git on a nfs","fromName":"Linus Torvalds","fromEmail":"torvalds@linux-foundation.org","sentAt":"2008-11-14T05:01:20Z","receivedAt":"2008-11-14T05:01:20Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Thu, 13 Nov 2008, James Pickens wrote:\n> \n> I wonder if there are other completely different parts of git that could\n> benefit from multi threading when the work tree is on nfs?\n\nI'm sure there are. That said, threading things is usually really quite \npainful. The only reason this preloading was easy to do was that we really \nhad all the data structures laid out beautifully for this, and I had spent \na lot of effort earlier on a whole series of \"avoid duplicate lstat()\" \nchanges, which gave us that whole ce_uptodate() thing, and all normal \ncases already taking advantage of it, and the \"uptodate\" bit being \npercolated along all the paths.\n\nIf it hadn't been for that, it would have been much nastier to do.\n\nAs it was, there was literally just a simple little extra phase to fill in \nall teh data structures that we already had set up in parallel.\n\n> I'm thinking specifically of 'git checkout', since while testing this \n> patch I happened to do a 'git pull' that resulted in several thousand \n> new files being created, and the \"Checking out files\" part took \n> *forever* to run.\n\nNow, the good news is that the actual work-tree part of checking things \nout is probably pretty amenable to the same kind of parallelization, for \nlargely the same reasons: the whole checking out thing is already done in \nmultiple phases with all error handling done before-hand. So we will have \nbuilt up all our data structures earlier, and set the CE_UPDATE bit, and \nthen there's just a final \"push it all out\" phase.\n\nSo CE_UPTODATE and CE_UPDATE are really very similar in that sense - \nexcept at opposite ends of the pipeline. The CE_UPTODATE bit marks a name \nentry as matching the filesystem data (and allows all later phases to \navoid doing the expensive lstat()s), while the CE_UPDATE (and CE_REMOVE) \nbits allow us to do all our complex work in-memory without committing it \nto disk, and then we push it out in one go.\n\nSo if you want to multi-thread checkout, you literally need to just thread \nthe last for-loop in unpack-trees.c:check_updates() (the CE_UPDATE loop \nthat does \"checkout_entry()\" over the whole index). \n\n> And FWIW, I timed 50 iterations of 'git diff', and the average runtime\n> dropped from 11.7s to 2.8s after this patch. A nice improvement.\n\nVery impressive. That said, I suspect you get a \"superlinear\" improvement \nbecause once it gets faster, the kernel cache also works better, since you \ncan do more loops without having the NFS attributes time out.\n\nWhether that kind of effect happens much in actual practice is debatable, \nalthough it's quite possible that it will work the same way in some \nscripting schenarios.\n\n\t\t\tLinus\n"},{"id":"95782","messageId":"491D76A7.9090809@drmicha.warpmail.net","threadId":"16279","inReplyTo":"alpine.LFD.2.00.0811131707090.3468@nehalem.linux-foundation.org","subject":"Re: hosting git on a nfs","fromName":"Michael J Gruber","fromEmail":"git@drmicha.warpmail.net","sentAt":"2008-11-14T13:01:27Z","receivedAt":"2008-11-14T13:01:27Z","isPatch":false,"sender":{"key":"git@grubix.eu","avatar":"https://avatars.githubusercontent.com/u/233215?v=4"},"body":"Linus Torvalds venit, vidit, dixit 14.11.2008 02:15:\n> \n> On Thu, 13 Nov 2008, Linus Torvalds wrote:\n>> I'll clean it up a bit and make a less hacky version. And I'll try to make \n>> it work for \"git status\" and friends too.\n> \n> Ok, this is a no-longer-totally-hacky thing, which also adds support for \n> doing the same for \"git status\". I haven't actually done any timings, but \n> the preload algorithm is all the same. The interface is just a much more \n> natural one.\n> \n> NOTE NOTE NOTE! It may not be totally hacky, but it's still not \n> \"finished\". There's a few more things to look at:\n\nDoing nicely already. Over here, cold cache (i.e. sleep 70) \"git status\"\ngoes from 0.8s to around 0.4s. Nice. Hot cache goes from 0.18s to 0.23s.\nThis is really worthwhile if you hack for more than 60s between\ndiffs/stats ;)\n\nThis is already with the server doing some caching (first status was 8s\nor so; hardly reproducible), which I can't control.\n\nMichael\n"},{"id":"95789","messageId":"f73f7ab80811140631h44f0c712h5656cd6664d91380@mail.gmail.com","threadId":"16279","inReplyTo":"alpine.LFD.2.00.0811131707090.3468@nehalem.linux-foundation.org","subject":"Re: hosting git on a nfs","fromName":"Kyle Moffett","fromEmail":"kyle@moffetthome.net","sentAt":"2008-11-14T14:31:09Z","receivedAt":"2008-11-14T14:31:09Z","isPatch":false,"sender":{"key":"kyle@moffetthome.net","avatar":null},"body":"On Thu, Nov 13, 2008 at 8:15 PM, Linus Torvalds\n wrote:\n> - It would be really cool to find some way to automatically notice when\n> the tree is hot-cached and we might as well just be linear. I don't\n> know exactly what it might be, but one of the nice things is that the\n> preloading is _entirely_ optimistic, and we could just stop it in the\n> middle if we notice that there's no upside.\n\nPerhaps, rather... notice when the tree is *cold*-cached. At that\npoint you're already in a pseudo-slowpath and starting several threads\nwon't be noticeable at all compared to the time savings. You could\nprobably very easily do that by keeping track of the time as you start\nwith the normal single-threaded scan of directory entries. If the\nentries are all equally slow/fast, there most likely won't be any\nbenefit at all to spawning extra threads. You might use some cutoff\non number of entries to test before you assume everything is hot and\nstop worrying about the time. If there's a lot of variability in the\nfirst few thousand then it means some caches are cold and spawning\nextra threads will probably help.\n\nCheers,\nKyle Moffett\n"},{"id":"95810","messageId":"vzAozXmaOLEpyz-7DHx4nMusAdaTsFp7iZ8xfFsgAIraex6_wfvyuw@cipher.nrlssc.navy.mil","threadId":"16279","inReplyTo":"alpine.LFD.2.00.0811131707090.3468@nehalem.linux-foundation.org","subject":"Re: hosting git on a nfs","fromName":"Brandon Casey","fromEmail":"casey@nrlssc.navy.mil","sentAt":"2008-11-14T18:32:01Z","receivedAt":"2008-11-14T18:32:01Z","isPatch":false,"sender":{"key":"drafnel@gmail.com","avatar":"https://avatars.githubusercontent.com/u/921167?v=4"},"body":"Linus Torvalds wrote:\n> \n> On Thu, 13 Nov 2008, Linus Torvalds wrote:\n>> I'll clean it up a bit and make a less hacky version. And I'll try to make \n>> it work for \"git status\" and friends too.\n> \n> Ok, this is a no-longer-totally-hacky thing, which also adds support for \n> doing the same for \"git status\". I haven't actually done any timings, but \n> the preload algorithm is all the same. The interface is just a much more \n> natural one.\n\ngit status > /dev/null\n\nBefore:\n\n 0.06user 0.37system 0:03.04elapsed 14%CPU\n 0.07user 0.36system 0:03.25elapsed 13%CPU\n 0.07user 0.36system 0:03.08elapsed 14%CPU\n\nAfter:\n\n 0.06user 0.53system 0:01.02elapsed 58%CPU\n 0.05user 0.54system 0:01.01elapsed 58%CPU\n 0.06user 0.52system 0:01.04elapsed 57%CPU\n\n\ngit diff > /dev/null\n\nBefore:\n\n 0.02user 0.31system 0:02.88elapsed 11%CPU\n 0.01user 0.32system 0:02.53elapsed 13%CPU\n 0.01user 0.28system 0:02.78elapsed 10%CPU\n\nAfter:\n\n 0.01user 0.47system 0:00.52elapsed 92%CPU\n 0.01user 0.48system 0:00.52elapsed 94%CPU\n 0.01user 0.47system 0:00.54elapsed 88%CPU\n\n\nI have no explanation for why the diff numbers are different from yesterday.\nCould be that there was some nightly cron job running last night which\nslowed things down. Still, the same ~5x speedup is observed!\n\nWow! Thanks!\n\n-brandon\n"},{"id":"95816","messageId":"alpine.LFD.2.00.0811141109580.3468@nehalem.linux-foundation.org","threadId":"16279","inReplyTo":"vzAozXmaOLEpyz-7DHx4nMusAdaTsFp7iZ8xfFsgAIraex6_wfvyuw@cipher.nrlssc.navy.mil","subject":"Re: hosting git on a nfs","fromName":"Linus Torvalds","fromEmail":"torvalds@linux-foundation.org","sentAt":"2008-11-14T19:23:01Z","receivedAt":"2008-11-14T19:23:01Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Fri, 14 Nov 2008, Brandon Casey wrote:\n> \n> I have no explanation for why the diff numbers are different from \n> yesterday. Could be that there was some nightly cron job running last \n> night which slowed things down. Still, the same ~5x speedup is observed!\n\nOne of the things I changed (inadvertently - it was just meant to be a \ntest) was that the last patch had MAX_PARALLEL set to 100 rather than 10. \nThat's definitely overkill.\n\nI also think the thread cost was wrong: it did\n\n\tthreads = index->cache_nr / 100;\n\nto give a first-order \"how many threads do we want\", but the thread \nstartup is likely to be higher than 100 lstat calls, so we probably want \nfewer threads than that. It doesn't much matter for something like the \nLinux kernel, where there are so many files that we'll end up maxing out \nthe threads anyway, but for smaller projects, I suspect a thread cost of \n\"one thread per 500 files\" is more reasonable. You almost certainly don't \nwant to thread anything at all for fewer than a few hundred files.\n\nSo here's a slight incremental update to my patch from yesterday. It also \nadds the config variable, and it defaults to off, so to actually see this \nin action, you now need to add\n\n\t[core]\n\t\tPreloadIndex = true\n\nto your ~/.gitconfig file.\n\nTotally untested. As usual. Maybe it works. Maybe it doesn't.\n\n\t\tLinus\n\n---\n Documentation/config.txt | 12 +++++++++++-\n cache.h | 1 +\n config.c | 5 +++++\n environment.c | 3 +++\n preload-index.c | 18 +++++++++++++-----\n 5 files changed, 33 insertions(+), 6 deletions(-)\n\ndiff --git a/Documentation/config.txt b/Documentation/config.txt\nindex 32dcd64..9260121 100644\n--- a/Documentation/config.txt\n+++ b/Documentation/config.txt\n@@ -411,7 +411,17 @@ core.fsyncobjectfiles::\n This is a total waste of time and effort on a filesystem that orders\n data writes properly, but can be useful for filesystems that do not use\n journalling (traditional UNIX filesystems) or that only journal metadata\n-and not file contents (OS X's HFS+, or Linux ext3 with \"data=writeback\").\n+and not file contents (OS X's HFS+, or Linux ext3 with\n+\"data=writeback\").\n+\n+core.preloadindex::\n+\tEnable parallel index preload for operations like 'git diff'\n++\n+This can speed up operations like 'git diff' and 'git status' especially\n+on filesystems like NFS that have weak caching semantics and thus\n+relatively high IO latencies. With this set to 'true', git will do the\n+index comparison to the filesystem data in parallel, allowing\n+overlapping IO's.\n \n alias.*::\n \tCommand aliases for the linkgit:git[1] command wrapper - e.g.\ndiff --git a/cache.h b/cache.h\nindex 64239fb..685a866 100644\n--- a/cache.h\n+++ b/cache.h\n@@ -460,6 +460,7 @@ extern size_t packed_git_limit;\n extern size_t delta_base_cache_limit;\n extern int auto_crlf;\n extern int fsync_object_files;\n+extern int core_preload_index;\n \n enum safe_crlf {\n \tSAFE_CRLF_FALSE = 0,\ndiff --git a/config.c b/config.c\nindex 67cc1dc..d2fc8f5 100644\n--- a/config.c\n+++ b/config.c\n@@ -490,6 +490,11 @@ static int git_default_core_config(const char *var, const char *value)\n \t\treturn 0;\n \t}\n \n+\tif (!strcmp(var, \"core.preloadindex\")) {\n+\t\tcore_preload_index = git_config_bool(var, value);\n+\t\treturn 0;\n+\t}\n+\n \t/* Add other config variables here and to Documentation/config.txt. */\n \treturn 0;\n }\ndiff --git a/environment.c b/environment.c\nindex bb96ac0..e278bce 100644\n--- a/environment.c\n+++ b/environment.c\n@@ -43,6 +43,9 @@ unsigned whitespace_rule_cfg = WS_DEFAULT_RULE;\n enum branch_track git_branch_track = BRANCH_TRACK_REMOTE;\n enum rebase_setup_type autorebase = AUTOREBASE_NEVER;\n \n+/* Parallel index stat data preload? */\n+int core_preload_index = 0;\n+\n /* This is set by setup_git_dir_gently() and/or git_default_config() */\n char *git_work_tree_cfg;\n static char *work_tree;\ndiff --git a/preload-index.c b/preload-index.c\nindex e322c27..3ce42e0 100644\n--- a/preload-index.c\n+++ b/preload-index.c\n@@ -4,9 +4,14 @@\n #include \n #include \"cache.h\"\n \n-/* Hacky hack-hack start */\n-#define MAX_PARALLEL (100)\n-int parallel_lstats = 1;\n+/*\n+ * Mostly randomly chosen maximum thread counts: we\n+ * cap the parallelism to 20 threads, and we want\n+ * to have at least 500 lstat's per thread for it to\n+ * be worth starting a thread.\n+ */\n+#define MAX_PARALLEL (20)\n+#define THREAD_COST (500)\n \n struct thread_data {\n \tpthread_t pthread;\n@@ -47,10 +52,13 @@ static void *preload_thread(void *_data)\n \n static void preload_index(struct index_state *index, const char **pathspec)\n {\n-\tint i, work, offset;\n-\tint threads = index->cache_nr / 100;\n+\tint threads, i, work, offset;\n \tstruct thread_data data[MAX_PARALLEL];\n \n+\tif (!core_preload_index)\n+\t\treturn;\n+\n+\tthreads = index->cache_nr / THREAD_COST;\n \tif (threads < 2)\n \t\treturn;\n \tif (threads > MAX_PARALLEL)\n"},{"id":"95825","messageId":"7v63mq9iao.fsf@gitster.siamese.dyndns.org","threadId":"16279","inReplyTo":"alpine.LFD.2.00.0811141109580.3468@nehalem.linux-foundation.org","subject":"Re: hosting git on a nfs","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2008-11-14T20:14:23Z","receivedAt":"2008-11-14T20:14:23Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Linus Torvalds writes:\n\n> I also think the thread cost was wrong: it did\n>\n> \tthreads = index->cache_nr / 100;\n>\n> to give a first-order \"how many threads do we want\", but the thread \n> startup is likely to be higher than 100 lstat calls, so we probably want \n> fewer threads than that. It doesn't much matter for something like the \n> Linux kernel, where there are so many files that we'll end up maxing out \n> the threads anyway, but for smaller projects, I suspect a thread cost of \n> \"one thread per 500 files\" is more reasonable. You almost certainly don't \n> want to thread anything at all for fewer than a few hundred files.\n\nIf you have 1000 files in a single directory, do you still want 2 threads\nfollowing the \"1/500\" rule, or they would compete reading the same\ndirectory and using a single thread is better off?\n"},{"id":"95838","messageId":"alpine.LFD.2.00.0811141505100.3468@nehalem.linux-foundation.org","threadId":"16279","inReplyTo":"7v63mq9iao.fsf@gitster.siamese.dyndns.org","subject":"Re: hosting git on a nfs","fromName":"Linus Torvalds","fromEmail":"torvalds@linux-foundation.org","sentAt":"2008-11-14T23:10:24Z","receivedAt":"2008-11-14T23:10:24Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Fri, 14 Nov 2008, Junio C Hamano wrote:\n> \n> If you have 1000 files in a single directory, do you still want 2 threads\n> following the \"1/500\" rule, or they would compete reading the same\n> directory and using a single thread is better off?\n\nWell, first off, the \"single directory\" thing is really a Linux kernel \ndeficiency, and it's entirely possible that it doesn't even exist on other \nsystems. Linux has a very special directory cache (dcache) model that is \npretty unique - it's part of why cached 'lstat()' calls are so cheap on \nLinux - but it is also part of the reason for why we serialize lookups \nwhen we do miss in the cache (*).\n\nSecondly, anybody who has a thousand tracked files in a single directory \ncan damn well blame themselves for being stupid. So I don't think it's a \ncase that is worth worrying too much about. Git will slow down for that \nkind of situation for other reasons (ie a lot of the tree pruning \noptimization won't work for projects that have large flat directories).\n\nSo i wouldn't worry about it. That said, with the second patch, we default \nto having people enable this explicitly, so it's something that people can \ndecide on their own.\n\n\t\t\tLinus\n\n(*) That said - the Linux dcache consistency is just _one_ reason why we \nserialize lookups. I would not be in the least surprised if other OS's \nhave the exact same issue. I'd love to fix it in Linux, but quiet \nhonestly, it has never actually come up before now, and we've literally \nworked on multi-threading the _cached_ case, not the uncached one.\n"},{"id":"95874","messageId":"7vtza95h01.fsf@gitster.siamese.dyndns.org","threadId":"16279","inReplyTo":"alpine.LFD.2.00.0811141109580.3468@nehalem.linux-foundation.org","subject":"[PATCH] Makefile: introduce NO_PTHREADS","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2008-11-15T12:08:14Z","receivedAt":"2008-11-15T12:08:14Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"This introduces make variable NO_PTHREADS for platforms that lack the\nsupport for pthreads library or people who do not want to use it for\nwhatever reason. When defined, it makes the multi-threaded index\npreloading into a no-op, and also disables threaded delta searching by\npack-objects.\n\nSigned-off-by: Junio C Hamano \n---\n\n * I notice a handful platforms do not define THREADED_DELTA_SEARCH, and\n on them Linus's preload-index.c is the first source file that includes\n , which may result in breakages.\n\n Makefile | 10 +++++++++-\n config.mak.in | 1 +\n configure.ac | 5 +++++\n preload-index.c | 9 +++++++++\n 4 files changed, 24 insertions(+), 1 deletions(-)\n\ndiff --git c/Makefile w/Makefile\nindex acac0ae..ffc9531 100644\n--- c/Makefile\n+++ w/Makefile\n@@ -90,6 +90,8 @@ all::\n #\n # Define NO_MMAP if you want to avoid mmap.\n #\n+# Define NO_PTHREADS if you do not have or do not want to use Pthreads.\n+#\n # Define NO_PREAD if you have a problem with pread() system call (e.g.\n # cygwin.dll before v1.5.22).\n #\n@@ -1018,9 +1020,15 @@ ifdef INTERNAL_QSORT\n \tCOMPAT_OBJS += compat/qsort.o\n endif\n \n+ifdef NO_PTHREADS\n+\tTHREADED_DELTA_SEARCH =\n+\tBASIC_CFLAGS += -DNO_PTHREADS\n+else\n+\tEXTLIBS += $(PTHREAD_LIBS)\n+endif\n+\n ifdef THREADED_DELTA_SEARCH\n \tBASIC_CFLAGS += -DTHREADED_DELTA_SEARCH\n-\tEXTLIBS += $(PTHREAD_LIBS)\n \tLIB_OBJS += thread-utils.o\n endif\n ifdef DIR_HAS_BSD_GROUP_SEMANTICS\ndiff --git c/config.mak.in w/config.mak.in\nindex ea7705c..14dfb21 100644\n--- c/config.mak.in\n+++ w/config.mak.in\n@@ -51,4 +51,5 @@ OLD_ICONV=@OLD_ICONV@\n NO_DEFLATE_BOUND=@NO_DEFLATE_BOUND@\n FREAD_READS_DIRECTORIES=@FREAD_READS_DIRECTORIES@\n SNPRINTF_RETURNS_BOGUS=@SNPRINTF_RETURNS_BOGUS@\n+NO_PTHREADS=@NO_PTHREADS@\n PTHREAD_LIBS=@PTHREAD_LIBS@\ndiff --git c/configure.ac w/configure.ac\nindex 4256742..8821b50 100644\n--- c/configure.ac\n+++ w/configure.ac\n@@ -490,6 +490,8 @@ AC_SUBST(NO_MKDTEMP)\n # Define NO_SYMLINK_HEAD if you never want .git/HEAD to be a symbolic link.\n # Enable it on Windows. By default, symrefs are still used.\n #\n+# Define NO_PTHREADS if we do not have pthreads\n+#\n # Define PTHREAD_LIBS to the linker flag used for Pthread support.\n AC_LANG_CONFTEST([AC_LANG_PROGRAM(\n [[#include ]],\n@@ -502,9 +504,12 @@ else\n ${CC} -lpthread conftest.c -o conftest.o > /dev/null 2>&1\n if test $? -eq 0;then\n PTHREAD_LIBS=\"-lpthread\"\n+ else\n+ NO_PTHREADS=UnfortunatelyYes\n fi\n fi\n AC_SUBST(PTHREAD_LIBS)\n+AC_SUBST(NO_PTHREADS)\n \n ## Site configuration (override autodetection)\n ## --with-PACKAGE[=ARG] and --without-PACKAGE\ndiff --git c/preload-index.c w/preload-index.c\nindex 6253578..3ae83dc 100644\n--- c/preload-index.c\n+++ w/preload-index.c\n@@ -2,6 +2,14 @@\n * Copyright (C) 2008 Linus Torvalds\n */\n #include \"cache.h\"\n+\n+#ifdef NO_PTHREADS\n+static void preload_index(struct index_state *index, const char **pathspec)\n+{\n+\t; /* nothing */\n+}\n+#else\n+\n #include \n \n /*\n@@ -81,6 +89,7 @@ static void preload_index(struct index_state *index, const char **pathspec)\n \t\t\tdie(\"unable to join threaded lstat\");\n \t}\n }\n+#endif\n \n int read_index_preload(struct index_state *index, const char **pathspec)\n {\n"},{"id":"95891","messageId":"alpine.LFD.2.00.0811150915240.3468@nehalem.linux-foundation.org","threadId":"16279","inReplyTo":"7vtza95h01.fsf@gitster.siamese.dyndns.org","subject":"Re: [PATCH] Makefile: introduce NO_PTHREADS","fromName":"Linus Torvalds","fromEmail":"torvalds@linux-foundation.org","sentAt":"2008-11-15T17:15:52Z","receivedAt":"2008-11-15T17:15:52Z","isPatch":true,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Sat, 15 Nov 2008, Junio C Hamano wrote:\n>\n> This introduces make variable NO_PTHREADS for platforms that lack the\n> support for pthreads library or people who do not want to use it for\n> whatever reason. When defined, it makes the multi-threaded index\n> preloading into a no-op, and also disables threaded delta searching by\n> pack-objects.\n\nAck. Makes sense.\n\n\t\tLinus\n"},{"id":"96016","messageId":"e2b179460811170203v41e54ecclc3d6526bcc0fe928@mail.gmail.com","threadId":"16279","inReplyTo":"alpine.LFD.2.00.0811150915240.3468@nehalem.linux-foundation.org","subject":"Re: [PATCH] Makefile: introduce NO_PTHREADS","fromName":"Mike Ralphson","fromEmail":"mike.ralphson@gmail.com","sentAt":"2008-11-17T10:03:18Z","receivedAt":"2008-11-17T10:03:18Z","isPatch":true,"sender":{"key":"mike.ralphson@gmail.com","avatar":"https://avatars.githubusercontent.com/u/21603?v=4"},"body":"2008/11/15 Linus Torvalds :\n>\n> On Sat, 15 Nov 2008, Junio C Hamano wrote:\n>>\n>> This introduces make variable NO_PTHREADS for platforms that lack the\n>> support for pthreads library or people who do not want to use it for\n>> whatever reason. When defined, it makes the multi-threaded index\n>> preloading into a no-op, and also disables threaded delta searching by\n>> pack-objects.\n>\n> Ack. Makes sense.\n\nI'd be minded to make this the default on AIX to keep the prerequisite\nlist as small as possible, then people can opt-in for the performance\nbenefits if required.\n\nI'll wait a little while to see if anyone else reports the same for\nother platforms and then submit a patch.\n\nMike\n"},{"id":"96020","messageId":"7vd4guzmdz.fsf@gitster.siamese.dyndns.org","threadId":"16279","inReplyTo":"e2b179460811170203v41e54ecclc3d6526bcc0fe928@mail.gmail.com","subject":"Re: [PATCH] Makefile: introduce NO_PTHREADS","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2008-11-17T10:18:16Z","receivedAt":"2008-11-17T10:18:16Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Mike Ralphson\" writes:\n\n> 2008/11/15 Linus Torvalds :\n>>\n>> On Sat, 15 Nov 2008, Junio C Hamano wrote:\n>>>\n>>> This introduces make variable NO_PTHREADS for platforms that lack the\n>>> support for pthreads library or people who do not want to use it for\n>>> whatever reason. When defined, it makes the multi-threaded index\n>>> preloading into a no-op, and also disables threaded delta searching by\n>>> pack-objects.\n>>\n>> Ack. Makes sense.\n>\n> I'd be minded to make this the default on AIX to keep the prerequisite\n> list as small as possible, then people can opt-in for the performance\n> benefits if required.\n>\n> I'll wait a little while to see if anyone else reports the same for\n> other platforms and then submit a patch.\n\nThanks.\n\nI expect to be slow this week til just before Thanksgiving, so the more\npeople we have to keep an eye on the areas they excel at, the better for\nall of us and certainly it would help me a lot.\n"},{"id":"96023","messageId":"492148AD.1090604@viscovery.net","threadId":"16279","inReplyTo":"e2b179460811170203v41e54ecclc3d6526bcc0fe928@mail.gmail.com","subject":"Re: [PATCH] Makefile: introduce NO_PTHREADS","fromName":"Johannes Sixt","fromEmail":"j.sixt@viscovery.net","sentAt":"2008-11-17T10:34:21Z","receivedAt":"2008-11-17T10:34:21Z","isPatch":true,"sender":{"key":"j6t@kdbg.org","avatar":"https://avatars.githubusercontent.com/u/14810926?v=4"},"body":"Mike Ralphson schrieb:\n> 2008/11/15 Linus Torvalds :\n>> On Sat, 15 Nov 2008, Junio C Hamano wrote:\n>>> This introduces make variable NO_PTHREADS for platforms that lack the\n>>> support for pthreads library or people who do not want to use it for\n>>> whatever reason. When defined, it makes the multi-threaded index\n>>> preloading into a no-op, and also disables threaded delta searching by\n>>> pack-objects.\n>> Ack. Makes sense.\n> \n> I'd be minded to make this the default on AIX to keep the prerequisite\n> list as small as possible, then people can opt-in for the performance\n> benefits if required.\n\nIs pthreads not a standard shipment on AIX? I would set NO_PTHREADS only\nif we know in advance that there are many installations without pthreads.\n(And I don't know what the situation is.)\n\nBTW, this needs to be squashed in, because we don't have pthreads on Windows:\n\ndiff --git a/Makefile b/Makefile\nindex ffc9531..3a30b8c 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -769,6 +769,7 @@ ifneq (,$(findstring MINGW,$(uname_S)))\n \tNO_STRCASESTR = YesPlease\n \tNO_STRLCPY = YesPlease\n \tNO_MEMMEM = YesPlease\n+\tNO_PTHREADS = YesPlease\n \tNEEDS_LIBICONV = YesPlease\n \tOLD_ICONV = YesPlease\n \tNO_C99_FORMAT = YesPlease\n\n-- Hannes\n"},{"id":"96026","messageId":"e2b179460811170245t1845cc66h7cb2a18c43a79359@mail.gmail.com","threadId":"16279","inReplyTo":"492148AD.1090604@viscovery.net","subject":"Re: [PATCH] Makefile: introduce NO_PTHREADS","fromName":"Mike Ralphson","fromEmail":"mike.ralphson@gmail.com","sentAt":"2008-11-17T10:45:28Z","receivedAt":"2008-11-17T10:45:28Z","isPatch":true,"sender":{"key":"mike.ralphson@gmail.com","avatar":"https://avatars.githubusercontent.com/u/21603?v=4"},"body":"2008/11/17 Johannes Sixt :\n> Mike Ralphson schrieb:\n>> 2008/11/15 Linus Torvalds :\n>>> On Sat, 15 Nov 2008, Junio C Hamano wrote:\n>>>> This introduces make variable NO_PTHREADS for platforms that lack the\n>>>> support for pthreads library or people who do not want to use it for\n>>>> whatever reason. When defined, it makes the multi-threaded index\n>>>> preloading into a no-op, and also disables threaded delta searching by\n>>>> pack-objects.\n>>> Ack. Makes sense.\n>>\n>> I'd be minded to make this the default on AIX to keep the prerequisite\n>> list as small as possible, then people can opt-in for the performance\n>> benefits if required.\n>\n> Is pthreads not a standard shipment on AIX? I would set NO_PTHREADS only\n> if we know in advance that there are many installations without pthreads.\n> (And I don't know what the situation is.)\n\nI should have dug a bit further, it seems to be present on my 5.3\nmachines but I still need to determine whether it got installed by\ndefault. Either way it must need some other link flags...\n\n> BTW, this needs to be squashed in, because we don't have pthreads on Windows:\n>\n> diff --git a/Makefile b/Makefile\n> index ffc9531..3a30b8c 100644\n> --- a/Makefile\n> +++ b/Makefile\n> @@ -769,6 +769,7 @@ ifneq (,$(findstring MINGW,$(uname_S)))\n> NO_STRCASESTR = YesPlease\n> NO_STRLCPY = YesPlease\n> NO_MEMMEM = YesPlease\n> + NO_PTHREADS = YesPlease\n> NEEDS_LIBICONV = YesPlease\n> OLD_ICONV = YesPlease\n> NO_C99_FORMAT = YesPlease\n>\n\nTa. Ok to add your S-o-B on a squashed patch?\n\nMike\n"},{"id":"96035","messageId":"4921548E.6070802@viscovery.net","threadId":"16279","inReplyTo":"e2b179460811170245t1845cc66h7cb2a18c43a79359@mail.gmail.com","subject":"Re: [PATCH] Makefile: introduce NO_PTHREADS","fromName":"Johannes Sixt","fromEmail":"j.sixt@viscovery.net","sentAt":"2008-11-17T11:25:02Z","receivedAt":"2008-11-17T11:25:02Z","isPatch":true,"sender":{"key":"j6t@kdbg.org","avatar":"https://avatars.githubusercontent.com/u/14810926?v=4"},"body":"Mike Ralphson schrieb:\n> 2008/11/17 Johannes Sixt :\n>> Mike Ralphson schrieb:\n>>> 2008/11/15 Linus Torvalds :\n>>>> On Sat, 15 Nov 2008, Junio C Hamano wrote:\n>>>>> This introduces make variable NO_PTHREADS for platforms that lack the\n>>>>> support for pthreads library or people who do not want to use it for\n>>>>> whatever reason. When defined, it makes the multi-threaded index\n>>>>> preloading into a no-op, and also disables threaded delta searching by\n>>>>> pack-objects.\n>>>> Ack. Makes sense.\n>>> I'd be minded to make this the default on AIX to keep the prerequisite\n>>> list as small as possible, then people can opt-in for the performance\n>>> benefits if required.\n>> Is pthreads not a standard shipment on AIX? I would set NO_PTHREADS only\n>> if we know in advance that there are many installations without pthreads.\n>> (And I don't know what the situation is.)\n> \n> I should have dug a bit further, it seems to be present on my 5.3\n> machines but I still need to determine whether it got installed by\n> default. Either way it must need some other link flags...\n\nI tried compiling with THREADED_DELTA_SEARCH=Yes, and it fails with\n\n CC builtin-pack-objects.o\nIn file included from /usr/include/sys/pri.h:29,\n from /usr/include/sys/sched.h:38,\n from /usr/include/sched.h:52,\n from /usr/include/pthread.h:43,\n from builtin-pack-objects.c:22:\n/usr/include/sys/proc.h:203: parse error before \"crid_t\"\n/usr/include/sys/proc.h:212: parse error before \"p_class\"\n/usr/include/sys/proc.h:355: parse error before '}' token\n\n:-( Maybe NO_PTHREADS is indeed the safer choice? I'm not going to dig\ninto this today, though. (I'm on AIX 4.3.something.)\n\n>> BTW, this needs to be squashed in, because we don't have pthreads on Windows:\n>>\n>> diff --git a/Makefile b/Makefile\n>> index ffc9531..3a30b8c 100644\n>> --- a/Makefile\n>> +++ b/Makefile\n>> @@ -769,6 +769,7 @@ ifneq (,$(findstring MINGW,$(uname_S)))\n>> NO_STRCASESTR = YesPlease\n>> NO_STRLCPY = YesPlease\n>> NO_MEMMEM = YesPlease\n>> + NO_PTHREADS = YesPlease\n>> NEEDS_LIBICONV = YesPlease\n>> OLD_ICONV = YesPlease\n>> NO_C99_FORMAT = YesPlease\n>>\n> \n> Ta. Ok to add your S-o-B on a squashed patch?\n\nSure. Use this address please:\n\nSigned-off-by: Johannes Sixt \n\n-- Hannes\n"},{"id":"96042","messageId":"7vvdumwbnn.fsf@gitster.siamese.dyndns.org","threadId":"16279","inReplyTo":"alpine.LFD.2.00.0811150915240.3468@nehalem.linux-foundation.org","subject":"Re: [PATCH] Makefile: introduce NO_PTHREADS","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2008-11-17T16:38:20Z","receivedAt":"2008-11-17T16:38:20Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Linus Torvalds writes:\n\n> On Sat, 15 Nov 2008, Junio C Hamano wrote:\n>>\n>> This introduces make variable NO_PTHREADS for platforms that lack the\n>> support for pthreads library or people who do not want to use it for\n>> whatever reason. When defined, it makes the multi-threaded index\n>> preloading into a no-op, and also disables threaded delta searching by\n>> pack-objects.\n>\n> Ack. Makes sense.\n\nHmm, I started getting random segfaults that sometimes reproduces. For\nexample, this is what I just got from \"git diff --stat $some_commit\".\n\n#0 0x00002b952568b090 in strlen () from /lib/libc.so.6\n#1 0x000000000044ed42 in git_checkattr (path=0x2b95284b3970 \"parse-options.c\",\n num=3, check=0x41000e90) at attr.c:512\n#2 0x0000000000458921 in convert_to_git (\n path=0x2b95284b3970 \"parse-options.c\",\n src=0x2aaaaaabb000
, len=12594,\n dst=0x41000f30, checksafe=SAFE_CRLF_FALSE) at convert.c:578\n#3 0x0000000000489ac3 in index_mem (\n sha1=0x41000ff0 \"\\210f�⽡x�\\207�� 7R}\\217\\032��\", buf=0x2aaaaaabb000,\n size=12594, write_object=0, type=,\n path=0x2f2f2f2f2f2f2f2f
)\n at sha1_file.c:2451\n#4 0x0000000000489c3d in index_fd (\n sha1=0x41000ff0 \"\\210f�⽡x�\\207�� 7R}\\217\\032��\", fd=5,\n st=, write_object=0, type=OBJ_BLOB,\n path=0x2b95284b3970 \"parse-options.c\") at sha1_file.c:2483\n#5 0x000000000047857a in ce_modified_check_fs (ce=0x2b95284b3930,\n st=0x41001080) at read-cache.c:92\n#6 0x00000000004786a2 in ie_match_stat (istate=0x71c860, ce=0x2b95284b3930,\n st=0x41001080, options=) at read-cache.c:282\n#7 0x0000000000497e65 in preload_thread (_data=)\n at preload-index.c:46\n#8 0x00002b9525964017 in start_thread () from /lib/libpthread.so.0\n#9 0x00002b95256da5bd in clone () from /lib/libc.so.6\n#10 0x0000000000000000 in ?? ()\n\nI suspect that the callpath around ce_modified_check_fs() uses a buffer\nobtained from path.c:get_pathname() and parallel threads stomp on each\nother, but I do not have time to debug this right now (I will be on a\n14-hour flight in a few hours).\n"},{"id":"96043","messageId":"7vskpqwbhz.fsf@gitster.siamese.dyndns.org","threadId":"16279","inReplyTo":"alpine.LFD.2.00.0811150915240.3468@nehalem.linux-foundation.org","subject":"Re: [PATCH] Makefile: introduce NO_PTHREADS","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2008-11-17T16:41:44Z","receivedAt":"2008-11-17T16:41:44Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Linus Torvalds writes:\n\n> On Sat, 15 Nov 2008, Junio C Hamano wrote:\n>>\n>> This introduces make variable NO_PTHREADS for platforms that lack the\n>> support for pthreads library or people who do not want to use it for\n>> whatever reason. When defined, it makes the multi-threaded index\n>> preloading into a no-op, and also disables threaded delta searching by\n>> pack-objects.\n>\n> Ack. Makes sense.\n\nHmm, I started getting random segfaults that sometimes reproduce. For\nexample, this is what I just got from \"git diff --stat $some_commit\".\n\n#0 0x00002b952568b090 in strlen () from /lib/libc.so.6\n#1 0x000000000044ed42 in git_checkattr (path=0x2b95284b3970 \"parse-options.c\",\n num=3, check=0x41000e90) at attr.c:512\n#2 0x0000000000458921 in convert_to_git (\n path=0x2b95284b3970 \"parse-options.c\",\n src=0x2aaaaaabb000
, len=12594,\n dst=0x41000f30, checksafe=SAFE_CRLF_FALSE) at convert.c:578\n#3 0x0000000000489ac3 in index_mem (\n sha1=0x41000ff0 \"\\210f�⽡x�\\207�� 7R}\\217\\032��\", buf=0x2aaaaaabb000,\n size=12594, write_object=0, type=,\n path=0x2f2f2f2f2f2f2f2f
)\n at sha1_file.c:2451\n#4 0x0000000000489c3d in index_fd (\n sha1=0x41000ff0 \"\\210f�⽡x�\\207�� 7R}\\217\\032��\", fd=5,\n st=, write_object=0, type=OBJ_BLOB,\n path=0x2b95284b3970 \"parse-options.c\") at sha1_file.c:2483\n#5 0x000000000047857a in ce_modified_check_fs (ce=0x2b95284b3930,\n st=0x41001080) at read-cache.c:92\n#6 0x00000000004786a2 in ie_match_stat (istate=0x71c860, ce=0x2b95284b3930,\n st=0x41001080, options=) at read-cache.c:282\n#7 0x0000000000497e65 in preload_thread (_data=)\n at preload-index.c:46\n#8 0x00002b9525964017 in start_thread () from /lib/libpthread.so.0\n#9 0x00002b95256da5bd in clone () from /lib/libc.so.6\n#10 0x0000000000000000 in ?? ()\n\nUnfortunately, I do not have time to debug this right now (I will be on a\n14-hour flight in a few hours).\n"},{"id":"96045","messageId":"alpine.LFD.2.00.0811170846390.3468@nehalem.linux-foundation.org","threadId":"16279","inReplyTo":"7vvdumwbnn.fsf@gitster.siamese.dyndns.org","subject":"Re: [PATCH] Makefile: introduce NO_PTHREADS","fromName":"Linus Torvalds","fromEmail":"torvalds@linux-foundation.org","sentAt":"2008-11-17T16:47:19Z","receivedAt":"2008-11-17T16:47:19Z","isPatch":true,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Mon, 17 Nov 2008, Junio C Hamano wrote:\n> \n> I suspect that the callpath around ce_modified_check_fs() uses a buffer\n> obtained from path.c:get_pathname() and parallel threads stomp on each\n> other, but I do not have time to debug this right now (I will be on a\n> 14-hour flight in a few hours).\n\nOh, damn. I had forgotten that check_fs() doesn't just do a \"lstat()\" any \nmore. You're right.\n\nLet me look at it. \n\n\t\tLinus\n"},{"id":"96047","messageId":"alpine.LFD.2.00.0811170850170.3468@nehalem.linux-foundation.org","threadId":"16279","inReplyTo":"alpine.LFD.2.00.0811170846390.3468@nehalem.linux-foundation.org","subject":"Fix index preloading for racy dirty case","fromName":"Linus Torvalds","fromEmail":"torvalds@linux-foundation.org","sentAt":"2008-11-17T17:01:20Z","receivedAt":"2008-11-17T17:01:20Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nIn the threaded index preloading case, we must be sure to always use the \nCE_MATCH_RACY_IS_DIRTY flag when calling ie_match_stat(), in order to make \nsure that we only ever look at the stat() data, and don't try to do \nanything fancy.\n\nBecause most of git internals are not thread-safe, and must not be called \nin parallel.\n\nOtherwise, what happens is that if the timestamps indicate that an entry \n_might_ be dirty, we might start actually comparing filesystem data with \nthe object database. And we mustn't do that, because that would involve\nlooking up and creating the object structure, and that whole code sequence \nwith read_sha1_file() where we look up and add objects to the hashes is \ndefinitely not thread-safe.\n\nNor do we want to add locking, because the whole point of the preload was \nto be simple and not affect anything else. With CE_MATCH_RACY_IS_DIRTY, we \nget what we wanted, and we'll just leave the hard cases well alone, to be \ndone later in the much simpler serial case.\n\nSigned-off-by: Linus Torvalds \n---\n\nOn Mon, 17 Nov 2008, Linus Torvalds wrote:\n> \n> Oh, damn. I had forgotten that check_fs() doesn't just do a \"lstat()\" any \n> more. You're right.\n\nNever mind the \"any more\". I don't think it ever did.\n\nBut I do think that this is trivially fixed, and I should have thought \nabout it. And while I didn't reproduce your SIGSEGV, I think this trivial \npatch should fix it.\n\nSorry about the mindfart.\n\n preload-index.c | 2 +-\n 1 files changed, 1 insertions(+), 1 deletions(-)\n\ndiff --git a/preload-index.c b/preload-index.c\nindex 3ce42e0..a6a6bdb 100644\n--- a/preload-index.c\n+++ b/preload-index.c\n@@ -43,7 +43,7 @@ static void *preload_thread(void *_data)\n \t\t\tcontinue;\n \t\tif (lstat(ce->name, &st))\n \t\t\tcontinue;\n-\t\tif (ie_match_stat(index, ce, &st, 0))\n+\t\tif (ie_match_stat(index, ce, &st, CE_MATCH_RACY_IS_DIRTY))\n \t\t\tcontinue;\n \t\tce_mark_uptodate(ce);\n \t} while (--nr > 0);\n"},{"id":"96812","messageId":"4933A058.3050101@viscovery.net","threadId":"16279","inReplyTo":"4921548E.6070802@viscovery.net","subject":"Re: [PATCH] Makefile: introduce NO_PTHREADS","fromName":"Johannes Sixt","fromEmail":"j.sixt@viscovery.net","sentAt":"2008-12-01T08:29:12Z","receivedAt":"2008-12-01T08:29:12Z","isPatch":true,"sender":{"key":"j6t@kdbg.org","avatar":"https://avatars.githubusercontent.com/u/14810926?v=4"},"body":"Johannes Sixt schrieb:\n> Mike Ralphson schrieb:\n>> 2008/11/17 Johannes Sixt :\n>>> Mike Ralphson schrieb:\n>>>> 2008/11/15 Linus Torvalds :\n>>>>> On Sat, 15 Nov 2008, Junio C Hamano wrote:\n>>>>>> This introduces make variable NO_PTHREADS for platforms that lack the\n>>>>>> support for pthreads library or people who do not want to use it for\n>>>>>> whatever reason. When defined, it makes the multi-threaded index\n>>>>>> preloading into a no-op, and also disables threaded delta searching by\n>>>>>> pack-objects.\n>>>>> Ack. Makes sense.\n>>>> I'd be minded to make this the default on AIX to keep the prerequisite\n>>>> list as small as possible, then people can opt-in for the performance\n>>>> benefits if required.\n>>> Is pthreads not a standard shipment on AIX? I would set NO_PTHREADS only\n>>> if we know in advance that there are many installations without pthreads.\n>>> (And I don't know what the situation is.)\n>> I should have dug a bit further, it seems to be present on my 5.3\n>> machines but I still need to determine whether it got installed by\n>> default. Either way it must need some other link flags...\n> \n> I tried compiling with THREADED_DELTA_SEARCH=Yes, and it fails with\n> \n> CC builtin-pack-objects.o\n> In file included from /usr/include/sys/pri.h:29,\n> from /usr/include/sys/sched.h:38,\n> from /usr/include/sched.h:52,\n> from /usr/include/pthread.h:43,\n> from builtin-pack-objects.c:22:\n> /usr/include/sys/proc.h:203: parse error before \"crid_t\"\n> /usr/include/sys/proc.h:212: parse error before \"p_class\"\n> /usr/include/sys/proc.h:355: parse error before '}' token\n> \n> :-( Maybe NO_PTHREADS is indeed the safer choice? I'm not going to dig\n> into this today, though. (I'm on AIX 4.3.something.)\n> \n>>> BTW, this needs to be squashed in, because we don't have pthreads on Windows:\n>>>\n>>> diff --git a/Makefile b/Makefile\n>>> index ffc9531..3a30b8c 100644\n>>> --- a/Makefile\n>>> +++ b/Makefile\n>>> @@ -769,6 +769,7 @@ ifneq (,$(findstring MINGW,$(uname_S)))\n>>> NO_STRCASESTR = YesPlease\n>>> NO_STRLCPY = YesPlease\n>>> NO_MEMMEM = YesPlease\n>>> + NO_PTHREADS = YesPlease\n>>> NEEDS_LIBICONV = YesPlease\n>>> OLD_ICONV = YesPlease\n>>> NO_C99_FORMAT = YesPlease\n>>>\n>> Ta. Ok to add your S-o-B on a squashed patch?\n> \n> Sure. Use this address please:\n> \n> Signed-off-by: Johannes Sixt \n\nMike,\n\nyou said you would resend the patch, but I think you forgot about it.\nWould you do that now, please?\n\n-- Hannes\n"},{"id":"96813","messageId":"e3f230850812010048x4b8038fegaa97b247f7851a3e@mail.gmail.com","threadId":"16279","inReplyTo":"4933A058.3050101@viscovery.net","subject":"Re: [PATCH] Makefile: introduce NO_PTHREADS","fromName":"dhruva","fromEmail":"dhruvakm@gmail.com","sentAt":"2008-12-01T08:48:00Z","receivedAt":"2008-12-01T08:48:00Z","isPatch":true,"sender":{"key":"dhruvakm@gmail.com","avatar":"https://gravatar.com/avatar/96fe022a95b60fd0de7f9f521364d964cdc7c46be5cfef279f8d4379e49e19a6?d=mp&s=160"},"body":"Hello,\n I am able to compile and use git by defining THREADED_DELTA_SEARCH\nand copying setjmp.h from mingw (www.mingw.org) to msys include folder\nwith small changes to it. If it does add value, we should try to\nenable this by default.\n\n-dhruva\n\nOn Mon, Dec 1, 2008 at 1:59 PM, Johannes Sixt wrote:\n> Johannes Sixt schrieb:\n>> Mike Ralphson schrieb:\n>>> 2008/11/17 Johannes Sixt :\n>>>> Mike Ralphson schrieb:\n>>>>> 2008/11/15 Linus Torvalds :\n>>>>>> On Sat, 15 Nov 2008, Junio C Hamano wrote:\n>>>>>>> This introduces make variable NO_PTHREADS for platforms that lack the\n>>>>>>> support for pthreads library or people who do not want to use it for\n>>>>>>> whatever reason. When defined, it makes the multi-threaded index\n>>>>>>> preloading into a no-op, and also disables threaded delta searching by\n>>>>>>> pack-objects.\n>>>>>> Ack. Makes sense.\n>>>>> I'd be minded to make this the default on AIX to keep the prerequisite\n>>>>> list as small as possible, then people can opt-in for the performance\n>>>>> benefits if required.\n>>>> Is pthreads not a standard shipment on AIX? I would set NO_PTHREADS only\n>>>> if we know in advance that there are many installations without pthreads.\n>>>> (And I don't know what the situation is.)\n>>> I should have dug a bit further, it seems to be present on my 5.3\n>>> machines but I still need to determine whether it got installed by\n>>> default. Either way it must need some other link flags...\n>>\n>> I tried compiling with THREADED_DELTA_SEARCH=Yes, and it fails with\n>>\n>> CC builtin-pack-objects.o\n>> In file included from /usr/include/sys/pri.h:29,\n>> from /usr/include/sys/sched.h:38,\n>> from /usr/include/sched.h:52,\n>> from /usr/include/pthread.h:43,\n>> from builtin-pack-objects.c:22:\n>> /usr/include/sys/proc.h:203: parse error before \"crid_t\"\n>> /usr/include/sys/proc.h:212: parse error before \"p_class\"\n>> /usr/include/sys/proc.h:355: parse error before '}' token\n>>\n>> :-( Maybe NO_PTHREADS is indeed the safer choice? I'm not going to dig\n>> into this today, though. (I'm on AIX 4.3.something.)\n>>\n>>>> BTW, this needs to be squashed in, because we don't have pthreads on Windows:\n>>>>\n>>>> diff --git a/Makefile b/Makefile\n>>>> index ffc9531..3a30b8c 100644\n>>>> --- a/Makefile\n>>>> +++ b/Makefile\n>>>> @@ -769,6 +769,7 @@ ifneq (,$(findstring MINGW,$(uname_S)))\n>>>> NO_STRCASESTR = YesPlease\n>>>> NO_STRLCPY = YesPlease\n>>>> NO_MEMMEM = YesPlease\n>>>> + NO_PTHREADS = YesPlease\n>>>> NEEDS_LIBICONV = YesPlease\n>>>> OLD_ICONV = YesPlease\n>>>> NO_C99_FORMAT = YesPlease\n>>>>\n>>> Ta. Ok to add your S-o-B on a squashed patch?\n>>\n>> Sure. Use this address please:\n>>\n>> Signed-off-by: Johannes Sixt \n>\n> Mike,\n>\n> you said you would resend the patch, but I think you forgot about it.\n> Would you do that now, please?\n>\n> -- Hannes\n>\n> --\n> To unsubscribe from this list: send the line \"unsubscribe git\" in\n> the body of a message to majordomo@vger.kernel.org\n> More majordomo info at http://vger.kernel.org/majordomo-info.html\n>\n\n\n\n-- \nContents reflect my personal views only!\n"},{"id":"96823","messageId":"e2b179460812010157y29ca5405ta8ff7efea3f2a167@mail.gmail.com","threadId":"16279","inReplyTo":"4933A058.3050101@viscovery.net","subject":"Re: [PATCH] Makefile: introduce NO_PTHREADS","fromName":"Mike Ralphson","fromEmail":"mike.ralphson@gmail.com","sentAt":"2008-12-01T09:57:56Z","receivedAt":"2008-12-01T09:57:56Z","isPatch":true,"sender":{"key":"mike.ralphson@gmail.com","avatar":"https://avatars.githubusercontent.com/u/21603?v=4"},"body":"2008/12/1 Johannes Sixt \n>\n> Johannes Sixt schrieb:\n> > Mike Ralphson schrieb:\n> >> 2008/11/17 Johannes Sixt :\n> >>> Mike Ralphson schrieb:\n> >>>> 2008/11/15 Linus Torvalds :\n> >>>>> On Sat, 15 Nov 2008, Junio C Hamano wrote:\n> >>>>>> This introduces make variable NO_PTHREADS for platforms that lack the\n> >>>>>> support for pthreads library or people who do not want to use it for\n> >>>>>> whatever reason. When defined, it makes the multi-threaded index\n> >>>>>> preloading into a no-op, and also disables threaded delta searching by\n> >>>>>> pack-objects.\n> >>>>> ...\n> >\n> > :-( Maybe NO_PTHREADS is indeed the safer choice? I'm not going to dig\n> > into this today, though. (I'm on AIX 4.3.something.)\n> >\n> >>> BTW, this needs to be squashed in, because we don't have pthreads on Windows:\n> >>>...\n>\n> you said you would resend the patch, but I think you forgot about it.\n> Would you do that now, please?\n\nNot forgotten, just slow. The current state I believe is we should\nhave NO_PTHREADS for AIX < v5. THREADED_DELTA_SEARCH and the new\nmulti-threaded lstat are actually ok on AIX 5.3 at least - though I\ncouldn't see any performance benefit on my repo / hardware\ncombination.\n\nBut now after dhruva's observation I'm unsure what the desired change\nis for Windows. 8-(\n\nMike\n"},{"id":"96866","messageId":"1228147782-9370-1-git-send-email-mike@abacus.co.uk","threadId":"16279","inReplyTo":"4933A058.3050101@viscovery.net","subject":"[PATCH] Makefile: introduce NO_PTHREADS","fromName":"Mike Ralphson","fromEmail":"mike@abacus.co.uk","sentAt":"2008-12-01T16:09:42Z","receivedAt":"2008-12-01T16:09:42Z","isPatch":true,"sender":{"key":"mike@abacus.co.uk","avatar":"https://gravatar.com/avatar/717865f1e9a9197ec082b9f6a12e048210a9da06878e47f165b470c43a9e9175?d=mp&s=160"},"body":"From: Junio C Hamano \n\nSigned-off-by: Junio C Hamano \nSigned-off-by: Johannes Sixt \nSigned-off-by: Mike Ralphson \n---\n\n* I notice a handful platforms do not define THREADED_DELTA_SEARCH, and\n on them Linus's preload-index.c is the first source file that includes\n , which may result in breakages.\n\n Made the default for AIX <5 and Mingw\n\n Author should still show as Junio, apologies if not\n\n Makefile | 17 ++++++++++++++++-\n config.mak.in | 1 +\n configure.ac | 5 +++++\n preload-index.c | 9 +++++++++\n 4 files changed, 31 insertions(+), 1 deletions(-)\n\ndiff --git a/Makefile b/Makefile\nindex d1e2116..5a69a41 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -90,6 +90,8 @@ all::\n #\n # Define NO_MMAP if you want to avoid mmap.\n #\n+# Define NO_PTHREADS if you do not have or do not want to use Pthreads.\n+#\n # Define NO_PREAD if you have a problem with pread() system call (e.g.\n # cygwin.dll before v1.5.22).\n #\n@@ -164,6 +166,7 @@ uname_M := $(shell sh -c 'uname -m 2>/dev/null || echo not')\n uname_O := $(shell sh -c 'uname -o 2>/dev/null || echo not')\n uname_R := $(shell sh -c 'uname -r 2>/dev/null || echo not')\n uname_P := $(shell sh -c 'uname -p 2>/dev/null || echo not')\n+uname_V := $(shell sh -c 'uname -v 2>/dev/null || echo not')\n \n # CFLAGS and LDFLAGS are for the users to override from the command line.\n \n@@ -722,6 +725,11 @@ ifeq ($(uname_S),AIX)\n \tINTERNAL_QSORT = UnfortunatelyYes\n \tNEEDS_LIBICONV=YesPlease\n \tBASIC_CFLAGS += -D_LARGE_FILES\n+\tifneq ($(shell expr \"$(uname_V)\" : '[1234]'),1)\n+\t\tTHREADED_DELTA_SEARCH = YesPlease\n+\telse\n+\t\tNO_PTHREADS = YesPlease\n+\tendif\n endif\n ifeq ($(uname_S),GNU)\n \t# GNU/Hurd\n@@ -766,6 +774,7 @@ ifneq (,$(findstring MINGW,$(uname_S)))\n \tNO_STRCASESTR = YesPlease\n \tNO_STRLCPY = YesPlease\n \tNO_MEMMEM = YesPlease\n+\tNO_PTHREADS = YesPlease\n \tNEEDS_LIBICONV = YesPlease\n \tOLD_ICONV = YesPlease\n \tNO_C99_FORMAT = YesPlease\n@@ -1017,9 +1026,15 @@ ifdef INTERNAL_QSORT\n \tCOMPAT_OBJS += compat/qsort.o\n endif\n \n+ifdef NO_PTHREADS\n+\tTHREADED_DELTA_SEARCH =\n+\tBASIC_CFLAGS += -DNO_PTHREADS\n+else\n+\tEXTLIBS += $(PTHREAD_LIBS)\n+endif\n+\n ifdef THREADED_DELTA_SEARCH\n \tBASIC_CFLAGS += -DTHREADED_DELTA_SEARCH\n-\tEXTLIBS += $(PTHREAD_LIBS)\n \tLIB_OBJS += thread-utils.o\n endif\n ifdef DIR_HAS_BSD_GROUP_SEMANTICS\ndiff --git a/config.mak.in b/config.mak.in\nindex ea7705c..14dfb21 100644\n--- a/config.mak.in\n+++ b/config.mak.in\n@@ -51,4 +51,5 @@ OLD_ICONV=@OLD_ICONV@\n NO_DEFLATE_BOUND=@NO_DEFLATE_BOUND@\n FREAD_READS_DIRECTORIES=@FREAD_READS_DIRECTORIES@\n SNPRINTF_RETURNS_BOGUS=@SNPRINTF_RETURNS_BOGUS@\n+NO_PTHREADS=@NO_PTHREADS@\n PTHREAD_LIBS=@PTHREAD_LIBS@\ndiff --git a/configure.ac b/configure.ac\nindex 4256742..8821b50 100644\n--- a/configure.ac\n+++ b/configure.ac\n@@ -490,6 +490,8 @@ AC_SUBST(NO_MKDTEMP)\n # Define NO_SYMLINK_HEAD if you never want .git/HEAD to be a symbolic link.\n # Enable it on Windows. By default, symrefs are still used.\n #\n+# Define NO_PTHREADS if we do not have pthreads\n+#\n # Define PTHREAD_LIBS to the linker flag used for Pthread support.\n AC_LANG_CONFTEST([AC_LANG_PROGRAM(\n [[#include ]],\n@@ -502,9 +504,12 @@ else\n ${CC} -lpthread conftest.c -o conftest.o > /dev/null 2>&1\n if test $? -eq 0;then\n PTHREAD_LIBS=\"-lpthread\"\n+ else\n+ NO_PTHREADS=UnfortunatelyYes\n fi\n fi\n AC_SUBST(PTHREAD_LIBS)\n+AC_SUBST(NO_PTHREADS)\n \n ## Site configuration (override autodetection)\n ## --with-PACKAGE[=ARG] and --without-PACKAGE\ndiff --git a/preload-index.c b/preload-index.c\nindex a685583..88edc5f 100644\n--- a/preload-index.c\n+++ b/preload-index.c\n@@ -2,6 +2,14 @@\n * Copyright (C) 2008 Linus Torvalds\n */\n #include \"cache.h\"\n+\n+#ifdef NO_PTHREADS\n+static void preload_index(struct index_state *index, const char **pathspec)\n+{\n+\t; /* nothing */\n+}\n+#else\n+\n #include \n \n /*\n@@ -81,6 +89,7 @@ static void preload_index(struct index_state *index, const char **pathspec)\n \t\t\tdie(\"unable to join threaded lstat\");\n \t}\n }\n+#endif\n \n int read_index_preload(struct index_state *index, const char **pathspec)\n {\n-- \n1.6.0.2.229.g1293c.dirty\n"},{"id":"96867","messageId":"1228148005-9404-1-git-send-email-mike@abacus.co.uk","threadId":"16279","inReplyTo":"4933A058.3050101@viscovery.net","subject":"[PATCH] Makefile: introduce NO_PTHREADS","fromName":"Mike Ralphson","fromEmail":"mike@abacus.co.uk","sentAt":"2008-12-01T16:13:25Z","receivedAt":"2008-12-01T16:13:25Z","isPatch":true,"sender":{"key":"mike@abacus.co.uk","avatar":"https://gravatar.com/avatar/717865f1e9a9197ec082b9f6a12e048210a9da06878e47f165b470c43a9e9175?d=mp&s=160"},"body":"From: Junio C Hamano \n\n This introduces make variable NO_PTHREADS for platforms that lack the\n support for pthreads library or people who do not want to use it for\n whatever reason. When defined, it makes the multi-threaded index\n preloading into a no-op, and also disables threaded delta searching by\n pack-objects.\n\nSigned-off-by: Junio C Hamano \nSigned-off-by: Johannes Sixt \nSigned-off-by: Mike Ralphson \n---\n\n* I notice a handful platforms do not define THREADED_DELTA_SEARCH, and\n on them Linus's preload-index.c is the first source file that includes\n , which may result in breakages.\n\n Made the default for AIX <5 and Mingw\n\n With correct commit message. Sorry for the previous attempt.\n\n Makefile | 17 ++++++++++++++++-\n config.mak.in | 1 +\n configure.ac | 5 +++++\n preload-index.c | 9 +++++++++\n 4 files changed, 31 insertions(+), 1 deletions(-)\n\ndiff --git a/Makefile b/Makefile\nindex d1e2116..5a69a41 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -90,6 +90,8 @@ all::\n #\n # Define NO_MMAP if you want to avoid mmap.\n #\n+# Define NO_PTHREADS if you do not have or do not want to use Pthreads.\n+#\n # Define NO_PREAD if you have a problem with pread() system call (e.g.\n # cygwin.dll before v1.5.22).\n #\n@@ -164,6 +166,7 @@ uname_M := $(shell sh -c 'uname -m 2>/dev/null || echo not')\n uname_O := $(shell sh -c 'uname -o 2>/dev/null || echo not')\n uname_R := $(shell sh -c 'uname -r 2>/dev/null || echo not')\n uname_P := $(shell sh -c 'uname -p 2>/dev/null || echo not')\n+uname_V := $(shell sh -c 'uname -v 2>/dev/null || echo not')\n \n # CFLAGS and LDFLAGS are for the users to override from the command line.\n \n@@ -722,6 +725,11 @@ ifeq ($(uname_S),AIX)\n \tINTERNAL_QSORT = UnfortunatelyYes\n \tNEEDS_LIBICONV=YesPlease\n \tBASIC_CFLAGS += -D_LARGE_FILES\n+\tifneq ($(shell expr \"$(uname_V)\" : '[1234]'),1)\n+\t\tTHREADED_DELTA_SEARCH = YesPlease\n+\telse\n+\t\tNO_PTHREADS = YesPlease\n+\tendif\n endif\n ifeq ($(uname_S),GNU)\n \t# GNU/Hurd\n@@ -766,6 +774,7 @@ ifneq (,$(findstring MINGW,$(uname_S)))\n \tNO_STRCASESTR = YesPlease\n \tNO_STRLCPY = YesPlease\n \tNO_MEMMEM = YesPlease\n+\tNO_PTHREADS = YesPlease\n \tNEEDS_LIBICONV = YesPlease\n \tOLD_ICONV = YesPlease\n \tNO_C99_FORMAT = YesPlease\n@@ -1017,9 +1026,15 @@ ifdef INTERNAL_QSORT\n \tCOMPAT_OBJS += compat/qsort.o\n endif\n \n+ifdef NO_PTHREADS\n+\tTHREADED_DELTA_SEARCH =\n+\tBASIC_CFLAGS += -DNO_PTHREADS\n+else\n+\tEXTLIBS += $(PTHREAD_LIBS)\n+endif\n+\n ifdef THREADED_DELTA_SEARCH\n \tBASIC_CFLAGS += -DTHREADED_DELTA_SEARCH\n-\tEXTLIBS += $(PTHREAD_LIBS)\n \tLIB_OBJS += thread-utils.o\n endif\n ifdef DIR_HAS_BSD_GROUP_SEMANTICS\ndiff --git a/config.mak.in b/config.mak.in\nindex ea7705c..14dfb21 100644\n--- a/config.mak.in\n+++ b/config.mak.in\n@@ -51,4 +51,5 @@ OLD_ICONV=@OLD_ICONV@\n NO_DEFLATE_BOUND=@NO_DEFLATE_BOUND@\n FREAD_READS_DIRECTORIES=@FREAD_READS_DIRECTORIES@\n SNPRINTF_RETURNS_BOGUS=@SNPRINTF_RETURNS_BOGUS@\n+NO_PTHREADS=@NO_PTHREADS@\n PTHREAD_LIBS=@PTHREAD_LIBS@\ndiff --git a/configure.ac b/configure.ac\nindex 4256742..8821b50 100644\n--- a/configure.ac\n+++ b/configure.ac\n@@ -490,6 +490,8 @@ AC_SUBST(NO_MKDTEMP)\n # Define NO_SYMLINK_HEAD if you never want .git/HEAD to be a symbolic link.\n # Enable it on Windows. By default, symrefs are still used.\n #\n+# Define NO_PTHREADS if we do not have pthreads\n+#\n # Define PTHREAD_LIBS to the linker flag used for Pthread support.\n AC_LANG_CONFTEST([AC_LANG_PROGRAM(\n [[#include ]],\n@@ -502,9 +504,12 @@ else\n ${CC} -lpthread conftest.c -o conftest.o > /dev/null 2>&1\n if test $? -eq 0;then\n PTHREAD_LIBS=\"-lpthread\"\n+ else\n+ NO_PTHREADS=UnfortunatelyYes\n fi\n fi\n AC_SUBST(PTHREAD_LIBS)\n+AC_SUBST(NO_PTHREADS)\n \n ## Site configuration (override autodetection)\n ## --with-PACKAGE[=ARG] and --without-PACKAGE\ndiff --git a/preload-index.c b/preload-index.c\nindex a685583..88edc5f 100644\n--- a/preload-index.c\n+++ b/preload-index.c\n@@ -2,6 +2,14 @@\n * Copyright (C) 2008 Linus Torvalds\n */\n #include \"cache.h\"\n+\n+#ifdef NO_PTHREADS\n+static void preload_index(struct index_state *index, const char **pathspec)\n+{\n+\t; /* nothing */\n+}\n+#else\n+\n #include \n \n /*\n@@ -81,6 +89,7 @@ static void preload_index(struct index_state *index, const char **pathspec)\n \t\t\tdie(\"unable to join threaded lstat\");\n \t}\n }\n+#endif\n \n int read_index_preload(struct index_state *index, const char **pathspec)\n {\n-- \n1.6.0.2.229.g1293c.dirty\n"},{"id":"96931","messageId":"4934E6BC.9040203@viscovery.net","threadId":"16279","inReplyTo":"1228148005-9404-1-git-send-email-mike@abacus.co.uk","subject":"Re: [PATCH] Makefile: introduce NO_PTHREADS","fromName":"Johannes Sixt","fromEmail":"j.sixt@viscovery.net","sentAt":"2008-12-02T07:41:48Z","receivedAt":"2008-12-02T07:41:48Z","isPatch":true,"sender":{"key":"j6t@kdbg.org","avatar":"https://avatars.githubusercontent.com/u/14810926?v=4"},"body":"Mike Ralphson schrieb:\n> From: Junio C Hamano \n> \n> This introduces make variable NO_PTHREADS for platforms that lack the\n> support for pthreads library or people who do not want to use it for\n> whatever reason. When defined, it makes the multi-threaded index\n> preloading into a no-op, and also disables threaded delta searching by\n> pack-objects.\n> \n> Signed-off-by: Junio C Hamano \n> Signed-off-by: Johannes Sixt \n> Signed-off-by: Mike Ralphson \n> ---\n\nYou can add\n\nTested-by: Johannes Sixt (AIX 4.3.x)\n\n-- Hannes\n"},{"id":"97009","messageId":"7v4p1mj92e.fsf@gitster.siamese.dyndns.org","threadId":"16279","inReplyTo":"4934E6BC.9040203@viscovery.net","subject":"Re: [PATCH] Makefile: introduce NO_PTHREADS","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2008-12-03T02:18:01Z","receivedAt":"2008-12-03T02:18:01Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Thanks, both.\n"}]}