{"thread":{"id":"34117","subject":"Is Git multithreaded ?","startedAt":"2013-06-12T18:28:52Z","lastAt":"2013-06-13T07:28:33Z","messageCount":3,"participants":["Laurent Alebarde","Jeff King"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"220656","messageId":"51B8BDE4.7030008@free.fr","threadId":"34117","inReplyTo":"51B8BBC0.6020908@free.fr","subject":"Is Git multithreaded ?","fromName":"Laurent Alebarde","fromEmail":"l.alebarde@free.fr","sentAt":"2013-06-12T18:28:52Z","receivedAt":"2013-06-12T18:28:52Z","isPatch":false,"sender":{"key":"l.alebarde@free.fr","avatar":"https://gravatar.com/avatar/fda8e675abcdda4aa434c0ba02e5765afadb490c4017a9e9d03cc63f0fa290f0?d=mp&s=160"},"body":"Hi list,\n\nI wonder if Git is multithreaded ? For example, during a commit, does it \nprocess the files one after one, or does it use a set of threads, say \n10, to process 10 files in parrallel ?\n\nIn the Git_Guide (http://wiki.sourcemage.org/Git_Guide.html), I can read \nthis :\n\n\"T/o enable aut-detection for number of threads to use (good for \nmulti-CPU or multi-core computers) for packing repositories, use: /////\n\n////\n\n///$ git config --global pack.threads \"0\"/  \"\n\nBut it is not a lot explanatory (to me). In particular, if Git is \nmultithreded and can be configured regarding the number of workers, I \nwonder in which operations it uses it ?\n\nA clear view of this may be of interrest for hooks or filters that may \nuse a daemon, in order to not have this daemon be a bottle-neck. It \nwould make sense to start as many workers in the daemon that there are \nauthorized ones in Git.\n\nCheers,\n\n\nLaurent.\n"},{"id":"220661","messageId":"20130612193836.GB4898@sigill.intra.peff.net","threadId":"34117","inReplyTo":"51B8BDE4.7030008@free.fr","subject":"Re: Is Git multithreaded ?","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2013-06-12T19:38:37Z","receivedAt":"2013-06-12T19:38:37Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Wed, Jun 12, 2013 at 08:28:52PM +0200, Laurent Alebarde wrote:\n\n> I wonder if Git is multithreaded ?\n\nA few selected operations are multi-threaded if you compile with\nthread support (i.e., do not set NO_PTHREADS when you build).\n\n> For example, during a commit, does it process the files one after one,\n> or does it use a set of threads, say 10, to process 10 files in\n> parrallel ?\n\nCommit is not multi-threaded, for example.\n\n> In the Git_Guide (http://wiki.sourcemage.org/Git_Guide.html), I can\n> read this :\n> \n> \"T/o enable aut-detection for number of threads to use (good for\n> multi-CPU or multi-core computers) for packing repositories, use:\n\nBut object packing (used during fetch/push, and during git-gc) is\nmulti-threaded (at least the delta compression portion of it is).\n\n> But it is not a lot explanatory (to me). In particular, if Git is\n> multithreded and can be configured regarding the number of workers, I\n> wonder in which operations it uses it ?\n\nThere is no master list, and the set of threaded operations changes from\nversion to version. If you have a clone of the git source code, you can\nfind the places where threads are used with\n\n  git grep NO_PTHREADS\n\nas every threaded spot also has a single-threaded variant.\n\nThe current list is something like:\n\n  - finding delta candidates during pack-objects (gc, server side of\n    fetch, client side of push); controlled by pack.threads, which\n    defaults to \"number of CPUs you have\"\n\n  - resolving received objects in index-pack via fetch; controlled by\n    pack.threads\n\n  - git grep on a working tree (I do not recall the details, but I think\n    grepping a commit actually ends up slower when parallel); I do not\n    think there is config to control this\n\n  - when stat()-ing files to refresh the index. This is not about\n    parallel CPU performance, but about reducing latency on slow\n    filesystems (e.g., NFS) by pipelining requests; controlled by\n    core.preloadindex, which defaults to \"false\"\n\n  - git may fork to perform certain asynchronous operations (e.g.,\n    during a fetch, one process runs pack-objects to create the output,\n    and the other speaks the git protocol, mostly just passing through\n    the output to the client. On systems with threads, some of these\n    operations are performed using a thread rather than fork. This is\n    not about CPU performance, but about keeping the code simple (and\n    cannot be controlled with config).\n\nI hope that helps.\n\n-Peff\n"},{"id":"220697","messageId":"51B974A1.7000607@free.fr","threadId":"34117","inReplyTo":"51B971FC.3030404@free.fr","subject":"Re: Re: Is Git multithreaded ?","fromName":"Laurent Alebarde","fromEmail":"l.alebarde@free.fr","sentAt":"2013-06-13T07:28:33Z","receivedAt":"2013-06-13T07:28:33Z","isPatch":false,"sender":{"key":"l.alebarde@free.fr","avatar":"https://gravatar.com/avatar/fda8e675abcdda4aa434c0ba02e5765afadb490c4017a9e9d03cc63f0fa290f0?d=mp&s=160"},"body":"Yes it does help ! Thanks a lot Peff for your very complete answer.\n\nCheers,\n\nLaurent.\n\n\nLe 12/06/2013 21:38, Jeff King a écrit :\n> I hope that helps. -Peff \n"}]}