git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[git wiki PATCH 1/3] "Improving parallelism in various commands" project

From
Thomas Rast <trast@student.ethz.ch>
Date
Mar 2, 2012, 11:05 UTC
Message-ID
<57e8b4eb7a98af33982c2f3a763e18f62b1d6d6d.1330686331.git.trast@student.ethz.ch>
In-Reply-To
<20120302091114.GA3984@sigill.intra.peff.net>
---
 SoC-2012-Ideas.md |   31 +++++++++++++++++++++++++++++++
 1 file changed, 31 insertions(+)
diff --git a/SoC-2012-Ideas.md b/SoC-2012-Ideas.md
index 29a374a..145b379 100644
--- a/SoC-2012-Ideas.md
+++ b/SoC-2012-Ideas.md
@@ -68,3 +68,34 @@ work to be done:
    and only accessed on demand.
 
 Proposed mentor: Jeff King
+
+Improving parallelism in various commands
+-----------------------------------------
+
+Git is mostly written single-threaded, with a few commands having
+bolted-on extensions to support parallel operation (notably git-grep,
+git-pack-objects and the core.preloadIndex feature).
+
+We have recently looked into some of these areas and made a few
+optimizations, but a big roadblock is that pack access is entirely
+single-threaded.  The project would consist of the following steps:
+
+ * In preparation (the half-step): identify commands that could
+   benefit from parallelism.  `git grep --cached` and `git grep
+   COMMIT` come to mind, but most likely also `git diff` and `git log
+   -p`.  You can probably find more.
+
+ * Rework the pack access mechanisms to allow the maximum possible
+   parallel access.
+
+ * Rework the commands found in the first step to use parallel pack
+   access if possible.  Along the way, document the improvements with
+   performance tests.
+
+The actual programming must be done in C using pthreads for obvious
+reasons.  At the very least you should not be scared of low-level
+programming.  Prior experience and access to one or more multi-core
+computers is a plus.
+
+Proposed by: Thomas Rast
+Possible mentor(s): Thomas Rast
-- 
1.7.9.2.467.g7fee4
Next: Thomas Rast
Message 1 of 8 in “"Improving parallelism in various commands" project”
  1. 1/3 "Improving parallelism in various commands" projectThomas Rast, Mar 2, 2012
  2. 2/3 "Designing a faster index format" projectThomas Rast, Mar 2, 2012
  3. Jeff KingMar 2, 2012
  4. Junio C HamanoMar 2, 2012
  5. Nguyen Thai Ngoc DuyMar 3, 2012
  6. 3/3 "Improving the `git add -p` interface" projectThomas Rast, Mar 2, 2012
  7. Nguyen Thai Ngoc DuyMar 2, 2012
  8. James PickensMar 2, 2012

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.