git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v1 1/4] fastindex: speed up index load through parallelization

From
Jeff King <peff@peff.net>
Date
Nov 20, 2017, 15:38 UTC
Message-ID
<20171120153846.v5b7ho42yzrznqoh@sigill.intra.peff.net>
In-Reply-To
<20171120142035.bfwu24oegw27ucs3@sigill.intra.peff.net>
On Mon, Nov 20, 2017 at 09:20:35AM -0500, Jeff King wrote:
Show 6 quoted lines
> Out of curiosity, have you tried experimenting with any high-performance
> 3rd-party allocator libraries? I've often wondered if we could get a
> performance improvement from dropping in a new allocator, but was never
> able to measure any real benefit over glibc's ptmalloc2. The situation
> might be different on Windows, though (i.e., if the libc allocator isn't
> that great).
Just linking with tcmalloc, like:
diff --git a/Makefile b/Makefile
index ee9d5eb11e..4f299cd914 100644
--- a/Makefile
+++ b/Makefile
@@ -1018,7 +1018,7 @@ BUILTIN_OBJS += builtin/worktree.o
 BUILTIN_OBJS += builtin/write-tree.o
 
 GITLIBS = common-main.o $(LIB_FILE) $(XDIFF_LIB)
-EXTLIBS =
+EXTLIBS = -ltcmalloc
 
 GIT_USER_AGENT = git/$(GIT_VERSION)
 

seems to consistently show about 30% speedup on p0002:

Test                                          HEAD^             HEAD                  
--------------------------------------------------------------------------------------
0002.1: read_cache/discard_cache 1000 times   0.19(0.18+0.01)   0.13(0.12+0.01) -31.6%

I don't think we really even need a patch for it. You should be able to
just build with:

  make EXT_LIBS=-ltcmalloc

I can't think of any real advantage to a Makefile knob except
discoverability.

-Peff
Previous: Jeff KingNext: Ramsay Jones
Message 14 of 19 in “Speed up index load through parallelization”
  1. 0/4 Speed up index load through parallelizationBen Peart, Nov 9, 2017
  2. 1/4 fastindex: speed up index load through parallelizationBen Peart, Nov 9, 2017
  3. Junio C HamanoNov 10, 2017
  4. Ben PeartNov 13, 2017
  5. Junio C HamanoNov 14, 2017
  6. Ben PeartNov 14, 2017
  7. Junio C HamanoNov 14, 2017
  8. Ben PeartNov 14, 2017
  9. Junio C HamanoNov 15, 2017
  10. Ben PeartNov 15, 2017
  11. Junio C HamanoNov 15, 2017
  12. Ben PeartNov 20, 2017
  13. Jeff KingNov 20, 2017
  14. Jeff KingNov 20, 2017
  15. Ramsay JonesNov 20, 2017
  16. Ben PeartNov 21, 2017
  17. 2/4 update-index: add fastindex support to update-indexBen Peart, Nov 9, 2017
  18. 3/4 fastindex: add test tools and a test scriptBen Peart, Nov 9, 2017
  19. 4/4 fastindex: add documentation for the fastindex extensionBen Peart, Nov 9, 2017

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.