git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v1 1/4] fastindex: speed up index load through parallelization

From
Ramsay Jones <ramsay@ramsayjones.plus.com>
Date
Nov 20, 2017, 23:51 UTC
Message-ID
<63053f5d-0089-4165-0470-ed0d88a87e9e@ramsayjones.plus.com>
In-Reply-To
<92b0b0ff-6878-003a-b26f-3b4c2c857be3@gmail.com>
On 20/11/17 14:01, Ben Peart wrote:
> Further testing has revealed that switching from the regular heap to a refactored version of the mem_pool in fast-import.c produces similar gains as parallelizing do_index_load().  This appears to be a much simpler patch for similar gains so we will be pursuing that path.
> 
> Combining the two patches resulted in a further 25% reduction of do_index_load() but with index load times getting smaller, that only resulted in a 3% improvement in total git command time.  Given the greater complexity of this patch, we've decided to put it on hold for now.  We'll keep it in our back pocket in case we need it in the future.

Since you are withdrawing the 'bp/fastindex' branch, I won't send the patch to fix up the sparse warnings (unless you would like to see it before you put it in your back pocket). ;-)

ATB, Ramsay Jones

Previous: Jeff KingNext: Ben Peart
Message 15 of 19 in “Speed up index load through parallelization”
  1. 0/4 Speed up index load through parallelizationBen Peart, Nov 9, 2017
  2. 1/4 fastindex: speed up index load through parallelizationBen Peart, Nov 9, 2017
  3. Junio C HamanoNov 10, 2017
  4. Ben PeartNov 13, 2017
  5. Junio C HamanoNov 14, 2017
  6. Ben PeartNov 14, 2017
  7. Junio C HamanoNov 14, 2017
  8. Ben PeartNov 14, 2017
  9. Junio C HamanoNov 15, 2017
  10. Ben PeartNov 15, 2017
  11. Junio C HamanoNov 15, 2017
  12. Ben PeartNov 20, 2017
  13. Jeff KingNov 20, 2017
  14. Jeff KingNov 20, 2017
  15. Ramsay JonesNov 20, 2017
  16. Ben PeartNov 21, 2017
  17. 2/4 update-index: add fastindex support to update-indexBen Peart, Nov 9, 2017
  18. 3/4 fastindex: add test tools and a test scriptBen Peart, Nov 9, 2017
  19. 4/4 fastindex: add documentation for the fastindex extensionBen Peart, Nov 9, 2017

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.