git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: performance on repack

From
Jon Smirl <jonsmirl@gmail.com>
Date
Aug 30, 2007, 16:17 UTC
Message-ID
<9e4733910708300917w46d25c61n387e89dc3b2985f9@mail.gmail.com>
In-Reply-To
<alpine.LFD.0.999.0708300033540.16727@xanadu.home>

Practicing on the kernel, without the patch: Q6600 with 4GB RAM

jonsmirl@terra:/home/linux$ time git repack -a -f --window=200 --depth=200
Generating pack...
Counting objects: 518288
Done counting 546466 objects.
Deltifying 546466 objects...
 100% (546466/546466) done
Writing 546466 objects...
 100% (546466/546466) done
Total 546466 (delta 452329), reused 0 (delta 0)
Pack pack-04c98effa112233951acbb2d8486eefac17a5a97 created.

real 16m27.752s user 16m23.829s sys 0m2.192s

With the patch it didn't balance very well. First two threads finished in 2min, thrid in 5min, fourth in 10min.

So for fun I set it up to 16 threads, that kept all fours cores running until the end. It added 1:20 in overhead, but it finished in 5m vs 16:30 unthreaded. Threading is an obvious win for this code.

jonsmirl@terra:/home/linux$ time git repack -a -f --window=200 --depth=200
Generating pack...
Counting objects: 502267
Done counting 546466 objects.
Deltifying 545660 objects...
Deltifying 545660 objects...
Deltifying 545660 objects...
Deltifying 545660 objects...
Deltifying 545660 objects...
Deltifying 545660 objects...
Deltifying 545660 objects...
Deltifying 545660 objects...
Deltifying 545660 objects...
Deltifying 545660 objects...
Deltifying 545660 objects...
Deltifying 545660 objects...
Deltifying 545660 objects...
Deltifying 545660 objects...
Deltifying 545660 objects...
Deltifying 545660 objects...
   6% (34104/545660) done
   6% (34099/545660) done
   1% (6929/545660) donee
   5% (30037/545660) done
   6% (34101/545660) done
   4% (25222/545660) done
   6% (34102/545660) done
   3% (19999/545660) done
   6% (34102/545660) done
   4% (27203/545660) done
   6% (34102/545660) done
   4% (23377/545660) done
   6% (34042/545660) done
   6% (34099/545660) done
   6% (32740/545660) done

real 5m0.360s user 17m41.822s sys 0m5.596s jonsmirl@terra:/home/linux$

-- 
Jon Smirl
jonsmirl@gmail.com
Previous: Nicolas PitreNext: Jon Smirl
Message 29 of 30 in “performance on repack”
  1. Jon SmirlAug 11, 2007
  2. David KastrupAug 11, 2007
  3. Linus TorvaldsAug 11, 2007
  4. Jon SmirlAug 11, 2007
  5. Martin KoeglerAug 12, 2007
  6. Jon SmirlAug 12, 2007
  7. Shawn O. PearceAug 14, 2007
  8. Jon SmirlAug 14, 2007
  9. Shawn O. PearceAug 14, 2007
  10. Jon SmirlAug 14, 2007
  11. Nicolas PitreAug 14, 2007
  12. Nicolas PitreAug 14, 2007
  13. Jon SmirlAug 15, 2007
  14. Nicolas PitreAug 15, 2007
  15. Shawn O. PearceAug 15, 2007
  16. Jon SmirlAug 15, 2007
  17. Martin KoeglerAug 15, 2007
  18. Jon SmirlAug 15, 2007
  19. Nicolas PitreAug 15, 2007
  20. Jon SmirlAug 15, 2007
  21. David KastrupAug 16, 2007
  22. Nicolas PitreAug 16, 2007
  23. Jon SmirlAug 16, 2007
  24. Nicolas PitreAug 16, 2007
  25. Nicolas PitreAug 15, 2007
  26. Nicolas PitreAug 15, 2007
  27. Nicolas PitreAug 30, 2007
  28. Nicolas PitreAug 30, 2007
  29. Jon SmirlAug 30, 2007
  30. Jon SmirlSep 1, 2007

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.