git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: cloning the kernel - why long time in "Resolving 313037 deltas"

From
Shawn Pearce <spearce@spearce.org>
Date
Dec 19, 2006, 08:49 UTC
Message-ID
<20061219084908.GG2511@spearce.org>
In-Reply-To
<20061219084017.GB30448@fiberbit.xs4all.nl>
Marco Roeland <marco.roeland@xs4all.nl> wrote:
Show 13 quoted lines
> On Tuesday December 19th 2006 at 03:32 Shawn Pearce wrote:
> 
> > > Anyway, try the pread() version first, see if that fixes the OS X problem.
> > 
> > It does.  Without pread() (aka stock 'next') it takes me over an
> > hour to index a pack of linux-2.6.  With pread() its 1m6s to run
> > index-pack on the same pack file.  The indexes are (of course)
> > identically produced.
> 
> I see the same here. From an (estimated) time of 37 minutes down to 52
> seconds with the pread() patch. Running the profiler (shark) on the old
> version showed that 85% of the time was spent in the Mac OS X mmap()
> system call.

More testing on Linux is probably needed, but if using pread() on Linux is breakeven or slightly faster (as suggested by Johannes' LilyPond test) this 60x performance improvement on initial clone of largish projects on Mac OS X would be nice to have.

Not that I personally frequently clone large projects on Mac OS X. But new users to Git might. :-)

Previous: Marco RoelandNext: Marco Roeland
Message 24 of 51 in “Re: [PATCH] fetch-pack: avoid fixing thin packs when unnecessary”
  1. Johannes SchindelinDec 18, 2006
  2. Nicolas PitreDec 18, 2006
  3. Randal L. SchwartzDec 18, 2006
  4. Nicolas PitreDec 18, 2006
  5. Randal L. SchwartzDec 18, 2006
  6. Nicolas PitreDec 18, 2006
  7. Linus TorvaldsDec 18, 2006
  8. Randal L. SchwartzDec 18, 2006
  9. Martin LanghoffDec 18, 2006
  10. Kyle MoffettDec 22, 2006
  11. Shawn PearceDec 22, 2006
  12. Marco RoelandDec 22, 2006
  13. Andreas EricssonJan 3, 2007
  14. Linus TorvaldsDec 18, 2006
  15. Nicolas PitreDec 19, 2006
  16. Theodore TsoDec 19, 2006
  17. Shawn PearceDec 19, 2006
  18. Linus TorvaldsDec 19, 2006
  19. Shawn PearceDec 19, 2006
  20. Marco RoelandDec 19, 2006
  21. Shawn PearceDec 19, 2006
  22. Shawn PearceDec 19, 2006
  23. Marco RoelandDec 19, 2006
  24. Shawn PearceDec 19, 2006
  25. Marco RoelandDec 19, 2006
  26. Alex RiesenDec 19, 2006
  27. Juergen RuehleDec 21, 2006
  28. Theodore TsoDec 19, 2006
  29. Linus TorvaldsDec 19, 2006
  30. Shawn PearceDec 20, 2006
  31. Shawn PearceDec 20, 2006
  32. Linus TorvaldsDec 19, 2006
  33. Johannes SchindelinDec 19, 2006
  34. Junio C HamanoDec 19, 2006
  35. Jeff KingDec 19, 2006
  36. Andy WhitcroftDec 19, 2006
  37. index-pack usage of mmap() is unacceptably slower on many OSes other than LinuxNicolas Pitre, Dec 19, 2006
  38. Junio C HamanoDec 19, 2006
  39. Nicolas PitreDec 19, 2006
  40. Linus TorvaldsDec 19, 2006
  41. Randal L. SchwartzDec 19, 2006
  42. Randal L. SchwartzDec 19, 2006
  43. Jeff GarzikDec 19, 2006
  44. Junio C HamanoDec 20, 2006
  45. Linus TorvaldsDec 20, 2006
  46. Jeff GarzikDec 20, 2006
  47. Junio C HamanoDec 20, 2006
  48. Junio C HamanoDec 20, 2006
  49. Linus TorvaldsDec 20, 2006
  50. Junio C HamanoDec 20, 2006
  51. Nikolai WeibullDec 20, 2006

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.