git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: pack operation is thrashing my server

From
Shawn O. Pearce <spearce@spearce.org>
Date
Aug 11, 2008, 15:59 UTC
Message-ID
<20080811155959.GG26363@spearce.org>
In-Reply-To
<32541b130808110840p1287426fpeef967a9ff4fb094@mail.gmail.com>
Avery Pennarun <apenwarr@gmail.com> wrote:
Show 12 quoted lines
> On Mon, Aug 11, 2008 at 11:01 AM, Shawn O. Pearce <spearce@spearce.org> wrote:
> > On the up side, packing should only be consuming huge memory like
> > this when it needs to move loose objects into a pack file.  I think
> > Martin Langhoff suggested packing this on your laptop then using
> > rsync over SSH to copy the pack file and .idx file to the server, so
> > the server didn't have to spend time figuring out the deltas itself.
> 
> Do you need to also introduce a ".keep" file to get the benefit from
> this?  I had a repo with some very large objects, and it was killing
> my low-memory server *every* time I did "git gc", until I repacked on
> another system, created the .keep file, and rsynced it back.  Does
> that make sense?

No, the ".keep" file wouldn't have an impact. Delta reuse (the feature I was alluding to) works whether or not there is a .keep file present.

I wonder if your "git gc" was using --aggressive?
-- 
Shawn.
Previous: Avery PennarunNext: Ken Pratt
Message 10 of 80 in “pack operation is thrashing my server”
  1. Ken PrattAug 10, 2008
  2. Martin LanghoffAug 10, 2008
  3. Ken PrattAug 10, 2008
  4. Martin LanghoffAug 10, 2008
  5. Ken PrattAug 10, 2008
  6. Shawn O. PearceAug 11, 2008
  7. Ken PrattAug 11, 2008
  8. Shawn O. PearceAug 11, 2008
  9. Avery PennarunAug 11, 2008
  10. Shawn O. PearceAug 11, 2008
  11. Ken PrattAug 11, 2008
  12. Andi KleenAug 11, 2008
  13. Ken PrattAug 11, 2008
  14. Nicolas PitreAug 13, 2008
  15. Andi KleenAug 13, 2008
  16. Shawn O. PearceAug 13, 2008
  17. Shawn O. PearceAug 11, 2008
  18. Ken PrattAug 11, 2008
  19. Shawn O. PearceAug 11, 2008
  20. Andi KleenAug 11, 2008
  21. Geert BoschAug 13, 2008
  22. Shawn O. PearceAug 13, 2008
  23. Geert BoschAug 13, 2008
  24. Nicolas PitreAug 13, 2008
  25. Jakub NarebskiAug 13, 2008
  26. Shawn O. PearceAug 13, 2008
  27. David TweedAug 13, 2008
  28. Martin LanghoffAug 13, 2008
  29. David TweedAug 14, 2008
  30. Johan HerlandAug 13, 2008
  31. Ken PrattAug 13, 2008
  32. Nicolas PitreAug 13, 2008
  33. Nicolas PitreAug 13, 2008
  34. Shawn O. PearceAug 13, 2008
  35. Nicolas PitreAug 13, 2008
  36. Shawn O. PearceAug 13, 2008
  37. Nicolas PitreAug 13, 2008
  38. Shawn O. PearceAug 13, 2008
  39. Andreas EricssonAug 14, 2008
  40. Thomas RastAug 14, 2008
  41. Andreas EricssonAug 14, 2008
  42. Shawn O. PearceAug 14, 2008
  43. Nicolas PitreAug 15, 2008
  44. Nicolas PitreAug 14, 2008
  45. Linus TorvaldsAug 14, 2008
  46. Linus TorvaldsAug 14, 2008
  47. Nicolas PitreAug 14, 2008
  48. Linus TorvaldsAug 14, 2008
  49. Andi KleenAug 14, 2008
  50. Linus TorvaldsAug 15, 2008
  51. Nicolas PitreAug 14, 2008
  52. Linus TorvaldsAug 14, 2008
  53. Björn SteinbrinkAug 14, 2008
  54. Linus TorvaldsAug 15, 2008
  55. Linus TorvaldsAug 15, 2008
  56. Björn SteinbrinkAug 16, 2008
  57. Linus TorvaldsAug 16, 2008
  58. Junio C HamanoSep 7, 2008
  59. Linus TorvaldsSep 7, 2008
  60. Junio C HamanoSep 7, 2008
  61. Nicolas PitreSep 7, 2008
  62. Junio C HamanoSep 7, 2008
  63. Jon SmirlSep 7, 2008
  64. Linus TorvaldsSep 7, 2008
  65. Jon SmirlSep 7, 2008
  66. Linus TorvaldsSep 7, 2008
  67. Jon SmirlSep 7, 2008
  68. Nicolas PitreSep 7, 2008
  69. Jon SmirlSep 7, 2008
  70. Nicolas PitreSep 8, 2008
  71. Jon SmirlSep 8, 2008
  72. Jon SmirlSep 8, 2008
  73. Andreas EricssonSep 7, 2008
  74. Mike HommeySep 7, 2008
  75. Nicolas PitreAug 14, 2008
  76. Linus TorvaldsAug 14, 2008
  77. Geert BoschAug 13, 2008
  78. Dana HowAug 13, 2008
  79. Nicolas PitreAug 13, 2008
  80. Jakub NarebskiAug 13, 2008

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.