git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: pack operation is thrashing my server

From
Ken Pratt <ken@kenpratt.net>
Date
Aug 13, 2008, 17:38 UTC
Message-ID
<a6b6acf60808131038s1ae7fb69s2b703c25766a82c0@mail.gmail.com>
In-Reply-To
<200808131810.19914.johan@herland.net>
Show 7 quoted lines
> As for how to estimate entropy, isn't that just a matter of feeding it
> through zlib and compare the output size to the input size? Especially
> if we're already about to feed it through zlib anyway... In other
> words, feed (an initial part of) the data through zlib, and if the
> compression ratio so far looks good, keep going and write out the
> compressed object, otherwise abort zlib and write out the original
> object with compression level 0.

This if probably off topic now, but as the OP, I'd like to mention that I tried setting pack.compression = 0 and it did not solve my memory issues. So it seems to be that the packing itself that is sucking up all the memory -- not the compression.

Thanks for all the insightful replies!
-Ken
Previous: Johan HerlandNext: Nicolas Pitre
Message 31 of 80 in “pack operation is thrashing my server”
  1. Ken PrattAug 10, 2008
  2. Martin LanghoffAug 10, 2008
  3. Ken PrattAug 10, 2008
  4. Martin LanghoffAug 10, 2008
  5. Ken PrattAug 10, 2008
  6. Shawn O. PearceAug 11, 2008
  7. Ken PrattAug 11, 2008
  8. Shawn O. PearceAug 11, 2008
  9. Avery PennarunAug 11, 2008
  10. Shawn O. PearceAug 11, 2008
  11. Ken PrattAug 11, 2008
  12. Andi KleenAug 11, 2008
  13. Ken PrattAug 11, 2008
  14. Nicolas PitreAug 13, 2008
  15. Andi KleenAug 13, 2008
  16. Shawn O. PearceAug 13, 2008
  17. Shawn O. PearceAug 11, 2008
  18. Ken PrattAug 11, 2008
  19. Shawn O. PearceAug 11, 2008
  20. Andi KleenAug 11, 2008
  21. Geert BoschAug 13, 2008
  22. Shawn O. PearceAug 13, 2008
  23. Geert BoschAug 13, 2008
  24. Nicolas PitreAug 13, 2008
  25. Jakub NarebskiAug 13, 2008
  26. Shawn O. PearceAug 13, 2008
  27. David TweedAug 13, 2008
  28. Martin LanghoffAug 13, 2008
  29. David TweedAug 14, 2008
  30. Johan HerlandAug 13, 2008
  31. Ken PrattAug 13, 2008
  32. Nicolas PitreAug 13, 2008
  33. Nicolas PitreAug 13, 2008
  34. Shawn O. PearceAug 13, 2008
  35. Nicolas PitreAug 13, 2008
  36. Shawn O. PearceAug 13, 2008
  37. Nicolas PitreAug 13, 2008
  38. Shawn O. PearceAug 13, 2008
  39. Andreas EricssonAug 14, 2008
  40. Thomas RastAug 14, 2008
  41. Andreas EricssonAug 14, 2008
  42. Shawn O. PearceAug 14, 2008
  43. Nicolas PitreAug 15, 2008
  44. Nicolas PitreAug 14, 2008
  45. Linus TorvaldsAug 14, 2008
  46. Linus TorvaldsAug 14, 2008
  47. Nicolas PitreAug 14, 2008
  48. Linus TorvaldsAug 14, 2008
  49. Andi KleenAug 14, 2008
  50. Linus TorvaldsAug 15, 2008
  51. Nicolas PitreAug 14, 2008
  52. Linus TorvaldsAug 14, 2008
  53. Björn SteinbrinkAug 14, 2008
  54. Linus TorvaldsAug 15, 2008
  55. Linus TorvaldsAug 15, 2008
  56. Björn SteinbrinkAug 16, 2008
  57. Linus TorvaldsAug 16, 2008
  58. Junio C HamanoSep 7, 2008
  59. Linus TorvaldsSep 7, 2008
  60. Junio C HamanoSep 7, 2008
  61. Nicolas PitreSep 7, 2008
  62. Junio C HamanoSep 7, 2008
  63. Jon SmirlSep 7, 2008
  64. Linus TorvaldsSep 7, 2008
  65. Jon SmirlSep 7, 2008
  66. Linus TorvaldsSep 7, 2008
  67. Jon SmirlSep 7, 2008
  68. Nicolas PitreSep 7, 2008
  69. Jon SmirlSep 7, 2008
  70. Nicolas PitreSep 8, 2008
  71. Jon SmirlSep 8, 2008
  72. Jon SmirlSep 8, 2008
  73. Andreas EricssonSep 7, 2008
  74. Mike HommeySep 7, 2008
  75. Nicolas PitreAug 14, 2008
  76. Linus TorvaldsAug 14, 2008
  77. Geert BoschAug 13, 2008
  78. Dana HowAug 13, 2008
  79. Nicolas PitreAug 13, 2008
  80. Jakub NarebskiAug 13, 2008

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.