git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: RAM consumption when working with the gcc repo

From
Jeff King <peff@peff.net>
Date
Dec 7, 2007, 21:39 UTC
Message-ID
<20071207213928.GA11613@coredump.intra.peff.net>
In-Reply-To
<9e4733910712071207p750c14f4h7abc5d637da3a478@mail.gmail.com>
On Fri, Dec 07, 2007 at 03:07:05PM -0500, Jon Smirl wrote:
Show 8 quoted lines
> I noticed two things when doing a repack of the gcc repo. First is
> that the git process is getting to be way too big. Turning off the
> delta caches had minimal impact. Why does the process still grow to
> 4.8GB?
> 
> Putting this in perspective, this is a 4.8GB process constructing a
> 330MB file. Something isn't right. Memory leak or inefficient data
> structure?

Keep in mind that you are trying many different deltas, which are being held in memory, to find the right one and generate the 330MB file. And when you multiply that times N threads going at once, _each one_ is using a bunch of memory.

As Nico suggested, you could probably drop the memory usage by reducing the size of the delta cache.

-Peff
Previous: Martin KoeglerNext: Jeff King
Message 13 of 15 in “RAM consumption when working with the gcc repo”
  1. Jon SmirlDec 7, 2007
  2. david@lang.hmDec 7, 2007
  3. Marco CostalbaDec 7, 2007
  4. Nicolas PitreDec 7, 2007
  5. Jon SmirlDec 7, 2007
  6. Marco CostalbaDec 7, 2007
  7. Johannes SchindelinDec 8, 2007
  8. Marco CostalbaDec 8, 2007
  9. Jon SmirlDec 7, 2007
  10. Jon SmirlDec 7, 2007
  11. Jon SmirlDec 7, 2007
  12. Martin KoeglerDec 8, 2007
  13. Jeff KingDec 7, 2007
  14. Jeff KingDec 7, 2007
  15. Jon SmirlDec 7, 2007

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.