git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: git tag triggers out-of-memory killer

From
Olaf Hering <olaf@aepfle.de>
Date
Jul 11, 2022, 15:15 UTC
Message-ID
<20220711171537.19d058da.olaf@aepfle.de>
In-Reply-To
<YswuaPx6Mk7YkIim@coredump.intra.peff.net>
Mon, 11 Jul 2022 10:06:32 -0400 Jeff King <peff@peff.net>:
Show 5 quoted lines
> How did you measure?  Are you sure that each one is allocating a
> gigabyte itself, or might some of it be shared between the processes?
> Git will mmap the packfiles on disk, which will count against RSS
> (assuming the memory even gets faulted in). But multiple processes on
> the same repository will share those read-only pages.

I ran top(1), all the git processes were competing for memory. There was most of the time no memory shared, according to top.

This tool to process a single repository exists since a few years. In the past I ran it on a 12cpu/64GB machine. Today it was running the first time on a 96cpu/64GB machine. There is a slim chance the issue did not show up because the 12 processes always had access to enough memory.

I will try your suggestions.

Thanks. Olaf

Previous: Jeff KingNext: Jeff King
Message 3 of 5 in “git tag triggers out-of-memory killer”
  1. Olaf HeringJul 11, 2022
  2. Jeff KingJul 11, 2022
  3. Olaf HeringJul 11, 2022
  4. Jeff KingJul 11, 2022
  5. Olaf HeringJul 12, 2022

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.