git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: If I were redoing git from scratch...

From
Linus Torvalds <torvalds@osdl.org>
Date
Nov 4, 2006, 22:44 UTC
Message-ID
<Pine.LNX.4.64.0611041436050.25218@g5.osdl.org>
In-Reply-To
<200611042329.46556.robin.rosenberg.lists@dewire.com>
On Sat, 4 Nov 2006, Robin Rosenberg wrote:
> 
> The overhead is eigth bytes per object on a 32-bit platform and 16 on a 64-bit 
> platform for Sun's JDK. The granularity is eight bytes in both cases. 

Note that the object structure itself is just 24 bytes (regardless of whether we're on a 32-bit or 64-bit architecture).

In addition to that, we need one pointer per hash entry, and in order to keep the hash list size down we need that hash array to be about 25% free, so say 1.5 pointers per object: ~6 bytes or ~12 bytes depending on whether it's a 32- or 64-bit architecture.

So 8 or 16 bytes overhead per object is roughly a 25% or 50% pure overhead. In contrast, right now we have pretty much _zero_ overhead (we do end up having some malloc cost, but since we batch them, it's on the order of a few bytes per _thousand_ objects, so we're talking fractions of a percent).

Of course, the memory footprint isn't _just_ the objects, but it's a big part of it, for some important apps (not just repack, but git-send-pack obviously also has this). So on a git server, keeping the memory use down is likely the #1 concern - even if the server might have gigabytes of RAM.

Previous: Robin RosenbergNext: Linus Torvalds
Message 6 of 7 in “If I were redoing git from scratch...”
  1. Junio C HamanoNov 4, 2006
  2. Jakub NarebskiNov 4, 2006
  3. Linus TorvaldsNov 4, 2006
  4. Shawn PearceNov 4, 2006
  5. Robin RosenbergNov 4, 2006
  6. Linus TorvaldsNov 4, 2006
  7. Linus TorvaldsNov 4, 2006

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.