git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: git-daemon on NSLU2

From
Linus Torvalds <torvalds@linux-foundation.org>
Date
Aug 24, 2007, 23:28 UTC
Message-ID
<alpine.LFD.0.999.0708241616390.25853@woody.linux-foundation.org>
In-Reply-To
<9e4733910708241417l44c55306xaa322afda69c6beb@mail.gmail.com>
On Fri, 24 Aug 2007, Jon Smirl wrote:
> 
> We're going something wrong in git-daemon.
Nope.
Or rather, it's mostly by design.
> I can clone the tree in five minutes using the http protocol. Using the 
> git protocol would take 24hrs if I let it finish.

The http side doesn't actually do any global verification, the way git-daemon does. So to it, everything is just temporary buffers, and you don't need any memory at all, really.

git-daemon will create a packfile. That means that it has to generate the *global* object reachability, and will then optimize the object packing etc etc. That's a minimum of something like 48 bytes per object for just the object chains, and the kernel has a *lot* of objects (over half a million).

In addition to the object chains yourself, the native protocol will also obviously have to actually *look* at and parse all the tree and commit objects while it does all this, so while it doesn't necessarily keep all of those in memory all the time, it will need to access them, and if you don't have enough memory to cache them, that will add its own set of IO.

So I haven't checked exactly how much memory you really want to have to serve big projects, but with some handwavy guesstimate, if you actually want to do a good job I'd guess that you really want to have at least as much memory as the size of largest project you are serving, and probably add at least 10-20% on top of that.

So for the kernel, at a guess, you'd probably want to have at least 256MB of RAM to do a half-way good job. 512MB is likely nicer and allows you to actually cache the stuff over multiple accesses.

But I haven't actually tested. Maybe it might be bearable at 128M.
			Linus
Previous: Nicolas PitreNext: Jon Smirl
Message 16 of 30 in “git-daemon on NSLU2”
  1. Jon SmirlAug 24, 2007
  2. Shawn O. PearceAug 24, 2007
  3. Jon SmirlAug 24, 2007
  4. Nicolas PitreAug 24, 2007
  5. Jon SmirlAug 24, 2007
  6. Nicolas PitreAug 24, 2007
  7. Jon SmirlAug 24, 2007
  8. Jakub NarebskiAug 24, 2007
  9. Junio C HamanoAug 24, 2007
  10. Jakub NarebskiAug 24, 2007
  11. Jon SmirlAug 24, 2007
  12. Junio C HamanoAug 25, 2007
  13. David KastrupAug 25, 2007
  14. Salikh ZakirovAug 25, 2007
  15. Nicolas PitreAug 25, 2007
  16. Linus TorvaldsAug 24, 2007
  17. Jon SmirlAug 25, 2007
  18. Jeff KingAug 26, 2007
  19. Jon SmirlAug 26, 2007
  20. Linus TorvaldsAug 26, 2007
  21. Jon SmirlAug 26, 2007
  22. Linus TorvaldsAug 26, 2007
  23. Jon SmirlAug 26, 2007
  24. Linus TorvaldsAug 26, 2007
  25. Junio C HamanoAug 26, 2007
  26. Theodore TsoAug 27, 2007
  27. Linus TorvaldsAug 27, 2007
  28. Daniel HulmeAug 26, 2007
  29. Jakub NarebskiAug 27, 2007
  30. Jon SmirlAug 24, 2007

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.