git/list[1] front-page[2] threads[3] people[4] search[5] about
 

git-repack & big files

From
Pietro Battiston <me@pietrobattiston.it>
Date
Jan 11, 2011, 07:37 UTC
Message-ID
<1294731438.3300.973.camel@voubian.casa>
Hello,

first, I do know git is not optimized for big files, and that's fine. But it is able, on my machine with 3 GB of RAM, to succesfully backup my home directory¹, which contains, among others, several files of several hundreds of megabytes each. And I like that a lot.

Since it perfectly does what it is not optimized to do... I then wonder when it does not do what it declares: if I run git-repack² with the parameter --window-memory set to, for instance, "100m", it takes hundreds and hundreds of MB of memory until it runs out of memory, fails a malloc and aborts. So, two questions:

1) is there a bug, is the documentation about that parameter a bit too
optimistic or did I just not understand it?
2) do I have any hope that in one way or another my 500+ MB mailboxes
with relatively small changes over time are archived smartly (=diffs) by
git at the current state of development? If I understand correctly, the
project git-bigfiles³ would just "solve" my problems by not making
differences of big files.
thanks for the clarifications
Pietro

¹ Just for the records: through gibak: http://eigenclass.org/hiki/gibak-0.3.0

² git version 1:1.7.2.3-2.2 on Debian
³ http://caca.zoy.org/wiki/git-bigfiles
Next: Phillip Susi
Message 1 of 6 in “git-repack & big files”
  1. Pietro BattistonJan 11, 2011
  2. Phillip SusiJan 11, 2011
  3. Pietro BattistonJan 11, 2011
  4. Stephen BashJan 11, 2011
  5. Phillip SusiJan 11, 2011
  6. Pietro BattistonJan 13, 2011

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.