Bug - crash on large commit

Hi...

I'm a new user, and thinking that git is better than sliced bread. So, today I had to set up a new computer (MacOSX), and thought: "hey, I could put the whole system into git!"

Well, ok, technically yes, you can do that. Go into the root directory, type 'git init', then 'git add .', then 'git commit -a'.

Bug 1: git complained about a symbolic link to nowhere and halted... several hours into the 'add'. If I want to store a broken link, let me; maybe warn me.

Bug 2: git died with an out-of-memory error on the commit:
------
# git commit -a
[master (root-commit) 62b52e2] The initial checkin of the whole hard disk.
 Committer: System Administrator <root@Mac-Pro.local>
Your name and email address were configured automatically based
on your username and hostname. Please check that they are accurate.
You can suppress this message by setting them explicitly:

    git config --global user.name "Your Name"
    git config --global user.email you@example.com

If the identity used for this commit is wrong, you can fix it with:

    git commit --amend --author='Your Name <you@example.com>'

git(4667) malloc: *** mmap(size=1964838912) failed (error code=12) *** error: can't allocate region *** set a breakpoint in malloc_error_break to debug git(4667) malloc: *** mmap(size=1964838912) failed (error code=12) *** error: can't allocate region *** set a breakpoint in malloc_error_break to debug fatal: Out of memory, malloc failed [Mac-Pro:/] shadministrator# git version git version 1.7.0.2 -----

Anyhoo, it's not really the right tool for the job, but Apple hasn't got a
package management system to speak of (yes, I know about bom files --
don't speak of it), and their filesystem loses bits now and then, so I
wanted the crypto checksums and the ability to spot and roll back changes.
 I suppose TimeMachine will have to do.

Thanks for a great program!

On Tue, Mar 23, 2010 at 12:36 AM, P Fudd <fink@ch.pkts.ca> wrote:

Is your git built as a 32-bit program? If so, it can't handle files larger than ~1-2 gb, which looks like it might be the case here...

On Mon, Mar 22, 2010 at 04:36:25PM -0700, P Fudd wrote:

Git does let you store broken links. Without seeing the actual error message you got, it's hard to say what actually happened. But you should be able to:

  $ mkdir foo && cd foo && git init
  $ ln -s bogus link
  $ git add .
  $ git commit -m 'add bogus link'
  $ git show
  [...]
  diff --git a/link b/link
  new file mode 120000
  index 0000000..882b604
  --- /dev/null
  +++ b/link
  @@ -0,0 +1 @@
  +bogus
  \ No newline at end of file

Note that it actually did make the commit, which means that the problem you saw happened while git was generating a diffstat summary of the commit.

It's hard to say exactly what it was trying to mmap without seeing a stack trace. But note that it is trying to malloc almost 2 gigabytes. In general, git assumes that it can fit any file you throw at it into memory during a diff. So presumably you have some gigantic file, and that is the problem.

Which, as you noted, means git is not really suitable for just throwing your whole hard disk at it. The topic of better handling gigantic files comes up occasionally on the list, but I don't know that anyone is working on anything concrete right now.

Avery Pennarun's "bup" tool may be closer to what you want, but I've never used it myself:

  http://github.com/apenwarr/bup

-Peff