Git Garbage Collect Error.

Hi,

Hopefully this is the right place to send bug reports... The community page "http://git-scm.com/community" suggests that it is.

      Introduction

I am creating a large GIT repository fetching from a large SVN repository, as an experiment initially. I usually use GIT repositories interfacing to parts of the SVN repository.

      Defect

During garbage collection (git gc) it encountered the following error:

git gc | git gc --prune :

    Counting objects: 856758, done.
    Delta compression using up to 2 threads.
    fatal: Out of memory, malloc failed (tried to allocate 303237121 bytes)
    error: failed to run repack

git gc --aggressive:

    Counting objects: 856758, done.
    Delta compression using up to 2 threads.
    fatal: Out of memory, malloc failed (tried to allocate 291942401 bytes)
    error: failed to run repack

At the moment the bare repository is about 4Gb in size and about 2/3rds the way through fetching.

The compression gets over 90% of the way through before this error occurs, but I don't think any compression results are kept, because when you repeat it has the same amount of work to do.

Initially this happen during an automatic gc during the fetch process. This aborted the fetch.

My system is XP64 2 core with 4Gb of memory and plenty of virtual memory.

      Comments

If this a genuine limitation due to the size of an object and memory handling limitations, then perhaps the error could be caught and the successful results kept. Ie. do a partial compression. That way the process could continue.

      Background

My requirement is to have GIT repositories of a source directory with all SVN branches included so that I can more easily merge and compare branches using GIT. However for even small source directories it takes weeks to fetch from the SVN respository (including all tags and branches), whereas fetching just the trunk takes a few hours. The SVN repository has over 90000 revisions. I am aware that I can fetch a sub-set of revisions (I don't want to at the moment), but I've found no way to fetch a sub-set of branches.

My config is as follows:

[svn-remote "svn"]
         url = svn://svn
         fetch = trunk:refs/remotes/svn/trunk
         branches = branches/*:refs/remotes/svn/*
         tags = tags/*:refs/remotes/svn/tags/*

I set this up using:

git svn init --prefix=svn/ --stdlayout --no-minimize-url svn://svn

To do this for individual directories I have to do the following:

git svn init --prefix=svn/ --stdlayout --no-minimize-url svn://svn/trunk/source/<dir>

and then edit the config manually so that:

[svn-remote "svn"]
         url = svn://svn
         fetch = trunk/source/<dir>:refs/remotes/svn/trunk
         branches = branches/*/source/<dir>:refs/remotes/svn/*
         tags = tags/*/source/<dir>:refs/remotes/svn/tags/*

This works ok but I couldn't get this result by using "git svn init" directly. Maybe I've missed something.

Regards, Tom.