Re: [PATCH 2/2] index-pack: reduce memory usage when the pack has large blobs

From

Nguyen Thai Ngoc Duy <pclouds@gmail.com>

Date

Feb 25, 2012, 01:49 UTC

Message-ID

<CACsJy8C-8dvXpNTU=JpdupSpS8OuqqTpGvDs6s1ASeKdk9d5Dg@mail.gmail.com>

In-Reply-To

<20120224161613.GH9526@pomac.netswarm.net>

2012/2/24 Ian Kumlien <pomac@vapor.com>:

> Writing objects: 100% (1425/1425), 56.06 MiB | 4.62 MiB/s, done.
> Total 1425 (delta 790), reused 1425 (delta 790)
> fatal: Out of memory, malloc failed (tried to allocate 3310214315 bytes)
> fatal: Out of memory, malloc failed (tried to allocate 3310214315 bytes)
> fatal: Out of memory, malloc failed (tried to allocate 3310214315 bytes)
> fatal: Out of memory, malloc failed (tried to allocate 3310214315 bytes)
> To ../test_data/
>  ! [remote rejected] master -> master (missing necessary objects)
>  ! [remote rejected] origin/HEAD -> origin/HEAD (missing necessary objects)
>  ! [remote rejected] origin/master -> origin/master (missing necessary objects)
> error: failed to push some refs to '../test_data/'
>
> So there are additional code paths to look at... =(

-- 8< --

diff --git a/builtin/rev-list.c b/builtin/rev-list.c
index 264e3ae..6dc46eb 100644
--- a/builtin/rev-list.c
+++ b/builtin/rev-list.c
@@ -183,7 +183,8 @@ static void show_object(struct object *obj,
        struct rev_list_info *info = cb_data;

        finish_object(obj, path, component, cb_data);
-       if (info->revs->verify_objects && !obj->parsed && obj->type !=
OBJ_COMMIT)
+       if (info->revs->verify_objects && !obj->parsed &&
+           obj->type != OBJ_COMMIT && obj->type != OBJ_BLOB)
                parse_object(obj->sha1);
        show_object_with_name(stdout, obj, path, component);
 }
-- 8< --

If not, you might need to apply this to generate coredump, then look
and see where that failed malloc comes from

-- 8< --
diff --git a/wrapper.c b/wrapper.c
index 85f09df..03f423e 100644
--- a/wrapper.c
+++ b/wrapper.c
@@ -40,9 +40,11 @@ void *xmalloc(size_t size)
                ret = malloc(size);
                if (!ret && !size)
                        ret = malloc(1);
-               if (!ret)
+               if (!ret) {
+                       *(char*)0 = 1;
                        die("Out of memory, malloc failed (tried to
allocate %lu bytes)",
                            (unsigned long)size);
+               }
        }
 #ifdef XMALLOC_POISON
        memset(ret, 0xA5, size);
-- 8< --

-- 
Duy