Re: git gc --aggressive led to about 40 times slower "git log --raw"

From: Junio C Hamano <gitster@pobox.com>
Date: Feb 21, 2014, 17:47 UTC
Message-ID: <xmqqzjlk4atj.fsf@gitster.dls.corp.google.com>
In-Reply-To: <CACsJy8AeZWPz=39ySQr9MrSUiLyJDQbs02sumS9VjbbGWzP9pw@mail.gmail.com>

Duy Nguyen <pclouds@gmail.com> writes:

> For old projects, commits older than 1-2 years is probably less often
> accessed and could use some aggressive packing.

I used to repack older part of history manually with a deeper depth, mark the result with the .keep bit, and then repack the whole thing again to have the remainder in a shallower depth. Something like:

	git rev-list --objects v1.5.3 |
        git pack-objects --depth=128 --delta-base-offset pack

would give me the first pack (in real life, I would use a larger window size like 4096), and then after placing the resulting .pack and .idx files along with a .keep file in .git/objects/pack/, running "git repack -a -d" to pack the rest.

> This still hits git-blame badly. We could even make sure all
> objects "on the blame surface" have short delta chain. But that
> may be pushing pack-objects too much.

Yes, you can do a similar trick by blaming all the paths that ever existed in the project, parse its --porcelain output to learn all the commits and paths involved, to find the objects that need quicker access. Pack such objects in a pack with a shallow depth, tentatively mark that pack with .keep, repack the remainder with a deep depth, remove .keep from the first pack and mark the new pack with .keep to prevent it from getting repacked, or something like that.