Re: Repacking a repository uses up all available disk space

From: Duy Nguyen <pclouds@gmail.com>
Date: Jun 13, 2016, 00:24 UTC
Message-ID: <CACsJy8Awd2oCm0puh=bnKu9snOZr85+kVRe0D5DUhP6NhmiwcQ@mail.gmail.com>
In-Reply-To: <20160612221309.GC5428@sigill.intra.peff.net>

On Mon, Jun 13, 2016 at 5:13 AM, Jeff King <peff@peff.net> wrote:

Show 16 quoted lines

> On Sun, Jun 12, 2016 at 05:54:36PM -0400, Konstantin Ryabitsev wrote:
>
>> >   git gc --prune=now
>>
>> You are correct, this solves the problem, however I'm curious. The usual
>> maintenance for these repositories is a regular run of:
>>
>> - git fsck --full
>> - git repack -Adl -b --pack-kept-objects
>> - git pack-refs --all
>> - git prune
>>
>> The reason it's split into repack + prune instead of just gc is because
>> we use alternates to save on disk space and try not to prune repos that
>> are used as alternates by other repos in order to avoid potential
>> corruption.

Isn't this what extensions.preciousObjects is for? It looks like prune just refuses to run in precious objects mode though, and repack is skipped by gc, but if that repack command works, maybe we should do something like that in git-gc?

BTW Jeff, I think we need more documentation for extensions.preciousObjects. It's only documented in technical/ which is practically invisible to all users. Maybe include::repository-version.txt in config.txt, or somewhere close to alternates?

Show 14 quoted lines

> [2] It's unclear to me if you're passing any options to git-prune, but
>     you may want to pass "--expire" with a short grace period. Without
>     any options it prunes every unreachable thing, which can lead to
>     races if the repository is actively being used.
>
>     At GitHub we actually have a patch to `repack` that keeps all
>     objects, reachable or not, in the pack, and use it for all of our
>     automated maintenance. Since we don't drop objects at all, we can't
>     ever have such a race. Aside from some pathological cases, it wastes
>     much less space than you'd expect. We turn the flag off for special
>     cases (e.g., somebody has rewound history and wants to expunge a
>     sensitive object).
>
>     I'm happy to share the "keep everything" patch if you're interested.

Ah ok, I guess this is why we just skip repack. I guess '-Adl -b --pack-kept-objects' is not enough then.

-- 
Duy