Re: Ideas to speed up repacking

From: Junio C Hamano <gitster@pobox.com>
Date: Dec 3, 2013, 07:17 UTC
Message-ID: <xmqqli02pfnf.fsf@gitster.dls.corp.google.com>
In-Reply-To: <CACsJy8DJU2YTE1iNdb=fvo0fVOgLUK2mKXUhjcoJh8Ac0wW_EA@mail.gmail.com>

Duy Nguyen <pclouds@gmail.com> writes:

Show 22 quoted lines

>> If nothing else has happened in the repository, perhaps, but I
>> suspect that the real problem is how you would prove it.  For
>> example, I am guessing that your Scenario 4 could be something like:
>>
>>     : setup #1
>>     $ git repack -a -d -f
>>     $ git prune
>>
>>     : scenario #4
>>     $ git commit --allow-empty -m 'new commit'
>>
>> which would add a single loose object to the repository, advancing
>> the current branch ref by one commit, fast-forwarding relative to
>> the state you were in after setup #1.
>>
>> But how would you efficiently prove that it was the only thing that
>> happened?
>
> Shawn mentioned elsewhere that we could generate bundle header in and
> keep it in pack-XXX.bh file at pack creation time. With that
> information we could verify if a ref has been reset, just fast
> forwarded or even deleted.

With what information? If you keep the back-then-current information and nothing else, how would you differentiate between the simple scenario #4 above vs 'lost and new' two commit versions of the scenario? The endpoints should both show that one ref (and only one ref) advanced by one commit, but one has cruft in the object database while the other does not.

Show 9 quoted lines

>> Also with Scenario #2, how would you prove that the new pack does
>> not contain any cruft that is not reachable?  When receiving a pack
>> and updating our refs, we only prove that we have all the objects
>> needed to complete updated refs---we do not reject packs with crufts
>> that are not necessary.
>
> We trust the pack producer to do it correctly, I guess. If a pack
> producer guarantees not to store any cruft, it could mark the pack
> somehow.

That is not an answer. Since when do we design to blindly trust anybody on the other end of the wire?