Re: [PATCH v3 05/10] config doc: elaborate on fetch.fsckObjects security

From: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Date: Jul 28, 2018, 14:09 UTC
Message-ID: <871sbnv4d4.fsf@evledraar.gmail.com>
In-Reply-To: <xmqqeffomphy.fsf@gitster-ct.c.googlers.com>

On Fri, Jul 27 2018, Junio C Hamano wrote:

Show 9 quoted lines

> Ævar Arnfjörð Bjarmason  <avarab@gmail.com> writes:
>
>> +For now, the paranoid need to find some way to emulate the quarantine
>> +environment if they'd like the same protection as "push". E.g. in the
>
> We probably should mention that you can immediately prune, as these
> unwanted crufts are unreferenced.  That would probably be a lot easier
> workaround for the intended readers of this document than "find some
> way to emulate".

I'll mention that as well in v6 that "git prune" will get rid of these objects.

For what it's worth I was imagining something like a system where you're mirroring every push to some unpatched-git-host.com repo in-house, by doing a local "git fetch" when you see new data, and you're paranoid that someone's trying to introduce something like the .gitmodules security issue to your local mirror, even if you have transfer.fsckObjects set.

In a case like that, relying on "git prune" is much more fragile. You'd need to implement your mirror as some loop that does (pseudocode):

    while ref = poll_new_refs()
        git fetch upstream
        git prune --expire=now

As opposed to:

    while ref = poll_new_refs()
        (git fetch upstream && git prune --expire=now) &

As you might find in some event-based system. I.e. every time you fetch you need to stop the world and run a full prune, because the potentially malicious upstream can craft a series of ref updates where one ref update (which you'll refuse) contains the bad data, but at that point you have some of those blobs/trees/commits it in your object store, and then a second ref update references that already existing data and causes you to update the ref.

It's also much slower and I/O heavy, on an already-pruned linux.git running 'git prune --expire=now' takes 40 seconds on my machine, as opposed to:

    while ref = poll_new_refs()
        (git fetch && git push internal-mirror --mirror) &

Which could take as little time as a second for the whole operation, can safely be run in parallel, and would be protected because the actually published internal mirror gets its refs via receive-pack, which uses the quarantine.