Re: [RFC/PATCH] gc: run more pre-detach operations under lock

From: Junio C Hamano <gitster@pobox.com>
Date: Jun 20, 2019, 18:55 UTC
Message-ID: <xmqqef3onl29.fsf@gitster-ct.c.googlers.com>
In-Reply-To: <20190619191037.GE28145@sigill.intra.peff.net>

Jeff King <peff@peff.net> writes:

> I think there may be room for both approaches. Yours fixes the repeated
> message in the more general case, but Duy's suggestion is the most
> efficient thing.

Yeah, not just the most efficient, but it is a low-hanging-fruit that is very obvious.

> I agree that the "thousands of remotes" case means we might want to gc
> in the interim. But we probably ought to do that deterministically
> rather than hoping that the pattern of lock contention makes sense.

In the process chain starting from the topmost "git fetch <multiple>", that calls multiple "git fetch <one>" for individual remotes, that does network transfer and calls "git index-pack" (or "git unpack-objects"), the bottommost layer knows how much cruft one step among "thousands of remotes" created for a later "gc", but there is no mechanism for them to report the information back to their callers. Unlike "git svn" that periodically calls the auto gc every N commits and can claim to be aware of its cruft creation rate ;-), the information available to the topmost "git fetch" is only the number of underlying fetches, counting a no-op fetch and a fetch that is close to a full clone equally, which is not a good way to gauge the cruft creation rate X-<.

A useful deteministic triggering would be a good thing to aim in the longer run, but would be somewhat involved to design and implement, I am afraid.