Re: Race condition between repack and loose-objects maintenance task

From: Taylor Blau <me@ttaylorr.com>
Date: Jun 29, 2022, 17:21 UTC
Message-ID: <YryKCl5J1Em89d3e@nand.local>
In-Reply-To: <CAKQoGakSFaNm10ZeTKc8XtTcD0JW19CZP1OwA4j7W__iBQaJfg@mail.gmail.com>

On Wed, Jun 29, 2022 at 10:16:38AM -0700, Gregory Szorc wrote:

Show 9 quoted lines

> > In either case, my recommendation would be to keep those unreachable
> > objects which haven't yet aged out of the repository around for longer,
> > which will decrease the likelihood of seeing the race.
>
> We had to lower gc.pruneExpire from its default of 1 hour because
> otherwise this would result in the creation of thousands of loose
> objects. This is normally acceptable. However, on NFS, the churn from
> lots of file creation and deletion resulted in acceptable performance
> degradation. We had to lower gc.pruneExpire to minimize the damage.

Makes sense. Having a shorter gc.pruneExpire makes the race much more likely to occur in practice, so I'm glad that we have a plausible confirmation that that's what's going on in your repository.

Show 9 quoted lines

> > See Documentation/technical/cruft-packs.txt for more information about
> > cruft packs.
>
> Yes, I saw this new feature in the Git release this week and am hoping
> to roll it out to better mitigate this general problem! With cruft
> packs, I anticipate being able to restore a longer gc.pruneExpire
> value as they should mitigate the NFS performance badness we're
> working around. Thank you for confirming it should help with the
> problem.

That should definitely help. Let me know if you have any questions, and thanks for trying it out!

Thanks, Taylor