Re: Question about `git gc` locking files

> > When this bug was fixed, the maintenance runs that triggered during
> > usage of the external tool, repo[1], would lock the HEAD file in the
> > Android manifest repository thereby erroring out `repo`. Additionally,
> > long running maintenance tasks would also cause users to frequently
> > run into this issue when using git commands that are written to HEAD.
>
> It is a bit surprising that HEAD would need to be locked in the first
> place. As far as I am aware, the only step where we end up locking refs
> in the context of git-gc(1) would be when we decide to repack refs via
> git-pack-refs(1). And that command shouldn't ever end up packing the
> HEAD file, as that loose reference must exist
>
> Digging a bit deeper surfaces that it's `git reflog expire --all` that
> causes us to lock HEAD, which is... unfortunate. Seemingly, relfogs are
> locked by locking the corresponding reference.

> > We can fix this easily temporarily by pushing out config changes to
> > run in the foreground, however, I was under the impression that `git
> > gc`, whether invoked normally or through `git maintenance`, would be
> > able to run in parallel with other git commands and therefore not
> > lock. There is no mention of this in the documentation for `git gc`,
> > but I do see it in the `git maintenance` documentation. So should `git
> > gc` be locking the HEAD file in the first place? And if so, is there a
> > way for `git gc` to have less of a dependence on HEAD.lock?
>
> So what seems to be happening is that you have two processes racing with
> each other: one that is trying to expire entries from your "HEAD"
> reflog, and the one invoked by the user to update "HEAD". By default,
> Git will wait up to 100ms for the "HEAD" lock to be released, but it
> seems like expiring the reflog for your "HEAD" takes longer than that.
> You can work around that issue by increasing "core.filesRefLockTimeout".
>
> But this whole thing uncovers an issue with git-maintenance(1) itself.
> The above commit fixed how git-maintenance(1) behaves such that we
> detach at the correct point in time. But what it neglects is that some
> tasks are more special than others and should be run synchronously
> whereas others can be run asynchronously. Packing refs and expiring the
> reflog are tasks that should be run synchronously to minimize the impact
> on users.
>
> This all demonstrates that git-maintenance(1) needs to get some more
> love. You have uncovered this issue with git-gc(1) as configured task,
> but we have a similar fundamental issue with the git-pack-refs(1)
> subtask. So I guess we'll have to classify those subtasks into two
> categories, where one category needs to be executed before detaching
> itself and another category can be executed asynchronously after we have
> detached.