threads / discuss / 66618

maintenance: stale maintenance.lock after the detached child is killed silently disables auto-maintenance (more likely since 2.55)

Subject: maintenance: stale maintenance.lock after the detached child is killed silently disables auto-maintenance (more likely since 2.55)

## tl;dr

One message between Oct 21, 2026 and Oct 21, 2026.

replies: 0people: 1as markdown or json

Masaya Kazama· Oct 21, 2026, 10:16 UTC · lore

[Resending as plain text; the first attempt was rejected by vger for containing an HTML part. Sorry for the duplicate to those on Cc.]

Hi,

Since 6e95b07e5f (builtin/maintenance: fix locking with "--detach", 2026-05-13), the detached "git maintenance" child owns objects/maintenance.lock until its background tasks finish. If that child is killed in a way that skips the tempfile cleanup (SIGKILL, power loss), the lock stays forever. Every later "git maintenance run --auto" then finds the lock and returns without running anything and without any message, so auto-maintenance stops for good and nobody notices.

The stale lock itself was reported in 2024 [1], where Stolee called it "unfortunately a common occurrence". What is new is that 6e95b07e5f extends the window in which the lock is held from the foreground phase to the whole background run, so a kill during the (possibly long) background tasks now leaves the lock behind. Before 6e95b07e5f the parent removed the lock when it exited right after daemonizing.

Reproduction (git 2.56.0 on macOS; same FIFO trick as Taylor's test in the "fix locking with --detach" discussion [2]):

    git init -q repro && cd repro
    git commit -q --allow-empty -m base
    git config maintenance.auto false
    git remote add origin /does/not/exist
    git config remote.origin.uploadpack \
        "echo \$PPID >\"$PWD/fetch.pid\" && cat \"$PWD/fifo\""
    mkfifo fifo
    git maintenance run --task=prefetch --detach
    while ! test -s fetch.pid; do sleep 0.1; done
    daemon=$(ps -o ppid= -p "$(cat fetch.pid)" | tr -d ' ')
    kill -9 "$daemon"
    : >fifo
    ls .git/objects/maintenance.lock      # still there
    GIT_TRACE2_EVENT="$PWD/trace" \
        git -c maintenance.loose-objects.auto=1 \
        maintenance run --auto --task=loose-objects
    grep -c '"region_enter".*"category":"maintenance"' trace   # 0
    git maintenance run --no-quiet --task=pack-refs
    # warning: lock file '.git/objects/maintenance' exists,
    #          skipping maintenance

After "rm .git/objects/maintenance.lock" the same --auto run enters one maintenance region. With Apple Git 2.50.1 (before 6e95b07e5f) the lock is already gone while the detached child runs, so killing it leaves nothing behind.

How I hit it: in one large partial clone (blob:none, ~425 branches) the lock was created on 2026-06-30, a few minutes after I upgraded to 2.55.0, and stayed until I found it on 2026-10-21. I do not know what killed the detached child; no git crash reports exist for that day. During those four months a GUI tool fetched the repository every minute; the pack count grew to 3,550 and the packs to 13.9 GB. Almost all of it was duplication: "count-objects -v" reported 94 million in-pack entries, but only 1.46 million remained after a manual "git maintenance run", which brought the packs down to 0.68 GB (GitHub reports about 0.4 GB for the repository). The commit-graph stayed at its April state, which made "git log --topo-order" over all refs take seconds. Two other repositories on the same machine had the same stale lock, created on 2026-09-29 and 2026-10-04; both are regular (non-partial) clones, so partial clone is not a condition; it likely made the packs pile up faster, since every promisor fetch is kept as a pack. Other projects report the same symptom, e.g. [3] (auto gc "silently disabled for a week") and [4].

A note on core.lockfilePid: it cannot simply be used to detect a stale maintenance.lock. With 2.56.0 and core.lockfilePid=true, while the detached child is alive, maintenance~pid.lock contains the pid of the parent, which has already exited. A liveness check based on that file would treat a live lock as stale.

Some options, roughly from least to most work (I have not tried to implement any of them):

 - When "maintenance run --auto" skips because of the lock, leave a
   trace that a user can find later (for example like the gc.log that
   a daemonized "git gc --auto" leaves), instead of returning silently.
 - Record the pid and hostname of the actual holder (the detached
   child) and ignore the lock when that process is gone on the same
   host, like lock_repo_for_gc() does for gc.pid.
 - Use an OS-level lock that is released when the process dies, as
   suggested in [1].

Thanks, miyaoka

[1] https://lore.kernel.org/git/CAN477tHJnVnOKfUsG5G9QAVdzYvmUuC8A8Vxt8mtHB23fd=hAQ@mail.gmail.com/ [2] https://lore.kernel.org/git/20260511201049.GB22912@coredump.intra.peff.net/ [3] https://github.com/craigoley/remudero/pull/9555 [4] https://github.com/solidDoWant/vault-sync/pull/115

← back to recent threads