BUG: fetch in certain repo always gives "did not send all necessary objects"

[I cannot share the local repo and had to modify output of commands slightly to redact hostnames, branch names, etc. I think I haven't messed anything up, but it's possible...]

Two people in the last week have come to me after running into a case where they could not update their repo because any call to git fetch or git pull would error out with

  fatal: bad object HEAD
  error: gerrit.server:reponame did not send all necessary objects"

which apparently has been reported before at both of:

https://public-inbox.org/git/CAE5ih78nLL6UhKPObvFEA9xQZUtc1XpPvGJNaYTH9fJ0RyFRvA@mail.gmail.com/ and

https://public-inbox.org/git/CAGZ79kYP0z1G_H3nwfmSHraWHMBOcik5LepUXKj0nveeBrihiw@mail.gmail.com/

but without a resolution and it appears to have been difficult to reproduce. I can reproduce reliably with a copy of one of their repos, and have dug in a little bit and may be able to provide more info. The pieces I've found that look relevant:

* Both users were using git-2.16.1 (suggesting they update and try new
things often)
* "git fsck --no-dangling" reports no problem on either client or server.
* The error bisects to commit d0c39a49cc ("revision.c: --all adds HEAD
from all worktrees", 2017-08-23).
* The developer has 3 directories in use corresponding to different
worktrees, but four entries under .git/worktrees, one of which appears
that it was once in use but no longer is
* The unused entry under .git/worktrees has a detached HEAD that
points to an object that doesn't exist; I suspect it was garbage
collected.

More details about the last two items:

$ ls .. master/ 1.0-release/ 2.0-release/ 3.0-release/

$ for i in ../*/.git; do echo $i $(cat $i/.git 2>&1); done ../master/.git cat: ../master/.git: Is a directory ../1.0-release/.git gitdir: /clone/master/.git/worktrees/1.0-release ../2.0-release/.git gitdir: /clone/master/.git/worktrees/2.0-release ../3.0-release/.git gitdir: /clone/master/.git/worktrees/3.0-release1

$ for i in .git/worktrees/*/gitdir; do echo $i $(cat $i); done .git/worktrees/1.0-release/gitdir /clone/1.0-release/.git .git/worktrees/2.0-release/gitdir /clone/2.0-release/.git .git/worktrees/3.0-release1/gitdir /clone/3.0-release/.git .git/worktrees/3.0-release/gitdir /clone/3.0-release/.git

$ (cd .git && for i in worktrees/*/HEAD; do echo $i $(git rev-parse $i^{object}); done) worktrees/1.0-release/HEAD c483c6806fe19c2fa9bcda892e0ddb470f8e9d65 worktrees/2.0-release/HEAD ab4c699ec9d1eab143c7e8ef51a0a6c451fcd4ea worktrees/3.0-release1/HEAD 55e52e24b24b2f81484de9c6ea7d4520238c5fd5 fatal: ambiguous argument 'worktrees/3.0-release/HEAD^{object}': unknown revision or path not in the working tree. Use '--' to separate paths from revisions, like this: 'git <command> [<revision>...] -- [<file>...]' worktrees/3.0-release/HEAD worktrees/3.0-release/HEAD^{object}

So, it appears both worktrees/3.0-release and worktrees/3.0-release1 believe they are used by /clone/3.0-release/, but /clone/3.0-release uses worktree-3.0-release1. That leaves worktrees/3.0-release unused, and in fact its HEAD is some sha1 that does not exist in the repo -- possibly GC'ed.

Also, after the investigation above and based on the hint in one of the other email threads linked above, we tried 'git worktree prune' but it didn't remove the unused directory under .git/worktree (perhaps because it thought something was using it?). The fetch bug didn't go away until doing an 'rm -rf .git/worktree/3.0-release'.

I have no experience prior to today with git worktree, and the developer in question doesn't remember how things got to the current state. It was all a while ago.

Does anyone have an idea what may have happened here or how to avoid it?