Unable to de-init stubborn submodule

What if we wanted to drop sha1collisiondetection/ as a submodule and replace it with a copy of what's now in sha1dc/? I ran into this with another project, but here's a way to reproduce it on git.git:

    (
        rm -rf /tmp/git &&
        git clone git@github.com:git/git.git /tmp/git
        cd /tmp/git &&
        git tag nuke-before &&
        git submodule update --init &&
        git rm -r .gitmodules sha1collisiondetection &&
        git commit -m"Nuke sha1dc submodule" &&
        cp -Rvp sha1dc sha1collisiondetection &&
        git add sha1collisiondetection &&
        git commit -m"Now it's not a submodule" &&
        git tag nuke-after &&
        git reset --hard nuke-before &&
        git submodule update --init && # skip this and the below won't fail
        git reset --hard nuke-after && # Emulate someone doing a pull
        git ls-tree HEAD | grep sha1collisiondetection && # OK, shows "tree" not "commit"
        test $(git rev-parse HEAD) == $(git -C sha1collisiondetection/ log -1 --pretty=format:%H) && echo OK || echo WTF
    )

This results in a really bizarre state where according to ls-tree sha1collisiondetection is a tree at the current commit:

    040000 tree 81583289d96bdde4b366c243ab524ea28d895ea5    sha1collisiondetection

But git still believes there's a submodule there for some reason, and shows the log for the upstream sha1collisiondetection project:

    git -C sha1collisiondetection/ log -1
    commit 19d97bf (HEAD, origin/master, origin/HEAD, master)
    Merge: 3f14d1b c93f0b4
    Author: Dan Shumow <shumow@gmail.com>
    Date:   Sat Jul 1 12:36:15 2017 -0700

        Merge pull request #37 from avar/fixup-pull-request-34

        Fix endian detection logic for Sparc, little endian BSD etc.

Doing:

    git submodule deinit sha1collisiondetection

Does nothing to help, then I thought it might be:

    git config -f .git/config -l|grep ^submodule
    submodule.sha1collisiondetection.active=true
    submodule.sha1collisiondetection.url=https://github.com/cr-marcstevens/sha1collisiondetection.git

But running:

    git config --remove-section submodule.sha1collisiondetection

Doesn't help either, neither does removing the index:

    rm .git/index &&
    git reset --hard

If you then do:

    rm -rf .git/modules

You'll get this error:

    git -C sha1collisiondetection/ log -1
    fatal: Not a git repository: /tmp/git/sha1collisiondetection/../.git/modules/sha1collisiondetection

But I can't see what's still referencing it.

This problem is avoided if, as noted with a comment I skip:

    git submodule update --init

But I shouldn't need to remember to de-init a submodule before moving to a new commit that doesn't have it, least I end up in some seemingly unrecoverable state.

Am I missing something obvious here?