Repo seems broken after deleting submodule

I included a submodule in my project then decided I didn't like submodules and deleted it again. I followed the advice of delting .gitmodules, the bit from .git/config and then git rm'ing the submodule. Seemed to work. I then copied files with the same directory name into where the submodule was. However, I can't add them.

Doing git add /path/to/old/submodule - does nothing, files are not staged, no error messages no nothing. If I try to git rm /path/to/old/submodule - it just says 'did not match any files'.

I'm a bit stuck... any suggestions for things I can try much appreciated.