Yes, I believe they use `/bin/sh`, except on Windows, where they may look up things in the PATH. It is very much intentional that they use sh because it means that aliases work in a consistent way across platforms and that they work at all.
On Linux, I have mksh, but my work machine on macOS does not, so specifying `core.sh = /bin/mksh` would not work on all my systems. If you have system-level aliases as well as user-level aliases, then you need to pick a particular shell and letting the user choose will likely break the system-level aliases. This is not a hypothetical: my last company shipped system-level aliases on development VMs while users were free to customize their user-level aliases (and shell).
This will absolutely be a problem on Windows, where you have CMD, multiple versions of PowerShell, and bash, all with different syntax. Some versions of PowerShell also corrupt non-Unicode data sent through pipelines as well, which means they're effectively useless for general-purpose scripting.
It can also lead to odd behaviour on some shells because, for instance, zsh runs a config file with `-c`, `.zshenv`, that can make the shell very much less POSIX compliant, so behaviour can differ substantially from sh.
I'll also note that Git doesn't handle aliases specially. Git has a with-shell and a without-shell mode for running commands, and we use the former for aliases. We definitely do not want the with-shell mode to generically allow arbitrary shells because we cannot handle non-POSIX shells and will not quote correctly for them (almost certainly leading to security vulnerabilities). We also format the shell command as "%s \"$@\"", which also basically requires a POSIX shell to work, and we don't want to work around people's use of non-POSIX shells; that way madness lies.
However, because mksh is POSIXy, you could build Git on your system with `SHELL_PATH=/bin/mksh`, which would use `/bin/mksh` everywhere that the shell is invoked. That feature exists primarily for systems where `sh` is not POSIXy or doesn't support `local` (which I know mksh does), but there's no reason you can't use it yourself.
There has been some discussion about this issue in the past (in the context of PowerShell) at https://lore.kernel.org/git/CAAXzdLXt4+-34+OhS=Jn=-VeORN3Y2jMzzg9+bhyn88aN4hm0A@mail.gmail.com/ .