{"thread":{"id":"64601","subject":"UX failure: SSH authentication failure diagnostics","startedAt":"2025-12-09T05:08:10Z","lastAt":"2025-12-12T21:46:13Z","messageCount":2,"participants":["Ryan Johnson","D. Ben Knoble"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"531885","messageId":"DS0PR03MB729012244C8A65D318FDC205A3A3A@DS0PR03MB7290.namprd03.prod.outlook.com","threadId":"64601","inReplyTo":null,"subject":"UX failure: SSH authentication failure diagnostics","fromName":"Ryan Johnson","fromEmail":"ryan.johnson.code@gmail.com","sentAt":"2025-12-09T05:08:06Z","receivedAt":"2025-12-09T05:08:10Z","isPatch":false,"sender":{"key":"ryan.johnson.code@gmail.com","avatar":null},"body":"When Git fails SSH authentication, the error message provides no indication\nthat Git may be using a different SSH client than the user expects.\n\nPROBLEM\n\nMultiple SSH implementations commonly coexist on a single system:\n\n  Windows:\n    - Windows OpenSSH: C:\\Windows\\System32\\OpenSSH\\ssh.exe\n    - Git's bundled SSH: C:\\Program Files\\Git\\usr\\bin\\ssh.exe\n\n  macOS:\n    - System SSH: /usr/bin/ssh\n    - Homebrew SSH: /opt/homebrew/bin/ssh\n\n  Linux:\n    - System SSH: /usr/bin/ssh\n    - Snap/Flatpak-packaged Git may bundle its own SSH\n    - Alternative installations: /usr/local/bin/ssh\n\nThese may use separate key stores and agents. On Windows, the system\nssh-agent service is inaccessible to Git's bundled MSYS2 SSH.\n\nA user who runs:\n\n  ssh -T git@github.com    # Works - uses one SSH binary\n  git push                 # Fails - uses different SSH binary\n\nreceives only:\n\n  git@github.com: Permission denied (publickey).\n  fatal: Could not read from remote repository.\n\nThis error gives no indication that Git is using a different SSH binary than\nthe one the user just tested. The user has no reason to suspect this is the\ncause. Debugging this issue typically costs hours of research.\n\nSOLUTION\n\nWhen SSH authentication fails, Git should:\n\n1. Print which SSH command it invoked:\n\n     Using SSH: /opt/homebrew/bin/ssh\n\n2. Detect if multiple ssh binaries exist in PATH or common locations. If so:\n\n     Note: Multiple SSH clients detected on this system.\n     Git is using: C:\\Program Files\\Git\\usr\\bin\\ssh.exe\n     Also found:   C:\\Windows\\System32\\OpenSSH\\ssh.exe\n     \n     To use a different SSH client:\n       git config --global core.sshCommand \"/path/to/preferred/ssh\"\n\nThis diagnostic should only appear on authentication failure, not on success.\n\nRATIONALE\n\nGit for Windows bundles MSYS2 tools for cross-platform consistency. Homebrew\nand Snap/Flatpak may install SSH binaries that shadow or conflict with system\nSSH. These are reasonable packaging decisions, but the resulting SSH client\nmismatch is a known, common failure mode that produces no actionable\ndiagnostic information.\n\nThe fix is a one-line config change. The problem is that users have no way\nto discover this without external research. Surfacing this information at\nthe point of failure would eliminate significant friction for beginners\nas well as veterans. Seasoned programmers and beginners alike complain about\nUX failures like this one all the time. Considering your tool has become de-facto\nstandard, you should take care of these problems. Dealing with this problem\n is the responsibility of the tooling creators, not the users. Do not shunt\nresponsibility onto every user to sit and spend an entire day of research and\nheadache unraveling your poorly-communicated configuration nuances.\n\n--\nRyan"},{"id":"532092","messageId":"CALnO6CDB8aHZ96emgX43GOVAzZxz_7-ZkOqhasob=zf+Hot0fw@mail.gmail.com","threadId":"64601","inReplyTo":"DS0PR03MB729012244C8A65D318FDC205A3A3A@DS0PR03MB7290.namprd03.prod.outlook.com","subject":"Re: UX failure: SSH authentication failure diagnostics","fromName":"D. Ben Knoble","fromEmail":"ben.knoble@gmail.com","sentAt":"2025-12-12T21:46:02Z","receivedAt":"2025-12-12T21:46:13Z","isPatch":false,"sender":{"key":"ben.knoble@gmail.com","avatar":"https://avatars.githubusercontent.com/u/22802209?v=4"},"body":"On Tue, Dec 9, 2025 at 12:08 AM Ryan Johnson\n<ryan.johnson.code@gmail.com> wrote:\n>\n> When Git fails SSH authentication, the error message provides no indication\n> that Git may be using a different SSH client than the user expects.\n>\n> PROBLEM\n>\n> Multiple SSH implementations commonly coexist on a single system:\n>\n>   Windows:\n>     - Windows OpenSSH: C:\\Windows\\System32\\OpenSSH\\ssh.exe\n>     - Git's bundled SSH: C:\\Program Files\\Git\\usr\\bin\\ssh.exe\n>\n>   macOS:\n>     - System SSH: /usr/bin/ssh\n>     - Homebrew SSH: /opt/homebrew/bin/ssh\n>\n>   Linux:\n>     - System SSH: /usr/bin/ssh\n>     - Snap/Flatpak-packaged Git may bundle its own SSH\n>     - Alternative installations: /usr/local/bin/ssh\n>\n> These may use separate key stores and agents. On Windows, the system\n> ssh-agent service is inaccessible to Git's bundled MSYS2 SSH.\n>\n> A user who runs:\n>\n>   ssh -T git@github.com    # Works - uses one SSH binary\n>   git push                 # Fails - uses different SSH binary\n>\n> receives only:\n>\n>   git@github.com: Permission denied (publickey).\n>   fatal: Could not read from remote repository.\n>\n> This error gives no indication that Git is using a different SSH binary than\n> the one the user just tested. The user has no reason to suspect this is the\n> cause. Debugging this issue typically costs hours of research.\n\nI'm a bit confused on how this could be the case, but my knowledge\nhere is admittedly murky: Git looks for SSH in the following places\n\n- the value of GIT_SSH_COMMAND in the environment\n- the value configured for core.sshcommand\n- the value of GIT_SSH in the environment (historical compatibility,\naccording to source comments)\n- finally, it uses the command \"ssh\" (which I assume is looked up in PATH)\n\nGranted, Git does a little bit of PATH manipulation to find its own\nbinaries, so is this the culprit? That might explain why the issue\nappears on Windows (bundled MSYS2 SSH)?\n\n>\n> SOLUTION\n>\n> When SSH authentication fails, Git should:\n>\n> 1. Print which SSH command it invoked:\n>\n>      Using SSH: /opt/homebrew/bin/ssh\n>\n> 2. Detect if multiple ssh binaries exist in PATH or common locations. If so:\n>\n>      Note: Multiple SSH clients detected on this system.\n>      Git is using: C:\\Program Files\\Git\\usr\\bin\\ssh.exe\n>      Also found:   C:\\Windows\\System32\\OpenSSH\\ssh.exe\n>\n>      To use a different SSH client:\n>        git config --global core.sshCommand \"/path/to/preferred/ssh\"\n>\n> This diagnostic should only appear on authentication failure, not on success.\n\nI think a simpler thing to do might be to make GIT_TRACE2* output\nlookup the pah for the command that's being run and log it, since I\ncan see (using either GIT_TRACE2 or GIT_TRACE2_EVENT) the invocations\nof ssh for e.g. git-fetch. They just appear as \"ssh\" for me, although\nat the time of the trace events we don't know (at least on *nix)\nwhether we're going to exec the program directly or fall back to\nexecuting it through the shell.\n\nWhile poking at the exec calls, I saw we also have locate_in_PATH\n(non-Windows!), so perhaps we could also use that here. And we could\nadd advice of the kind you mentioned (although I'm not sure it makes\nsense to search the system for other ssh implementations?), though it\nmight not be the case that the solution to a failed SSH attempt is the\nclient (it could be something else).\n\nStepping back a moment: I realized you wrote \"When SSH authentication\nfails\"---I'm not sure if Git sees that. I suspect it only sees that\nSSH failed (\"could not read from remote\"). And authentication could\nfail for any number of reasons. Hm. What to do?\n\n> RATIONALE\n>\n> Git for Windows bundles MSYS2 tools for cross-platform consistency. Homebrew\n> and Snap/Flatpak may install SSH binaries that shadow or conflict with system\n> SSH. These are reasonable packaging decisions, but the resulting SSH client\n> mismatch is a known, common failure mode that produces no actionable\n> diagnostic information.\n>\n> The fix is a one-line config change. The problem is that users have no way\n> to discover this without external research. Surfacing this information at\n> the point of failure would eliminate significant friction for beginners\n> as well as veterans. Seasoned programmers and beginners alike complain about\n> UX failures like this one all the time. Considering your tool has become de-facto\n> standard, you should take care of these problems. Dealing with this problem\n>  is the responsibility of the tooling creators, not the users. Do not shunt\n> responsibility onto every user to sit and spend an entire day of research and\n> headache unraveling your poorly-communicated configuration nuances.\n>\n> --\n> Ryan\n\nI can't speak for anyone else, but I find the statements at the end\nunhelpful to a fruitful collaboration for improvement. \"Your tool\",\n\"you should\", (paraphrasing) it's your problem, not mine… how are we\nto collaborate in such an atmosphere? (This kind of thing is where\nthat trite \"we don't owe you anything\" comes from [1].) In a\ncharitable reading, I'd just skip the last few sentences. It sounds\nlike you've found a problem in documentation, or maybe in the way\ncertain errors are presented. Great! Help us improve that for the next\nperson, or maybe write down what you found and share it widely so\nothers won't spend as much time researching the problem as you did.\n\n[1]: https://mikemcquaid.com/open-source-maintainers-owe-you-nothing/\n\n-- \nD. Ben Knoble\n"}]}