threads / discuss / 52130

Support for non-ascii urls broken in some cases, help needed for Windows

Subject: Support for non-ascii urls broken in some cases, help needed for Windows

## tl;dr

One message between Oct 26, 2019 and Oct 26, 2019.

replies: 0people: 1as markdown or json

Mike Hommey· Oct 26, 2019, 23:13 UTC · lore
Hi,

I don't actually know of any git hosting that lives on a non-ascii domain and with non-ascii parts in the path, but I nevertheless gave it a shot because I was wondering how a conversion of git-cinnabar to python 3 should handle them.

Multiple problems arose when trying `git clone https://テスト.com`:
- with NO_GETTEXT set, setlocale is never called, and when curl calls
  libidn for conversion of the domain name, it fails with
  "Failed to convert テスト.com to ACE; could not convert string to
  UTF-8" because, while the string *is* UTF-8, it thinks it's ASCII and
  wants to convert it somehow.
  Calling setlocale(LC_CTYPE, "") before the curl request happens fixes
  this. It feels like git_setup_gettext in the NO_GETTEXT case should
  take care of this.
- On Windows with a Japanese locale, it fails with the same error.
  Earlier versions of git failed with a different error message. I
  suspect 090d1e84771bb4a310e3fe8291ec71b0ddb03d4f is involved in the
  change in error message but I'm not sure. Anyways, I think the problem
  here is that what git gets as input from the command line is not UTF-8,
  and curl fails to convert it however it tries to. However, for
  non-ascii path parts, it seems a conversion to UTF-8 happens correctly
  at some stage, so I'm not entirely sure. That is,
  `git clone http://localhost:8000/テスト` does send a UTF-8 テスト to
  the HTTP server (which, interestingly, is not what
  `curl http://localhost:8000/テスト` does).

I'm happy to look at fixing the NO_GETTEXT unix part, but I'm not equipped to build git for windows the way it's shipped to look at it there.

Mike

← back to recent threads