{"thread":{"id":"21379","subject":"[PATCH 1/4] Add routines for filenames encoding <-> UTF-8","startedAt":"2009-10-27T13:54:00Z","lastAt":"2009-10-28T18:10:34Z","messageCount":8,"participants":["Timur Sufiev","Peter Krefting","Jeff King"],"isPatch":true,"patchVersion":1,"patchTotal":4},"messages":[{"id":"126015","messageId":"1256651643-18382-1-git-send-email-timur@iris-comp.ru","threadId":"21379","inReplyTo":null,"subject":"[PATCH 1/4] Add routines for filenames encoding <-> UTF-8","fromName":"Timur Sufiev","fromEmail":"timur@iris-comp.ru","sentAt":"2009-10-27T13:54:00Z","receivedAt":"2009-10-27T13:54:00Z","isPatch":true,"sender":{"key":"timur@iris-comp.ru","avatar":null},"body":"The point is to make Git aware of filenames local encoding and make it\nkeep all filenames in UTF-8 internally. If\n`i18n.filenameslocalencoding' option was set via git-config to a\ncorrect encoding, 2 things should be done:\n\n1. Translate all filenames read by READDIR from into UTF-8.\n\n2. Translate all filenames passed to IO-routines from UTF-8 into\n.\n\nSigned-off-by: Timur Sufiev \n---\n cache.h | 1 +\n config.c | 3 ++\n environment.c | 1 +\n io-i18n.c | 82 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++\n 4 files changed, 87 insertions(+), 0 deletions(-)\n create mode 100644 io-i18n.c\n\ndiff --git a/cache.h b/cache.h\nindex 96840c7..7f19f7a 100644\n--- a/cache.h\n+++ b/cache.h\n@@ -919,6 +919,7 @@ extern int user_ident_explicitly_given;\n \n extern const char *git_commit_encoding;\n extern const char *git_log_output_encoding;\n+extern const char *git_filenames_local_encoding;\n extern const char *git_mailmap_file;\n \n /* IO helper functions */\ndiff --git a/config.c b/config.c\nindex c644061..2be6531 100644\n--- a/config.c\n+++ b/config.c\n@@ -539,6 +539,9 @@ static int git_default_i18n_config(const char *var, const char *value)\n \tif (!strcmp(var, \"i18n.logoutputencoding\"))\n \t\treturn git_config_string(&git_log_output_encoding, var, value);\n \n+\tif (!strcmp(var, \"i18n.filenameslocalencoding\"))\n+\t return git_config_string(&git_filenames_local_encoding, var, value);\n+\n \t/* Add other config variables here and to Documentation/config.txt. */\n \treturn 0;\n }\ndiff --git a/environment.c b/environment.c\nindex 5de6837..b101f7b 100644\n--- a/environment.c\n+++ b/environment.c\n@@ -24,6 +24,7 @@ int warn_ambiguous_refs = 1;\n int repository_format_version;\n const char *git_commit_encoding;\n const char *git_log_output_encoding;\n+const char *git_filenames_local_encoding;\n int shared_repository = PERM_UMASK;\n const char *apply_default_whitespace;\n const char *apply_default_ignorewhitespace;\ndiff --git a/io-i18n.c b/io-i18n.c\nnew file mode 100644\nindex 0000000..4dcc2db\n--- /dev/null\n+++ b/io-i18n.c\n@@ -0,0 +1,82 @@\n+#include \"utf8.h\"\n+#include \"cache.h\"\n+\n+inline static int is_string_ascii(const char *str)\n+{\n+\tint is_ascii = 1;\n+\n+\tfor (; *str && is_ascii; str++)\n+\t\tis_ascii &= isascii(*str);\n+\n+\treturn is_ascii;\n+}\n+\n+static char *filename_to_utf8(const char *filename)\n+{\n+\tchar *out;\n+\n+\tif (is_string_ascii(filename))\n+\t\treturn NULL;\n+\n+#ifndef NO_ICONV\n+\tif (git_filenames_local_encoding && !is_utf8(filename)) {\n+\t\tout = reencode_string(filename,\n+\t\t\t\t \"utf-8\", git_filenames_local_encoding);\n+#ifdef DEBUG_I18N\n+\t\tfprintf(stderr, \"Local -> UTF8 encoding: <%s> -> <%s>\\n\",\n+\t\t\tfilename, out);\n+#endif\n+\t\treturn out;\n+\t} else if (git_filenames_local_encoding && is_utf8(filename)) {\n+#ifdef DEBUG_I18N\n+\t\tfprintf(stderr,\n+\t\t\t\"Filename <%s> is already utf8-encoded, doing nothing...\\n\",\n+\t\t\tfilename);\n+#endif\n+\t\treturn NULL;\n+\t} else {\n+#ifdef DEBUG_I18N\n+\t\tfprintf(stderr, \"No local encoding set, doing nothing...\\n\");\n+#endif\n+\t\treturn NULL;\n+\t}\n+#else /* #ifdef NO_ICONV */\n+\twarning(\"No iconv support, doing nothing...\\n\");\n+\treturn NULL;\n+#endif\n+}\n+\n+char *filename_to_local(const char *filename)\n+{\n+\tchar *out;\n+\n+\tif (is_string_ascii(filename))\n+\t\treturn NULL;\n+\n+#ifndef NO_ICONV\n+\tif (git_filenames_local_encoding && is_utf8(filename)) {\n+\t\tout = reencode_string(filename,\n+\t\t\t\t git_filenames_local_encoding, \"utf-8\");\n+#ifdef DEBUG_I18N\n+\t\tfprintf(stderr, \"UTF8 -> local encoding: <%s> -> <%s>\\n\",\n+\t\t\tfilename, out);\n+#endif\n+\t\treturn out;\n+\t} else if (git_filenames_local_encoding && !is_utf8(filename)) {\n+#ifdef DEBUG_I18N\n+\t\tfprintf(stderr,\n+\t\t\t\"Filename <%s> is already local-encoded, doing nothing...\\n\",\n+\t\t\tfilename);\n+#endif\n+\t\treturn NULL;\n+\t} else {\n+#ifdef DEBUG_I18N\n+\t\tfprintf(stderr, \"No local encoding set, doing nothing...\\n\");\n+#endif\n+\t\treturn NULL;\n+\t}\n+#else /* #ifdef NO_ICONV */\n+\twarning(\"No iconv support, doing nothing...\\n\");\n+\treturn NULL;\n+#endif\n+}\n-- \n1.6.5.1\n"},{"id":"126016","messageId":"1256651643-18382-2-git-send-email-timur@iris-comp.ru","threadId":"21379","inReplyTo":"1256651643-18382-1-git-send-email-timur@iris-comp.ru","subject":"[PATCH 2/4] Add I18N-wrappers for low-level IO-routines","fromName":"Timur Sufiev","fromEmail":"timur@iris-comp.ru","sentAt":"2009-10-27T13:54:01Z","receivedAt":"2009-10-27T13:54:01Z","isPatch":true,"sender":{"key":"timur@iris-comp.ru","avatar":null},"body":"Signed-off-by: Timur Sufiev \n---\n Makefile | 2 +\n io-i18n.c | 129 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++\n io-i18n.h | 23 +++++++++++\n 3 files changed, 154 insertions(+), 0 deletions(-)\n create mode 100644 io-i18n.h\n\ndiff --git a/Makefile b/Makefile\nindex 42b7d60..ac8e807 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -459,6 +459,7 @@ LIB_H += tree-walk.h\n LIB_H += unpack-trees.h\n LIB_H += userdiff.h\n LIB_H += utf8.h\n+LIB_H += io-i18n.h\n LIB_H += wt-status.h\n \n LIB_OBJS += abspath.o\n@@ -562,6 +563,7 @@ LIB_OBJS += unpack-trees.o\n LIB_OBJS += usage.o\n LIB_OBJS += userdiff.o\n LIB_OBJS += utf8.o\n+LIB_OBJS += io-i18n.o\n LIB_OBJS += walker.o\n LIB_OBJS += wrapper.o\n LIB_OBJS += write_or_die.o\ndiff --git a/io-i18n.c b/io-i18n.c\nindex 4dcc2db..9d89ac3 100644\n--- a/io-i18n.c\n+++ b/io-i18n.c\n@@ -1,3 +1,4 @@\n+#include \"io-i18n.h\"\n #include \"utf8.h\"\n #include \"cache.h\"\n \n@@ -80,3 +81,131 @@ char *filename_to_local(const char *filename)\n \treturn NULL;\n #endif\n }\n+\n+int stat_i18n(const char *filename, struct stat *buf)\n+{\n+\tint ret;\n+\tchar *out = filename_to_local(filename);\n+\n+\tif (out != NULL) {\n+\t\tret = stat(out, buf);\n+\t\tfree(out);\n+\t} else\n+\t\tret = stat(filename, buf);\n+\treturn ret;\n+}\n+\n+int lstat_i18n(const char *filename, struct stat *buf)\n+{\n+\n+\tint ret;\n+\tchar *out = filename_to_local(filename);\n+\n+\tif (out != NULL) {\n+\t\tret = lstat(out, buf);\n+\t\tfree(out);\n+\t} else\n+\t\tret = lstat(filename, buf);\n+\treturn ret;\n+}\n+\n+DIR *opendir_i18n(const char *dirname)\n+{\n+\tDIR *dir;\n+\tchar *out = filename_to_local(dirname);\n+\n+\tif (out != NULL) {\n+\t\tdir = opendir(out);\n+\t\tfree(out);\n+\t} else\n+\t\tdir = opendir(dirname);\n+\treturn dir;\n+}\n+\n+struct dirent *readdir_i18n(DIR * dirstream)\n+{\n+\tstruct dirent *de = readdir(dirstream);\n+\n+\tif (de) {\n+\t\tchar *out = filename_to_utf8(de->d_name);\n+\n+\t\tif (out) {\n+\t\t\tint len = strlen(out);\n+\t\t\tif (len >= NAME_MAX) {\n+\t\t\t\twarning(\"readdir_i18n: converted dir entry name length exceeds NAME_MAX and will be truncated\\n\");\n+\t\t\t\tlen = NAME_MAX - 1;\n+\t\t\t}\n+\t\t\tmemcpy(de->d_name, out, len);\n+\t\t\tde->d_name[len] = '\\0';\n+\t\t\tfree(out);\n+\t\t}\n+\t\treturn de;\n+\t} else\n+\t\treturn NULL;\n+}\n+\n+int open_i18n(const char *filename, int flags, mode_t mode)\n+{\n+\tint ret;\n+\tchar *out = filename_to_local(filename);\n+\n+\tif (out != NULL) {\n+\t\tret = open(out, flags, mode);\n+\t\tfree(out);\n+\t} else\n+\t\tret = open(filename, flags, mode);\n+\treturn ret;\n+}\n+\n+FILE *fopen_i18n(const char *filename, const char *opentype)\n+{\n+\tFILE *file;\n+\tchar *out = filename_to_local(filename);\n+\n+\tif (out != NULL) {\n+\t\tfile = fopen(out, opentype);\n+\t\tfree(out);\n+\t} else\n+\t\tfile = fopen(filename, opentype);\n+\treturn file;\n+}\n+\n+int chmod_i18n(const char *filename, mode_t mode)\n+{\n+\tint ret;\n+\tchar *out = filename_to_local(filename);\n+\n+\tif (out != NULL) {\n+\t\tret = chmod(out, mode);\n+\t\tfree(out);\n+\t} else\n+\t\tret = chmod(filename, mode);\n+\treturn ret;\n+}\n+\n+int link_i18n(const char *oldname, const char *newname)\n+{\n+\tchar *old_out = filename_to_local(oldname);\n+\tchar *new_out = filename_to_local(newname);\n+\tint ret = link(old_out ? old_out : oldname,\n+\t\t new_out ? new_out : newname);\n+\n+\tif (old_out)\n+\t\tfree(old_out);\n+\tif (new_out)\n+\t\tfree(new_out);\n+\treturn ret;\n+}\n+\n+int unlink_i18n(const char *filename)\n+{\n+\tchar *out = filename_to_local(filename);\n+\tint ret;\n+\n+\tif (out) {\n+\t\tret = unlink(out);\n+\t\tfree(out);\n+\t} else\n+\t\tret = unlink(filename);\n+\treturn ret;\n+}\ndiff --git a/io-i18n.h b/io-i18n.h\nnew file mode 100644\nindex 0000000..c386e20\n--- /dev/null\n+++ b/io-i18n.h\n@@ -0,0 +1,23 @@\n+#ifndef GIT_IO_I18N_H\n+#define GIT_IO_I18N_H\n+\n+#define _FILE_OFFSET_BITS 64\n+\n+#include \n+#include \n+#include \n+\n+#define DEFAULT_OPEN_MODE 0\n+\n+char *filename_to_local (const char* filename);\n+int stat_i18n(const char *filename, struct stat *buf);\n+int lstat_i18n(const char *filename, struct stat *buf);\n+DIR* opendir_i18n(const char *dirname);\n+struct dirent *readdir_i18n(DIR *dirstream);\n+int open_i18n(const char *filename, int flags, mode_t mode);\n+FILE *fopen_i18n(const char *filename, const char *opentype);\n+int chmod_i18n(const char *filename, mode_t mode);\n+int link_i18n(const char *oldname, const char *newname);\n+int unlink_i18n(const char *filename);\n+\n+#endif /* GIT_IO_I18N_H */\n-- \n1.6.5.1\n"},{"id":"126017","messageId":"1256651643-18382-4-git-send-email-timur@iris-comp.ru","threadId":"21379","inReplyTo":"1256651643-18382-3-git-send-email-timur@iris-comp.ru","subject":"[PATCH 4/4] Make mingw-compatibility layer to be aware of I18N-wrappers","fromName":"Timur Sufiev","fromEmail":"timur@iris-comp.ru","sentAt":"2009-10-27T13:54:03Z","receivedAt":"2009-10-27T13:54:03Z","isPatch":true,"sender":{"key":"timur@iris-comp.ru","avatar":null},"body":"Signed-off-by: Timur Sufiev \n---\n compat/fopen.c | 5 +++--\n compat/mingw.c | 24 ++++++++++++++++++------\n compat/mingw.h | 12 ++++++++++++\n compat/mkstemps.c | 3 ++-\n compat/regex/regex.c | 9 ++++++++-\n compat/regex/regex.h | 3 +++\n compat/win32.h | 13 +++++++++++--\n io-i18n.h | 8 ++++++++\n 8 files changed, 65 insertions(+), 12 deletions(-)\n\ndiff --git a/compat/fopen.c b/compat/fopen.c\nindex b5ca142..9136a14 100644\n--- a/compat/fopen.c\n+++ b/compat/fopen.c\n@@ -10,6 +10,7 @@\n */\n #undef FREAD_READS_DIRECTORIES\n #include \"../git-compat-util.h\"\n+#include \"io-i18n.h\"\n \n FILE *git_fopen(const char *path, const char *mode)\n {\n@@ -17,9 +18,9 @@ FILE *git_fopen(const char *path, const char *mode)\n \tstruct stat st;\n \n \tif (mode[0] == 'w' || mode[0] == 'a')\n-\t\treturn fopen(path, mode);\n+\t\treturn fopen_i18n(path, mode);\n \n-\tif (!(fp = fopen(path, mode)))\n+\tif (!(fp = fopen_i18n(path, mode)))\n \t\treturn NULL;\n \n \tif (fstat(fileno(fp), &st)) {\ndiff --git a/compat/mingw.c b/compat/mingw.c\nindex 6b5b5b2..9cb135a 100644\n--- a/compat/mingw.c\n+++ b/compat/mingw.c\n@@ -253,7 +253,7 @@ int mingw_utime (const char *file_name, const struct utimbuf *times)\n \tint fh, rc;\n \n \t/* must have write permission */\n-\tif ((fh = open(file_name, O_RDWR | O_BINARY)) < 0)\n+\tif ((fh = open_i18n(file_name, O_RDWR | O_BINARY, DEFAULT_OPEN_MODE)) < 0)\n \t\treturn -1;\n \n \ttime_t_to_filetime(times->modtime, &mft);\n@@ -278,7 +278,7 @@ int mkstemp(char *template)\n \tchar *filename = mktemp(template);\n \tif (filename == NULL)\n \t\treturn -1;\n-\treturn open(filename, O_RDWR | O_CREAT, 0600);\n+\treturn open_i18n(filename, O_RDWR | O_CREAT, 0600);\n }\n \n int gettimeofday(struct timeval *tv, void *tz)\n@@ -519,7 +519,7 @@ static const char *parse_interpreter(const char *cmd)\n \tif (n >= 4 && !strcasecmp(cmd+n-4, \".exe\"))\n \t\treturn NULL;\n \n-\tfd = open(cmd, O_RDONLY);\n+\tfd = open_i18n(cmd, O_RDONLY, DEFAULT_OPEN_MODE);\n \tif (fd < 0)\n \t\treturn NULL;\n \tn = read(fd, buf, sizeof(buf)-1);\n@@ -1135,10 +1135,14 @@ sig_handler_t mingw_signal(int sig, sig_handler_t handler)\n static const char *make_backslash_path(const char *path)\n {\n \tstatic char buf[PATH_MAX + 1];\n+\tchar *out = filename_to_local(path);\n \tchar *c;\n \n-\tif (strlcpy(buf, path, PATH_MAX) >= PATH_MAX)\n-\t\tdie(\"Too long path: %.*s\", 60, path);\n+\tif ( strlcpy(buf, out ? out : path, PATH_MAX) >= PATH_MAX) {\n+\t die(\"Too long path: %.*s\", 60, out ? out : path);\n+\t free(out);\n+\t}\n+\tfree(out);\n \n \tfor (c = buf; *c; c++) {\n \t\tif (*c == '/')\n@@ -1158,6 +1162,9 @@ int link(const char *oldpath, const char *newpath)\n {\n \ttypedef BOOL (WINAPI *T)(const char*, const char*, LPSECURITY_ATTRIBUTES);\n \tstatic T create_hard_link = NULL;\n+\tchar *old_out = filename_to_local(oldpath);\n+\tchar *new_out = filename_to_local(newpath);\n+\n \tif (!create_hard_link) {\n \t\tcreate_hard_link = (T) GetProcAddress(\n \t\t\tGetModuleHandle(\"kernel32.dll\"), \"CreateHardLinkA\");\n@@ -1168,10 +1175,15 @@ int link(const char *oldpath, const char *newpath)\n \t\terrno = ENOSYS;\n \t\treturn -1;\n \t}\n-\tif (!create_hard_link(newpath, oldpath, NULL)) {\n+\tif (!create_hard_link(new_out ? new_out : newpath,\n+\t\t\t old_out ? old_out : oldpath, NULL)) {\n+\t\tfree(new_out);\n+\t\tfree(old_out);\n \t\terrno = err_win_to_posix(GetLastError());\n \t\treturn -1;\n \t}\n+\tfree(new_out);\n+\tfree(old_out);\n \treturn 0;\n }\n \ndiff --git a/compat/mingw.h b/compat/mingw.h\nindex 5b5258b..0d1dc83 100644\n--- a/compat/mingw.h\n+++ b/compat/mingw.h\n@@ -1,5 +1,16 @@\n+#ifndef GIT_MINGW_H\n+#define GIT_MINGW_H\n #include \n \n+/* 3 following lines are taken from io-i18n.h because we cannot\n+ include it here: if we do so, readdir_i18n prototype will\n+ be inconsistent with with other code which properly uses all\n+ mingw defines\n+*/\n+char *filename_to_local (const char* filename);\n+int open_i18n(const char *filename, int flags, mode_t mode);\n+#define DEFAULT_OPEN_MODE 0\n+\n /*\n * things that are not available in header files\n */\n@@ -273,3 +284,4 @@ struct mingw_dirent\n #define readdir(x) mingw_readdir(x)\n struct dirent *mingw_readdir(DIR *dir);\n #endif // !NO_MINGW_REPLACE_READDIR\n+#endif // GIT_MINGW_H\ndiff --git a/compat/mkstemps.c b/compat/mkstemps.c\nindex 14179c8..0cdd42d 100644\n--- a/compat/mkstemps.c\n+++ b/compat/mkstemps.c\n@@ -1,4 +1,5 @@\n #include \"../git-compat-util.h\"\n+#include \"../io-i18n.h\"\n \n /* Adapted from libiberty's mkstemp.c. */\n \n@@ -47,7 +48,7 @@ int gitmkstemps(char *pattern, int suffix_len)\n \t\ttemplate[4] = letters[v % num_letters]; v /= num_letters;\n \t\ttemplate[5] = letters[v % num_letters]; v /= num_letters;\n \n-\t\tfd = open(pattern, O_CREAT | O_EXCL | O_RDWR, 0600);\n+\t\tfd = open_i18n(pattern, O_CREAT | O_EXCL | O_RDWR, 0600);\n \t\tif (fd > 0)\n \t\t\treturn fd;\n \t\t/*\ndiff --git a/compat/regex/regex.c b/compat/regex/regex.c\nindex 67d5c37..2deec55 100644\n--- a/compat/regex/regex.c\n+++ b/compat/regex/regex.c\n@@ -450,8 +450,15 @@ static int debug = 0;\n #define DEBUG_PRINT_DOUBLE_STRING(w, s1, sz1, s2, sz2)\t\t\t\\\n if (debug) print_double_string (w, s1, sz1, s2, sz2)\n \n-\n+#ifdef __USE_MINGW_ACCESS\n+void\n+ printchar(char c)\n+{\n+ putc(c, stderr);\n+}\n+#else\n extern void printchar ();\n+#endif // __USE_MINGW_ACCESS\n \n /* Print the fastmap in human-readable form. */\n \ndiff --git a/compat/regex/regex.h b/compat/regex/regex.h\nindex 6eb64f1..a5f2809 100644\n--- a/compat/regex/regex.h\n+++ b/compat/regex/regex.h\n@@ -29,6 +29,9 @@\n #include \n #endif\n \n+#ifdef __USE_MINGW_ACCESS\n+#include \n+#endif\n \n /* The following bits are used to determine the regexp syntax we\n recognize. The set/not-set meanings are chosen so that Emacs syntax\ndiff --git a/compat/win32.h b/compat/win32.h\nindex 8ce9104..f71e36d 100644\n--- a/compat/win32.h\n+++ b/compat/win32.h\n@@ -20,8 +20,17 @@ static inline int file_attr_to_st_mode (DWORD attr)\n \n static inline int get_file_attr(const char *fname, WIN32_FILE_ATTRIBUTE_DATA *fdata)\n {\n-\tif (GetFileAttributesExA(fname, GetFileExInfoStandard, fdata))\n-\t\treturn 0;\n+\tchar *out = filename_to_local(fname);\n+\tint ret;\n+\n+\tif ( out != NULL ) {\n+\t ret = GetFileAttributesExA(out, GetFileExInfoStandard, fdata);\n+\t free(out);\n+\t} else\n+\t ret = GetFileAttributesExA(fname, GetFileExInfoStandard, fdata);\n+\n+\tif ( ret )\n+\t return 0;\n \n \tswitch (GetLastError()) {\n \tcase ERROR_ACCESS_DENIED:\ndiff --git a/io-i18n.h b/io-i18n.h\nindex c386e20..7329105 100644\n--- a/io-i18n.h\n+++ b/io-i18n.h\n@@ -6,6 +6,14 @@\n #include \n #include \n #include \n+#include \n+\n+#ifdef __USE_MINGW_ACCESS\n+#include \n+#define NAME_MAX MAX_PATH\n+#define stat _stati64\n+#define dirent mingw_dirent\n+#endif\n \n #define DEFAULT_OPEN_MODE 0\n \n-- \n1.6.5.1\n"},{"id":"126019","messageId":"alpine.DEB.2.00.0910271515270.17168@ds9.cixit.se","threadId":"21379","inReplyTo":"1256651643-18382-1-git-send-email-timur@iris-comp.ru","subject":"Re: [PATCH 1/4] Add routines for filenames encoding <-> UTF-8","fromName":"Peter Krefting","fromEmail":"peter@softwolves.pp.se","sentAt":"2009-10-27T14:16:59Z","receivedAt":"2009-10-27T14:16:59Z","isPatch":true,"sender":{"key":"peter@softwolves.pp.se","avatar":"https://avatars.githubusercontent.com/u/990764?v=4"},"body":"Timur Sufiev:\n\n> The point is to make Git aware of filenames local encoding and make it \n> keep all filenames in UTF-8 internally.\n\nGood.\n\n> If `i18n.filenameslocalencoding' option was set via git-config to a \n> correct encoding, 2 things should be done:\n\nWindows supports UTF-16 file names, but need to use wchar_t APIs for fopen() \nand friends. Have you looked at any of that?\n\n-- \n\\\\// Peter - http://www.softwolves.pp.se/\n"},{"id":"126059","messageId":"20091027210817.GA1577@sigill.intra.peff.net","threadId":"21379","inReplyTo":"1256651643-18382-2-git-send-email-timur@iris-comp.ru","subject":"Re: [PATCH 2/4] Add I18N-wrappers for low-level IO-routines","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2009-10-27T21:08:17Z","receivedAt":"2009-10-27T21:08:17Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Tue, Oct 27, 2009 at 04:54:01PM +0300, Timur Sufiev wrote:\n\n> Signed-off-by: Timur Sufiev \n\nHmm. Two questions about this series:\n\n 1. Patch 3/4 didn't seem to make it to the list. Presumably that is\n where you actually use these routines in git? Or are they just for\n mingw?\n\n 2. I seem to recall that Linus added a filename translation layer for\n doing much more, like handling unicode normalizations (but I\n confess I haven't looked closely at that code). Should this be part\n of that system?\n\n-Peff\n"},{"id":"126092","messageId":"4ae7ef9d.0f1abc0a.5b53.1a73@mx.google.com","threadId":"21379","inReplyTo":"alpine.DEB.2.00.0910271515270.17168@ds9.cixit.se","subject":"Re: [PATCH 1/4] Add routines for filenames encoding <-> UTF-8","fromName":"Timur Sufiev","fromEmail":"tsufiev@gmail.com","sentAt":"2009-10-28T07:15:29Z","receivedAt":"2009-10-28T07:15:29Z","isPatch":true,"sender":{"key":"tsufiev@gmail.com","avatar":"https://gravatar.com/avatar/e4a068ddcf68fc9b207d8be5cd98db47642d7c3d465191cdab7237538371144d?d=mp&s=160"},"body":">> If `i18n.filenameslocalencoding' option was set via git-config to a \n>> correct encoding, 2 things should be done:\n> \n> Windows supports UTF-16 file names, but need to use wchar_t APIs for fopen() \n> and friends. Have you looked at any of that?\n\nNo, I didn't look towards Windows handling of Unicode, because Git under\nWindows works well enough without that: I've built Git in MinGW/MSYS\nenvironmment (using the patches we're discussing) and it handles\ncyrillic filenames in Windows ANSI CP1251 codepage just as was planned:\nwrites filenames to tree objects in UTF-8, checks out them into working\ndir in CP1251, other stuff as git-clone, git-diff, git-status, etc also\nworks ok. One remaining issue (offtopic here, as far as I understand) to\nmake TortoiseGit work with UTF-8 Git's output.\n \n> -- \n> \\\\// Peter - http://www.softwolves.pp.se/\n\n-- \nTimur Sufiev\n"},{"id":"126164","messageId":"4ae886f4.0b38560a.6cfb.5ac4@mx.google.com","threadId":"21379","inReplyTo":"20091027210817.GA1577@sigill.intra.peff.net","subject":"Re: [PATCH 2/4] Add I18N-wrappers for low-level IO-routines","fromName":"Timur Sufiev","fromEmail":"tsufiev@gmail.com","sentAt":"2009-10-28T18:01:21Z","receivedAt":"2009-10-28T18:01:21Z","isPatch":true,"sender":{"key":"tsufiev@gmail.com","avatar":"https://gravatar.com/avatar/e4a068ddcf68fc9b207d8be5cd98db47642d7c3d465191cdab7237538371144d?d=mp&s=160"},"body":"> Hmm. Two questions about this series:\n> \n> 1. Patch 3/4 didn't seem to make it to the list. Presumably that is\n> where you actually use these routines in git? Or are they just for\n> mingw?\n\nYes, it actually haven't made it to the list. Perhaps this was due to\npatch size: it was approx. 3300 lines long (BTW, what's the message size\nlimit?) So I've rewritten the patch to make it more compact, using mingw\napproach with macros. Subj prefix for a patch series is 'PATCH I18N\nfilenames v2'. \n\n> 2. I seem to recall that Linus added a filename translation layer for\n> doing much more, like handling unicode normalizations (but I\n> confess I haven't looked closely at that code). Should this be part\n> of that system?\n\nI've heard nothing about that :(. Could you point me directly at Linus'\nchanges?\n\n> -Peff\n\n-- \nTimur Sufiev\n"},{"id":"126169","messageId":"20091028181033.GA17225@sigill.intra.peff.net","threadId":"21379","inReplyTo":"4ae886f4.0b38560a.6cfb.5ac4@mx.google.com","subject":"Re: [PATCH 2/4] Add I18N-wrappers for low-level IO-routines","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2009-10-28T18:10:34Z","receivedAt":"2009-10-28T18:10:34Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Wed, Oct 28, 2009 at 09:01:21PM +0300, Timur Sufiev wrote:\n\n> Yes, it actually haven't made it to the list. Perhaps this was due to\n> patch size: it was approx. 3300 lines long (BTW, what's the message size\n> limit?) So I've rewritten the patch to make it more compact, using mingw\n> approach with macros. Subj prefix for a patch series is 'PATCH I18N\n> filenames v2'. \n\nThanks. The rules for vger are here:\n\n http://vger.kernel.org/majordomo-info.html\n\nThe max size is 100K, but you may also be triggering something from the\ntaboo list accidentally.\n\n> > 2. I seem to recall that Linus added a filename translation layer for\n> > doing much more, like handling unicode normalizations (but I\n> > confess I haven't looked closely at that code). Should this be part\n> > of that system?\n> \n> I've heard nothing about that :(. Could you point me directly at Linus'\n> changes?\n\nTry looking at this series:\n\n http://thread.gmane.org/gmane.comp.version-control.git/119222\n\n-Peff\n"}]}