git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 2/3] Win32: Unicode file name support (dirent)

From
Stepan Kasal <kasal@ucw.cz>
Date
Jul 15, 2014, 13:43 UTC
Message-ID
<1405431797-20899-3-git-send-email-kasal@ucw.cz>
In-Reply-To
<1405431797-20899-1-git-send-email-kasal@ucw.cz>
From: Karsten Blees <blees@dcon.de>
Date: Sat, 14 Jan 2012 22:01:09 +0100

Changes opendir/readdir to use Windows Unicode APIs and convert between UTF-8/UTF-16.

Removes parameter checks that are already covered by xutftowcs_path. This changes detection of ENAMETOOLONG from MAX_PATH - 2 to MAX_PATH (matching is_dir_empty in mingw.c). If name + "/*" or the resulting absolute path is too long, FindFirstFile fails and errno is set through err_win_to_posix.

Increases the size of dirent.d_name to accommodate the full WIN32_FIND_DATA.cFileName converted to UTF-8 (UTF-16 to UTF-8 conversion may grow by factor three in the worst case).

Signed-off-by: Karsten Blees <blees@dcon.de>
Signed-off-by: Stepan Kasal <kasal@ucw.cz>
---
 compat/win32/dirent.c | 30 ++++++++++--------------------
 compat/win32/dirent.h |  2 +-
 2 files changed, 11 insertions(+), 21 deletions(-)
diff --git a/compat/win32/dirent.c b/compat/win32/dirent.c
index 82a515c..52420ec 100644
--- a/compat/win32/dirent.c
+++ b/compat/win32/dirent.c
@@ -6,10 +6,10 @@ struct DIR {
 	int dd_stat;          /* 0-based index */
 };
 
-static inline void finddata2dirent(struct dirent *ent, WIN32_FIND_DATAA *fdata)
+static inline void finddata2dirent(struct dirent *ent, WIN32_FIND_DATAW *fdata)
 {
-	/* copy file name from WIN32_FIND_DATA to dirent */
-	memcpy(ent->d_name, fdata->cFileName, sizeof(ent->d_name));
+	/* convert UTF-16 name to UTF-8 */
+	xwcstoutf(ent->d_name, fdata->cFileName, sizeof(ent->d_name));
 
 	/* Set file type, based on WIN32_FIND_DATA */
 	if (fdata->dwFileAttributes & FILE_ATTRIBUTE_DIRECTORY)
@@ -20,25 +20,15 @@ static inline void finddata2dirent(struct dirent *ent, WIN32_FIND_DATAA *fdata)
 
 DIR *opendir(const char *name)
 {
-	char pattern[MAX_PATH];
-	WIN32_FIND_DATAA fdata;
+	wchar_t pattern[MAX_PATH + 2]; /* + 2 for '/' '*' */
+	WIN32_FIND_DATAW fdata;
 	HANDLE h;
 	int len;
 	DIR *dir;
 
-	/* check that name is not NULL */
-	if (!name) {
-		errno = EINVAL;
+	/* convert name to UTF-16 and check length < MAX_PATH */
+	if ((len = xutftowcs_path(pattern, name)) < 0)
 		return NULL;
-	}
-	/* check that the pattern won't be too long for FindFirstFileA */
-	len = strlen(name);
-	if (len + 2 >= MAX_PATH) {
-		errno = ENAMETOOLONG;
-		return NULL;
-	}
-	/* copy name to temp buffer */
-	memcpy(pattern, name, len + 1);
 
 	/* append optional '/' and wildcard '*' */
 	if (len && !is_dir_sep(pattern[len - 1]))
@@ -47,7 +37,7 @@ DIR *opendir(const char *name)
 	pattern[len] = 0;
 
 	/* open find handle */
-	h = FindFirstFileA(pattern, &fdata);
+	h = FindFirstFileW(pattern, &fdata);
 	if (h == INVALID_HANDLE_VALUE) {
 		DWORD err = GetLastError();
 		errno = (err == ERROR_DIRECTORY) ? ENOTDIR : err_win_to_posix(err);
@@ -72,8 +62,8 @@ struct dirent *readdir(DIR *dir)
 	/* if first entry, dirent has already been set up by opendir */
 	if (dir->dd_stat) {
 		/* get next entry and convert from WIN32_FIND_DATA to dirent */
-		WIN32_FIND_DATAA fdata;
-		if (FindNextFileA(dir->dd_handle, &fdata)) {
+		WIN32_FIND_DATAW fdata;
+		if (FindNextFileW(dir->dd_handle, &fdata)) {
 			finddata2dirent(&dir->dd_dir, &fdata);
 		} else {
 			DWORD lasterr = GetLastError();
diff --git a/compat/win32/dirent.h b/compat/win32/dirent.h
index 8838cd6..058207e 100644
--- a/compat/win32/dirent.h
+++ b/compat/win32/dirent.h
@@ -10,7 +10,7 @@ typedef struct DIR DIR;
 
 struct dirent {
 	unsigned char d_type;      /* file type to prevent lstat after readdir */
-	char d_name[MAX_PATH];     /* file name */
+	char d_name[MAX_PATH * 3]; /* file name (* 3 for UTF-8 conversion) */
 };
 
 DIR *opendir(const char *dirname);

-- 
2.0.0.9635.g0be03cb

-- 
-- 
*** Please reply-to-all at all times ***
*** (do not pretend to know who is subscribed and who is not) ***
*** Please avoid top-posting. ***
The msysGit Wiki is here: https://github.com/msysgit/msysgit/wiki - Github accounts are free.

You received this message because you are subscribed to the Google
Groups "msysGit" group.
To post to this group, send email to msysgit@googlegroups.com
To unsubscribe from this group, send email to
msysgit+unsubscribe@googlegroups.com
For more options, and view previous threads, visit this group at
http://groups.google.com/group/msysgit?hl=en_US?hl=en

--- 
You received this message because you are subscribed to the Google Groups "msysGit" group.
To unsubscribe from this group and stop receiving emails from it, send an email to msysgit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Previous: Stepan KasalNext: Stepan Kasal
Message 7 of 46 in “Topic sk/mingw-unicode-spawn-args breaks tests”
  1. Johannes SixtJul 10, 2014
  2. Karsten BleesJul 11, 2014
  3. Stepan KasalJul 12, 2014
  4. Johannes SchindelinJul 15, 2014
  5. 0/3 fix test suite with mingw-unicode patchesStepan Kasal, Jul 15, 2014
  6. 1/3 Win32: Unicode file name support (except dirent)Stepan Kasal, Jul 15, 2014
  7. 2/3 Win32: Unicode file name support (dirent)Stepan Kasal, Jul 15, 2014
  8. 3/3 tests: do not pass iso8859-1 encoded parameterStepan Kasal, Jul 15, 2014
  9. Stepan KasalJul 15, 2014
  10. Junio C HamanoJul 15, 2014
  11. Karsten BleesJul 15, 2014
  12. 1/2 MinGW: fix compile error due to missing ELOOPKarsten Blees, Jul 15, 2014
  13. Jonathan NiederJul 15, 2014
  14. Karsten BleesJul 15, 2014
  15. Junio C HamanoJul 16, 2014
  16. 2/2 config: use chmod() instead of fchmod()Karsten Blees, Jul 15, 2014
  17. Johannes SixtJul 16, 2014
  18. Karsten BleesJul 16, 2014
  19. Junio C HamanoJul 16, 2014
  20. Karsten BleesJul 17, 2014
  21. Stepan KasalJul 16, 2014
  22. Thomas BraunJul 16, 2014
  23. 0/6 mingw test fixesStepan Kasal, Jul 17, 2014
  24. 1/6 MinGW: Skip test redirecting to fd 4Stepan Kasal, Jul 17, 2014
  25. Junio C HamanoJul 17, 2014
  26. Johannes SchindelinJul 17, 2014
  27. Karsten BleesJul 17, 2014
  28. Junio C HamanoJul 17, 2014
  29. 2/6 Disable t0110's high-bit test on WindowsStepan Kasal, Jul 17, 2014
  30. Karsten BleesJul 17, 2014
  31. Junio C HamanoJul 18, 2014
  32. t0110/MinGW: skip tests that pass arbitrary bytes on the command lineKarsten Blees, Jul 19, 2014
  33. Junio C HamanoJul 21, 2014
  34. 3/6 MinGW: disable legacy encoding testsStepan Kasal, Jul 17, 2014
  35. 4/6 t4210: skip command-line encoding tests on mingwStepan Kasal, Jul 17, 2014
  36. Erik Faye-LundJul 18, 2014
  37. Junio C HamanoJul 21, 2014
  38. Johannes SchindelinJul 21, 2014
  39. Johannes SixtJul 21, 2014
  40. Junio C HamanoJul 21, 2014
  41. Junio C HamanoJul 21, 2014
  42. Junio C HamanoJul 21, 2014
  43. 2/2 test prerequisites: enumerate with commasJunio C Hamano, Jul 21, 2014
  44. 5/6 t9902: mingw-specific fix for gitfile link filesStepan Kasal, Jul 17, 2014
  45. Junio C HamanoJul 17, 2014
  46. 6/6 t800[12]: work around MSys limitationStepan Kasal, Jul 17, 2014

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.