git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v2 2/3] compat/mingw: allow deletion of most opened files

From
Johannes Sixt <j6t@kdbg.org>
Date
Oct 27, 2024, 13:17 UTC
Message-ID
<a769d5a6-0d0f-4df4-b581-539d00aacbc9@kdbg.org>
In-Reply-To
<c308eafbbb5a7c853b8126750c4c44a8b8635192.1729770140.git.ps@pks.im>
Am 24.10.24 um 13:46 schrieb Patrick Steinhardt:
Show 24 quoted lines
> On Windows, we emulate open(3p) via `mingw_open()`. This function
> implements handling of some platform-specific quirks that are required
> to make it behave as closely as possible like open(3p) would, but for
> most cases we just call the Windows-specific `_wopen()` function.
> 
> This function has a major downside though: it does not allow us to
> specify the sharing mode. While there is `_wsopen()` that allows us to
> pass sharing flags, those sharing flags are not the same `FILE_SHARE_*`
> flags as `CreateFileW()` accepts. Instead, `_wsopen()` only allows
> concurrent read- and write-access, but does not allow for concurrent
> deletions. Unfortunately though, we have to allow concurrent deletions
> if we want to have POSIX-style atomic renames on top of an existing file
> that has open file handles.
> 
> Implement a new function that emulates open(3p) for existing files via
> `CreateFileW()` such that we can set the required sharing flags.
> 
> While we have the same issue when calling open(3p) with `O_CREAT`,
> implementing that mode would be more complex due to the required
> permission handling. Furthermore, atomic updates via renames typically
> write to exclusive lockfile and then perform the rename, and thus we
> don't have to handle the case where the locked path has been created
> with `O_CREATE`. So while it would be nice to have proper POSIX
> semantics in all paths, we instead aim for a minimum viable fix here.
Agreed!
Show 38 quoted lines
> 
> Signed-off-by: Patrick Steinhardt <ps@pks.im>
> ---
>  compat/mingw.c | 58 ++++++++++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 58 insertions(+)
> 
> diff --git a/compat/mingw.c b/compat/mingw.c
> index e326c6fcd2d..6c75fe36b15 100644
> --- a/compat/mingw.c
> +++ b/compat/mingw.c
> @@ -532,6 +532,62 @@ static int mingw_open_append(wchar_t const *wfilename, int oflags, ...)
>  	return fd;
>  }
>  
> +/*
> + * Ideally, we'd use `_wopen()` to implement this functionality so that we
> + * don't have to reimplement it, but unfortunately we do not have tight control
> + * over the share mode there. And while `_wsopen()` and friends exist that give
> + * us _some_ control over the share mode, this family of functions doesn't give
> + * us the ability to enable FILE_SHARE_DELETE, either. But this is a strict
> + * requirement for us though so that we can unlink or rename over files that
> + * are held open by another process.
> + *
> + * We are thus forced to implement our own emulation of `open()`. To make our
> + * life simpler we only implement basic support for this, namely opening
> + * existing files for reading and/or writing. This means that newly created
> + * files won't have their sharing mode set up correctly, but for now I couldn't
> + * find any case where this matters. We may have to revisit that in the future
> + * though based on our needs.
> + */
> +static int mingw_open_existing(const wchar_t *filename, int oflags, ...)
> +{
> +	SECURITY_ATTRIBUTES security_attributes = {
> +		.nLength = sizeof(security_attributes),
> +		.bInheritHandle = !(oflags & O_NOINHERIT),
> +	};
> +	HANDLE handle;
> +	int access;

I would have made this variable DWORD or unsigned instead of plain int, because it receives a bit pattern and would match the parameter type of CreateFileW() better; but no big deal.

Show 14 quoted lines
> +	int fd;
> +
> +	/* We only support basic flags. */
> +	if (oflags & ~(O_ACCMODE | O_NOINHERIT)) {
> +		errno = ENOSYS;
> +		return -1;
> +	}
> +
> +	if (oflags & O_RDWR)
> +		access = GENERIC_READ | GENERIC_WRITE;
> +	else if (oflags & O_WRONLY)
> +		access = GENERIC_WRITE;
> +	else
> +		access = GENERIC_READ;

O_RDWR, O_WRONLY and O_RDONLY are not flags, but values occupying two bits of oflags. This must be:

	if ((oflags & O_ACCMODE) == O_RDWR)
		access = GENERIC_READ | GENERIC_WRITE;
	else if ((oflags & O_ACCMODE) == O_WRONLY)
		access = GENERIC_WRITE;
	else
		access = GENERIC_READ;
or similar.
Show 8 quoted lines
> +
> +	handle = CreateFileW(filename, access,
> +			     FILE_SHARE_WRITE | FILE_SHARE_READ | FILE_SHARE_DELETE,
> +			     &security_attributes, OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL, NULL);
> +	if (handle == INVALID_HANDLE_VALUE) {
> +		DWORD err = GetLastError();
> +		if (err == ERROR_INVALID_PARAMETER)
> +			err = ERROR_PATH_NOT_FOUND;

First I wondered what this is about, but then noticed that it is just copied from the mingw_open_append() implementation catering to some bogus network filesystems. Good.

Show 22 quoted lines
> +		errno = err_win_to_posix(err);
> +		return -1;
> +	}
> +
> +	fd = _open_osfhandle((intptr_t)handle, oflags | O_BINARY);
> +	if (fd < 0)
> +		CloseHandle(handle);
> +	return fd;
> +}
> +
>  /*
>   * Does the pathname map to the local named pipe filesystem?
>   * That is, does it have a "//./pipe/" prefix?
> @@ -567,6 +623,8 @@ int mingw_open (const char *filename, int oflags, ...)
>  
>  	if ((oflags & O_APPEND) && !is_local_named_pipe_path(filename))
>  		open_fn = mingw_open_append;
> +	else if (!(oflags & ~(O_ACCMODE | O_NOINHERIT)))
> +		open_fn = mingw_open_existing;
>  	else
>  		open_fn = _wopen;
>  
Makes sense!
-- Hannes
Previous: Patrick SteinhardtNext: Patrick Steinhardt
Message 25 of 44 in “compat/mingw: implement POSIX-style atomic renames”
  1. 0/3 compat/mingw: implement POSIX-style atomic renamesPatrick Steinhardt, Oct 23, 2024
  2. 1/3 compat/mingw: share file handles created via `CreateFileW()`Patrick Steinhardt, Oct 23, 2024
  3. Kristoffer HaugsbakkOct 23, 2024
  4. Taylor BlauOct 23, 2024
  5. Taylor BlauOct 23, 2024
  6. Taylor BlauOct 23, 2024
  7. Patrick SteinhardtOct 24, 2024
  8. Johannes SixtOct 27, 2024
  9. Taylor BlauOct 27, 2024
  10. 2/3 compat/mingw: allow deletion of most opened filesPatrick Steinhardt, Oct 23, 2024
  11. Kristoffer HaugsbakkOct 23, 2024
  12. Taylor BlauOct 23, 2024
  13. Patrick SteinhardtOct 24, 2024
  14. Taylor BlauOct 23, 2024
  15. 3/3 compat/mingw: support POSIX semantics for atomic renamesPatrick Steinhardt, Oct 23, 2024
  16. Kristoffer HaugsbakkOct 23, 2024
  17. Patrick SteinhardtOct 24, 2024
  18. Kristoffer HaugsbakkOct 24, 2024
  19. Patrick SteinhardtOct 24, 2024
  20. Taylor BlauOct 23, 2024
  21. Taylor BlauOct 23, 2024
  22. 0/3 compat/mingw: implement POSIX-style atomic renamesPatrick Steinhardt, Oct 24, 2024
  23. 1/3 compat/mingw: share file handles created via `CreateFileW()`Patrick Steinhardt, Oct 24, 2024
  24. 2/3 compat/mingw: allow deletion of most opened filesPatrick Steinhardt, Oct 24, 2024
  25. Johannes SixtOct 27, 2024
  26. Patrick SteinhardtOct 27, 2024
  27. Taylor BlauOct 27, 2024
  28. Taylor BlauOct 27, 2024
  29. 3/3 compat/mingw: support POSIX semantics for atomic renamesPatrick Steinhardt, Oct 24, 2024
  30. Johannes SixtOct 27, 2024
  31. Patrick SteinhardtOct 27, 2024
  32. Johannes SixtOct 27, 2024
  33. Patrick SteinhardtOct 27, 2024
  34. Johannes SixtOct 27, 2024
  35. Taylor BlauOct 27, 2024
  36. Taylor BlauOct 24, 2024
  37. Johannes SixtOct 27, 2024
  38. 0/3 compat/mingw: implement POSIX-style atomic renamesPatrick Steinhardt, Oct 27, 2024
  39. 1/3 compat/mingw: share file handles created via `CreateFileW()`Patrick Steinhardt, Oct 27, 2024
  40. 2/3 compat/mingw: allow deletion of most opened filesPatrick Steinhardt, Oct 27, 2024
  41. 3/3 compat/mingw: support POSIX semantics for atomic renamesPatrick Steinhardt, Oct 27, 2024
  42. Junio C HamanoNov 6, 2024
  43. Johannes SixtNov 6, 2024
  44. Junio C HamanoNov 6, 2024

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.