git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v2 3/3] compat/mingw: support POSIX semantics for atomic renames

From
Patrick Steinhardt <ps@pks.im>
Date
Oct 27, 2024, 15:38 UTC
Message-ID
<Zx5eg2SkHMmNhQRh@pks.im>
In-Reply-To
<542b306a-523d-424b-bc9f-c63bb7040beb@kdbg.org>
On Sun, Oct 27, 2024 at 02:23:28PM +0100, Johannes Sixt wrote:
Show 35 quoted lines
> Am 24.10.24 um 13:46 schrieb Patrick Steinhardt:
> > By default, Windows restricts access to files when those files have been
> > opened by another process. As explained in the preceding commits, these
> > restrictions can be loosened such that reads, writes and/or deletes of
> > files with open handles _are_ allowed.
> > 
> > While we set up those sharing flags in most relevant code paths now, we
> > still don't properly handle POSIX-style atomic renames in case the
> > target path is open. This is failure demonstrated by t0610, where one of
> > our tests spawns concurrent writes in a reftable-enabled repository and
> > expects all of them to succeed. This test fails most of the time because
> > the process that has acquired the "tables.list" lock is unable to rename
> > it into place while other processes are busy reading that file.
> > 
> > Windows 10 has introduced the `FILE_RENAME_FLAG_POSIX_SEMANTICS` flag
> > that allows us to fix this usecase [1]. When set, it is possible to
> > rename a file over a preexisting file even when the target file still
> > has handles open. Those handles must have been opened with the
> > `FILE_SHARE_DELETE` flag, which we have ensured in the preceding
> > commits.
> > > Careful readers might have noticed that [1] does not mention the above
> > flag, but instead mentions `FILE_RENAME_POSIX_SEMANTICS`. This flag is
> > not for use with `SetFileInformationByHandle()` though, which is what we
> > use. And while the `FILE_RENAME_FLAG_POSIX_SEMANTICS` flag exists, it is
> > not documented on [2] or anywhere else as far as I can tell.
> 
> The Windows 10 SDK defines FILE_RENAME_FLAG_REPLACE_IF_EXISTS and
> FILE_RENAME_FLAG_POSIX_SEMANTICS for SetFileInformationByHandle(). That
> the documentation lacks "_FLAG_" in the names must be an error in the
> documentation.
> 
> I found the mention of FILE_RENAME_POSIX_SEMANTICS quite distracting,
> because it is a flag to be used with CreateFileW() and basically only
> has to do with case-sensitivity, but nothing with POSIX semantics of
> renaming.

I'd still prefer to mention this, because otherwise an astute reader might notice that I'm using a different flag name than what is documented in the docs and figure out that I defined the wrong flag name.

[snip]
Show 17 quoted lines
> > +		HANDLE old_handle = INVALID_HANDLE_VALUE;
> > +		BOOL success;
> > +
> > +		old_handle = CreateFileW(wpold, DELETE,
> > +					 FILE_SHARE_WRITE | FILE_SHARE_READ | FILE_SHARE_DELETE,
> > +					 NULL, OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL, NULL);
> > +		if (old_handle == INVALID_HANDLE_VALUE) {
> > +			errno = err_win_to_posix(GetLastError());
> > +			return -1;
> > +		}
> > +
> > +		rename_info.Flags = FILE_RENAME_FLAG_REPLACE_IF_EXISTS |
> > +				    FILE_RENAME_FLAG_POSIX_SEMANTICS;
> > +		rename_info.FileNameLength = wpnew_len * sizeof(WCHAR);
> 
> Size is in bytes, not in characters, and without the NUL. Good. I read
> one comment on SO, which said that this value is ignored...

Yeah, I noticed at one point that it didn't really make a difference what I pass here.

Show 6 quoted lines
> > +		memcpy(rename_info.FileName, wpnew, wpnew_len * sizeof(WCHAR));
> 
> ... which makes it all the more important that this path is
> NUL-terminated. Yet, this does not copy the NUL. We are still good,
> because the buffer is zero-initialized and xutftowcs_path() ensures that
> wpnew_len is at most MAX_PATH-1.
Yup.
[snip]
> The general structure of the patch makes a lot of sense!

Great, thanks for your review! I'll send a revised version of this series where I adapt the second patch.

Patrick
Previous: Johannes SixtNext: Johannes Sixt
Message 31 of 44 in “compat/mingw: implement POSIX-style atomic renames”
  1. 0/3 compat/mingw: implement POSIX-style atomic renamesPatrick Steinhardt, Oct 23, 2024
  2. 1/3 compat/mingw: share file handles created via `CreateFileW()`Patrick Steinhardt, Oct 23, 2024
  3. Kristoffer HaugsbakkOct 23, 2024
  4. Taylor BlauOct 23, 2024
  5. Taylor BlauOct 23, 2024
  6. Taylor BlauOct 23, 2024
  7. Patrick SteinhardtOct 24, 2024
  8. Johannes SixtOct 27, 2024
  9. Taylor BlauOct 27, 2024
  10. 2/3 compat/mingw: allow deletion of most opened filesPatrick Steinhardt, Oct 23, 2024
  11. Kristoffer HaugsbakkOct 23, 2024
  12. Taylor BlauOct 23, 2024
  13. Patrick SteinhardtOct 24, 2024
  14. Taylor BlauOct 23, 2024
  15. 3/3 compat/mingw: support POSIX semantics for atomic renamesPatrick Steinhardt, Oct 23, 2024
  16. Kristoffer HaugsbakkOct 23, 2024
  17. Patrick SteinhardtOct 24, 2024
  18. Kristoffer HaugsbakkOct 24, 2024
  19. Patrick SteinhardtOct 24, 2024
  20. Taylor BlauOct 23, 2024
  21. Taylor BlauOct 23, 2024
  22. 0/3 compat/mingw: implement POSIX-style atomic renamesPatrick Steinhardt, Oct 24, 2024
  23. 1/3 compat/mingw: share file handles created via `CreateFileW()`Patrick Steinhardt, Oct 24, 2024
  24. 2/3 compat/mingw: allow deletion of most opened filesPatrick Steinhardt, Oct 24, 2024
  25. Johannes SixtOct 27, 2024
  26. Patrick SteinhardtOct 27, 2024
  27. Taylor BlauOct 27, 2024
  28. Taylor BlauOct 27, 2024
  29. 3/3 compat/mingw: support POSIX semantics for atomic renamesPatrick Steinhardt, Oct 24, 2024
  30. Johannes SixtOct 27, 2024
  31. Patrick SteinhardtOct 27, 2024
  32. Johannes SixtOct 27, 2024
  33. Patrick SteinhardtOct 27, 2024
  34. Johannes SixtOct 27, 2024
  35. Taylor BlauOct 27, 2024
  36. Taylor BlauOct 24, 2024
  37. Johannes SixtOct 27, 2024
  38. 0/3 compat/mingw: implement POSIX-style atomic renamesPatrick Steinhardt, Oct 27, 2024
  39. 1/3 compat/mingw: share file handles created via `CreateFileW()`Patrick Steinhardt, Oct 27, 2024
  40. 2/3 compat/mingw: allow deletion of most opened filesPatrick Steinhardt, Oct 27, 2024
  41. 3/3 compat/mingw: support POSIX semantics for atomic renamesPatrick Steinhardt, Oct 27, 2024
  42. Junio C HamanoNov 6, 2024
  43. Johannes SixtNov 6, 2024
  44. Junio C HamanoNov 6, 2024

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.