threads / patch / 19749

patchExpand the rename(2) workaround in mingw to cover case change in filename

Subject: [PATCH] Expand the rename(2) workaround in mingw to cover case change in filename

## tl;dr

7 messages between Jun 8, 2009 and Jun 9, 2009. Diffs are folded; open one to read it.

replies: 6people: 3as markdown or json

Alex Riesen· Jun 8, 2009, 20:32 UTC · lore

Windows, being a case-confused operating system, sometimes has problems renaming directory entries with only change in the case of their characters. Try to work the problem around by using an intermediate file.

---
NOT TESTED. Can't. My Windows broke again. Not even compile-tested.

And sorry for somewhat extended Cc:. Thought the problem need a little more experienced attention (then again, maybe it doesn't).

That's a response to Bug 228 (which is properly "WONTFIX", if you ask my opinion):

    http://code.google.com/p/msysgit/issues/detail?id=228

I don't like it, but it is the same what people usually forced to do anyway, so here it is.

It has a racing problem which can cause the original file be renamed into a temporary name (as returned by mkstemp):

    - the rename of the source name to temp name succeeds
    - the rename of temp name to destination name fails (i.e. it
      exists and is open)

It is not fixable (because the atomicity of rename(2) is broken by the intermediate name and two separate calls to rename involved). I could have tried to rename back to source, but got fed up with another Windows stupidity and gave up.

 compat/mingw.c |   65 +++++++++++++++++++++++++++++++++++++++++++++++--------
 1 files changed, 55 insertions(+), 10 deletions(-)
Show changes to compat/mingw.c +55 −10
diff --git a/compat/mingw.c b/compat/mingw.c
index e190fdd..ad60dff 100644
--- a/compat/mingw.c
+++ b/compat/mingw.c
@@ -931,21 +931,12 @@ int mingw_connect(int sockfd, struct sockaddr *sa, size_t sz)
 	return connect(s, sa, sz);
 }
 
-#undef rename
-int mingw_rename(const char *pold, const char *pnew)
+static int move_file_replace(const char *pold, const char *pnew)
 {
 	DWORD attrs, gle;
 	int tries = 0;
 	static const int delay[] = { 0, 1, 10, 20, 40 };
 
-	/*
-	 * Try native rename() first to get errno right.
-	 * It is based on MoveFile(), which cannot overwrite existing files.
-	 */
-	if (!rename(pold, pnew))
-		return 0;
-	if (errno != EEXIST)
-		return -1;
 repeat:
 	if (MoveFileEx(pold, pnew, MOVEFILE_REPLACE_EXISTING))
 		return 0;
@@ -982,6 +973,60 @@ repeat:
 	return -1;
 }
 
+#undef rename
+int mingw_rename(const char *pold, const char *pnew)
+{
+	int fd, rc = -1;
+	char *p, *tmp = NULL;
+
+	/*
+	 * Try native rename() first to get errno right.
+	 * It is based on MoveFile(), which cannot overwrite existing files.
+	 */
+	if (!rename(pold, pnew))
+		return 0;
+	if (errno != EEXIST)
+		return -1;
+	/*
+	 * Windows' case-insensitivity does not allow it to directly
+	 * do a rename where the only change in the file name is
+	 * the change of a letter case. Work this around with a
+	 * temporary file.
+	 */
+	if (!dst) {
+		errno = EINVAL;
+		goto fail;
+	}
+	tmp = malloc(strlen(dst) + 7 /* reserved for template */);
+	if (!tmp) {
+		errno = ENOMEM;
+		goto fail;
+	}
+	strcpy(tmp, dst);
+	p = tmp + strlen(dst);
+	for (p = tmp + strlen(dst); p > tmp; --p)
+		if ('\\' == *p || '/' == *p) {
+			++p;
+			break;
+		}
+	strcpy(p, "XXXXXX")
+	fd = mkstemp(tmp);
+	if (fd < 0)
+		goto fail;
+	close(fd);
+	if (move_file_replace(src, tmp)) {
+		rc = errno;
+		unlink(tmp);
+		errno = rc;
+		rc = -1;
+		goto fail;
+	}
+	rc = move_file_replace(tmp, dst);
+fail:
+	free(tmp);
+	return rc;
+}
+
 struct passwd *getpwuid(int uid)
 {
 	static char user_name[100];
-- 
1.6.3.2.214.g82a9d
Johannes Schindelin· Jun 8, 2009, 21:59 UTC · re: Alex Riesen · lore

Re: [PATCH] Expand the rename(2) workaround in mingw to cover case change in filename

Hi,
On Mon, 8 Jun 2009, Alex Riesen wrote:
Show 8 quoted lines
> Windows, being a case-confused operating system, sometimes has
> problems renaming directory entries with only change in the case
> of their characters. Try to work the problem around by using an
> intermediate file.
> 
> ---
> 
> NOT TESTED. Can't. My Windows broke again. Not even compile-tested.

Yes, that is pretty easy to see as you first used pold/pnew and then src/dst.

I minimally fixed up your patch (it now uses strbuf), and added a test-case.

This test-case is actually crucial: it shows that your patch is not enough: the culprit is this code in builtin-mv.c:167--168:

                else if (lstat(dst, &st) == 0) {
                        bad = "destination exists";

Ciao, Dscho

Johannes Schindelin· Jun 8, 2009, 22:00 UTC · re: Johannes Schindelin · lore

Re: [PATCH] Expand the rename(2) workaround in mingw to cover case change in filename

Hi,
On Mon, 8 Jun 2009, Johannes Schindelin wrote:
Show 16 quoted lines
> On Mon, 8 Jun 2009, Alex Riesen wrote:
> 
> > Windows, being a case-confused operating system, sometimes has
> > problems renaming directory entries with only change in the case
> > of their characters. Try to work the problem around by using an
> > intermediate file.
> > 
> > ---
> > 
> > NOT TESTED. Can't. My Windows broke again. Not even compile-tested.
> 
> Yes, that is pretty easy to see as you first used pold/pnew and then 
> src/dst.
> 
> I minimally fixed up your patch (it now uses strbuf), and added a 
> test-case.
I probably should have mentioned that you can see it here:
http://repo.or.cz/w/git/mingw/4msysgit.git?a=shortlog;h=refs/heads/work/rename

Ciao, Dscho

Alex Riesen· Jun 8, 2009, 22:08 UTC · re: Johannes Schindelin · lore

Re: [PATCH] Expand the rename(2) workaround in mingw to cover case change in filename

2009/6/8 Johannes Schindelin <Johannes.Schindelin@gmx.de>:
Show 5 quoted lines
> On Mon, 8 Jun 2009, Alex Riesen wrote:
>> NOT TESTED. Can't. My Windows broke again. Not even compile-tested.
>
> Yes, that is pretty easy to see as you first used pold/pnew and then
> src/dst.
Yep.
Show 8 quoted lines
> I minimally fixed up your patch (it now uses strbuf), and added a
> test-case.
>
> This test-case is actually crucial: it shows that your patch is not
> enough: the culprit is this code in builtin-mv.c:167--168:
>
>                else if (lstat(dst, &st) == 0) {
>                        bad = "destination exists";
Ah, thanks. Missed that completely.
Johannes Sixt· Jun 9, 2009, 05:54 UTC · re: Alex Riesen · lore

Re: [PATCH] Expand the rename(2) workaround in mingw to cover case change in filename

Alex Riesen schrieb:
Show 18 quoted lines
> 2009/6/8 Johannes Schindelin <Johannes.Schindelin@gmx.de>:
>> On Mon, 8 Jun 2009, Alex Riesen wrote:
>>> NOT TESTED. Can't. My Windows broke again. Not even compile-tested.
>> Yes, that is pretty easy to see as you first used pold/pnew and then
>> src/dst.
> 
> Yep.
> 
>> I minimally fixed up your patch (it now uses strbuf), and added a
>> test-case.
>>
>> This test-case is actually crucial: it shows that your patch is not
>> enough: the culprit is this code in builtin-mv.c:167--168:
>>
>>                else if (lstat(dst, &st) == 0) {
>>                        bad = "destination exists";
> 
> Ah, thanks. Missed that completely.

That's the reason why I think any work-around for this problem is not worth it.

If you want to be cross-platfrom, make up your mind about file names in advance.

-- Hannes
Alex Riesen· Jun 9, 2009, 06:04 UTC · re: Johannes Sixt · lore

Re: [PATCH] Expand the rename(2) workaround in mingw to cover case change in filename

2009/6/9 Johannes Sixt <j.sixt@viscovery.net>:
Show 22 quoted lines
> Alex Riesen schrieb:
>> 2009/6/8 Johannes Schindelin <Johannes.Schindelin@gmx.de>:
>>> On Mon, 8 Jun 2009, Alex Riesen wrote:
>>>> NOT TESTED. Can't. My Windows broke again. Not even compile-tested.
>>> Yes, that is pretty easy to see as you first used pold/pnew and then
>>> src/dst.
>>
>> Yep.
>>
>>> I minimally fixed up your patch (it now uses strbuf), and added a
>>> test-case.
>>>
>>> This test-case is actually crucial: it shows that your patch is not
>>> enough: the culprit is this code in builtin-mv.c:167--168:
>>>
>>>                else if (lstat(dst, &st) == 0) {
>>>                        bad = "destination exists";
>>
>> Ah, thanks. Missed that completely.
>
> That's the reason why I think any work-around for this problem is not
> worth it.
What is the lstat is for, anyway?
> If you want to be cross-platfrom, make up your mind about file names in
> advance.

I suspect it is the other way around. Some platforms just hate the idea of being ported to. Or its designers have poor imagination and never though about the fact that other platforms exist.

Johannes Schindelin· Jun 9, 2009, 07:27 UTC · re: Alex Riesen · lore

Re: [PATCH] Expand the rename(2) workaround in mingw to cover case change in filename

Hi,
On Tue, 9 Jun 2009, Alex Riesen wrote:
Show 25 quoted lines
> 2009/6/9 Johannes Sixt <j.sixt@viscovery.net>:
> > Alex Riesen schrieb:
> >> 2009/6/8 Johannes Schindelin <Johannes.Schindelin@gmx.de>:
> >>> On Mon, 8 Jun 2009, Alex Riesen wrote:
> >>>> NOT TESTED. Can't. My Windows broke again. Not even compile-tested.
> >>> Yes, that is pretty easy to see as you first used pold/pnew and then
> >>> src/dst.
> >>
> >> Yep.
> >>
> >>> I minimally fixed up your patch (it now uses strbuf), and added a
> >>> test-case.
> >>>
> >>> This test-case is actually crucial: it shows that your patch is not
> >>> enough: the culprit is this code in builtin-mv.c:167--168:
> >>>
> >>>                else if (lstat(dst, &st) == 0) {
> >>>                        bad = "destination exists";
> >>
> >> Ah, thanks. Missed that completely.
> >
> > That's the reason why I think any work-around for this problem is not
> > worth it.
> 
> What is the lstat is for, anyway?
It is to avoid overwriting existing files.
Show 6 quoted lines
> > If you want to be cross-platfrom, make up your mind about file names 
> > in advance.
> 
> I suspect it is the other way around. Some platforms just hate the idea 
> of being ported to. Or its designers have poor imagination and never 
> though about the fact that other platforms exist.

Actually, think about it as a brilliant business plan: make developers be stuck with developing for your platform, then they lack the time to take care of other platforms.

The bigger question with the patch is if the lstat() call can tell us if this is the same file (really the same, not just a hardlink (= same contents)).

Ciao, Dscho

← back to recent threads