threads / patch / 42797

patchnotes-merge: use O_EXCL to avoid overwriting existing files

Subject: [PATCH] notes-merge: use O_EXCL to avoid overwriting existing files

## tl;dr

4 messages between Jul 7, 2016 and Jul 8, 2016. Diffs are folded; open one to read it.

replies: 3people: 3as markdown or json

René Scharfe· Jul 7, 2016, 20:08 UTC · lore

Use the open(2) flag O_EXCL to ensure the file doesn't already exist instead of (racily) calling stat(2) through file_exists(). While at it switch to xopen() to reduce code duplication and get more consistent error messages.

Signed-off-by: Rene Scharfe <l.s.r@web.de>
---
 notes-merge.c | 6 +-----
 1 file changed, 1 insertion(+), 5 deletions(-)
Show changes to notes-merge.c +1 −5
diff --git a/notes-merge.c b/notes-merge.c
index b7814c9..2b29fc4 100644
--- a/notes-merge.c
+++ b/notes-merge.c
@@ -298,12 +298,8 @@ static void write_buf_to_worktree(const unsigned char *obj,
 	char *path = git_pathdup(NOTES_MERGE_WORKTREE "/%s", sha1_to_hex(obj));
 	if (safe_create_leading_directories_const(path))
 		die_errno("unable to create directory for '%s'", path);
-	if (file_exists(path))
-		die("found existing file at '%s'", path);
 
-	fd = open(path, O_WRONLY | O_TRUNC | O_CREAT, 0666);
-	if (fd < 0)
-		die_errno("failed to open '%s'", path);
+	fd = xopen(path, O_WRONLY | O_EXCL | O_CREAT, 0666);
 
 	while (size > 0) {
 		long ret = write_in_full(fd, buf, size);

-- 
2.9.0
Jeff King· Jul 7, 2016, 20:38 UTC · re: René Scharfe · lore

Re: [PATCH] notes-merge: use O_EXCL to avoid overwriting existing files

On Thu, Jul 07, 2016 at 10:08:30PM +0200, René Scharfe wrote:
> Use the open(2) flag O_EXCL to ensure the file doesn't already exist
> instead of (racily) calling stat(2) through file_exists().  While at it
> switch to xopen() to reduce code duplication and get more consistent
> error messages.

This is definitely an improvement, as it behaves the same except for the TOCTOU race. But not being very familiar with the notes-merge code, I have to wonder if this is a system of a larger design issue.

Why do we care that the file exists? Should we instead be using the lockfile code to get exclusive access to it? That would also switch us to doing the write-to-tempfile-and-rename dance, but that seems like it would be a good thing. If we hit a write() error in the code now, we leave a partially-written file in the notes worktree.

I dunno. From my cursory reading of the code, it seems like we'd never really expect this file_exists() to trigger in the first place, so perhaps it's not worth thinking too hard about it.

-Peff
Junio C Hamano· Jul 7, 2016, 21:10 UTC · re: Jeff King · lore

Re: [PATCH] notes-merge: use O_EXCL to avoid overwriting existing files

Jeff King <peff@peff.net> writes:
Show 5 quoted lines
> Why do we care that the file exists? Should we instead be using the
> lockfile code to get exclusive access to it? That would also switch us
> to doing the write-to-tempfile-and-rename dance, but that seems like it
> would be a good thing. If we hit a write() error in the code now, we
> leave a partially-written file in the notes worktree.
Yeah, I had the same thought when I saw the change.
> I dunno. From my cursory reading of the code, it seems like we'd never
> really expect this file_exists() to trigger in the first place, so
> perhaps it's not worth thinking too hard about it.
Perhaps.
Jeff King· Jul 8, 2016, 09:37 UTC · re: René Scharfe · lore

Re: [PATCH] notes-merge: use O_EXCL to avoid overwriting existing files

On Thu, Jul 07, 2016 at 10:08:30PM +0200, René Scharfe wrote:
Show 15 quoted lines
> diff --git a/notes-merge.c b/notes-merge.c
> index b7814c9..2b29fc4 100644
> --- a/notes-merge.c
> +++ b/notes-merge.c
> @@ -298,12 +298,8 @@ static void write_buf_to_worktree(const unsigned char *obj,
>  	char *path = git_pathdup(NOTES_MERGE_WORKTREE "/%s", sha1_to_hex(obj));
>  	if (safe_create_leading_directories_const(path))
>  		die_errno("unable to create directory for '%s'", path);
> -	if (file_exists(path))
> -		die("found existing file at '%s'", path);
>  
> -	fd = open(path, O_WRONLY | O_TRUNC | O_CREAT, 0666);
> -	if (fd < 0)
> -		die_errno("failed to open '%s'", path);
> +	fd = xopen(path, O_WRONLY | O_EXCL | O_CREAT, 0666);

While working on write_file_buf() elsewhere, I wondered if this was a good candidate for conversion. But it's not because of the O_EXCL.

I was tempted by something like:
  write_file_buf(path, O_EXCL, buf, len);

but I think that just makes the interface more cluttered for the other callers. If you are doing something clever with O_EXCL, you probably should just do it by hand.

However, we can make the loop less painful, as below (on top of your patch).

-- >8 --
Subject: [PATCH] notes-merge: use write_or_die()

This output loop is overkill. For one thing, we do not need to loop on write_in_full(); it's entire purpose is to avoid looping, and it even includes the "disk full?" check itself.

Secondly, the check for EPIPE is pointless. We know this is an on-disk file that we just opened, so short of somebody sneaking a FIFO into our notes-merge working tree, it will not be a pipe. And even if it is, we would generally die of SIGPIPE before hitting this code. And even if we somehow ignored SIGPIPE, dying (rather than silently ignoring the error) seems like the only sensible action anyway.

So what we're left with is calling write_in_full() and dying if it fails. And that's exactly what write_or_die() does.

Signed-off-by: Jeff King <peff@peff.net>
---
 notes-merge.c | 16 +---------------
 1 file changed, 1 insertion(+), 15 deletions(-)
Show changes to notes-merge.c +1 −15
diff --git a/notes-merge.c b/notes-merge.c
index 2b29fc4..a659a62 100644
--- a/notes-merge.c
+++ b/notes-merge.c
@@ -300,21 +300,7 @@ static void write_buf_to_worktree(const unsigned char *obj,
 		die_errno("unable to create directory for '%s'", path);
 
 	fd = xopen(path, O_WRONLY | O_EXCL | O_CREAT, 0666);
-
-	while (size > 0) {
-		long ret = write_in_full(fd, buf, size);
-		if (ret < 0) {
-			/* Ignore epipe */
-			if (errno == EPIPE)
-				break;
-			die_errno("notes-merge");
-		} else if (!ret) {
-			die("notes-merge: disk full?");
-		}
-		size -= ret;
-		buf += ret;
-	}
-
+	write_or_die(fd, buf, size);
 	close(fd);
 	free(path);
 }

-- 
2.9.0.393.g704e522

← back to recent threads