Re: [RFC PATCH] merge-recursive: create new files with O_EXCL

From: Junio C Hamano <gitster@pobox.com>
Date: Mar 10, 2021, 23:01 UTC
Message-ID: <xmqqblbqipeh.fsf@gitster.g>
In-Reply-To: <f6cd9386-8a58-ee52-4c7b-60d9bd14a51d@gmail.com>

Ephrim Khong <dr.khong@gmail.com> writes:

Show 12 quoted lines

> When re-writing the content of a file during a merge, the file
> is first unlinked and then re-created. Since it is a new file
> at the time of the open call, O_EXCL should be passed to clarify
> that the file is expected to be new.
>
> Signed-off-by: Ephrim Khong <dr.khong@gmail.com>
> ---
> This is actually a fix for an issue we ran into on an nfs4
> mount. Files created with O_TRUNC instead of O_EXCL sometimes
> had their permissions wrong. However, it appears to be a safe
> thing to change this, especially since other parts of the
> git codebase also prefer the O_EXCL flag.

Interesting.

It seems that this part of the code has always used O_TRUNC since its inception at 6d297f81 (Status update on merge-recursive in C, 2006-07-08) when two thieves smuggled it into our codebase. Back then, this was the only use of O_TRUNC|O_WRONLY|O_CREAT combination in the production codebase (as you observed, O_CREAT|O_EXCL|O_WRONLY was what all the other codepaths use). But the use of O_TRUNC has spread over time to other parts of the codebase, perhaps cargo culted. If you look at hits from

    $ git grep -e O_TRUNC -e O_EXCL

you see the combination of CREAT/WRONLY/TRUNC used all over the place [*], especially in newer parts of the code.

So it becomes very curious why this one location needs to be so special and you are not patching other uses of O_TRUNC.

I do not think we mind fixing the use of O_TRUNC with "remove and then create with O_EXCL", but we'd probably want to

 * understand why only this place matters, or perhaps other uses of
   O_TRUNC needs the same fix to work "correctly" with your NFS
   mounts, in which case we'd need all of them addressed in the same
   series of patches, and

 * understand why your NFS mount is broken and give a better
   explanation as to why we need to have a workaround in our code.

before doing so.

Thanks.

[Footnote]

* It is understandable that CREAT/WRONLY/TRUNC are used in
  combination, as that is documented as a synonym for creat().

Show 16 quoted lines

>  merge-recursive.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/merge-recursive.c b/merge-recursive.c
> index f736a0f632..f72a376c5b 100644
> --- a/merge-recursive.c
> +++ b/merge-recursive.c
> @@ -974,7 +974,7 @@ static int update_file_flags(struct merge_options *opt,
>  			int fd;
>  			int mode = (contents->mode & 0100 ? 0777 : 0666);
>
> -			fd = open(path, O_WRONLY | O_TRUNC | O_CREAT, mode);
> +			fd = open(path, O_WRONLY | O_EXCL | O_CREAT, mode);
>  			if (fd < 0) {
>  				ret = err(opt, _("failed to open '%s': %s"),
>  					  path, strerror(errno));