Re: [PATCH v2 1/4] Introduce a function to lock/unlock file descriptors when appending

From: Johannes Schindelin <johannes.schindelin@gmx.de>
Date: Aug 10, 2018, 21:31 UTC
Message-ID: <nycvar.QRO.7.76.6.1808102327080.71@tvgsbejvaqbjf.bet>
In-Reply-To: <xmqqzhxu6l81.fsf@gitster-ct.c.googlers.com>

Hi Junio,

On Fri, 10 Aug 2018, Junio C Hamano wrote:

Show 22 quoted lines

> "Johannes Schindelin via GitGitGadget" <gitgitgadget@gmail.com>
> writes:
> 
> > +
> > +#ifndef GIT_WINDOWS_NATIVE
> > +int lock_or_unlock_fd_for_appending(int fd, int lock_it)
> > +{
> > +	struct flock flock;
> > +
> > +	flock.l_type = lock_it ? F_WRLCK : F_UNLCK;
> > +
> > +	/* (un-)lock the whole file */
> > +	flock.l_whence = SEEK_SET;
> > +	flock.l_start = 0;
> > +	flock.l_len = 0;
> > +
> > +	return fcntl(fd, F_SETLKW, &flock);
> > +}
> > +#endif
> 
> I think people already told you that this is not needed on systems
> with properly working O_APPEND [*1*]

I also wanted to safeguard against other (well-behaved) programs writing to the trace file.

Granted, this is a bit theoretical, but it is quite possible that other Git implementations do things differently than our trace.c. But with the file locking that even that POSIX man page I quoted recommends, all this is not an issue.

And this patch series implements that locking.

> 	Side note #1: and with network filesystems where O_APPEND
>         may not work reliably, fcntl based range locking would not
>         work either, so having this would not help.

At least on Windows, it would help, though.

And if there *is* a way to lock via NFS (which is the only network filesystem I am aware of that has these locking issues, at least *some* others are just fine), we would at least already have that function where to implement it.

> I saw other Johannes and other Jeff peeking into fixing O_APPEND;
> I do not know how well that effort goes, but it would be preferrable
> if we can successfully go that route.

As I pointed out previously, my mail provider is losing mails left and right for me. Could I ask for a pointer?

Show 11 quoted lines

> As I said in my review of the first patch in v1 series, I am not
> fundamentally opposed to a few "lock here to work around lack of
> O_APPEND" and "unlock here for the same reason" calls to limited
> codepaths as a workaround, as the damage is limited (that is why I
> earlier looked at our use of O_APPEND), but that would be the last
> resort if O_APPEND cannot be made to work reliably on Windows.
> 
> But even if we end up doing so, on systems with POSIX O_APPEND
> working, I think that function should be
> 
>     #define lock_or_unlock_for_appending(fd, lock) 0 /* nothing to do */

I think that that would be only appropriate if there were no other Git implementations.

Ciao, Dscho