Re: [PATCH v3 2/6] core.fsyncobjectfiles: batched disk flushes
- From
Junio C Hamano <gitster@pobox.com>
- Date
- Sep 14, 2021, 19:34 UTC
- Message-ID
- <xmqqfsu70x58.fsf@gitster.g>
- In-Reply-To
- <f8b5b709e9edc363b2de7d4afa443deec0120ca0.1631590725.git.gitgitgadget@gmail.com>
"Neeraj Singh via GitGitGadget" <gitgitgadget@gmail.com> writes:
Show 17 quoted lines
> diff --git a/config.c b/config.c
> index cb4a8058bff..9fe3602e1c4 100644
> --- a/config.c
> +++ b/config.c
> @@ -1509,7 +1509,13 @@ static int git_default_core_config(const char *var, const char *value, void *cb)
> }
>
> if (!strcmp(var, "core.fsyncobjectfiles")) {
> - fsync_object_files = git_config_bool(var, value);
> + if (!value)
> + return config_error_nonbool(var);
> + if (!strcasecmp(value, "batch"))
> + fsync_object_files = FSYNC_OBJECT_FILES_BATCH;
> + else
> + fsync_object_files = git_config_bool(var, value)
> + ? FSYNC_OBJECT_FILES_ON : FSYNC_OBJECT_FILES_OFF;
> return 0;The original code used to allow the short-and-sweet valueless true
[core] fsyncobjectfiles
but it no longer does by calling it a nonbool error. This breaks existing users' repositories that have been happily working, doesn't it?
Perhaps
if (value && !strcmp(value, "batch")) fsync_object_files = FSYNC_OBJECT_FILES_BATCH; else if (git_config_bool(var, value)) fsync_object_files = FSYNC_OBJECT_FILES_ON; else fsync_object_files = FSYNC_OBJECT_FILES_OFF;
Show 30 quoted lines
> -/* Finalize a file on disk, and close it. */
> -static void close_loose_object(int fd)
> -{
> - if (fsync_object_files)
> - fsync_or_die(fd, "loose object file");
> - if (close(fd) != 0)
> - die_errno(_("error when closing loose object file"));
> -}
> -
> /* Size of directory component, including the ending '/' */
> static inline int directory_size(const char *filename)
> {
> @@ -1973,17 +1964,8 @@ static int write_loose_object(const struct object_id *oid, char *hdr,
> die(_("confused by unstable object source data for %s"),
> oid_to_hex(oid));
>
> - close_loose_object(fd);
> -
> - if (mtime) {
> - struct utimbuf utb;
> - utb.actime = mtime;
> - utb.modtime = mtime;
> - if (utime(tmp_file.buf, &utb) < 0)
> - warning_errno(_("failed utime() on %s"), tmp_file.buf);
> - }
> -
> - return finalize_object_file(tmp_file.buf, filename.buf);
> + return fsync_and_close_loose_object_bulk_checkin(fd, tmp_file.buf,
> + filename.buf, mtime);
> }This block of code looked familiar and I was about to complain "why add it in one step and remove it in another?"
But it is a different instance from the one that was added in one of the previous patches ;-).
Show 28 quoted lines
> +int git_fsync(int fd, enum fsync_action action)
> +{
> + if (action == FSYNC_WRITEOUT_ONLY) {
> +#ifdef __APPLE__
> + /*
> + * on Mac OS X, fsync just causes filesystem cache writeback but does not
> + * flush hardware caches.
> + */
> + return fsync(fd);
> +#endif
> +
> +#ifdef HAVE_SYNC_FILE_RANGE
> + /*
> + * On linux 2.6.17 and above, sync_file_range is the way to issue
> + * a writeback without a hardware flush. An offset of 0 and size of 0
> + * indicates writeout of the entire file and the wait flags ensure that all
> + * dirty data is written to the disk (potentially in a disk-side cache)
> + * before we continue.
> + */
> +
> + return sync_file_range(fd, 0, 0, SYNC_FILE_RANGE_WAIT_BEFORE |
> + SYNC_FILE_RANGE_WRITE |
> + SYNC_FILE_RANGE_WAIT_AFTER);
> +#endif
> +
> + errno = ENOSYS;
> + return -1;
> + }This allows the caller that can take advantage of writeout-only mode to naturally fall back on the full sync per each file if we cannot do a writeout-only sync. OK.
Show 6 quoted lines
> +#ifdef __APPLE__ > + return fcntl(fd, F_FULLFSYNC); > +#else > + return fsync(fd); > +#endif > +}
If we are introducing "enum fsync_action", we should have some way to make it clear that we are covering all the possible values of "action".
Switching on action, i.e.
switch (action) {
case FSYNC_WRITEOUT_ONLY:
...
break;
case FSYNC_HARDWARE_FLUSH:
...
break;
default:
BUG("unexpected git_fsync(%d) call", action);
}would be one way to do so.
Thanks.