Re: [PATCH 2/2] core.fsyncobjectfiles: batch disk flushes

From: Christoph Hellwig <hch@lst.de>
Date: Aug 25, 2021, 05:38 UTC
Message-ID: <20210825053839.GA27037@lst.de>
In-Reply-To: <d1e68d4a2afc1d0ba74af64680bea09f412f21cc.1629856293.git.gitgitgadget@gmail.com>

On Wed, Aug 25, 2021 at 01:51:32AM +0000, Neeraj Singh via GitGitGadget wrote:

Show 13 quoted lines

> From: Neeraj Singh <neerajsi@microsoft.com>
> 
> When adding many objects to a repo with core.fsyncObjectFiles set to
> true, the cost of fsync'ing each object file can become prohibitive.
> 
> One major source of the cost of fsync is the implied flush of the
> hardware writeback cache within the disk drive. Fortunately, Windows,
> MacOS, and Linux each offer mechanisms to write data from the filesystem
> page cache without initiating a hardware flush.
> 
> This patch introduces a new 'core.fsyncObjectFiles = 2' option that
> takes advantage of the bulk-checkin infrastructure to batch up hardware
> flushes.

Another interesting way to flush on linux would be the syncfs call, which syncs all files on a file system. Once you write more than handful or two of files that tends to win out over a batch of fsync calls.