Re: reftable [v5]: new ref storage format

From: Howard Chu <hyc@symas.com>
Date: Aug 14, 2017, 12:30 UTC
Message-ID: <576a2361-1a3d-4bb2-1d31-f095f9e3c708@symas.com>
In-Reply-To: <ee8f70bd-6f9e-3fb6-67be-ba26b6d5bf16@symas.com>

Howard Chu wrote:

Show 6 quoted lines

> The primary issue with using LMDB over NFS is with performance. All reads are 
> performed thru accesses of mapped memory, and in general, NFS implementations 
> don't cache mmap'd pages. I believe this is a consequence of the fact that 
> they also can't guarantee cache coherence, so the only way for an NFS client 
> to see a write from another NFS client is by always refetching pages whenever 
> they're accessed.

> LMDB's read lock management also wouldn't perform well over NFS; it also uses an mmap'd file. On a local filesystem LMDB read locks are zero cost since they just atomically update a word in the mmap. Over NFS, each update to the mmap would also require an msync() to propagate the change back to the server. This would seriously limit the speed with which read transactions may be opened and closed. (Ordinarily opening and closing a read txn can be done with zero system calls.)

All that aside, we could simply add an EXCLUSIVE open-flag to LMDB, and prevent multiple processes from using the DB concurrently. In that case, maintaining coherence with other NFS clients is a non-issue. It strikes me that git doesn't require concurrent multi-process access anyway, and any particular process would only use the DB for a short time before closing it and going away.

-- 
   -- Howard Chu
   CTO, Symas Corp.           http://www.symas.com
   Director, Highland Sun     http://highlandsun.com/hyc/
   Chief Architect, OpenLDAP  http://www.openldap.org/project/