Re: [PATCH v3 03/16] index-helper: new daemon for caching index and related stuff

From: Junio C Hamano <gitster@pobox.com>
Date: Apr 12, 2016, 18:05 UTC
Message-ID: <xmqqfuuqr9ca.fsf@gitster.mtv.corp.google.com>
In-Reply-To: <1460482108.5540.59.camel@twopensource.com>

David Turner <dturner@twopensource.com> writes:

Show 7 quoted lines

>> I avoided actual files for two reasons
>> 
>>  - disk error rate is higher than memory one, and we might need
>> trailing SHA-1 back
>
> This only matters if we ever *read* the mmap off disk.  But that will
> rarely happen -- usually, everything will stay in memory.

True, and I do not think I'd terribly mind if we decided to use a on-disk file mmaped with MAP_SHARED as a more portable substitute for shm_* for this reason.

> Also, we
> never worry about disk errors for other git objects (e.g. blobs,
> commits, or trees), so it seems silly to worry for the index.

The objects are protected by checksums and fsck will notice errors, so the argument itself is bogus, I'd think, but you do not have to depend on that argument to support "let's use a mmaped file as a more portable substitute for shm*" in the first place, I would say.

>>  - access is slow (unless cached, but we can't be sure)
>
> We could solve this (and the other problem) with mlock.

Probably you meant madvise(2)?

For something of a size comparable to the index file held by index-helper-daemon in-core, I'd expect we wouldn't page too badly.