Re: Excessive mmap [was Git server eats all memory]

>> Looking some more into it today the bulk of the memory allocation
>> happens in write_pack_file in the following loop.
>>
>> for (; i < nr_objects; i++) {
>>    if (!write_one(f, objects + i, &offset))
>>        break;
>>    display_progress(progress_state, written);
>> }
>>
>> This eventually calls write_object, here I am wondering if the
>> unuse_pack function is doing its job. As far as I can tell it writes a
>> null in memory, that I think is not enough to reclaim memory.
>
> What do you mean by the "memory allocation" happens here?  How are you
> measuring it?

>> I also looked at the use_pack function where the mmap is
>> happening. Would it be worth refactoring this function so that it uses
>> an index withing a file instead of mmap?
>>
>> Unless I hear of a better idea I'll be trying that tomorrow...
>
> I wouldn't expect this to help, but I would be interested to hear if
> it does.

> If the problem is simply that you're flooding the kernel disk cache
> with data you'll use only once, to the detriment of everything else on
> the system, then one thing that might help could be posix_fadvise:
>
>     posix_fadvise(fd, ofs, len, POSIX_FADV_DONTNEED);