From: Enrico Weigelt Date: Tue, 05 Oct 2010 22:51:50 GMT Subject: Re: large files and low memory Message-ID: <20101005225150.GA27633@nibiru.local> In-Reply-To: * Nicolas Pitre wrote: > > > The mmap() isn't the problem. Its the allocation of a buffer that is > > > larger than the file in order to hold the result of deflating the file > > > before it gets written to disk. > > > When the file is bigger than physical memory, the kernel has to > > > page in parts of the file as well as swap in and out parts of > > > that allocated buffer to hold the deflated file. > > > > What are the access pattern of these memory areas ? > > Perfectly linear. In this case, I wonder why my machine goes into thrashing so easily (P3 w/ 256MB ram). Seems the mmu/paging code doesnt recognize that the previously-used pages can be kicked-off quickly ;-o Perhaps I should talk to the kernel folks. > > Perhaps madvise() could help ? > > Perhaps. hmm, so we should try it ;-p where'd be the right place to add it ? cu -- ---------------------------------------------------------------------- Enrico Weigelt, metux IT service -- http://www.metux.de/ phone: +49 36207 519931 email: weigelt@metux.de mobile: +49 151 27565287 icq: 210169427 skype: nekrad666 ---------------------------------------------------------------------- Embedded-Linux / Portierung / Opensource-QM / Verteilte Systeme ----------------------------------------------------------------------