Re: [RFC \ WISH] Add -o option to git-rev-list

On Sun, 10 Dec 2006, Linus Torvalds wrote:

Btw, I should obviously clarify that. You should _always_ worry about low-memory situations, but the only real issue is really "working set size", not "data in files or in memory".

If the file representation is very dense (like the git pack-files are, for example), then it may well make sense to keep the data in a file, just because it's smaller there than if you keep it in expanded form in memory.

Also, it's nice to keep stuff in the filesystem rather than in process VM memory, because filesystem data that is cached in memory is useful for _other_ processes, ie it has a lifetime that is longer than the process itself. However, that's obviously only true for long-lived files that are shared among processes, it's _not_ true for temporary files.

For temporary files, the memory footprint of a temp-file is usually _larger_ than the memory footprint of the same data kept in memory. You have things like page-cache alignment, and often the issue of marshalling data into ASCII etc. So temp-files are almost never a better solution than keeping things in memory (unless you use those temp-files to truly _share_ data between processes, ie you do a shared mmap and they can re-use the same pages actively in a way they couldn't otherwise).