Media repositories and memory usage

Hi,

There is another aspect I did not see Tim mention in his slides: memory usage with large blobs. Git has tradition of loading the whole blob in memory for easy manipulation. It could consume a lot of memory in large blob case (and particularly worse in mingw port because it does not support mmap).

I see these operations that need access to blobs:
 1. checkout blobs
 2. checkin blobs
 3. diff/delta blobs

Diffing blobs should be avoided any way for large blobs. Checking in blobs does not require lots of memory. I'm working on checkout case to reduce memory footprint. Is there any other case that will need full blob in memory?

PS. For checkout/checkin case, if you do any conversion, full blob in memory is still needed. But I guess that is rare for large blobs.