Re: [PATCH] don't use mmap() to hash files

From: Johannes Schindelin <johannes.schindelin@gmx.de>
Date: Feb 14, 2010, 19:28 UTC
Message-ID: <alpine.DEB.1.00.1002142025160.20986@pacific.mpi-cbg.de>
In-Reply-To: <alpine.DEB.1.00.1002142021100.20986@pacific.mpi-cbg.de>

Hi,

On Sun, 14 Feb 2010, Johannes Schindelin wrote:

Show 29 quoted lines

> On Sun, 14 Feb 2010, Dmitry Potapov wrote:
> 
> > On Sun, Feb 14, 2010 at 9:10 PM, Johannes Schindelin 
> > <Johannes.Schindelin@gmx.de> wrote:
> > >
> > > On Sun, 14 Feb 2010, Dmitry Potapov wrote:
> > >
> > >> On Sun, Feb 14, 2010 at 02:53:58AM +0100, Johannes Schindelin 
> > >> wrote:
> > >> > On Sun, 14 Feb 2010, Dmitry Potapov wrote:
> > >> >
> > >> > > + if (strbuf_read(&sbuf, fd, 4096) >= 0)
> > >> >
> > >> > How certain are you at this point that all of fd's contents fit 
> > >> > into your memory?
> > >>
> > >> You can't be sure... In fact, we know mmap() also may fail for huge 
> > >> files, so can strbuf_read().
> > >
> > > That's comparing oranges to apples. In one case, the address space 
> > > runs out, in the other the available memory. The latter is much more 
> > > likely.
> > 
> > "much more likely" is not a very qualitative characteristic...
> 
> Git was touted as a "content tracker". So I use it as such.
> 
> Concrete example: in one of my repositories, the average file size is 
> well over 2 gigabytes.

Just to make extremely sure that you undertand the issue: adding these files on a computer with 512 megabyte RAM works at the moment. Can you guarantee that there is no regression in that respect _with_ your patch?

Git is in wide use. If you provide a patch, it is not good enough anymore if it Works For You(tm), when it Does Not Work Somewhere Else Anymore(tm).

Ciao, Dscho