From: Nicolas Pitre <nico@fluxnic.net>
Date: Mon, 22 Feb 2010 18:14:29 GMT
Subject: Re: [PATCH] Teach "git add" and friends to be paranoid
Message-ID: <alpine.LFD.2.00.1002221310130.1946@xanadu.home>
In-Reply-To: <37fcd2781002221005l2a8ecb64y3be84eaaacd27cdc@mail.gmail.com>

On Mon, 22 Feb 2010, Dmitry Potapov wrote:

> Currently, we calculate SHA-1, then lookup whether the object with this
> SHA-1 exists, and if it does not, then deflate and write it to the
> object storage. So, we avoid deflate and write cost if this object
> already exists. Moreover, when we deflate data, we create the temporary
> file in the same directory where the target object will be stored, thus
> avoiding cross-directory rename (which is important for some reason, but
> I don't remember why).  So, creating the temporary file requires the
> knowledge first two digits of SHA-1, which you cannot know without
> calculation SHA-1.

Even that initial SHA1 calculation can be done in chunks.  Worth doing 
for large enough files only though.


Nicolas