Re: [PATCH] Teach "git add" and friends to be paranoid

> Nicolas Pitre <nico@fluxnic.net> writes:
> 
> > We do have to make two separate passes anyway.  The first pass is to 
> > hash the data only, and if that hash already exists in the object store 
> > then we call it done and skip over the deflate process which is still 
> > the dominant cost.  And that happens quite often.
> >
> > However, with a really large file, then it becomes advantageous to 
> > simply do the hash and deflate in parallel one chunk at a time, and 
> > simply discard the newly created objects if it happens to already 
> > exists.  That's the whole idea behind the newly introduced 
> > core.bigFileThreshold config variable (but the code to honor it in 
> > sha1_file.c doesn't exist yet).
> 
> The core.bigFileThreshold could be used in sha1_file.c to decide writing
> straight into a new packfile; that would avoid both the later repacking
> cost and the cross directory rename issue for loose object files.