Re: git-fetching from a big repository is slow
- From
Shawn Pearce <spearce@spearce.org>
- Date
- Dec 14, 2006, 19:46 UTC
- Message-ID
- <20061214194636.GO1747@spearce.org>
- In-Reply-To
- <C287764F-6755-4291-A87A-3E8816E90B49@adacore.com>
Geert Bosch <bosch@adacore.com> wrote:
Show 20 quoted lines
> Such special magic based on filenames is always a bad idea. Tomorrow
> somebody
> comes with .zip files (oh, and of course .ZIP), then it's .jpg's other
> compressed content. In the end git will be doing lots of magic and
> still perform
> badly on unknown compressed content.
>
> There is a very simple way of detecting compressed files: just look
> at the
> size of the compressed blob and compare against the size of the
> expanded blob.
> If the compressed blob has a non-trivial size which is close to the
> expanded
> size, assume the file is not interesting as source or target for deltas.
>
> Example:
> if (compressed_size > expanded_size / 4 * 3 + 1024) {
> /* don't try to deltify if blob doesn't compress well */
> return ...;
> }And yet I get good delta compression on a number of ZIP formatted files which don't get good additional zlib compression (<3%). Doing the above would cause those packfiles to explode to about 10x their current size.