Re: Why SHA are 40 bytes? (aka looking for flames)

On 4/21/07, Marco Costalba <mcostalba@gmail.com> wrote:

When Shawn gets done with full compression the SHAs would get stored in the packfile once and then be replaced with a token generated from the compression algorithm. Compression tokens are designed to use the minimal number of bits depending on frequency of occurrence.

This doesn't happen with the current compression code since it doesn't have a global dictionary.

There are many other things that would benefit from a global dictionary. For example when working on the Mozilla repository the Mozilla license has gone through four major revisions. Each of these licenses is in the repository thousands of times. Making a copy of each license to a global dictionary and then replacing them with a token would yield megabytes of savings in the pack.

I would think that it is better to wait for a general compression solution rather than do a specific one for SHAs.