git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH WIP] sha1-lookup: make selection of 'middle' less aggressive

From
MCMarco Costalba <mcostalba@gmail.com>
Date
Jan 1, 2008, 09:01 UTC
Message-ID
<e5bfff550801010101w9d32122l1d7d76f2813a2eca@mail.gmail.com>
In-Reply-To
<e5bfff550801010040k59c9608do1acb9da548620f85@mail.gmail.com>
On Jan 1, 2008 9:40 AM, Marco Costalba <mcostalba@gmail.com> wrote:
Show 8 quoted lines
> On Jan 1, 2008 7:36 AM, Jeff King <peff@peff.net> wrote:
> >
> > The packfile is noticeably larger (55M versus 40M)
>
> Well 55M versus 40M is _only_  27% of compression ratio. It means that
> the compression algorithm is not so fundamental because the data is
> already, how to say, well packaged.
>

I think zlib is a very good general purpose algorithm, but is main strength is to give good final file sizes, it is mainly intended for files that are seldom decompressed.

For the use we do in git IMHO it would seem appropriate to look for algorithms used in the field of filesystem compression, where decompression penalty is a design goal. I know very little about this but I think among kernel people, expert and competent hackers should not be difficult to find, given that compressed filesystem are around from many years under linux/fs/ directory.

Marco
Previous: Marco CostalbaNext: Pierre Habouzit
Message 13 of 14 in “sha1-lookup: more memory efficient search in sorted list of SHA-1”
  1. sha1-lookup: more memory efficient search in sorted list of SHA-1Junio C Hamano, Dec 30, 2007
  2. sha1-lookup: make selection of 'middle' less aggressiveJunio C Hamano, Dec 30, 2007
  3. Marco CostalbaDec 30, 2007
  4. Marco CostalbaDec 30, 2007
  5. Shawn O. PearceDec 31, 2007
  6. Linus TorvaldsDec 30, 2007
  7. Junio C HamanoDec 30, 2007
  8. Marco CostalbaDec 30, 2007
  9. Linus TorvaldsDec 31, 2007
  10. Marco CostalbaDec 31, 2007
  11. Jeff KingJan 1, 2008
  12. Marco CostalbaJan 1, 2008
  13. Marco CostalbaJan 1, 2008
  14. Pierre HabouzitJan 1, 2008

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.