git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: Decompression speed: zip vs lzo

From
Sam Vilain <sam@vilain.net>
Date
Jan 9, 2008, 23:23 UTC
Message-ID
<47855765.9090001@vilain.net>
In-Reply-To
<7v4pdmfw27.fsf@gitster.siamese.dyndns.org>
Junio C Hamano wrote:
Show 11 quoted lines
> Note that the space nor time performance of compressing and
> uncompressing a single huge blob is not as interesting in the
> context of git as compressing/uncompressing millions of small
> pieces whose total size is comparable to the specimen of "huge
> single blob" experiment.  Obviously loose object files are
> compressed individually, and packfile contents are also
> individually and independently compressed.  Set-up cost for
> individual invocation of compression and uncompression on
> smaller data matters a lot more than an experiment on
> compressing and uncompressiong a single huge blob (this applies
> to both time and space).
Yes - and lzo will almost certainly win on all those counts!

I think to go forward this would need a prototype and benchmark figures for things like "annotate" and "fsck --full" - but bear in mind it would be a long road to follow-up to completion, as repository compatibility would need to be a primary concern and this essentially would create a new pack type AND a new *object* type. Not only that, but currently there is no header in the objects on disk which can be used to detect a gzip vs. an lzop stream. Not really worth it IMHO - gzip is already fast enough on even the most modern processor these days.

Sam.
Previous: Junio C HamanoNext: Johannes Schindelin
Message 3 of 39 in “Decompression speed: zip vs lzo”
  1. Marco CostalbaJan 9, 2008
  2. Junio C HamanoJan 9, 2008
  3. Sam VilainJan 9, 2008
  4. Johannes SchindelinJan 9, 2008
  5. Sam VilainJan 10, 2008
  6. Sam VilainJan 10, 2008
  7. Pierre HabouzitJan 10, 2008
  8. Nicolas PitreJan 10, 2008
  9. Linus TorvaldsJan 10, 2008
  10. Nicolas PitreJan 10, 2008
  11. Pierre HabouzitJan 11, 2008
  12. Sam VilainJan 10, 2008
  13. Linus TorvaldsJan 10, 2008
  14. Sam VilainJan 10, 2008
  15. Linus TorvaldsJan 10, 2008
  16. Sam VilainJan 11, 2008
  17. Linus TorvaldsJan 11, 2008
  18. Sam VilainJan 11, 2008
  19. Sam VilainJan 11, 2008
  20. Linus TorvaldsJan 11, 2008
  21. Sam VilainJan 12, 2008
  22. Nicolas PitreJan 12, 2008
  23. Sam VilainJan 12, 2008
  24. Nicolas PitreJan 12, 2008
  25. Johannes SchindelinJan 12, 2008
  26. Junio C HamanoJan 12, 2008
  27. Marco CostalbaJan 10, 2008
  28. Sam VilainJan 10, 2008
  29. Nicolas PitreJan 10, 2008
  30. Pierre HabouzitJan 11, 2008
  31. Nicolas PitreJan 11, 2008
  32. Morten WelinderJan 11, 2008
  33. Nicolas PitreJan 10, 2008
  34. Marco CostalbaJan 10, 2008
  35. Marco CostalbaJan 10, 2008
  36. Johannes SchindelinJan 10, 2008
  37. Marco CostalbaJan 10, 2008
  38. Dana HowJan 10, 2008
  39. Junio C HamanoJan 9, 2008

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.