git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: Decompression speed: zip vs lzo

From
Nicolas Pitre <nico@cam.org>
Date
Jan 10, 2008, 21:30 UTC
Message-ID
<alpine.LFD.1.00.0801101613050.3054@xanadu.home>
In-Reply-To
<alpine.LFD.1.00.0801101252030.3148@woody.linux-foundation.org>
On Thu, 10 Jan 2008, Linus Torvalds wrote:
Show 15 quoted lines
> 
> 
> On Thu, 10 Jan 2008, Nicolas Pitre wrote:
> > 
> > Here's my rather surprising results:
> > 
> > My kernel repo pack size without the patch:	184275401 bytes
> > Same repo with the above patch applied:		205204930 bytes
> > 
> > So it is only 11% larger.  I was expecting much more.
> 
> It's probably worth doing those statistics on some other projects.
> 
> Maybe the difference to other repositories isn't huge, and maybe the 
> kernel *is* a good test-case, but I just wouldn't take that for granted. 
Obviously.

This was a really crud test, and my initial goal was to quickly dismiss Pierre's assertion. Turns out that he wasn't that wrong after all, and if a significant increase in access speed by avoiding zlib for 82% of object accesses can also be demonstrated for the kernel, then we have an opportunity for some optimization tradeoff with no backward compatibility concerns.

Show 5 quoted lines
> Yes, delta's are bound to compress much less well than non-deltas, and 
> especially for tree objects (which is a large chunk of them) they probably 
> compress even less (because a big part of the delta is actually just the 
> SHA1 changes), but if it's 11% on the kernel, it could easily be 25% on 
> something else.

Right. But again this is not worth pursuing if a significant speed increase in repo access is not demonstrated at least with the kernel.

Nicolas
Previous: Linus TorvaldsNext: Pierre Habouzit
Message 10 of 39 in “Decompression speed: zip vs lzo”
  1. Marco CostalbaJan 9, 2008
  2. Junio C HamanoJan 9, 2008
  3. Sam VilainJan 9, 2008
  4. Johannes SchindelinJan 9, 2008
  5. Sam VilainJan 10, 2008
  6. Sam VilainJan 10, 2008
  7. Pierre HabouzitJan 10, 2008
  8. Nicolas PitreJan 10, 2008
  9. Linus TorvaldsJan 10, 2008
  10. Nicolas PitreJan 10, 2008
  11. Pierre HabouzitJan 11, 2008
  12. Sam VilainJan 10, 2008
  13. Linus TorvaldsJan 10, 2008
  14. Sam VilainJan 10, 2008
  15. Linus TorvaldsJan 10, 2008
  16. Sam VilainJan 11, 2008
  17. Linus TorvaldsJan 11, 2008
  18. Sam VilainJan 11, 2008
  19. Sam VilainJan 11, 2008
  20. Linus TorvaldsJan 11, 2008
  21. Sam VilainJan 12, 2008
  22. Nicolas PitreJan 12, 2008
  23. Sam VilainJan 12, 2008
  24. Nicolas PitreJan 12, 2008
  25. Johannes SchindelinJan 12, 2008
  26. Junio C HamanoJan 12, 2008
  27. Marco CostalbaJan 10, 2008
  28. Sam VilainJan 10, 2008
  29. Nicolas PitreJan 10, 2008
  30. Pierre HabouzitJan 11, 2008
  31. Nicolas PitreJan 11, 2008
  32. Morten WelinderJan 11, 2008
  33. Nicolas PitreJan 10, 2008
  34. Marco CostalbaJan 10, 2008
  35. Marco CostalbaJan 10, 2008
  36. Johannes SchindelinJan 10, 2008
  37. Marco CostalbaJan 10, 2008
  38. Dana HowJan 10, 2008
  39. Junio C HamanoJan 9, 2008

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.