git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Huge win, compressing a window of delta runs as a unit

From
Jon Smirl <jonsmirl@gmail.com>
Date
Aug 16, 2006, 17:20 UTC
Message-ID
<9e4733910608161020s6855140bs68aaab6e1bbd3bad@mail.gmail.com>

Shawn put together a new version of his import utility that packs all of the deltas from a run into a single blob instead of one blob per delta. The idea is to put 10 or more deltas into each delta entry instead of one. The index format would map the 10 sha1's to a single packed delta entry which would be expanded when needed. Note that you probably needed multiple entries out of the delta pack to generate the revision you were looking for so this is no real loss on extraction.

I ran it overnight on mozcvs. If his delta pack code is correct this is a huge win.

One entry per delta - 845,42,0150 Packed deltas - 295,018,474 65% smaller

The effect of packing the deltas is to totally eliminate many of the redundant zlib dictionaries.

This is without using a zlib dictionary which gains another 4% form a 4KB dictionary.

-- 
Jon Smirl
jonsmirl@gmail.com
Next: Shawn Pearce
Message 1 of 36 in “Huge win, compressing a window of delta runs as a unit”
  1. Jon SmirlAug 16, 2006
  2. Shawn PearceAug 17, 2006
  3. Johannes SchindelinAug 17, 2006
  4. Johannes SchindelinAug 17, 2006
  5. Jon SmirlAug 17, 2006
  6. Johannes SchindelinAug 17, 2006
  7. Nicolas PitreAug 17, 2006
  8. Johannes SchindelinAug 17, 2006
  9. Jon SmirlAug 17, 2006
  10. Nicolas PitreAug 17, 2006
  11. Jon SmirlAug 17, 2006
  12. Nicolas PitreAug 17, 2006
  13. Jon SmirlAug 17, 2006
  14. Nicolas PitreAug 17, 2006
  15. Jon SmirlAug 17, 2006
  16. Nicolas PitreAug 17, 2006
  17. Nicolas PitreAug 18, 2006
  18. Jon SmirlAug 18, 2006
  19. Nicolas PitreAug 18, 2006
  20. Jon SmirlAug 18, 2006
  21. Nicolas PitreAug 21, 2006
  22. Shawn PearceAug 21, 2006
  23. Jakub NarebskiAug 21, 2006
  24. Jon SmirlAug 21, 2006
  25. Jon SmirlAug 18, 2006
  26. Johannes SchindelinAug 18, 2006
  27. Jon SmirlAug 18, 2006
  28. Linus TorvaldsAug 19, 2006
  29. Nicolas PitreAug 18, 2006
  30. Shawn PearceAug 21, 2006
  31. Jon SmirlAug 21, 2006
  32. Nicolas PitreAug 21, 2006
  33. Jon SmirlAug 21, 2006
  34. Nicolas PitreAug 21, 2006
  35. Nicolas PitreAug 21, 2006
  36. Nicolas PitreAug 21, 2006

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.