git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: A look at some alternative PACK file encodings

From
A Large Angry SCM <gitzilla@gmail.com>
Date
Sep 6, 2006, 23:39 UTC
Message-ID
<44FF5C27.2040300@gmail.com>
In-Reply-To
<9e4733910609061623k73086dbey4a600ecf2852c024@mail.gmail.com>
Jon Smirl wrote:
Show 6 quoted lines
> On 9/6/06, A Large Angry SCM <gitzilla@gmail.com> wrote:
>> TREE objects do not delta or deflate well.
> 
> I can understand why they don't deflate, the path names are pretty
> much unique and the sha1s are incompressible. By why don't they delta
> well? Does sorting them by size mess up the delta process?

My guess would be the TREEs would only delta well against other TREE versions for the same path.

> Shawn is doing some prototype work on true dictionary based
> compression. I don't know how far along he is but it has potential for
> taking 30% off the Mozilla pack.

Just looking at the structures in non-BLOBS, I see a lot of potential for the use of a set dictionaries when deflating TREEs and another set of dictionaries when deflating COMMITs and TAGs. The low hanging fruit is to create dictionaries of the most referenced IDs across all TREE or COMMIT/TAG objects.

Previous: Jon SmirlNext: Linus Torvalds
Message 3 of 24 in “A look at some alternative PACK file encodings”
  1. A Large Angry SCMSep 6, 2006
  2. Jon SmirlSep 6, 2006
  3. A Large Angry SCMSep 6, 2006
  4. Linus TorvaldsSep 6, 2006
  5. Jon SmirlSep 7, 2006
  6. David LangSep 7, 2006
  7. A Large Angry SCMSep 7, 2006
  8. Linus TorvaldsSep 7, 2006
  9. Nicolas PitreSep 7, 2006
  10. Jon SmirlSep 7, 2006
  11. Shawn PearceSep 7, 2006
  12. Shawn PearceSep 7, 2006
  13. Nicolas PitreSep 7, 2006
  14. Jon SmirlSep 7, 2006
  15. Nicolas PitreSep 7, 2006
  16. A Large Angry SCMSep 7, 2006
  17. Junio C HamanoSep 7, 2006
  18. Nicolas PitreSep 7, 2006
  19. Shawn PearceSep 7, 2006
  20. Junio C HamanoSep 7, 2006
  21. Shawn PearceSep 7, 2006
  22. Junio C HamanoSep 7, 2006
  23. Shawn PearceSep 7, 2006
  24. A Large Angry SCMSep 7, 2006

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.