git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: A look at some alternative PACK file encodings

From
A Large Angry SCM <gitzilla@gmail.com>
Date
Sep 7, 2006, 00:19 UTC
Message-ID
<44FF6586.8080206@gmail.com>
In-Reply-To
<Pine.LNX.4.64.0609061651500.27779@g5.osdl.org>
Linus Torvalds wrote:
Show 18 quoted lines
> 
> On Wed, 6 Sep 2006, A Large Angry SCM wrote:
> 
>> Jon Smirl wrote:
>>> On 9/6/06, A Large Angry SCM <gitzilla@gmail.com> wrote:
>>>> TREE objects do not delta or deflate well.
>>> I can understand why they don't deflate, the path names are pretty
>>> much unique and the sha1s are incompressible. By why don't they delta
>>> well? Does sorting them by size mess up the delta process?
>> My guess would be the TREEs would only delta well against other TREE
>> versions for the same path.
> 
> That's what you'd normally have in a real project, though. I wonder if 
> your "pack mashup" lost the normal behaviour: we very much sort trees 
> together normally, thanks to the "sort-by-filename, then by size" 
> behaviour that git-pack-objects should have (for trees, the size normally 
> shouldn't change, so the sorting should basically boil down to "sort the 
> same directory together, keeping the ordering it had from git-rev-list").

The mashup is just all the projects in a single repository with a bushy refs tree so I can view the updates in a single gitk window.

The sorting by name, then by path may be breaking the object version relationship for wide graphs.

> Btw, that "keeping the ordering it had" part I'm not convinced we actually 
> enforce. That would depend on the sort algorithm used by "qsort()", I 
> think. So there might be room for improvement there in order to keep 
> things in recency order.
qsort() is not stable.
Show 8 quoted lines
>> Just looking at the structures in non-BLOBS, I see a lot of potential
>> for the use of a set dictionaries when deflating TREEs and another set
>> of dictionaries when deflating COMMITs and TAGs. The low hanging fruit
>> is to create dictionaries of the most referenced IDs across all TREE or
>> COMMIT/TAG objects.
>
> Is there any way to get zlib to just generate a suggested dictionary from 
> a given set of input?
The docs suggest "no".
Previous: David LangNext: Linus Torvalds
Message 7 of 24 in “A look at some alternative PACK file encodings”
  1. A Large Angry SCMSep 6, 2006
  2. Jon SmirlSep 6, 2006
  3. A Large Angry SCMSep 6, 2006
  4. Linus TorvaldsSep 6, 2006
  5. Jon SmirlSep 7, 2006
  6. David LangSep 7, 2006
  7. A Large Angry SCMSep 7, 2006
  8. Linus TorvaldsSep 7, 2006
  9. Nicolas PitreSep 7, 2006
  10. Jon SmirlSep 7, 2006
  11. Shawn PearceSep 7, 2006
  12. Shawn PearceSep 7, 2006
  13. Nicolas PitreSep 7, 2006
  14. Jon SmirlSep 7, 2006
  15. Nicolas PitreSep 7, 2006
  16. A Large Angry SCMSep 7, 2006
  17. Junio C HamanoSep 7, 2006
  18. Nicolas PitreSep 7, 2006
  19. Shawn PearceSep 7, 2006
  20. Junio C HamanoSep 7, 2006
  21. Shawn PearceSep 7, 2006
  22. Junio C HamanoSep 7, 2006
  23. Shawn PearceSep 7, 2006
  24. A Large Angry SCMSep 7, 2006

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.