git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: A look at some alternative PACK file encodings

From
Jon Smirl <jonsmirl@gmail.com>
Date
Sep 7, 2006, 12:57 UTC
Message-ID
<9e4733910609070557jd8cfc57nd4f7a8973b69f6ed@mail.gmail.com>
In-Reply-To
<20060907090756.30111.qmail@science.horizon.com>
On 7 Sep 2006 05:07:56 -0400, linux@horizon.com <linux@horizon.com> wrote:
Show 11 quoted lines
> > Support for 'thin' packs would pretty much require mixing IDs and
> > (relative) offsets in the same pack file.
>
> An alternative would be to create a small "placeholder" object that
> just gives an ID, then refer to it by offset.
>
> That would avoid the need for an id/offset bit with every offset,
> and possibly save more space if the same object was referenced
> multiple times.
>
> And it just seems simpler.

There are 2 million objects in the Mozilla pack. This table would take: 2M * (20b (sha) + 10b(object index/overhead) = 60MB This 60MB is pretty much incompressible and increases download time.

Much better if storage of the sha1s can be totally eliminated and replaced by something smaller. Alternatively this map could be stripped for transmission and rebuilt locally.

-- 
Jon Smirl
jonsmirl@gmail.com
Previous: linux@horizon.comNext: linux@horizon.com
Message 2 of 10 in “Re: A look at some alternative PACK file encodings”
  1. linux@horizon.comSep 7, 2006
  2. Jon SmirlSep 7, 2006
  3. linux@horizon.comSep 7, 2006
  4. Jon SmirlSep 7, 2006
  5. linux@horizon.comSep 7, 2006
  6. Richard CurnowSep 7, 2006
  7. Junio C HamanoSep 7, 2006
  8. A Large Angry SCMSep 7, 2006
  9. Nicolas PitreSep 7, 2006
  10. linux@horizon.comSep 7, 2006

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.