git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH/RFC] index-pack: produce pack index version 3

From
Junio C Hamano <gitster@pobox.com>
Date
Aug 16, 2012, 16:26 UTC
Message-ID
<7vwr0y247a.fsf@alter.siamese.dyndns.org>
In-Reply-To
<CAJo=hJtgq7j67RU28ziQN4xUX7S5-YcaiApmcpMSYdamV-0SgQ@mail.gmail.com>
Shawn Pearce <spearce@spearce.org> writes:
Show 19 quoted lines
> On Wed, Aug 15, 2012 at 10:42 PM, Junio C Hamano <gitster@pobox.com> wrote:
> ...
>> An obvious way to record the "delta chain" is to simply keep the
>> name_hash of each object in the pack, which would need 2 bytes per
>> object in the pack, that would bloat pack_idx_entry size from 32
>> bytes to 34 bytes per entry.  That way, after your bitmap discovers
>> an object that cannot reuse existing deltas, you can throw it, other
>> such objects with the same name-hash, and then objects that you know
>> will be available to the recipient (you mark the last category of
>> objects as "preferred base"), into the delta_list so that they are
>> close together in the delta window.
>
> Yes, this is one thought I had. Inside of JGit I think the name hash
> is 32 bits, not 16 bits. Storing the name hash into the *.idx file
> means we need to codify what the name hash algorithm is for a given
> *.idx file version, and compatible implementations of Git must use the
> same hash function. Thus far the name hash has been an in-memory
> transient concept that doesn't need to be persisted across runs of the
> packer. Storing it means we have to do that.

Let's not go there. We cannot resurrect the name hash out of *.pack stream, which means index-pack cannot recreate it after receiving objects over the network. We would need to instead teach index-pack to observe the delta chains, and come up with some "delta chain identifier" (unique name to identify what you called "delta cluster" in your response) on its own, and give it to each object when it writes the *.idx file out.

Previous: Shawn Pearce
Message 10 of 10 in “index-pack: produce pack index version 3”
  1. index-pack: produce pack index version 3Nguyễn Thái Ngọc Duy, Aug 12, 2012
  2. Junio C HamanoAug 12, 2012
  3. Junio C HamanoAug 12, 2012
  4. Nguyen Thai Ngoc DuyAug 13, 2012
  5. Junio C HamanoAug 13, 2012
  6. Shawn PearceAug 14, 2012
  7. Nguyen Thai Ngoc DuyAug 14, 2012
  8. Junio C HamanoAug 16, 2012
  9. Shawn PearceAug 16, 2012
  10. Junio C HamanoAug 16, 2012

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.