git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: proposal: delta based git archival

From
JMJaime Medrano <overflow5@gmail.com>
Date
Apr 22, 2005, 09:49 UTC
Message-ID
<b008c6a40504220249477e70ae@mail.gmail.com>
In-Reply-To
<20050422090341.GC22479@zoy.org>
On 4/22/05, Michel Lespinasse <walken@zoy.org> wrote:
Show 5 quoted lines
> I noticed people on this mailing list start talking about using blob deltas
> for compression, and the basic issue that the resulting files are too small
> for efficient filesystem storage. I thought about this a little and decided
> I should send out my ideas for discussion.
> 
I've been thinking in another simpler approach.

The main benefit of using deltas is reducing the bandwith use in pull/push. My idea is leaving the blob storage as it is by now and adding a new kind of object (remote) that acts as a link to an object in another repository.

So that, when you rsync, you don't have to get all the blobs (which can be a lot of data), but only the sha1 of the new objects created. Then a remote object is created for each new object in the local repository pointing to its location in the external repository.

Once the rsync is done, when git has to access any of the new objects they can be fetched from the original location, so that only necessary objects are transfered.

This way, the cost of a sync in terms of bandwith is nearly zero.

I've been working on this, so if you think it to be a good idea, I can send a patch when I get it fully working.

Regards, Jaime Medrano. http://jmedrano.sl-form.com

Previous: Jeffrey E. Hundstad
Message 3 of 3 in “proposal: delta based git archival”
  1. Michel LespinasseApr 22, 2005
  2. Jeffrey E. HundstadApr 22, 2005
  3. Jaime MedranoApr 22, 2005

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.