git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: Local clones aka forks disk size optimization

From
Enrico Weigelt <enrico.weigelt@vnc.biz>
Date
Nov 16, 2012, 18:04 UTC
Message-ID
<ccb8680a-97ad-4cf8-95d0-5e21f60494f4@zcs>
In-Reply-To
<50A622A9.4040709@drmicha.warpmail.net>
> Provide one "main" clone which is bare, pulls automatically, and is
> there to stay (no pruning), so that all others can use that as a
> reliable alternates source.

The problem here, IMHO, is the assumption, that the main repo will never be cleaned up. But what to do if you dont wanna let it grow forever ?

hmm, distributed GC is a tricky problem.
maybe it could be easier having two kind of alternates:
a) classical: gc+friends will drop local objects that are 
   already there
b) fallback: normal operations fetch objects if not accessible
   from anywhere else, but gc+friends do not skip objects from there.

And extend prune machinery to put some backup of the dropped objects to some separate store.

This way we could use some kind of rotating archive:
* GC'ed objects will be stored in the backup repo for some while
* there are multiple active (rotating) backups kept for some time,
  each cycle, only the oldest one is dropped (and maybe objects
  in a newer backup are removed from the older ones)
* downstream repos must be synced often enough, so removed objects
  are fetched back from the backups early enough
You could see this as some heap:
* the currently active objects (directly referenced) are always
  on the top
* once they're not referenced, they sink a lever deeper
* when the're referenced again, they immediately jump up to the top
* at some point in time unreferenced objects sink too deep that
  they're dropped completely
cu
-- 
Mit freundlichen Grüßen / Kind regards 

Enrico Weigelt 
VNC - Virtual Network Consult GmbH 
Head Of Development 

Pariser Platz 4a, D-10117 Berlin
Tel.: +49 (30) 3464615-20
Fax: +49 (30) 3464615-59

enrico.weigelt@vnc.biz; www.vnc.de 
Previous: Michael J GruberNext: Sitaram Chamarty
Message 9 of 13 in “Fwd: Local clones aka forks disk size optimization”
  1. Javier DomingoNov 14, 2012
  2. Andrew ArdillNov 15, 2012
  3. Javier DomingoNov 15, 2012
  4. Andrew ArdillNov 15, 2012
  5. Javier DomingoNov 15, 2012
  6. Andrew ArdillNov 15, 2012
  7. Sitaram ChamartyNov 15, 2012
  8. Michael J GruberNov 16, 2012
  9. Enrico WeigeltNov 16, 2012
  10. Sitaram ChamartyNov 18, 2012
  11. Enrico WeigeltNov 18, 2012
  12. Pyeron, Jason J CTR (US)Nov 16, 2012
  13. Jörg RosenkranzNov 18, 2012

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.