Re: [RFC] Support projects including other projects

I was thinking about this recently while reading an article on bittorrent and how it works and it occured to me that perhapse the network access model of git should be reexamined.

git produces a large pool of objects, there are two ways that people want to access these objects.

1. pull the current version of a project (either a straight 'ckeckout' 
type pull or a 'merge' to a local project)

2. pull the objects nessasary for past versions of a project (either all 
the way back to the beginning of time or back to some point, that point 
being a number of possibilities (date, version, things you don't have, 
etc)

in either case the important thing that's key are the indexes related to a particular project, the objects themselves could all be in one huge pool for all projects that ever existed (this doesn't make sense if you use rsync to copy repositories as Linux origionally did, but if you have a more git-aware transport it can make sense)

I believe that there are going to be quite a number of cases where the same object is used for multiple projects (either becouse the project is a fork of another project or becouse some functions (or include files) are so trivial that they are basicly boilerplate and get reused or recreated) if you think about a major mirror server distributing a dozen linux distros via git you will realize that in many cases the source files, scripts, and (in many cases) even the binaries are really going to be identical objects for all the distros so a ftp/http server that used a git filesystem could result in a pretty significant saveings in disk space.

In addition, when you are doing a pull you can accept data from non-authoritative sources since each object (and it's index info) includes enough info to validate the object hasn't been tampered with (at least until such time as the hashes are sufficiantly broken, but that's another debate, and we had that one :-). so a bittorrent-like peer sharing system to fetch objects identified by the index files would open the potential for saving significant bandwith on the master servers while not comprimising the trees at all.

Going back (somewhat) to the subject at hand, with something like this you should be able to combine as many projects as you want in one repository, and the only issue would be the work nessasary to go through that repository and all the index files that point at it when you want to prune old data out of the object pool to save disk space.

thoughts? unfortunnatly I don't have the time to even consider codeing something like this up, but hopefully it will spark interest for someone who does.

David Lang