Mozilla version control requirements and git

The Mozilla people have a web page describing what they are looking for in a new version control system. How does git stack up?

http://wiki.mozilla.org/Version_Control_System_Requirements

They have also recently put up a Subversion server for limited use.

http://weblogs.mozillazine.org/preed/2006/08/subversive_subversion_conversi.html

On 9/3/06, Jon Smirl <jonsmirl@gmail.com> wrote:

Hi Jon,

you've been playing with GIT quite a bit by now, so I guess you know the answer ;-) Is there anything in particular you are wondering about?

The one item that sticks out for me as not there is ACLs, but access controls can be implemented in hooks for direct pushes. Maintain an .htaccess-like file and have a perl script to check it on pushes to the repo.

Oh, and "partial tree pulls for localisers". Perhaps git-cvsserver can help there? Localisers can just use TortoiseCVS and get a checkout of the language pack subdir.

cheers,

martin

On 9/2/06, Martin Langhoff <martin.langhoff@gmail.com> wrote:

Shallow clones are a big problem. We have Mozilla down to 450MB is git, but that is still gigantic for anyone doing an initial check out, especially if they don't have good broadband. It is over 10x the data that CVS brings down. Even without doing shallow clones git still needs to be modified to restart an interrupted pack transfer.

I don't see ACLs as that big an issue. If you convert to model where everyone has their own repo and you just push changesets at each other, ACLs are much less important. You need ACLs when 2,000 people have commit privs.

Partial repo pulls and an issue to. The mozilla repo has much more than a browser in it, it also has a large mail/news program. A partial repo pull may not be what is needed for git, instead git needs a partial repo checkout.

Jon Smirl wrote:

Or better support for subprojects.

I think you can fake subproject support using separate repositories for subprojects, in the directory matching the one in the whole project, and the project repository, where we pull/push subprojects into.

On 9/3/06, Jakub Narebski <jnareb@gmail.com> wrote:

You can't really fake it. Many of the projects in Mozilla are dependent on each other. If you break them up into separate repositories you lose the ability to do a cross project commit. This is common when you are changing an interface between the subprojects.

Would it make sense for git to treat a tree of projects like you describe as a single entity? So a commit would transparently commit to all of the subprojects. Maybe the commit entry's sha1 could be set up so that it would be identical in all subprojects. That would let you string them back together.

On 03-09-2006 15:46, Jon Smirl wrote:

Scratch that. You can fake subproject support using something similar to the way Junio works with TODO branch. Namely, separate repositories for subprojects, each as if it would start from it's own directory (although you can manage without that, it only would be slightly harder to set it up), and main repository working directory composed from subproject repositories checked out to correct place. Add to this ingnoring object database (i.e. .git repositories), and you have whole repository... where you can do whole repository commits to.

E.g. the following directory structure

	main-project/
	main-project/.git
	main-project/subproject/
	main-project/subproject/.git
	main-project/subproject/...subproject files...

where second .git is in .gitignore or info/excludes, but the subrpoject/ directory itself isn't.

If this is not enough, check out ideas in Subpro.txt in the 'todo' branch
in git repository. It adds additional 'bind' header... the implementation
(search for 'gitlink') were if I remember correctly never fully implemented
and is not in git, although parts of it like git-read-tree and 
git-write-tree --prefix option made it.
  http://www.kernel.org/git/?p=git/git.git;a=blob;f=Subpro.txt;hb=todo

As commit id depends on the contents, including _parents_ and _tree_, it is I think impossible to set commit entry sha1 to be identical.

P.S. I just hope that it would pass VGER filter. I don't have much luck with it lately...

On 9/4/06, Jon Smirl <jonsmirl@gmail.com> wrote:

In many projects this is handled by having tags and generally versioning modules and interfaces. Being independent gives the subprojects/modules a lot more freedom to work/branch on wild tangents, and the versioned interfaces mean that -- with the contraints of the versioned interfaces -- you can mix and match branches/releases of the different subprojects.

In short, I'd definitely break those up :-)

cheers,

martin

On 9/3/06, Martin Langhoff <martin.langhoff@gmail.com> wrote:

Mozilla always ships as a complete system so there is no need to deal with old versions of the pieces. Only the external interfaces are frozen and versioned. This is similar to the Linux kernel. The user space API is frozen, but when an internal device driver interface changes all of the drivers in the kernel tree get updated in lock step. Doing it this way removes a lot of overhead in maintaining binary compatibility.

"Jon Smirl" <jonsmirl@gmail.com> writes:

That does not contradict with the underlying data model of git, and it would certainly be possible with a specialized Porcelain.

Since git tracks the entire tree, the tree object contained in the commit you create _must_ describe the entire project tree, but that does not necessarily mean that your working tree needs to have the entire tree checked out.

The git corelevel is designed to be operable with sparsely populated working tree, and the problem you are seeing is primarily coming from the fact that some wrappers you are seeing as "git UI" are specifically done to help workflows that _wants_ to have fully populated working tree (e.g. "git commit -a" and "git checkout"). There are other wrappers that are designed to work well in sparsely populated working tree, including "git-am" and "git-merge". They should be usable pretty much as they are.

To support "narrow checkout" workflow, some wrappers either need to be told about the "interesting area", or a set of different wrappers specifically designed to support "narrow checkout" workflow can be written. I am not sure which is more desirable. I have a gut feeling that "git commit" can be used as is (you cd into the area you are interested in, and say "git commit ." instead of saying "git commit -a"), while it would be less confusing to keep "git checkout" the whole-tree operation and have a specialized porcelain for a narrow checkout, but this is me saying things without thinking things through.

I imagine such a "narrow checkout" workflow Porcelain would work like this:

 - the user tells it which parts of the whole project she is
   interested in.

 - when checking things out, the Porcelain populates the index
   in full [*1*] and then checks out only specified subtrees to
   the working tree.

 - when checking things in, the changes outside the specified
   area are ignored (well, outside there should be no files in
   working tree so ignoring them should be trivial).  write-tree
   writes from the whole index and creates the tree object and
   then commit object just as usual.  The net effect is that the
   parts outside the areas of interest stay the same as the
   previous HEAD commit, which is exactly what the user wanted
   anyway.

You could optimize the above further by populating the index sparsely as well (i.e. *1* above is not a requirement), but you would need to merge in the changes made to that sparse index into the full tree read from the HEAD commit before writing the next tree object out anyway, so I doubt it is worth it.