From: Roman Shaposhnik Date: Thu, 03 Apr 2008 19:42:39 GMT Subject: Achieving efficient storage of weirdly structured repos Message-ID: <7BE3E865-C30D-49B8-A1D9-898109514990@sun.com> Hi Guys! First of all, I must admit that I got seriously hooked up on Git when I started to use it internally for FFmpeg development and my life as far as non company related projects are concerned hasn't been the same ever since. The way Git addresses issues like branches (both local and remote), merges and content tracking are, in my opinion, quite superior to the alternative SCMs such as Mercurial. In fact, that's why when I saw my coworkers struggling with exactly the same issues around NetBeans Mercurial repository my natural instinct was to introduce them to Git and show them that life doesn't have to be that complicated or troublesome. But before I can do that, there's a little issue that I either need to address or at least understand better. It seems that the storage requirements for the repository that a friend of mine and I created from: http:// hg.netbeans.org/main/ got doubled under Git: $ du -sh Mercurial- nb-main 491M Mercurial- nb-main $ du -sh Git-nb-main 1.1G Git-nb-main The repository was created using hg2git (the one based on git-fast- import) and it was GC'ed and REPACK'ed just in case. I spent some time trying to understand the issue and here's some statistics on the kind of objects that this repository consists of: 75053 commit 334572 blob 750003 tree The last item (trees) also seem to take the most space and the most reasonable explanation that I can offer is that NetBeans repository has a really weird structure where they have approximately 700 (yes, seven hundred!) top- level subdirectories there. They are clearly Submodules-shy, but that's another issue that I will need to address with them. So based on my preliminary analysis the biggest culprit seems to be that for every commit at least one pretty hefty tree object needs to be created (remember 700 top level subdirectories). These objects are all in 18-20K range but they are almost identical to each other except for one or two trees that they refer to. So here's my question: is there anything in Git that I can use to accommodate a repository like http://hg.netbeans.org/main in an efficient manner? Any kind of ideas (like particular command line options for repack, etc.) would be greatly appreciated! Thanks, Roman.