Re: how to reduce disk usage for large .git dirs?

From: Jeff King <peff@peff.net>
Date: Nov 13, 2014, 20:15 UTC
Message-ID: <20141113201542.GC3869@peff.net>
In-Reply-To: <CALKQrgeZYs9A-GZLuRczwzRWWapmfrjFvcvR8GN+YNKxajjDRw@mail.gmail.com>

On Thu, Nov 13, 2014 at 05:08:19PM +0100, Johan Herland wrote:

Show 12 quoted lines

> Can you not do this much simpler with --reference? Like this:
> 
>   $ git clone --bare git://host/repo.git repo-master
>   $ git clone -b branchA --reference repo-master git://host/repo.git
> repo-branchA
>   $ git clone -b branchB --reference repo-master git://host/repo.git
> repo-branchB
> 
> All three repos now push/pull directly to/from git://host/repo.git,
> but repo-branchA and repo-branchB reference objects from within the
> bare repo-master. You have to make use to never delete objects from
> repo-master

I think the "never delete" part is why we usually warn people off of using alternates. I think at the least you would have to "git config gc.auto 0" in the bare repository (otherwise your nightly fetches risk pruning). Of course you'd probably want to repack eventually for performance reasons. So maybe setting gc.pruneExpire is a better option (to something like "20.years.ago").

Show 5 quoted lines

> If you want to prevent the repos growing in size, you must devise a
> way to add new objects into repo-master before repo-branchA|B. (e.g. a
> nightly cron-job in repo-master that fetches from origin), so that
> when repo-branchA|B pulls, they will find most objects are already
> present in repo-master.

You can also fetch from the children into repo-master periodically. Like:

  cd repo-master &&
  for i in branchA branchB; do
    git fetch ../$i +refs/*:refs/remotes/$i/*
  done

after which it is actually safe to run "git gc" in the master (assuming there isn't simultaneous activity in the children). This is how we manage fork networks on GitHub (we take in objects to individual forks via push, and then migrate them to the master repo via fetch).

-Peff