Re: Using git as a general backup mechanism

From: Junio C Hamano <junkio@cox.net>
Date: Dec 12, 2006, 23:43 UTC
Message-ID: <7vfybkof3s.fsf@assigned-by-dhcp.cox.net>
In-Reply-To: <457F31E6.8090701@midwinter.com>

Steven Grimm <koreth@midwinter.com> writes:

Show 11 quoted lines

> What would be great for this would be to store each day's backup as a
> git revision; with a periodic repack, this would be much more
> space-efficient than the rsync hard links.
>
> The problem is that while that would give me a very efficient backup
> scheme, the repository would still grow over time. In rsync land, I
> solve the disk space issue by keeping two weeks' worth of daily
> snapshots, then six months' worth of weekly snapshots, then two years'
> worth of monthly snapshots; files that change daily have a constant
> number of revisions stored in my backups, and older files drop off the
> backup disk as they age.

Why not use N independent branches? I'd illustrate only with two levels below, but you could:

 (0) make a full tree snapshot.  Store the commit in 'daily'
     branch as its tip.

 (1) A new day comes.  Create an empty branch 'daily' if you
     do not already have one.  Make a full tree snapshot, and
     create a parentless commit for the day if the 'daily'
     branch did not exist, or make it a child of the 'daily'
     commit from the previous day if the branch existed.

 (2) End of week comes.  Create an empty branch 'weekly' if you
     do not already have one.  Make a full tree snapshot, and
     create a parentless commit for the week if the 'weekly'
     branch did not exist, or make it a child of the 'weekly'
     commit from the last week.  Discard 'lastweek' branch if
     you have one, and rename 'daily' branch to 'lastweek'.

At the end of month, you can rename 'weekly' to 'lastmonth'; if you discard previous 'lastmonth' at this point, you essentially made files older than two months drop off the backup disk. You can add more hierarchy with longer period to extend the scheme ad infinitum.