threads / discuss / 8127

RFD Use git for off-site backups

Subject: RFD Use git for off-site backups

## tl;dr

5 messages between May 13, 2007 and May 13, 2007.

replies: 4people: 4as markdown or json

Thomas Glanzmann· May 13, 2007, 09:34 UTC · lore

Hello, I am thinking about using git for doing differential backups of a whole machines. And I would like to know what is necessary to obtain that. In such a scenario the machine to be backed up should have a local index file but sends its objects directly to the remote machine via git-via-ssh or git protocol. Is it possible to store extra information like extended attributes / acls in git? I don't care about named pipes, sockets and device files for the moment.

        Thomas
Johannes Schindelin· May 13, 2007, 11:16 UTC · re: Thomas Glanzmann · lore

Re: RFD Use git for off-site backups

Hi,
On Sun, 13 May 2007, Thomas Glanzmann wrote:
Show 5 quoted lines
> I am thinking about using git for doing differential backups of a whole
> machines. And I would like to know what is necessary to obtain that. In
> such a scenario the machine to be backed up should have a local index
> file but sends its objects directly to the remote machine via git-via-ssh
> or git protocol.

I think that this setup (have objects remotely) will kill performance pretty quickly. You have to compare to other blobs in order to find reasonable deltas...

> Is it possible to store extra information like extended attributes / 
> acls in git?

There has been talk about pre-commit scripts which put this information into a file, and a post-checkout script to resurrect these metadata, but I haven't seen functional versions of these scripts yet.

Ciao, Dscho

Thomas Glanzmann· May 13, 2007, 11:18 UTC · re: Johannes Schindelin · lore

Re: RFD Use git for off-site backups

Hello,
> I think that this setup (have objects remotely) will kill performance
> pretty quickly. You have to compare to other blobs in order to find
> reasonable deltas...
so true. I thought about that myself just after I sent the eMail.
> There has been talk about pre-commit scripts which put this
> information into a file, and a post-checkout script to resurrect these
> metadata, but I haven't seen functional versions of these scripts yet.

I thought there is a way to add information without loosing compability with current git versions. But maybe I am just mistaken.

        Thomas
Jan Hudec· May 13, 2007, 12:34 UTC · re: Thomas Glanzmann · lore

Re: RFD Use git for off-site backups

On Sun, May 13, 2007 at 11:34:17 +0200, Thomas Glanzmann wrote:
Show 7 quoted lines
> I am thinking about using git for doing differential backups of a whole
> machines. And I would like to know what is necessary to obtain that. In
> such a scenario the machine to be backed up should have a local index
> file but sends its objects directly to the remote machine via git-via-ssh
> or git protocol. Is it possible to store extra information like extended
> attributes / acls in git? I don't care about named pipes, sockets and
> device files for the moment.

I don't actually think git is a good match for whole-machine backups. It's strenghts are in distributedness and branching, but you don't need either of those for large-scale backups (monitoring changes say in /etc is a different thing -- git is sensible match there -- see http://www.isisetup.ch/).

I can suggest two solutions:
 - Use CVS on the client (machine to back up) and git-cvsserver on the
   server. You will probably be able to find some scripts that do backups to
   CVS. On server you will still have git for digging in history. Git is also
   very space efficient.
   CVS does not cache any data on the client. It only keeps metadata to
   detect which files were modified. Therefore it will have minimal overhead
   in storage, but needs to transfer the modified files in their entirety.
 - Use Subversion. There is summer of code project to create git-svnserver,
   so you might be able to use git on the server-side in future with
   subversion client too. But for backups you should not need any of the
   version control features subversion does not have, so subversion should be
   suitable.
   Subversion stores copy of the previous revision on the client, so all data
   under it's control will take twice as much space on client, but it will
   only transfer deltas over the network.
-- 
						 Jan 'Bulb' Hudec <bulb@ucw.cz>
Shawn O. Pearce· May 13, 2007, 23:42 UTC · re: Jan Hudec · lore

Re: RFD Use git for off-site backups

Jan Hudec <bulb@ucw.cz> wrote:
Show 5 quoted lines
>  - Use Subversion. There is summer of code project to create git-svnserver,
>    so you might be able to use git on the server-side in future with
>    subversion client too. But for backups you should not need any of the
>    version control features subversion does not have, so subversion should be
>    suitable.

Unfortunately Google is not sponsering the git-svnserver summer of code project. We didn't get enough slots for us to get Google to pay for that one. At least one of the students has still expressed interest in working on it, but it won't be their full-time job this summer. :-(

-- 
Shawn.

← back to recent threads