Re: Incremental Backup of repositories using Git

From: Michal Suchánek <msuchanek@suse.de>
Date: May 9, 2025, 11:13 UTC
Message-ID: <aB3jYNAJAbKjSWAS@kitsune.suse.cz>
In-Reply-To: <PH7PR19MB70252D42F5D04FFC0331AB63C08E2@PH7PR19MB7025.namprd19.prod.outlook.com>

On Mon, May 05, 2025 at 02:35:43PM +0000, Akash S wrote:

Show 7 quoted lines

> Hi,
> 
> Currently we are backing up repositories using the "git clone -bare" command and save it to disk. If we want to restore, we just run git push -mirror from the repo that was saved during the backup.
> 
> Currently we are running full backups (run git clone -bare) everyday, which is taking a lot of disk space and time.
> 
> Are there any possible ways to backup only the incremental changes of a repository? And somehow construct the whole repository when we want to do a restore from the incremental backups?

Hello,

first, to make it easier to update the backup the clone should be done with --bare --mirror.

If your clone ends up having multiple packs and loose objects you can reduce its size with

git --git-dir=/path/to/clone repack -adk

This should give you a repository with a single pack and no loose objects.

The -k (or --cruft) option is required, using only -ad seems to corrupt repositories quite reliably.

To speed up the clone next time around you can make a copy of the previous backup and fetch from the remote repository but because there is no safe way I am aware of to eliminate no longer referenced objects you will accumulate cruft this way.

This is now a complete backup, and should be made readonly to not get corrupted with further operations.

The inrementeal backups are somewhat speculative, I have not tested this at all.

You can create a shared clone of the full backup, update the origin URL of the shared clone to the remote repository to backup, and do a fetch -p (which now should do the right thing because the initial clone was set up as mirror).

To repack you need to use the --local option in addition.

With this you should have a valid repository for each backup with the incremental backups sharing most objects with the full backup.

These can be inspected with git commands, exported over gitweb, or whatever.

Thanks

Michal