RE: Incremental Backup of repositories using Git

From: Abhishek Dalmia <adalmia@commvault.com>
Date: May 6, 2025, 12:44 UTC
Message-ID: <SJ1PR19MB6401D7734B73C453E491D54DAE89A@SJ1PR19MB6401.namprd19.prod.outlook.com>
In-Reply-To: <2dz3cema2mr5mrlvuroemnyeqyrglxfmusfdz2kaghv6rvj3ro@ti2dhu45fdmr>

Hi Justin

(My previous email got blocked due to HTML content)

Thanks for the recommendation. We want to backup all the repo contents, so could you please comment if the following steps will help us backup and restore everything, or we might miss some tags/references?

During backup:
- Create full bundle first time using: git bundle create <full-bundle-file-path> --all
- Create further incremental bundles using: git bundle create <inc-bundle-file-path> --since="<last-backup-time>" -all
	- making sure we don't miss out any time

During restore:
- Create the initial repo with: git clone -bare <full-bundle-file-path> - using the full bundle we created earlier
- For restoring further incremental bundle files
	- git fetch <inc-bundle-file-path> 'refs/*:refs/*'
	- I can't use --all here, that works only with remote repos

Will using 'refs/*:refs/*' restore everything, or is it possible any git data might get missed out?

Regards, Abhishek

-----Original Message-----
From: Justin Tobler <jltobler@gmail.com> 
Sent: 05 May 2025 21:49
To: Akash S <akashs@commvault.com>
Cc: git@vger.kernel.org; Adithya Urugudige <aurugudige@commvault.com>; Abhishek Dalmia <adalmia@commvault.com>
Subject: Re: Incremental Backup of repositories using Git

[Some people who received this message don't often get email from jltobler@gmail.com. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ]

External email. Inspect before opening.

On 25/05/05 02:35PM, Akash S wrote:

Show 7 quoted lines

> Hi,
>
> Currently we are backing up repositories using the "git clone -bare" command and save it to disk. If we want to restore, we just run git push -mirror from the repo that was saved during the backup.
>
> Currently we are running full backups (run git clone -bare) everyday, which is taking a lot of disk space and time.
>
> Are there any possible ways to backup only the incremental changes of a repository? And somehow construct the whole repository when we want to do a restore from the incremental backups?

You could look into using git-bundle(1) to create incremental bundles using exclusions. Examples:

  # Creates a bundle containing the last 10 commits for main.
  $ git bundle create inc-backup main~10..main

  # Creates incremental bundle based on time for all references.
  $ git bundle create inc-backup --all --since=7.days

These bundles can then be "unbundled" into a repository as long as the repo contains the required prerequisite objects.

-Justin

>
> Thanks,
> Akash
>