Re: Incremental Backup of repositories using Git

> On Thu, May 08, 2025 at 08:47:47PM +0200, Michal Suchánek wrote:
> 
> > If you have one of those filesystems that support deduplication on
> > filesystem level you could make each snapshot as a full repository with
> > all objects unpacked, and the filesystem would deduplicate the objects
> > for you.
> > 
> > The downside is that you have no way to do multiple full backups this
> > way, and you would have to use something else for that (such as those
> > bundles, or plain archiving the repository as files in a tar archive or
> > such.
> 
> This is tempting, but I suspect that storing the objects unpacked will
> become unfeasibly large, because you are missing out on delta
> compression in the packfiles. You can compare the on-disk and
> uncompressed sizes of objects in a repo like this:
> 
>   git cat-file --batch-all-objects --unordered \
>                --batch-check='%(objectsize:disk) %(objectsize)' |
>   perl -alne '
>     $disk += $F[0];
>     $true += $F[1];
>     END {
>       print "$true / $disk = ", int($true / $disk);
>     }
>   '
> 
> It's not entirely fair because the "true" size is missing out on zlib
> compression that loose objects would get. But that's at best going to be
> about 4:1 (and in practice worse, since trees are full of sha1 hashes
> that don't compress very well).
> 
> In my copy of linux.git, that yields ~135G versus ~2.4G, for a factor of
> 56. Even if we grant 4:1 compression from zlib, that's still inflating
> your on-disk repository by a factor of 14.