Re: [PATCH] RFC: git lazy clone proof-of-concept

From: Brian Downing <bdowning@lavos.net>
Date: Feb 14, 2008, 23:57 UTC
Message-ID: <20080214235747.GV27535@lavos.net>
In-Reply-To: <20080214235129.GU27535@lavos.net>

On Thu, Feb 14, 2008 at 05:51:29PM -0600, Brian Downing wrote:

Show 6 quoted lines

> Do you by chance have repack.usedeltabaseoffset turned on?  That has the
> unfortunate side effect of changing the output of verify-pack -v to be
> almost useless for my packinfo script (specifically, it no longer
> reports the parent SHA1 hash for deltas, and the script is basically all
> about deltra tree statistics.)  I suppose that should probably be fixed,
> but I never looked into it.

That being said, the most useful output for figuring out where all the space in the pack is going in my experience is gotten from:

git-verify-pack -v | packinfo.pl -tree -filenames

That will produce a huge amount of output, which is basically the tree structure of the delta chains in the file. If things aren't being deltified together properly, it's usually pretty obvious.

A delta chain in this output looks approximately like this:

# 0 blob 03156f21... 1767 1767 Documentation/git-lost-found.txt @ tags/v1.2.0~142 # 1 blob f52a9d7f... 10 1777 Documentation/git-lost-found.txt @ tags/v1.5.0-rc1~74 # 2 blob a8cc5739... 51 1828 Documentation/git-lost+found.txt @ tags/v0.99.9h^0 # 3 blob 660e90b1... 15 1843 Documentation/git-lost+found.txt @ master~3222^2~2 # 4 blob 0cb8e3bb... 33 1876 Documentation/git-lost+found.txt @ master~3222^2~3 # 2 blob e48607f0... 311 2088 Documentation/git-lost-found.txt @ tags/v1.5.2-rc3~4 # size: count 6 total 2187 min 10 max 1767 mean 364.50 median 51 std_dev 635.85 # path size: count 6 total 11179 min 1767 max 2088 mean 1863.17 median 1843 std_dev 107.26

# The first number after the sha1 is the object size, the second # number is the path size. The statistics are across all objects in # the previous delta tree. Obviously they are omitted for trees of # one object.

# A path size is the sum of the size of the delta chain, including the # base object. In other words, it's how many bytes need be read to # reassemble the file from deltas.

This is also quite slow, as it runs git-ls-tree -t -r on every commit in the repository to assign file names to blobs. You can leave out the -filenames option to not do this (if you don't care about seeing filenames, that is).

-bcd