Re: getting list of objects for packing

From: Brandon Casey <casey@nrlssc.navy.mil>
Date: Nov 1, 2008, 00:00 UTC
Message-ID: <1S3xpaVP1Cy1Rei_ODwlXsBdu64BGiPve-lj_4fN6cA@cipher.nrlssc.navy.mil>
In-Reply-To: <fAUegZ3bxPo8HquZjUM9syW-giYefuAzBtb1XXHQ-TwbmvJvMZvmDA@cipher.nrlssc.navy.mil>

Brandon Casey wrote:

Show 21 quoted lines

> Junio C Hamano wrote:
>> Brandon Casey <casey@nrlssc.navy.mil> writes:
>>
>>> Nicolas Pitre wrote:
>>>> On Fri, 31 Oct 2008, Brandon Casey wrote:
>>>>>   -The sed statement is stripping off anything after the sha1. Any way to
>>>>>    get rev-list to print out just the sha1 so that sed is not necessary?
>>>> If you strip the data after the SHA1 when pipping into pack-objects then 
>>>> you'll have horrible delta compression results.  The path names after 
>>>> each SHA1 is used to sort objects when trying to find best matches for 
>>>> delta compression. So you should preserve those and feed it back 
>>>> especially with those packs that you still want delta compression for.
>>> Ah, I'll have to rethink my script then. Thanks!
>> Yeah, but wasn't the purpose of your whole exercise to list objects that
>> do not delta nor compress well with each other, in which case the delta
>> compression order (aka name hash) would not matter, no?
> 
> The script I wrote actually starts up two pack-objects instances and I was
> writing the objects I wanted to pack _normally_ to one, and the ones that I
> did not want compressed/deltafied to the other (which was started with
> --no-reuse-object --window=0 --depth=0 --compression=0).

So, my script created two pack files: one packed normally, and one packed without compression or delta. I removed my original packs, and put these two new ones in my pack directory and ran 'git fsck --full' and it completed successfully. There are no loose objects in the repo.

I added a .keep file for each pack.

Since my script removed the extra info from rev-parse's output, I removed the .keep file from the appropriate pack and ran 'git gc --aggressive'.

The 1.7GB pack that had the .keep file removed has been replaced with a +3GB pack file. The other pack file which still has the .keep file is 2.3GB.

In another repo with 3 packs marked .keep, and one 388KB pack with ~300 objects in it, and 3 loose dangling objects, the 388KB pack was replaced with a 3.5GB pack.

It appears that the entire repository is being packed into the new pack file even though there are existing pack files with .keep files.

If I compare the output from 'git verify-pack -v' I can see that many of the objects in the packs marked with a .keep file are indeed in the new pack file. But not all of them.

-brandon