Re: git packs

From: Nicolas Pitre <nico@cam.org>
Date: Nov 12, 2007, 04:21 UTC
Message-ID: <alpine.LFD.0.9999.0711112307070.21255@xanadu.home>
In-Reply-To: <B298202C-3D54-498D-A348-0338914FBA46@mac.com>

On Sun, 11 Nov 2007, bob wrote:

Show 7 quoted lines

> I applied the patch and these commands:
> 
> cd rmwHtmlOld
> rm -fr .git
> git init
> git config core.compression 0
> git add .

Note that I did "git config core.compression 0" simply to disable zlib compression altogether when creating the test repo just so it gets created faster. even then, auto-generating and cloning a 8GB test repository isn't particularly quick.

> I then got the same error as before, "Bus error".  Rats!

Do you get that with a 32-bit or 64-bit build of Git?

> Then I modified your script since I do not have seq or
> your test-genrandom.

test-genrandom is built with Git.  It is just not installed anywhere.

Show 5 quoted lines

> I substituted:
> 
> dd count=XX  if=/dev/random of=file_$i
> 
> where XX is adjusted to meet dd's requirements.  Also,

Again I used test-genrandom instead of /dev/random or /dev/urandom simply because the former is much faster.

Show 10 quoted lines

> I found after searching for a while, that the following
> works just like your seq command:
> 
> xyzzy="1 2 3 4"
> for i in $xyzzy
> do
> ...
> done
> 
> Your script then ran flawlessly.

However 'seq -w 1 2 63' should be replaced with "01 03 05 07 09 11 13 15" and so on up to 63, and 'seq -w 2 2 64' is "02 04 06 08 10 12 16" and so on.

Show 5 quoted lines

> I looked through index-pack.c some more, but it is
> very hard to figure it out without doing a lot of research
> since there doesn't seem to be anything that describes
> the layout of a pack.  The link towards the end of the user's
> manual doesn't work for me.

Look at Documentation/technical/pack-format.txt in the Git source tree.

> The difference between your test and my data is that
> instead of having a few large files, I have 11,500 files
> of varying sizes.  On average though, the file size is
> about 370k.

Are you saying that the test repo with big files works for you but not your own data set?

Would you please recap what your problem is?

With my one line patch you should not get the "serious inflate inconsistency" error anymore. The bus error must be another issue.

Nicolas