Re: Huge performance bottleneck reading packs

On 10/13/2016 12:45 AM, Junio C Hamano wrote:
 > Vegard Nossum <vegard.nossum@oracle.com> writes:
 >
 >> A closer inspection reveals the problem to really be that this is an
 >> extremely hot path with more than -- holy cow -- 4,106,756,451
 >> iterations on the 'packed_git' list for a single 'git fetch' on my
 >> repository. I'm guessing the patch above just made the inner loop
 >> ever so slightly slower.
 >
 > Very plausible, and this ...
 >
 >> My .git/objects/pack/ has ~2088 files (1042 idx files, 1042 pack files,
 >> and 4 tmp_pack_* files).
 >
 > ... may explain why nobody else has seen a difference.
 >
 > Is there a reason why your repository has that many pack files?  Is
 > automatic GC not working for some reason?

Oops. I disabled gc a while ago; one reason I did that is that it takes a long time to run and it has a tendency to kick in at the worst time. I guess I should really put it in cron then.

I'm not sure if this is related, but I also had a problem with GitPython and large pack files in the past (" ValueError: Couldn't obtain fanout table or warning: packfile ./objects/pack/....pack cannot be accessed") and I have pack.packSizeLimit set to 512m to fix that. Although the whole repo is 17G so I guess it shouldn't be necessary to have that many pack files.

Will try Jeff's patch, then a gc. Thanks!

Vegard