Re: upload-pack is slow with lots of refs
- From
Ævar Arnfjörð Bjarmason <avarab@gmail.com>
- Date
- Oct 3, 2012, 22:32 UTC
- Message-ID
- <CACBZZX4Fb0OCkh5kwKvLC+_0xb7q-UB7LH2_WY=dFN5SYUeezQ@mail.gmail.com>
- In-Reply-To
- <20121003180324.GB27446@sigill.intra.peff.net>
On Wed, Oct 3, 2012 at 8:03 PM, Jeff King <peff@peff.net> wrote:
Show 20 quoted lines
> What version of git are you using? In the past year or so, I've made > several tweaks to speed up large numbers of refs, including: > > - cff38a5 (receive-pack: eliminate duplicate .have refs, v1.7.6); note > that this only helps if they are being pulled in by an alternates > repo. And even then, it only helps if they are mostly duplicates; > distinct ones are still O(n^2). > > - 7db8d53 (fetch-pack: avoid quadratic behavior in remove_duplicates) > a0de288 (fetch-pack: avoid quadratic loop in filter_refs) > Both in v1.7.11. I think there is still a potential quadratic loop > in mark_complete() > > - 90108a2 (upload-pack: avoid parsing tag destinations) > 926f1dd (upload-pack: avoid parsing objects during ref advertisement) > Both in v1.7.10. Note that tag objects are more expensive to > advertise than commits, because we have to load and peel them. > > Even with those patches, though, I found that it was something like ~2s > to advertise 100,000 refs.
FWIW I bisected between 1.7.9 and 1.7.10 and found that the point at which it went from 1.5/s to 2.5/s upload-pack runs on the pathological git.git repository was none of those, but:
ccdc6037fe - parse_object: try internal cache before reading object db