Re: Performance issue: initial git clone causes massive repack
- From
Björn Steinbrink <b.steinbrink@gmx.de>
- Date
- Apr 7, 2009, 20:27 UTC
- Message-ID
- <20090407202725.GC4413@atjola.homenet>
- In-Reply-To
- <alpine.LFD.2.00.0904071454250.6741@xanadu.home>
On 2009.04.07 14:56:41 -0400, Nicolas Pitre wrote:
Show 25 quoted lines
> On Tue, 7 Apr 2009, Björn Steinbrink wrote: > > > On 2009.04.07 13:48:02 -0400, Nicolas Pitre wrote: > > > The first low hanging fruit to help this case is to make upload-pack use > > > the --revs argument with pack-object to let it do the object enumeration > > > itself directly, instead of relying on the rev-list output through a > > > pipe. This is what 'git repack' does already. pack-objects has to > > > access the pack anyway, so this would eliminate an extra access from a > > > different process. > > > > Hm, for an initial clone that would end up as: > > git pack-objects --stdout --all > > right? > > > > If so, that doesn't look it it's going to work out as easily as one > > would hope. Robin said that both processes, git-upload-pack (which does > > the rev-list) and pack-objects peaked at ~2GB of RSS (which probably > > includes the mmapped packs). But the above pack-objects with --all peaks > > at 3.1G here, so it basically seems to keep all the stuff in memory that > > the individual processes had. But this way, it's all at once, not 2G > > first and then 2G in a second process, after the first one exitted. > > Right, and it is probably faster too. > > Can I get a copy of that repository somewhere?
http://git.overlays.gentoo.org/gitweb/?p=exp/gentoo-x86.git;a=summary
At least that's what I cloned ;-) I hope it's the right one, but it fits the description...
Björn