Re: Poor push performance with large number of refs

From: Shawn Pearce <spearce@spearce.org>
Date: Dec 10, 2014, 05:41 UTC
Message-ID: <CAJo=hJvKBvQvN=EV4y=ACz5pou9A0tD+txAn_8VR9L3KKtQSiA@mail.gmail.com>
In-Reply-To: <20141210003735.GA124293@vauxhall.crustytoothpaste.net>

On Tue, Dec 9, 2014 at 4:37 PM, brian m. carlson <sandals@crustytoothpaste.net> wrote:

Show 14 quoted lines

> I have a repository that's just under 2 GiB in size and contains over
> 20000 refs, with a copy of it on a server.  Both sides are using Git
> 2.1.2.  If I push a branch that contains a single commit, it takes about
> 15 seconds to push.  However, if everything is up-to-date, it completes
> within 2 seconds.  Notably, HTTPS performs the same as SSH.
>
> Most of the time is spent between the "Pushing to remote machine" and
> "Counting objects", running git pack-objects:
>
>   git pack-objects --all-progress-implied --revs --stdout --thin --delta-base-offset --progress
>
> Unfortunately, -vvv doesn't provide any helpful output.  I have some
> suspicions what's going on here, but no hard data.  Where should I
> be looking to determine the bottleneck?

My guess is the revision queue is struggling to insert 20,000 commits that the remote side "has", are uninteresting, and should not be transmitted. This queue insertion usually requires parsing the commit object out of the local object store to get the commit timestamp, then bubble sort inserting that commit into the queue.