Re: Partial-clone cause big performance impact on server
- From
Derrick Stolee <derrickstolee@github.com>
- Date
- Aug 15, 2022, 13:18 UTC
- Message-ID
- <85e6fd08-c741-26d4-1393-4b115133e687@github.com>
- In-Reply-To
- <YviaoXRctE9fg/mB@coredump.intra.peff.net>
On 8/14/2022 2:48 AM, Jeff King wrote:
Show 20 quoted lines
> On Thu, Aug 11, 2022 at 08:09:56AM +0000, 程洋 wrote: > >> 4. And we then track our server(which is gerrit with jgit). We >> found the server is couting objects. Then we check those 40k >> objects, most of them are blobs rather than commit. (which >> means they're not in bitmap) >> 5. We believe that's the root cause of our problem. Git sends too >> many "want SHA1" which are not in bitmap, cause the server to >> count objects frequently, which then slow down the server. > > I'd be surprised if bitmaps make a big difference either way here, since > blobs are very quick in the "counting" phase of pack-objects. They can't > link to anything else, so we should not be opening the object contents > at all! We just need to find them on disk, and then in many cases we can > send them over the wire without even decompressing (the exception is if > they are stored as deltas against an object the client doesn't have). > > I didn't generate a test case, but I'm pretty sure that is how git.git's > pack-objects should behave. But you mentioned that the server is jgit; > it's possible that it isn't as optimized in that area.
I just remembered that Gerrit specifically has branch-level security, where some branches are not visible to all users. For that reason, blobs cannot be served without first determining if they are reachable from a branch visible to the current user.
I'm not sure if that's the problem in this particular case, but it could be.
Thanks, -Stolee