Re: [PATCH RFC 0/3] cat-file: make remote-object-info request in batches
- From
Pablo Sabater <pabloosabaterr@gmail.com>
- Date
- Oct 27, 2026, 20:47 UTC
- Message-ID
- <DMFWYZSCTM1C.ZM3OE2DGCMPC@gmail.com>
- In-Reply-To
- <xmqqld7jj6u5.fsf@gitster.g>
On Tue Oct 27, 2026 at 7:12 PM WET, Junio C Hamano wrote:
Show 25 quoted lines
> Pablo Sabater <pabloosabaterr@gmail.com> writes: > >> A noticeable difference can be seen from a 10k OID batch to a 50k batch. >> After that the difference isn't as noticeable (the number of batches to >> be requested isn't that big). >> >> I propose 50k OIDs as the default number of OIDs for the >> remote-object-info batches. Coincidentally, git backfill's default batch >> size is 50k as well though it's unrelated. > > Interesting. We cannot share any code with backfill as we only > retrieve info on objects while backfill does grab the objects > themselves, but using the same number may make it easy to explain in > the documentation. > >> I have barely done any benchmarking before, so I would appreciate >> any feedback on the one in patch 2. Is there a better way to do it, >> or other things I should take into account? >> >> Should the server also check for a limit on the number of OIDs it >> accepts in a single object-info request? > > What I am more curious about is how this updated code interacts with > existing code (i.e., updated "cat-file --batch" clients with deployed > servers, and vice versa).
The code in this series only changes the client side, the server has no OID limit. Before this series, what prevented large requests was the client with its 10k hard limit, so the server is fine with any size without any change.
The server will see requests of at most 10k OIDs from old clients, and bigger ones from new clients.
The other change compared to an older client is that now multiple requests can be received on the same connection (instead of one per connection), but that should be fine, serve() keeps processing requests until EOF.
With this new option there's a risk (that was already there if someone modified their client) of setting a huge amount of OIDs in the same request, but I'm not sure how bad that is, a 1M OID request vs smaller ones on the same connection. Could you help me with this?
Anyway, the fix would be to add a cap on the server side. It can't be lower than 10k, or older clients would get an error they can't do anything about, and if it's lower than the client's batch size it should be advertised so newer clients can adapt.
IIRC clients ignore unknown capability keys, so advertising it shouldn't break older ones. In this case, it would be advertised on the handshake and the client would die() if the user sets a batch size bigger than what the server advertises. Ideally the default on the client (the proposed 50k) would be the min cap on the server, so someone that doesn't manually set the batch size and uses the default value doesn't get an error.
If we don't even want to die() we could set the batch size as: min(user-size, advertised-size)
Sorry for the brainstorm but I just realised that any client could modify the size and that a cap on the server side could be important.
Regards, Pablo