git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH] send-pack: use OBJECT_INFO_QUICK to check negative objects

From
PPPatrick Marlier (pamarlie) <pamarlie@cisco.com>
Date
Nov 29, 2019, 09:22 UTC
Message-ID
<CH2PR11MB4294A6C1AA71DF1A1721C94CCF460@CH2PR11MB4294.namprd11.prod.outlook.com>
In-Reply-To
<20191127123211.GG22221@sigill.intra.peff.net>
Show 10 quoted lines
> On Tue, Nov 19, 2019 at 01:12:51PM +0000, Patrick Marlier (pamarlie) wrote:
>> I am hitting a performance issue with "git push <remote> <refspec>".
>> The local repository has only few refs and the remote repository has a
>> lot of refs (1000+) with objects unknown to the local repository.
>> 
>> "git push" of only one refspec takes minutes to complete. A quick
>> analysis shows that a lot of time is spent in the client side.
>> A deeper analysis shows that the client receives the entire list of
>> refs on the remote, then the client is checking in its local
>> repository if the objects exist for all remote refs.
>Right, this is expected. The client send-pack feeds the list of remote
>objects (that it has) to pack-objects, which can then limit the size of
>the packfile it sends based on what the other side has.
Show 6 quoted lines
>So the patch you showed (to skip refs that aren't part of the push)
>would miss many opportunities for a smaller push. E.g., imagine I create
>a new branch "topic" from the remote branch "master", and then try to
>push it up. If I do not look at which objects are in "master", I'll end
>up sending the whole history again, when I really just need to send the
>differences between the two.

Thanks Jeff. Indeed it makes sense. (My very particular use-case would have a limited impact in this case)

Show 6 quoted lines
>> Since the local repository has a only few refs, most of the objects
>> are unknown.
>>
>> This issue is particularly amplified because the local repository is
>> using many alternates. Indeed for each unknown object, git will try to
>> find in all alternates too.
> I think the patch below would help you.
I will give a try next week and I keep you posted on how it goes.
Thanks a lot for your input and the patch!

Have a nice day, -- Pat

Previous: Jeff KingNext: Junio C Hamano
Message 4 of 12 in “Push a ref to a remote with many refs”
  1. Patrick Marlier (pamarlie)Nov 19, 2019
  2. Patrick Marlier (pamarlie)Nov 25, 2019
  3. send-pack: use OBJECT_INFO_QUICK to check negative objectsJeff King, Nov 27, 2019
  4. Patrick Marlier (pamarlie)Nov 29, 2019
  5. Junio C HamanoNov 30, 2019
  6. Jeff KingDec 3, 2019
  7. Jonathan TanDec 4, 2019
  8. Junio C HamanoDec 4, 2019
  9. Jonathan NiederDec 4, 2019
  10. Jeff KingDec 4, 2019
  11. Patrick Marlier (pamarlie)Dec 10, 2019
  12. Jeff KingDec 10, 2019

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.