Re: [RFC PATCH] push: perform negotiation before sending packfile

From: Junio C Hamano <gitster@pobox.com>
Date: Feb 18, 2021, 20:25 UTC
Message-ID: <xmqqy2fldsy4.fsf@gitster.g>
In-Reply-To: <xmqqwnv6gnuq.fsf@gitster.g>

Junio C Hamano <gitster@pobox.com> writes:

>> Any comments are welcome, especially if you have ideas about the overall
>> design or implementation, and/or if you've thought about similar things
>> before.

I think that it is a good way to prototype by adding a separate session with the upload-pack to only learn about what is common before talking to the receive-pack, presumably running on the same repository and may share common view with the previous upload-pack.

And the code seems to implement the idea just fine. I wonder how much we can cut out the unnecessary resource usage in pathological cases with this approach.

 - If the target ref got rewound, with the current send-pack
   approach, we may not find any common commit, or we may end up
   using a mediocre one---that would result in unnecessary packdata
   transfer, which we may be able to cut down significantly.

   I thought you guys have some server-side hacks to send a bit
   stale commit as a "fake" ref to help with this situation, though.
   So the comparison is really between how much each of these two
   approaches helps.

 - With this approach, we can cut down the initial ref advertisement
   from the receiving end to the minimum.  The protocol with
   negotiation could work correctly without receiving end
   advertising no refs, but I suspect that it would be very common
   in a publishing repository (not a shared everybody-pushes-into
   repository) that the tip of the target repository is known by the
   pusher, and it may help such a case to know where the ref being
   updated with this push originally points at.  But even in such a
   case, additional negotiation may help---the target branch may
   have not advanced, but may of the side branches the pusher have
   merged into the commit being pushed to the target branch may
   already be known by the receiver.

For a real implementation, I think we'd want to do the negotiation inside the conversation between send-pack and receive-pack, so that what is agreed to be common between two parties will not shift in the middle (in the same spirit that upload-pack grabs all the relevant refs first, advertises them, negotiates what is common and creates a pack, all using the same worldview of where the tips of refs are throughout the process, even if some refs change in the meantime).

Thanks for a fun read.