Re: [RFC] push: add documentation on push v2

From: Stefan Beller <sbeller@google.com>
Date: Jul 18, 2018, 16:56 UTC
Message-ID: <CAGZ79kbLn-uwQOXfqhtO46v0EWevY43Tf4W5Rz9gDD9_qbmX=A@mail.gmail.com>
In-Reply-To: <a7c43308-a388-e307-6bea-47e6df74b65c@gmail.com>

On Wed, Jul 18, 2018 at 6:31 AM Derrick Stolee <stolee@gmail.com> wrote:

Show 16 quoted lines

>
> On 7/17/2018 7:25 PM, Stefan Beller wrote:
> > On Tue, Jul 17, 2018 at 2:09 PM Brandon Williams <bmwill@google.com> wrote:
> >> Signed-off-by: Brandon Williams <bmwill@google.com>
> >> ---
> >>
> >> Since introducing protocol v2 and enabling fetch I've been thinking
> >> about what its inverse 'push' would look like.  After talking with a
> >> number of people I have a longish list of things that could be done to
> >> improve push and I think I've been able to distill the core features we
> >> want in push v2.
> > It would be nice to know which things you want to improve.
>
> Hopefully we can also get others to chime in with things they don't like
> about the existing protocol. What pain points exist, and what can we do
> to improve at the transport layer before considering new functionality?

Another thing that I realized last night was the possibility to chunk requests. The web of today is driven by lots of small http(s) requests. I know our server team fights with the internal tools all the time because the communication involved in git-fetch is usually a large http request (large packfile). So it would be nice to have the possibility of chunking the request. But I think that can be added as a capability? (Not sure how)

Show 25 quoted lines

> >>   Thankfully (due to the capability system) most of the
> >> other features/improvements can be added later with ease.
> >>
> >> What I've got now is a rough design for a more flexible push, more
> >> flexible because it allows for the server to do what it wants with the
> >> refs that are pushed and has the ability to communicate back what was
> >> done to the client.  The main motivation for this is to work around
> >> issues when working with Gerrit and other code-review systems where you
> >> need to have Change-Ids in the commit messages (now the server can just
> >> insert them for you and send back new commits) and you need to push to
> >> magic refs to get around various limitations (now a Gerrit server should
> >> be able to communicate that pushing to 'master' doesn't update master
> >> but instead creates a refs/changes/<id> ref).
> > Well Gerrit is our main motivation, but this allows for other workflows as well.
> > For example Facebook uses hg internally and they have a
> > "rebase-on-the-server-after-push" workflow IIRC as pushing to a single repo
> > brings up quite some contention. The protocol outlined below would allow
> > for such a workflow as well? (This might be an easier sell to the Git
> > community as most are not quite familiar with Gerrit)
>
> I'm also curious how this "change commits on push" would be helpful to
> other scenarios.
>
> Since I'm not familiar with Gerrit: what is preventing you from having a
> commit hook that inserts (or requests) a Change-Id when not present?

That is how you do it normally. But what if you just get started or want to send a one-off to the server (I wanted to upload a git patch to our internal Gerrit once, and as my repository is configured to work with upstream Git which doesn't carry change ids, I ran into this problem. I had to manually add it to have the server accept it)

> How
> can the server identify the Change-Id automatically when it isn't present?

The change id is just a randomly assigned id, which can be made up, but should stay consistent in further revisions. (Put another way: change ids solve the 'linear assignment problem' of range-diff at scale)

So once the protocol support is in, the client would need to get some UX update to replace its commits just pushed with the answer from the server to work well with server side generated change ids.

But as said I am not sure how much we want to discuss in that direction, but rather see if we could have other use cases: Instead of just rebasing to solve the contention problem server side, we could also offer a "coding helper as a service" - server. That would work similar as the change id workflow lines out above: You push to the server, the server performs some action (style formatting your code for example, linting) and you download it back and have it locallly again.

I think that would be pretty cool actually.

Thanks, Stefan