Re: [PATCH 0/3] protocol v2 and hidden refs

From: Jeff King <peff@peff.net>
Date: Dec 16, 2018, 10:40 UTC
Message-ID: <20181216104017.GB13704@sigill.intra.peff.net>
In-Reply-To: <87d0q21s8w.fsf@evledraar.gmail.com>

On Sat, Dec 15, 2018 at 08:53:35PM +0100, Ævar Arnfjörð Bjarmason wrote:

Show 8 quoted lines

> > So I'm a bit worried that the unified endpoint model is going to be a
> > dead end, at which point carrying around git-serve just makes things
> > more complicated.
> 
> This is from wetware memory of something discussed at a previous Git
> Merge, so I may have (inadvertently) made it up, but wasn't part of the
> idea of "git-serve" to have an extensible next-gen protocol where we
> could add new actions & verbs unrelated to sending or receiving packs?

Yes, I think that's a goal (and we already have upload-archive, which is a similar thing).

Show 5 quoted lines

> Of course that's not in itself an argument for having a general "serve"
> command, actually the opposite for the reasons you mention with locking
> down things. E.g. maybe I want to support server-side git-grep on my
> server, but not git-log, and if it's one command it becomes a hassle to
> do that via SSH config or HTTPD config for the reasons you mention.

Right, exactly. It pushes more of the information down into Git's own protocol. Of course we _can_ build mechanisms at that level for configuring which verbs are allowed. But if some context is available at the higher protocol level, then we can use the mechanisms at that higher level.

I think of it as a tradeoff. By including the endpoint in the transport protocol (e.g., in ssh the command name, in HTTP the URL), we get to use the mechanisms in those transports to make policy decisions on the server. But it also means we _have_ to implement those policies twice, once per transport.

IMHO having to deal with both transports is not that big a loss, considering that there are only two, and really not likely to be more. git:// is already unauthenticated, and IMHO is mostly a dead-end for future protocol work since it provides no advantage over HTTP, and the future is mostly HTTP, with ssh for people who really prefer its authentication mode.

Show 5 quoted lines

> The upside would be that once a host understands "git serve" I'm more
> likely to be able to get past whatever middle layer there is between my
> client and the "git" binary on the other side. E.g. if I have a newly
> compiled "git" client/server binary, but something like GitLab's
> "gitaly" sitting between the two of us.

But I think that's what makes it dangerous, too. :)

Gitaly (and we have our own equivalent at GitHub) is responsible for making those policy decisions about who can run what. Opening a pipe between the client and the backend that can issue arbitrary verbs is exactly what they _don't_ want to do.

So they have to intercept the conversation at least at the verb level. It _is_ nice if conversation for each verb is standardized (so once a verb is issued, they can just step out of the way and proxy bytes[1]), and v2 helps with that.

That's not too hard for a Git-aware endpoint to implement. But when that verb interception can be done at the HTTP/ssh level, then it's easy for tools that _aren't_ Git-aware to do handle it (again, like the Apache config we recommend in git-http-backend(1)).

-Peff

[1] Actually, we do much more intimate interception than that at GitHub
    already. The upload-pack conversation is mostly vanilla, but for
    receive-pack we handle replication at that layer. So your pack is
    streamed to 3-6 backend receive-packs simultaneously, and that
    endpoint layer handles quorum for updating refs, etc.