Re: "IMAP IDLE"-like long-polling "git fetch"

From: Stefan Beller <sbeller@google.com>
Date: Jan 9, 2019, 22:27 UTC
Message-ID: <CAGZ79kb9Tbnxe1mSnxpqT_FO6Gdi6wxd-r2YarHXRF1sVRyxLA@mail.gmail.com>
In-Reply-To: <20181229043858.GA28509@pure.paranoia.local>

On Fri, Dec 28, 2018 at 8:39 PM Konstantin Ryabitsev <konstantin@linuxfoundation.org> wrote:

Show 14 quoted lines

>
> On Sat, Dec 29, 2018 at 03:56:21AM +0000, Eric Wong wrote:
> > Hey all, I just added this to the TODO file for public-inbox[1] but
> > obviously it's intended for git.git (meta@public-inbox cc-ed):
> >
> > > +* Contribute something like IMAP IDLE for "git fetch".
> > > +  Inboxes (and any git repos) can be kept up-to-date without
> > > +  relying on polling.
> >
> > I would've thought somebody had done this by now, but I guess
> > it's dependent on a bunch of things (TLS layer nowadays, maybe
> > HTTP/2), so git-daemon support alone wouldn't cut it...
>
> Polling is not all bad, especially for large repository collections.

I disagree with that statement.

IIRC, More than half the bandwidth of Googles git servers are used for ls-remote calls (i.e. polling a lot of repos, most of them did *not* change, by build bots which are really eager to try again after a minute).

That is why we use a superproject, with all other repositories as a submodule for polling, as that would slash the ls-remote traffic approximately by the number of repositories.

There was an attempt in JGit to support this type of communication of long polling at https://git.eclipse.org/r/plugins/gitiles/jgit/jgit/+/2adc572628f9382ace5fbd791325dc64f7c968d3 but not a whole lot is left over in JGit as it was refactored at least once again.

IIRC the issues where in the lack of protocol definition that made it usable for a wider audience.