[ANNOUNCE] git-scm.com Heroku credits update

As some of you know, we have a website at https://git-scm.com, which has served as the Git project's de-facto homepage.

That site is hosted on Heroku, which has graciously provided the Git project with hosting credits for many years. That, combined with support from Cloudflare has meant that the maintenance cost for the site is effectively $0.

Unfortunately, Heroku no longer provides free hosting credits to organizations and open-source projects. I have been in an ongoing support thread with them, and their stance is pretty clear.

To that end, we have a few options about what to do with the website in the future:

  - Convert the Rails application into a static site that could be
    hosted elsewhere for free. The non-static portions of the site would
    have to become static in one way or another, and we'd have to come
    up with an alternative search mechanism to replace the existing one.

  - Explore ways to cheapen our Heroku bill (more on this later). Our
    footprint at Heroku currently is a dyno for the Rails site itself,
    along with a separate database (Postgres) and cache (Redis). We
    could explore eliminating the Rails-layer cache, and tune our
    Cloudflare settings to more aggressively cache parts of the website.

  - Pay Heroku using the Git project's funds held at Conservancy.

  - Find a cheaper Rails host.

Our Heroku bill varies slightly based on usage, but here's last month's bill, which is more or less representative:

              | Usage (months)       | Cost $/mo |  Total
    ----------+----------------------+-----------+---------
    scheduler | 0.004941199999999998 | $25.00/mo |   $0.12
    web       | 1                    | $50.00/mo |  $50.00
    redis     | 1                    | $30.00/mo |  $30.00
    postgres  | 1                    | $50.00/mo |  $50.00
    ----------+----------------------+-----------+---------
                                                   $131.58

(I'm omitting about $1.46 of charges that come from spinning up dynos to power "PR previews" when somebody makes a pull request to the git/git-scm.com repository on GitHub).

I think we could reasonably get rid of the Rails-layer caching using Redis and save some money there, but it may not be possible depending on how much content Cloudflare is willing to keep warm in its cache.

In the meantime, Heroku is billing Git's account at Conservancy [1] in order to keep the site running while we all figure out what to do next. I have some opinions about what a sustainable solution looks like, but I'll avoid sharing them just yet in order to avoid tainting the discussion.

Thanks, Taylor

[1]: I promised[2] an update on our financials, and am still working on it. The Git project currently has about ~$70k USD in its account. [2]: https://lore.kernel.org/git/Yg%2FsO63nhhtAXWvn@nand.local/

On Fri, Apr 01, 2022 at 12:22:14PM -0400, Taylor Blau wrote:

Dan Moore (cc'd) has graciously offered the support of his company FusionAuth [1] to help assist with our hosting costs. From Dan:

    [M]y company has benefited tremendously from git and would be happy
    to foot the monthly cost if you are looking for sponsors.

External support would go a long way towards keeping our existing funds available for things like sponsoring travel and lodging for Git Merge attendees who could not otherwise make it in person.

Dan: thank you very much for your gracious offer. Does anybody have
objections to having some external support? I think the only change to
the site would be adding an entry to the "Sponsors" section of the about
page [2].

Thanks, Taylor

[1]: https://fusionauth.io/ [2]: https://git-scm.com/site

On 5 April 2022 4:39:08 am IST, Taylor Blau <me@ttaylorr.com> wrote:

If that's all the change that's required, I would be in favour.

Thank you for offering to sponsor the git-scm site, Dan!

Hi Taylor and Dan

On 05/04/2022 00:09, Taylor Blau wrote:

That sounds like a good solution. Thank you Dan for your generous offer of support and thank you Taylor for looking after the website.

Best Wishes

Phillip

On Mon, Apr 04, 2022 at 07:09:08PM -0400, Taylor Blau wrote:

It's been a ~week since I sent this. In the meantime, a couple of folks have responded that they would be happy to take Dan up on his offer to support our monthly Heroku costs.

Any last objections? If I don't hear anything soon-ish, I'll work with Dan and Conservancy to set up the logistics and transition our Heroku billing off of our project funds.

Thanks, Taylor

On Mon, Apr 11, 2022 at 11:21:20AM -0400, Taylor Blau wrote:

I didn't hear any objections in the meantime, so I'll work with Dan and the fine folks at Conservancy offline to set up the remaining logistics.

Dan: thank you (again!) very much for your generous offer.

Thanks, Taylor

Hi Taylor,

On Fri, 1 Apr 2022, Taylor Blau wrote:

Thank you for all your efforts. I frequently point users to git-scm.com, be it for the manual pages or for other resources, and appreciate that you keep it running so smoothly. While it is understandable that the free tier is going away, it is nevertheless an unplanned work for us, and I thank you for doing it.

This is my preference (and I would love to contribute the time, but am quite short on that resource to help much).

It looks to me as if the only blocker is the site-search, and there are ways to pre-generate an index e.g. in a Jekyll site (which we could host on GitHub Pages, incidentally, which would be a very nice setup indeed).

The other thing the Rails app does is to regularly poll for updated versions, e.g. Git for Windows and Git for Mac. This strikes me as something that would benefit from the transparency provided by having a GitHub workflow to perform said polling instead of the opaque Rails app on Heroku with no public log of when it ran and whether it was successful.

Please let me know what you think, and whether you already looked into how automatable/parallelizable this conversion would be. Hopefully I _can_ be of use in bringing that about.

Thanks, Dscho

On Fri, Apr 8, 2022 at 12:39 AM Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:

I wonder if this could be an Outreachy (maybe next Winter) or a GSoC (2023) project.

Christian Couder <christian.couder@gmail.com> writes:

Good suggestion.

It would take talent quite different from what our past GSoC programs saw, from both mentors and "contributors" (we used to call them "students"). If we have mentors who can give good experience to mentees, I would welcome to see such a task for a change.

Thanks.

On Fri, Apr 08, 2022 at 11:12:37AM -0700, Junio C Hamano wrote:

I agree!

I would be happy to help mentor a new contributor as somebody vaguely familiar with the architecture of the existing site, though I have next to no experience / good judgement on current web best practices ;).

Thanks, Taylor

On Thu, Apr 07, 2022 at 11:54:18PM +0200, Johannes Schindelin wrote:

There are a few other smaller-ish things that we are using Rails for. But most of them are limited to "fetch this list of things from Postgres and format it in HTML" which could easily be automated and checked into the static version of git-scm.com's repo.

Yeah, agreed. Another "behind the scenes" task is running:

    heroku run -a git-scm rake preindex
    heroku run -a git-scm rake downloads
    heroku run -a git-scm rake search_index

, which I do after Junio pushes new tags. (FWIW, these jobs should run on a timer, too, but I like to update the site with new links around the time a release is tagged as opposed to having to wait for the automated job to kick off).

I imagine that this would be replaced with some kind of GitHub workflow, and the result (any new pages, updated search indexes, etc.) would be committed into the repository transparently. That workflow could be automated as well and get kicked off when new tags are pushed to git/git (I think).

Thanks, Taylor

Hi Taylor,

Yes, we would love to offer our monetary support if it makes sense for the community.

Cheers, Dan

On Mon, Apr 4, 2022 at 5:23 PM Dan Moore <dan@fusionauth.io> wrote: