patch, 13 partsgit-curl-compat: remove check for curl 7.21.5

For a long time, we ended up with protracted discussions on the mailing list about what versions of software we should support. Oftentimes, we broke long-obsolete operating system versions by using something shipped slightly more recently.

Fortunately, we now have a platform support policy to guide us in our approach to dependencies, so we can make updates without worrying about breaking systems that have not received security support in several years.

This series updates our requirements for libcurl to 7.61.0 (the version in RHEL 8) and for Perl to 5.26.0 (the version in 15.6). I considered the mainstream LTS versions of RHEL, Debian, Ubuntu, and SLES, but omitted consideration of paid support extended LTS, since we cannot expect Git developers to have to pay a large corporation lots of money just to test functionality. This is in conformance with our policy, which states that versions must be "in line with the version used by other long-term-support distributions", which does not include extended LTS distributions.

The libcurl dependency changes come in incremental patches so that if we have people on unsupported systems, they can simply revert the patches that they'd like to omit. It also makes the changes easier to review than one giant commit.

The Perl changes are a huge upgrade. 5.8.1, our former supported version, was from 2003. 5.26 has substantially improved Unicode support (including Unicode strings), s///r (to allow returning a modified value instead of modifying it in place), postderef syntax (which also provides better interpolation for complex expressions), and subroutine signatures (although these are experimental until 5.36). These allow us much more readable, modern Perl.

The final commit introduces a small but useful change that we can now take advantage of with our newly updated Perl dependency as an example of why this is a generally beneficial change. It can be omitted without problem if it is judged to be too noisy.

brian m. carlson (13):
  git-curl-compat: remove check for curl 7.21.5
  git-curl-compat: remove check for curl 7.25.0
  git-curl-compat: remove check for curl 7.34.0
  git-curl-compat: remove check for curl 7.39.0
  git-curl-compat: remove check for curl 7.43.0
  git-curl-compat: remove check for curl 7.44.0
  git-curl-compat: remove check for curl 7.52.0
  git-curl-compat: remove check for curl 7.53.0
  git-curl-compat: remove check for curl 7.56.0
  INSTALL: document requirement for libcurl 7.61.0
  Require Perl 5.26.0
  INSTALL: require Perl 5.26.0
  gitweb: make use of s///r

 INSTALL                                 | 13 +---
 contrib/diff-highlight/DiffHighlight.pm |  2 +-
 contrib/mw-to-git/Git/Mediawiki.pm      |  2 +-
 git-archimport.perl                     |  2 +-
 git-curl-compat.h                       | 98 -------------------------
 git-cvsexportcommit.perl                |  2 +-
 git-cvsimport.perl                      |  2 +-
 git-cvsserver.perl                      |  2 +-
 git-send-email.perl                     |  2 +-
 git-svn.perl                            |  2 +-
 gitweb/gitweb.perl                      |  6 +-
 http.c                                  | 58 ---------------
 imap-send.c                             |  4 -
 perl/Git.pm                             |  2 +-
 perl/Git/I18N.pm                        |  2 +-
 perl/Git/LoadCPAN.pm                    |  2 +-
 perl/Git/Packet.pm                      |  2 +-
 t/t0202/test.pl                         |  2 +-
 t/t5562/invoke-with-content-length.pl   |  2 +-
 t/t9700/test.pl                         |  2 +-
 t/test-terminal.perl                    |  2 +-
 21 files changed, 23 insertions(+), 188 deletions(-)

libcurl 7.21.5 was released in April 2011, which is well over ten years ago, and no major operating system vendor is still providing security support for it. Debian 7, RHEL 7, and Ubuntu 12.04, all of which are out of mainstream security support, have all supported a newer version.

Remove the check for this version and use this functionality unconditionally.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 git-curl-compat.h | 7 -------
 1 file changed, 7 deletions(-)

libcurl 7.25.0 was released in March 2012, which is well over ten years ago, and no major operating system vendor is still providing security support for it. Debian 8, RHEL 7, and Ubuntu 12.10, all of which are out of mainstream security support, have all supported a newer version.

Remove the check for this version and use this functionality unconditionally.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 git-curl-compat.h |  8 --------
 http.c            | 24 ------------------------
 2 files changed, 32 deletions(-)

libcurl 7.34.0 was released in December 2013, which is well over ten years ago, and no major operating system vendor is still providing security support for it. Debian 8 and Ubuntu 14.04, both of which are out of mainstream security support, have supported a newer version, and RHEL 8, which is still in support, also has a newer version.

Remove the check for this version and use this functionality unconditionally.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 git-curl-compat.h | 22 ----------------------
 http.c            |  2 --
 imap-send.c       |  4 ----
 3 files changed, 28 deletions(-)

libcurl 7.39.0 was released in November 2014, which is almost ten years ago, and no major operating system vendor is still providing security support for it. Debian 9 and Ubuntu 16.04, both of which are out of mainstream security support, have supported a newer version, and RHEL 8, which is still in support, also has a newer version.

Remove the check for this version and use this functionality unconditionally.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 git-curl-compat.h |  9 ---------
 http.c            | 11 -----------
 2 files changed, 20 deletions(-)

libcurl 7.43.0 was released in June 2015, which is over nine years ago, and no major operating system vendor is still providing security support for it. Debian 9 and Ubuntu 16.04, both of which are out of mainstream security support, have supported a newer version, and RHEL 8, which is still in support, also has a newer version.

Remove the check for this version and use this functionality unconditionally.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 git-curl-compat.h | 11 -----------
 http.c            |  5 -----
 2 files changed, 16 deletions(-)

libcurl 7.44.0 was released in August 2015, which is over nine years ago, and no major operating system vendor is still providing security support for it. Debian 9 and Ubuntu 16.04, both of which are out of mainstream security support, have supported a newer version, and RHEL 8, which is still in support, also has a newer version.

Remove the check for this version and use this functionality unconditionally.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 git-curl-compat.h | 10 ----------
 http.c            |  4 ----
 2 files changed, 14 deletions(-)

libcurl 7.52.0 was released in August 2017, which is over seven years ago, and no major operating system vendor is still providing security support for it. Debian 9 and Ubuntu 18.04, both of which are out of mainstream security support, have supported a newer version, and RHEL 8, which is still in support, also has a newer version.

Remove the check for this version and use this functionality unconditionally.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 git-curl-compat.h | 15 ---------------
 http.c            |  8 --------
 2 files changed, 23 deletions(-)

libcurl 7.56.0 was released in September 2017, which is over seven years ago, and no major operating system vendor is still providing security support for it. Debian 10, which is out of mainstream security support, has supported a newer version, and Ubuntu 20.04 and RHEL 8, which are still in support, also have a newer version.

Remove the check for this version and use this functionality unconditionally.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 git-curl-compat.h | 8 --------
 http.c            | 2 --
 2 files changed, 10 deletions(-)

On Thu, Oct 10, 2024 at 11:56:17PM +0000, brian m. carlson wrote:

I wonder whether we want to have something like the below patch to give people a better error message in case they have a version that is too old now.

Other than that I agree with the sentiment of this patch series. Supporting ancient dependency versions that aren't used by any still-supported and available distro doesn't feel sensible to me, and scenarios like this are why we have introduced the platform support policy in the first place.

Patrick

On Fri, Oct 11, 2024 at 08:48:51AM +0200, Patrick Steinhardt wrote:

IIRC we ran into some interesting situations in the past where some distros had older versions that had backported some features. So Git would continue to compile, even though it was not technically the version we said was needed. And a patch like the one above would break those systems, even they'd otherwise be OK.

Now possibly that is a little bit insane and not something we should worry about. I don't have good examples of what kinds of things got backported, but searching the archive for LIBCURL_VERSION_NUM and "backport" yielded this:

  https://lore.kernel.org/git/4d29d43d458f61c6dabca093f591ad8698ca2ceb.1502462884.git.tgc@jupiterrise.com/

and I seem to recall most of the discussion of this was around that author and RHEL/EPEL.

-Peff

On Fri, Oct 11, 2024 at 03:33:26AM -0400, Jeff King wrote:

Huh, interesting, thanks for the context! I'm not really sure whether we really should worry about such weird backports all that much. But in any case I'm okay with not pursuing the error.

Patrick

Patrick Steinhardt <ps@pks.im> writes:

Yup, the runtime die() would work it around for such versions of libcURL with silent backports.

The message should be made _("localizable"), though.

Thanks.

Our platform support policy states that we require "versions of dependencies which are generally accepted as stable and supportable, e.g., in line with the version used by other long-term-support distributions". Of Debian, Ubuntu, and RHEL, the three most common distributions that provide LTS versions, the version with mainstream long-term security support with the oldest libcurl is 7.61.0 in RHEL 8.

Update the documentation to state that this is the new base version for libcurl. Remove text that is no longer applicable to older versions.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 INSTALL | 11 +++--------
 1 file changed, 3 insertions(+), 8 deletions(-)

Our platform support policy states that we require "versions of dependencies which are generally accepted as stable and supportable, e.g., in line with the version used by other long-term-support distributions". Of Debian, Ubuntu, RHEL, and SLES, the four most common distributions that provide LTS versions, the version with mainstream long-term security support with the oldest Perl is 5.26.0 in SLES 15.6.

This is a major upgrade, since Perl 5.8.1, according to the Perl documentation, was released in September of 2003. It brings a lot of new features that we can choose to use, such as s///r to return the modified string, the postderef functionality, and subroutine signatures, although the latter was still considered experimental until 5.36.

Update the INSTALL file to reflect our new dependency requirement.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 INSTALL | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

On Thu, Oct 10, 2024 at 11:56:20PM +0000, brian m. carlson wrote:

any particular reason not to squash this into the parent commit? i see how the separation makes sense for the libcurl sub-series, but that doesn't seem applicable here.

regarding the actual `use` statements, you could make them somewhat more legible by using 'v5.26' as the version number.

another aspect to consider is that the statement doesn't just specify the minimal version, but also subtly changes behavior. for example, the `use strict;` statements become redundant.

cf. https://perldoc.perl.org/functions/use#use-VERSION

lastly, it would be nice to update the build systems to reflect the version requirements. though the only pre-existing version check i found is the libcurl one in contrib/buildsystems/CMakeLists.txt.

On 2024-10-11 at 09:38:38, Oswald Buddenhagen wrote:

Sure, I can do that.

Yes, I'll change that to a require v5.26 instead, since my goal isn't to change the behaviour.

I don't build with cmake, so I can't speak to the requirements for it. It doesn't actually work on Unix as far as I know, and I don't run Windows at all.

Our platform support policy states that we require "versions of dependencies which are generally accepted as stable and supportable, e.g., in line with the version used by other long-term-support distributions". Of Debian, Ubuntu, RHEL, and SLES, the four most common distributions that provide LTS versions, the version with mainstream long-term security support with the oldest Perl is 5.26.0 in SLES 15.6.

This is a major upgrade, since Perl 5.8.1, according to the Perl documentation, was released in September of 2003. It brings a lot of new features that we can choose to use, such as s///r to return the modified string, the postderef functionality, and subroutine signatures, although the latter was still considered experimental until 5.36.

This change was made with the following one-liner, which intentionally excludes modifying the vendored modules we include to avoid conflicts:

    git grep -l 'use 5.008001' | grep -v 'LoadCPAN/' | xargs perl -pi -e 's/use 5.008001/use 5.026000/'

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 contrib/diff-highlight/DiffHighlight.pm | 2 +-
 contrib/mw-to-git/Git/Mediawiki.pm      | 2 +-
 git-archimport.perl                     | 2 +-
 git-cvsexportcommit.perl                | 2 +-
 git-cvsimport.perl                      | 2 +-
 git-cvsserver.perl                      | 2 +-
 git-send-email.perl                     | 2 +-
 git-svn.perl                            | 2 +-
 gitweb/gitweb.perl                      | 2 +-
 perl/Git.pm                             | 2 +-
 perl/Git/I18N.pm                        | 2 +-
 perl/Git/LoadCPAN.pm                    | 2 +-
 perl/Git/Packet.pm                      | 2 +-
 t/t0202/test.pl                         | 2 +-
 t/t5562/invoke-with-content-length.pl   | 2 +-
 t/t9700/test.pl                         | 2 +-
 t/test-terminal.perl                    | 2 +-
 17 files changed, 17 insertions(+), 17 deletions(-)

libcurl 7.53.0 was released in February 2017, which is over seven years ago, and no major operating system vendor is still providing security support for it. Debian 10 and Ubuntu 18.04, both of which are out of mainstream security support, have supported a newer version, and RHEL 8, which is still in support, also has a newer version.

Remove the check for this version and use this functionality unconditionally.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 git-curl-compat.h | 8 --------
 http.c            | 2 --
 2 files changed, 10 deletions(-)

In Perl 5.14, released in May 2011, the r modifier was added to the s/// operator to allow it to return the modified string instead of modifying the string in place. This allows to write nicer, more succinct code in several cases, so let's do that here.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 gitweb/gitweb.perl | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

On Thu, Oct 10, 2024 at 11:56:08PM +0000, brian m. carlson wrote:

The libcurl changes all looked OK to me. I was a little surprised that we could move to 7.61.0, which is only 6 years old, since many long-term releases target 10 years. I guess the ones you looked at have had point releases with updated libcurl?

I don't have a strong opinion on the extended LTS issue. Like you, I don't really care about dealing with paid support. OTOH, I think in many cases there was little to no maintenance burden for these older versions, since we'd already done the work to #ifdef them. But I guess since you broke up the patches, they can always choose to revert or include what they want.

I'm OK with a move to perl 5.26. It does feel a little weird to be mass-updating the "require" lines in stuff in contrib/ (specifically I noticed diff-highlight, since I maintain it). But 5.008 is so absurdly old that I find it hard to believe anybody would ever notice the difference.

-Peff

Jeff King <peff@peff.net> writes:

Likewise for me, as 10 years was the random number floated around, if I remember correctly, back when we discussed the platform support policy.

Anybody who find that the update from 5.008 should be to 5.10, like I initially did, should feel absurdly old themselves ;-)

On Fri, Oct 11, 2024 at 3:40 AM Jeff King <peff@peff.net> wrote:

I may be in the minority here, but I'm fairly negative on this entire patch series. As you say, supporting these old versions is effectively zero-cost, so how does this project benefit from these changes which potentially "break" Git for users on older platforms? I see no upside here. The cover letter provides no strong justification for (potentially) inconveniencing people; the argument about being able to utilize more modern Perl features is weak[1] at best and is not convincing.

Although brian is (quite rightly) concerned about security (or lack thereof with older installations), it is not this project's responsibility to "force" people to upgrade their insecure installations. And it is not at all uncommon in the "Real World" for decade-or-more old installations to be running in production environments, and programmers need to work within those environments, however, those installations are, for various business reasons (such as cost-effectiveness and known stability), unlikely to (ever) be upgraded to more modern versions. I, personally, deal with such installations on a very regular basis, and in my experience, the only time upgrades are undertaken (in production settings) is when the systems break completely and there is no choice but to replace them.

Finally, there clearly are real-world cases[2] which benefit from Git continuing to support older platforms; why should we abandon them intentionally? And why should we turn down[3] the periodic trivial patch[4] which trickles in to help people on older platforms?

[1]: https://lore.kernel.org/git/xmqq1q0mh9gn.fsf@gitster.g/ [2]: https://lore.kernel.org/git/CAOO-Oz0NUA-YeyFT1MJ=XKyLWJvQoFH1b-F0EFOzvy8iWka3KA@mail.gmail.com/ [3]: https://lore.kernel.org/git/ZwhMmGt0kZvaSzSL@tapette.crustytoothpaste.net/ [4]: https://lore.kernel.org/git/CAOO-Oz1KhFcyErVx1Qb142PtPJS=UpgSD-FacckqNS4_okAtFQ@mail.gmail.com/

Eric Sunshine <sunshine@sunshineco.com> writes:

While I agree with all you said above, one thing I find missing is that even with #ifdef, we won't be shipping what we tested in real, as nobody, not just the author that touches the same file with the #ifdef we added 6 months ago is in, but all other developers who looked at the change. It merely is "we have #ifdef here and those with ancient version of the library shouldn't see this new code", which certainly is good enough for those of us who consider the ancient platform support as a "best effort" thing.

But that does not, in my dictionary, quite qualify for the verb "support". A variable declared only inside #ifdef may be used outside it, or a variable declared without initialization outside that is only assigned inside #ifdef may be used after matching #endif, which would not be noticed by anybody because nobody among us would be running such an ancient version without the feature #ifdef guards.

So I dunno.

Having said all that, I did find it was surprising that we raised to a merely 6-year old cutoff point. If it were discarding versions of libraries that are older than 12 years (instead of 6 years), would you be having the same reaction?

Thanks.

Junio C Hamano <gitster@pobox.com> writes:

Should I go ahead and send the patch series that I had planned to fix the breakage for old libcurl after all? I've gone ahead and built the latest version for one of the ancient platforms I inexplicably build git for, but am now dealing with breakage on another (SunOS 5.10).

(Specifically, the new unit test framework stuff was failing to generate a suite file, patch forthcoming, and depends on mkdtemp, which we check for in configure but use unconditionally in the newly-imported clar, and which I don't have here.)

-Alejandro

On Fri, Oct 11, 2024 at 2:35 PM Junio C Hamano <gitster@pobox.com> wrote:

I almost certainly would have had the same reaction had it been 12 years instead of 6. As one who "lives" with these old platforms both professionally and personally, I'm sensitive to the issue because I have been burned too many times by projects arbitrarily dropping support for older platforms (or, more generally, not taking their user population into consideration when making arbitrary changes).

I would be much more tolerant and understanding of changes with substantial and provable value, such as ridding the project of a high-cost maintenance burden, or eliminating some maldesign which impedes implementation of some new important feature (or even which impedes fixing some serious flaw). But the patch series under discussion does not fall into those categories; it (potentially) penalizes an arbitrary chunk of the Git user base without any obvious benefit to the project itself.

On 2024-10-11 at 18:09:14, Eric Sunshine wrote:

It is not effectively zero cost. When I want to write a patch, I must make sure that it works on all the platforms we support, or my patch will get reverted or not picked up. That means I have to expend additional effort when adding features to look through the supported versions of our dependencies and either conditionally check them or skip the feature. Sometimes I have to rewrite that feature in a different way, or ship a compatibility stub for a system that doesn't support it.

I have actually spent a decent amount of work getting things to work across older versions of software, both in Git and elsewhere. The more we honour the policy we have already made and agreed upon, the less work Git developers have to do to support adding and maintaining these features.

I should be clear that I do very much value the portability of Git across systems and architectures: my first laptop was a PowerPC Mac running Linux and I've owned UltraSPARC, ARM64, and MIPS hardware. I really try to write code that doesn't have weird portability problems across architectures or OSes, and that's relatively easy to do. But I'm not willing to do lots of extra work to reimplement features or work around ancient systems because people can't upgrade their OS and dependencies.

It isn't acceptable to run systems that don't have security updates applied that are connected to the Internet, period. In this day and age, it's very easy to have bugs in things like TLS or HTTP libraries that are written in C and have security-sensitive implications and that are exploitable remotely.

No matter how stable your systems may be, it's very easy for unpatched systems to quickly become part of a botnet, which is a problem for everyone else. Typically most businesses that sell to other businesses have to be in compliance with certain security policies, especially if they have user or corporate data. My employer simply cannot refuse to upgrade because we risk major legal problems (e.g., GDPR or PIPEDA problems) or loss of most of our corporate customers because they won't or can't (due to regulatory requirements) do business with people who have lax security. So I very much doubt that there is, in the general case, any compelling business reason not to upgrade to a patched OS.

Certainly we cannot force people to upgrade, but we also don't have to support those people. Git is an open-source project, and people are welcome to make changes that they want to it without our approval, as long as they comply with the license.

I've worked at multiple companies where we had obsolete systems that needed to be upgraded but hadn't been and have dealt with that pain, including when it negatively affected us shipping Git. I still think that this is the appropriate policy to have.

There's also discussion about adding Rust to Git. Assuming we do that, we're going to have to work with Rust's requirements for OSes, which usually follow major supported distros (for Linux) or upstream's policy (for the BSDs). So we're going to have the same problem in that people are actually going to have to upgrade to a supported OS, except it's really not going to be optional because the code simply won't compile. We might as well get used to doing that now.

On Fri, Oct 11, 2024 at 4:01 PM brian m. carlson <sandals@crustytoothpaste.net> wrote:

This attitude feels backward to me. It says that simplifying life for Git developers ("the few") is of paramount importance and that Git developers shouldn't care about inflicting pain/difficulty upon Git users ("the many"). This is especially disturbing considering the size of the Git user base.

Instead, for every proposed change to Git, we should be asking ourselves what possible positive and negative impacts the change will have on *users*, and if the negatives outweigh the positives, then the change should be considered with a very wary eye indeed.

I don't disagree with your opinions about security and that, in an ideal world, businesses should take these concerns seriously and should upgrade. However...

In my experience, it is very rare for the non-technical people responsible for allocating funds to be convinced that money/time should be spent on upgrading *working* systems. There are always more urgent tasks (in their minds) which take priority. So, while there may not be a compelling reason in the ideal world to forego upgrading, the real world works differently.

Ditto what I said above about this attitude feeling backward.

Moreover, as mentioned previously, it is not *this* project's responsibility to be forcing people to upgrade their insecure systems.

"Assuming we do that" is the key phrase. There have been proponents and opponents, but almost nothing convincing written in favor of adopting Rust according to a (mostly) outsider's summary of the discussion[1]. The only properly compelling point in favor of Rust came from Elijah; all other arguments for Rust had the flavor of someone evangelizing for his or her latest favorite language. We've seen such evangelizing before: numerous times with people insisting that Git needed to be rewritten in C++, and (somewhat) more recently when Felipe insisted, not only that Ruby be accepted into the project, but that parts of the project should be rewritten in Ruby. But mere evangelizing is not convincing. (Elijah's support for Rust was more compelling, not only because he was not evangelizing, but because, as usual with him, he backed up his position with solid, well-reasoned statements of experience directly applicable to the Git project.)

[1]: https://lore.kernel.org/git/CAPig+cQtxx=fQM2xHSt8AsxyWgjSiS9Kd5PtjA+jDoK5s9xh4A@mail.gmail.com/

On Tue, Oct 15, 2024 at 02:13:45AM -0400, Eric Sunshine wrote:

I agree with Eric that we should first and foremost consider the user-impact of any changes we make to Git.

I think in reality there must be a balance between the two. We should make reasonable decisions when presented a trade-off between supporting users and making the lives of Git developers easier. For instance, if there is some change we could make which would involve a manageable amount of additional effort, but would somehow benefit the lives of many users (e.g., supporting more versions of a dependency, improving performance, fixing a widespread bug, etc.), then we should do that thing.

On the other hand, if we are bending over backwards as developers to support a small portion of the user-base (e.g., by maintaining some ancient version of a dependency that is no longer reasonable because we can assume that 99.99% of users have a newer version), then we should consider our options and investigate. What are the ongoing costs to maintain that minimum version? What features are we missing? How many users would be affected by dropping support for that version, etc.?

I am not entirely sure whether the jump that brian is proposing is reasonable or not. The current minimum version of Perl, for example, is from 2003, but the proposed new minimum is from 2017. While the new version is certainly not new, I am not sure how many users would be affected by dragging the minimum version forward by some 14 years.

I do not think it is our responsibility to force people to upgrade their systems. But OTOH we should not bend over backwards here either to support ancient versions of dependencies when there are compelling reasons *not* to use those versions.

I agree with your earlier comment that there is a balance between thinking about this abstractly and applying it to the real world. But at some point we have to throw our hands up and stop spending effort supporting ancient/insecure versions of dependencies.

Indeed. Let's not worry about it for now.

Thanks, Taylor

On 2024-10-15 at 19:19:51, Taylor Blau wrote:

Right now, we have a clearly documented policy about what we support which was discussed extensively on the list. This is the project's policy, not mine. I agree with it, but I'm not the only person who has advocated for it or thought it was an acceptable policy.

The policy was going to be even stricter in that we were going to require people to set up CI in order to be a supported platform. My concern with that, which I mentioned at the time, is that less common architectures don't run in GitHub Actions or most other CI platforms (or don't run fast enough or correctly enough in emulation to be reasonably tested), and so we'd essentially be excluding all non-x86 processors, which I don't believe to be acceptable. That is a position that I think is definitely in the interests of users.

However, the fact is that nobody is testing the platforms I'm proposing to drop support for. Nobody has even bothered to set up a single CI job for any variant of those platforms at all or to request that one be set up, nor stepped up to be a maintainer. I should point out that setting up tests in a VM in GitHub Actions is very easy and I linked to an example I use in other projects in the thread where we adopted this policy.

Nobody, outside of the FreeBSD maintainer, has even bothered to set up CI for a platform other than the three major ones. The patches to fix SunOS 5.10 also don't include any tests or CI. I don't think it's reasonable for us to go out of our way to support these systems if nobody using those platforms has bothered to provide even the most rudimentary check that they work. How can we expect developers who don't use these systems to even know if they work without some basic tests, even if it's for only one architecture, especially given that in many cases it involves adding just three lines to the workflow file?

I think the answer is that we can't. Since we don't have anyone who has demonstrated that there's basic interest in helping the contributors support their platform by setting up tests or volunteering to be the maintainer, we can't support those platforms specifically and we're essentially left with just honouring the policy that we've set, which is what I'm doing here.

I don't think we can actually know in the general case. It will exclude people on obsolete systems, but it should not exclude anyone with an OS shipped in the past 5 years. The only major OS distributions that I see supporting more than a 5-year regular LTS life span are RHEL and SLES, and I've considered them here. Again, I don't think asking people to upgrade an OS every five years is in any way unreasonable, and I have even considered people farther back.

It's also reasonably easy to build new versions of Perl with things like perlbrew or other toolchain tools, and those are the common suggestion that people use when they have a toolchain that's out of date. I've worked at a company which did some very unusual things with Perl (specifically compiling it to C for performance) and who I think had at one point used the oldest Perl I'm aware of being used at a Perl shop (at the time, 5.6) for major development, and I know they're now using a modern Perl and wouldn't be affected. In fact, people doing Perl development professionally are overwhelmingly using very modern Perl, so the practical implication is that we only need to consider the distro Perl here, since everyone will be using something at least that new (or will have an easy way to build such a version).

I will point out that I specifically dropped it down from Perl 5.30 to 5.26 in the interests of SLES, even though I don't believe they're a major Linux distro anymore. I felt that given the fact that it was easy to support SLES, it would be better to do so, even if it sees relatively little use. I'm not aware of any other reasonably common distro supporting an older Perl.

As I said, nobody is supporting these systems. We, as contributors, cannot get a suitable (secure and functional, available at no charge) system to test on. Nobody has stepped up to volunteer to do this work and maintain these systems for the project. Our own policy, which we've discussed and agreed upon, is not to support them.

Absent somebody volunteering to do the work here, I'm proposing to drop support for them. I'm willing to do the work to adequately support Debian on all its architectures (to the best of my ability), and I'm willing to take into consideration other major platforms for which we have CI or for which I can reasonably be expected to test. I'm not willing to consider other systems where nobody has volunteered to step up and be responsible.

If other people in this thread are volunteering to be maintainers for these systems and add suitable CI jobs so that we can find problems before they land in `master`, I'll happily adjust my series accordingly. Please also propose a patch for the platform support policy which clearly states what our new policy should be so it can be discussed.

On Tue, Oct 15, 2024 at 7:57 PM brian m. carlson <sandals@crustytoothpaste.net> wrote:

The machine I use to build for SunOS takes, let's be generous and say an hour to build git from a fresh checkout. If I'm iterating on trying to fix something, run make, and see that it's building daemon.o, I know I've got another hour or so before I find out if my change worked, and maybe discover what the *next* new issue is. There are faster SunOS machines, but not the one I happen to have available. You would not want this machine in any sort of CI system. That said, until sometime this summer, I was building every release of git on that machine within days, often hours, of it being tagged, for *nearly 15 years*. If something broke, I'd fix it, test the build (which could take hours if I had to iterate), and submit a patch. You can find them in the logs. It was, fortunately, not that often, which is a testament to git remaining portable. Thank you all for that.

As I mentioned in my report regarding the SunOS build, I'm personally ready to abandon that particular use of my time, though if it's fixed, it'll go back onto my semi-automated build scripts for git releases, and I'll continue to submit patches as needed. It's not a CI, and no, I don't have notifications for and don't build RCs, but it's something.

Building a new perl is easy. Telling the system-controlled apache mod_perl to trust me and use my perl, less easy. (gitweb.perl.)

-Alejandro

On 10/11/24 4:01 PM, brian m. carlson wrote:

On a personal level I have a mild abhorrence of the general notion of bumping a version requirement in order to bump a requirement. I have a lot of sympathy for having a policy about what to expend effort on supporting, though!

Without getting into what the git project "should expend effort to support" (future efforts to be clear, not existing code that simply stays in place doing no significant harm)...

This patch series simplifies the codebase in order to remove workarounds for versions of curl < 7.56.0 -- it then documents the minimum supported version as 7.61.0 and there are even proposals to add a version check for that. Why? Is it currently believed that curl 7.56 through 7.60.x are going to break as a result of the modifications to ./INSTALL alone?

Instead I would suggest documenting the minimum version as 7.56 to align with reality.

Your general observation about respecting the platform support policy and not making developers expend time working around ancient dependency versions no one should be using... is something that I would say is a better fit for, well, the platform support policy.

You could instead add a section to the platform support policy detailing the minimum versions of dependencies which the git developers are willing to spend time supporting. A developer working on changes which would be onerous to backfill support for, would then have a simple, documented, easy to find policy about when it is acceptable to bump the version documented in ./INSTALL. The process would then look like:

- Code a new feature.

- Check the version table to see if maybe it was added basically
  yesterday in curl 8.7, or whether it is available in say, curl 7.75.

- Discover it was added in curl 7.59. Oh shoot! The ./INSTALL says we
  still support versions before that, but it's also super decrepit and
  nobody runs it anyway. But wait -- the platform support says we only
  care about 7.61.

- Shrug and grin. First patch in the series now bumps ./INSTALL to say
  the minimum required curl is 7.59, and if anyone disagrees then it's
  fair game to respond with. "fite me. The platform support says I don't
  have to care, we are making this change whether you like it or not".

The important distinction here is that in this model, the install requirements aren't about what you want to spend time on supporting, they are about truthfully communicating what *works* in point of fact.

Likewise, it does actually make sense to have a version check either in the build system or the code, but probably the build system, to ensure that the minimum required version which is necessary in order to successfully compile the codebase is available. It doesn't change what works and what fails -- it simply provides a clear error message. Instead of inscrutable compiler errors about CURLSSLSET_NO_BACKENDS not existing, you get:

Dependency libcurl found: NO. Found 7.51.0 but need: '>=7.56.0'

meson.build:642:7: ERROR: Dependency 'libcurl' is required but not found.

On 2024-10-22 at 03:34:25, Eli Schwartz wrote:

We already wrote this out. If it's supported in a major LTS (not extended LTS) distribution, then we support it; otherwise, we don't. I plan to add some specific CI jobs to cover common supported platforms in the future, so we actually test what we're supporting.

This is the approach we used to have, where we'd accept patches to support older systems if they weren't too invasive. It involved lots of heated discussions on the list that were unproductive and never came to a conclusion, and they'd repeat with some frequency. That's why we have the policy we have now: because it's clearer and more definitive and arguing extensively about what we were supporting was not in the interests of a healthy community for the project. It is also more honest in that we're clearly communicating to users whether they can expect things to work out of the box or whether they'll need to carry custom patches on their own.

Overall, people don't update the INSTALL documentation and it's routinely out of date. Should they? Yes, but practically they don't, and we don't test that, so we don't know if it's accurate.

While this sounds nice in principle, it doesn't work in practice. We don't test things like MIPS or UltraSPARC hardware because we don't have CI systems that use that hardware and they're extremely slow in emulation, but we do want to support them if they're on an otherwise supported OS. Similarly, we probably do want to support NetBSD, but have no tests for it.

We also don't have situations where, in general, people are willing to compile their own set of software from scratch. For example, I'm not compiling an arbitrary libcurl version to test a problem on the list. With very few exceptions, the versions people use are tied to their distribution or vendor. If someone asks to support libcurl 7.19, we either have to custom compile that to test or try to run CentOS 6, which no longer runs in a Docker container on a modern kernel and has no security support, so practically the answer is no.

So we don't know for certain what does and does work, but we do know what we're willing to fix and support.

On Thu, Oct 10, 2024 at 23:56:08 +0000, brian m. carlson wrote:

...

Strictly speaking, the first three of these in the series can be squashed, as support for libcurl older than 7.37.0 is already broken. Reverting any subset of these patches will not achieve the goal of allowing people to get back to a working build.

Personally, I'd still prefer to see support maintained, but on a more philosophical level, I agree that this patch series a better course of action.

-Alejandro

"brian m. carlson" <sandals@crustytoothpaste.net> writes:

Quite honestly, these two changes, each of which is a one-liner, are so boringly trivial for being "too noisy". But on the other hand, I am not sure it demonstrates why it is a "generally beneficial change" sufficiently well, either. The pre-s///r idiom

    (my $result = $orig_to_be_kept) =~ s/...//;

was concice enough that

    my $result = ($orig_to_be_kept =~ s/...//r);

does not make all that much improvement. Where it shines, I would imagine, is to rewrite an original that did not use the idiom using the 'r' modifier, but fortunately we didn't have such a code?

"brian m. carlson" <sandals@crustytoothpaste.net> writes:

This sounds compelling, however...

The change being made to illustrate the point is not at all compelling to me. This appears to be an update for the sake of an update, with very minor benefit at great compatibility cost.

I'm especially opposed to the change in gitweb/gitweb.perl, as that script is the one that is most likely to be used in a web-hosting environment where the user does not have control over the version of perl being used. And yes, those users would be better off hosting on a newer platform, but that's not a good reason to break them with no real gain for git.

-Alejandro

On Thu, Oct 10, 2024 at 11:56:08PM +0000, brian m. carlson wrote:

For what it's worth, this patch series breaks our GitLab pipeline because we still exercise Ubuntu 16.04, which uses an old version of curl that's not supported anymore after this patch series lands. We have just recently started adopted that job in GitLab because GitHub couldn't support it anymore, but we wanted to keep around the test coverage for such oldish platforms.

So if we want to declare Ubuntu 16.04 as unsupported, this patch series would also have to remove the CI job.

Patrick

For a long time, we ended up with protracted discussions on the mailing list about what versions of software we should support. Oftentimes, we broke long-obsolete operating system versions by using something shipped slightly more recently.

Fortunately, we now have a platform support policy to guide us in our approach to dependencies, so we can make updates without worrying about breaking systems that have not received security support in several years.

This series updates our requirements for libcurl to 7.61.0 (the version in RHEL 8) and for Perl to 5.26.0 (the version in 15.6). I considered the mainstream LTS versions of RHEL, Debian, Ubuntu, and SLES, but omitted consideration of paid support extended LTS, since we cannot expect Git developers to have to pay a large corporation lots of money just to test functionality. This is in conformance with our policy, which states that versions must be "in line with the version used by other long-term-support distributions", which does not include extended LTS distributions.

I plan to send a future series that will add some additional CI jobs in order to be sure that we're testing the major supported distros and avoid regressions.

Changes from v1:
* Use require instead of use for Perl to avoid enabling features.
* Use v5.26 instead of 5.026000.
* Squash the INSTALL documentation into the Perl changes.

brian m. carlson (12):
  git-curl-compat: remove check for curl 7.21.5
  git-curl-compat: remove check for curl 7.25.0
  git-curl-compat: remove check for curl 7.34.0
  git-curl-compat: remove check for curl 7.39.0
  git-curl-compat: remove check for curl 7.43.0
  git-curl-compat: remove check for curl 7.44.0
  git-curl-compat: remove check for curl 7.52.0
  git-curl-compat: remove check for curl 7.53.0
  git-curl-compat: remove check for curl 7.56.0
  INSTALL: document requirement for libcurl 7.61.0
  Require Perl 5.26.0
  gitweb: make use of s///r

 INSTALL                                 | 13 +---
 contrib/diff-highlight/DiffHighlight.pm |  2 +-
 contrib/mw-to-git/Git/Mediawiki.pm      |  2 +-
 git-archimport.perl                     |  2 +-
 git-curl-compat.h                       | 98 -------------------------
 git-cvsexportcommit.perl                |  2 +-
 git-cvsimport.perl                      |  2 +-
 git-cvsserver.perl                      |  2 +-
 git-send-email.perl                     |  2 +-
 git-svn.perl                            |  2 +-
 gitweb/gitweb.perl                      |  6 +-
 http.c                                  | 58 ---------------
 imap-send.c                             |  4 -
 perl/Git.pm                             |  2 +-
 perl/Git/I18N.pm                        |  2 +-
 perl/Git/LoadCPAN.pm                    |  2 +-
 perl/Git/Packet.pm                      |  2 +-
 t/t0202/test.pl                         |  2 +-
 t/t5562/invoke-with-content-length.pl   |  2 +-
 t/t9700/test.pl                         |  2 +-
 t/test-terminal.perl                    |  2 +-
 21 files changed, 23 insertions(+), 188 deletions(-)

libcurl 7.39.0 was released in November 2014, which is almost ten years ago, and no major operating system vendor is still providing security support for it. Debian 9 and Ubuntu 16.04, both of which are out of mainstream security support, have supported a newer version, and RHEL 8, which is still in support, also has a newer version.

Remove the check for this version and use this functionality unconditionally.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 git-curl-compat.h |  9 ---------
 http.c            | 11 -----------
 2 files changed, 20 deletions(-)

libcurl 7.34.0 was released in December 2013, which is well over ten years ago, and no major operating system vendor is still providing security support for it. Debian 8 and Ubuntu 14.04, both of which are out of mainstream security support, have supported a newer version, and RHEL 8, which is still in support, also has a newer version.

Remove the check for this version and use this functionality unconditionally.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 git-curl-compat.h | 22 ----------------------
 http.c            |  2 --
 imap-send.c       |  4 ----
 3 files changed, 28 deletions(-)

libcurl 7.21.5 was released in April 2011, which is well over ten years ago, and no major operating system vendor is still providing security support for it. Debian 7, RHEL 7, and Ubuntu 12.04, all of which are out of mainstream security support, have all supported a newer version.

Remove the check for this version and use this functionality unconditionally.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 git-curl-compat.h | 7 -------
 1 file changed, 7 deletions(-)

libcurl 7.25.0 was released in March 2012, which is well over ten years ago, and no major operating system vendor is still providing security support for it. Debian 8, RHEL 7, and Ubuntu 12.10, all of which are out of mainstream security support, have all supported a newer version.

Remove the check for this version and use this functionality unconditionally.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 git-curl-compat.h |  8 --------
 http.c            | 24 ------------------------
 2 files changed, 32 deletions(-)

libcurl 7.43.0 was released in June 2015, which is over nine years ago, and no major operating system vendor is still providing security support for it. Debian 9 and Ubuntu 16.04, both of which are out of mainstream security support, have supported a newer version, and RHEL 8, which is still in support, also has a newer version.

Remove the check for this version and use this functionality unconditionally.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 git-curl-compat.h | 11 -----------
 http.c            |  5 -----
 2 files changed, 16 deletions(-)

libcurl 7.44.0 was released in August 2015, which is over nine years ago, and no major operating system vendor is still providing security support for it. Debian 9 and Ubuntu 16.04, both of which are out of mainstream security support, have supported a newer version, and RHEL 8, which is still in support, also has a newer version.

Remove the check for this version and use this functionality unconditionally.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 git-curl-compat.h | 10 ----------
 http.c            |  4 ----
 2 files changed, 14 deletions(-)

libcurl 7.53.0 was released in February 2017, which is over seven years ago, and no major operating system vendor is still providing security support for it. Debian 10 and Ubuntu 18.04, both of which are out of mainstream security support, have supported a newer version, and RHEL 8, which is still in support, also has a newer version.

Remove the check for this version and use this functionality unconditionally.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 git-curl-compat.h | 8 --------
 http.c            | 2 --
 2 files changed, 10 deletions(-)

libcurl 7.56.0 was released in September 2017, which is over seven years ago, and no major operating system vendor is still providing security support for it. Debian 10, which is out of mainstream security support, has supported a newer version, and Ubuntu 20.04 and RHEL 8, which are still in support, also have a newer version.

Remove the check for this version and use this functionality unconditionally.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 git-curl-compat.h | 8 --------
 http.c            | 2 --
 2 files changed, 10 deletions(-)

In Perl 5.14, released in May 2011, the r modifier was added to the s/// operator to allow it to return the modified string instead of modifying the string in place. This allows to write nicer, more succinct code in several cases, so let's do that here.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 gitweb/gitweb.perl | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

On Wed, Oct 23, 2024 at 12:46:00AM +0000, brian m. carlson wrote:

"several" is a bit of an overstatement.

i'm a fan of "excess" parentheses where the syntax relies heavily on the binding and priority of operators:

   my $error = ($@ =~ s/ at \S+ line \d+.*\n?//r);

which is arguably semantically clearer than the old idiom, but not shorter.

i think

   map { "'".(s/(['!])/'\\$1'/gr)."'" } @_ );

should work, and is an actually significant improvement.

On 2024-10-23 at 12:34:13, Oswald Buddenhagen wrote:

I can rephrase if a v3 is necessary.

I don't think those are necessary. It's obvious to people who use the s///r idiom what's meant here, and in my experience most Perl code using that idiom doesn't use them.

I'm sorry, I don't necessarily like that much better than what we have now. It's not that I think it's awful, just that I don't think it's a significant improvement. If I do a v3, I can omit the `$_ =~`, though.

Our platform support policy states that we require "versions of dependencies which are generally accepted as stable and supportable, e.g., in line with the version used by other long-term-support distributions". Of Debian, Ubuntu, and RHEL, the three most common distributions that provide LTS versions, the version with mainstream long-term security support with the oldest libcurl is 7.61.0 in RHEL 8.

Update the documentation to state that this is the new base version for libcurl. Remove text that is no longer applicable to older versions.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 INSTALL | 11 +++--------
 1 file changed, 3 insertions(+), 8 deletions(-)

Our platform support policy states that we require "versions of dependencies which are generally accepted as stable and supportable, e.g., in line with the version used by other long-term-support distributions". Of Debian, Ubuntu, RHEL, and SLES, the four most common distributions that provide LTS versions, the version with mainstream long-term security support with the oldest Perl is 5.26.0 in SLES 15.6.

This is a major upgrade, since Perl 5.8.1, according to the Perl documentation, was released in September of 2003. It brings a lot of new features that we can choose to use, such as s///r to return the modified string, the postderef functionality, and subroutine signatures, although the latter was still considered experimental until 5.36.

This change was made with the following one-liner, which intentionally excludes modifying the vendored modules we include to avoid conflicts:

    git grep -l 'use 5.008001' | grep -v 'LoadCPAN/' | xargs perl -pi -e 's/use 5.008001/require v5.26/'

Use require instead of use to avoid changing the behavior as the latter enables features and the former does not.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 INSTALL                                 | 2 +-
 contrib/diff-highlight/DiffHighlight.pm | 2 +-
 contrib/mw-to-git/Git/Mediawiki.pm      | 2 +-
 git-archimport.perl                     | 2 +-
 git-cvsexportcommit.perl                | 2 +-
 git-cvsimport.perl                      | 2 +-
 git-cvsserver.perl                      | 2 +-
 git-send-email.perl                     | 2 +-
 git-svn.perl                            | 2 +-
 gitweb/gitweb.perl                      | 2 +-
 perl/Git.pm                             | 2 +-
 perl/Git/I18N.pm                        | 2 +-
 perl/Git/LoadCPAN.pm                    | 2 +-
 perl/Git/Packet.pm                      | 2 +-
 t/t0202/test.pl                         | 2 +-
 t/t5562/invoke-with-content-length.pl   | 2 +-
 t/t9700/test.pl                         | 2 +-
 t/test-terminal.perl                    | 2 +-
 18 files changed, 18 insertions(+), 18 deletions(-)

On October 22, 2024 8:46 PM, brian m. carlson wrote:

dependencies which

version used

SLES, the

SLES 15.6.

documentation, was

choose

functionality, and

experimental until

excludes

's/use

enables

Please be aware that the most recent version of Perl available on NonStop is currently 5.26.3. On the ia64 variant, we will not see a newer version *ever*. The x86 platform Supports 5.30.3 and may evolve. By the end of 2025, the ia64 platform goes away, so as long as we can keep 5.26.x as a minimum, that would be acceptable.

Thanks, Randall

libcurl 7.52.0 was released in August 2017, which is over seven years ago, and no major operating system vendor is still providing security support for it. Debian 9 and Ubuntu 18.04, both of which are out of mainstream security support, have supported a newer version, and RHEL 8, which is still in support, also has a newer version.

Remove the check for this version and use this functionality unconditionally.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 git-curl-compat.h | 15 ---------------
 http.c            |  8 --------
 2 files changed, 23 deletions(-)

On Wed, Oct 23, 2024 at 12:45:48AM +0000, brian m. carlson wrote:

Thanks, queued.

Thanks, Taylor

On Wed, Oct 23, 2024 at 04:16:59PM -0400, Taylor Blau wrote:

Note that this still breaks GitLab CI, as we exercise Ubuntu 16.04 there which doesn't have recent-enough versions of curl. This version has recently moved into Extended Security Maintenance mode, so the next LTS release would be Ubuntu 20.04.

So if this gets merged we should add something like the below patch on top.

Patrick

On 2024-10-24 at 06:05:49, Patrick Steinhardt wrote:

Yes, I think we should add that on top. I don't believe we should be testing or supporting Ubuntu 16.04, so 20.04 would be the right version to choose.