{"thread":{"id":"61422","subject":"[PATCH] ci: fix Python dependency on Ubuntu 24.04","startedAt":"2024-05-06T05:35:24Z","lastAt":"2024-05-07T04:08:40Z","messageCount":7,"participants":["Patrick Steinhardt","Justin Tobler","Junio C Hamano"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"494108","messageId":"cb8cefc20f373a3516695e7cbee975132553ea95.1714973381.git.ps@pks.im","threadId":"61422","inReplyTo":null,"subject":"[PATCH] ci: fix Python dependency on Ubuntu 24.04","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2024-05-06T05:35:17Z","receivedAt":"2024-05-06T05:35:24Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"Newer versions of Ubuntu have dropped Python 2 starting with Ubuntu\n23.04. By default though, our CI setups will try to use that Python\nversion on all Ubuntu-based jobs except for the \"linux-gcc\" one.\n\nWe didn't notice this issue due to two reasons:\n\n  - The \"ubuntu:latest\" tag always points to the latest LTS release.\n    Until a few weeks ago this was Ubuntu 22.04, which still had Python\n    2.\n\n  - Our Docker-based CI jobs had their own script to install\n    dependencies until 9cdeb34b96 (ci: merge scripts which install\n    dependencies, 2024-04-12), where we didn't even try to install\n    Python at all for many of them.\n\nSince the CI refactorings have originally been implemented, Ubuntu\n24.04 was released, and it being an LTS versions means that the \"latest\"\ntag now points to that Python-2-less version. Consequently, those jobs\nthat use \"ubuntu:latest\" broke.\n\nAddress this by using Python 2 on Ubuntu 20.04, only, whereas we use\nPython 3 on all other Ubuntu jobs. Eventually, we should think about\ndropping support for Python 2 completely.\n\nReported-by: Justin Tobler <jtobler@gmail.com>\nSigned-off-by: Patrick Steinhardt <ps@pks.im>\n---\n\nNote: this topic depends on ps/ci-test-with-jgit at 70b81fbf3c (t0612:\nadd tests to exercise Git/JGit reftable compatibility, 2024-04-12).\n\n ci/lib.sh | 8 ++++++--\n 1 file changed, 6 insertions(+), 2 deletions(-)\n\ndiff --git a/ci/lib.sh b/ci/lib.sh\nindex 473a2d0348..273f3540a6 100755\n--- a/ci/lib.sh\n+++ b/ci/lib.sh\n@@ -325,9 +325,13 @@ ubuntu-*)\n \t\tbreak\n \tfi\n \n-\tPYTHON_PACKAGE=python2\n-\tif test \"$jobname\" = linux-gcc\n+\t# Python 2 is end of life, and Ubuntu 23.04 and newer don't actually\n+\t# have it anymore. We thus only test with Python 2 on older LTS\n+\t# releases.\n+\tif \"$distro\" = \"ubuntu-20.04\"\n \tthen\n+\t\tPYTHON_PACKAGE=python2\n+\telse\n \t\tPYTHON_PACKAGE=python3\n \tfi\n \tMAKEFLAGS=\"$MAKEFLAGS PYTHON_PATH=/usr/bin/$PYTHON_PACKAGE\"\n\nbase-commit: b6db6b1598946fbf777e55ff0d187b11ff3bd21f\n-- \n2.45.0\n\n"},{"id":"494155","messageId":"c2dyqdqge3ypc7syissjm3mh6fqftojvincu4qizkuravc5ll6@bjgjlcas7n4k","threadId":"61422","inReplyTo":"cb8cefc20f373a3516695e7cbee975132553ea95.1714973381.git.ps@pks.im","subject":"Re: [PATCH] ci: fix Python dependency on Ubuntu 24.04","fromName":"Justin Tobler","fromEmail":"jltobler@gmail.com","sentAt":"2024-05-06T12:55:25Z","receivedAt":"2024-05-06T12:56:46Z","isPatch":true,"sender":{"key":"jltobler@gmail.com","avatar":"https://avatars.githubusercontent.com/u/53454972?v=4"},"body":"On 24/05/06 07:35AM, Patrick Steinhardt wrote:\n> Newer versions of Ubuntu have dropped Python 2 starting with Ubuntu\n> 23.04. By default though, our CI setups will try to use that Python\n> version on all Ubuntu-based jobs except for the \"linux-gcc\" one.\n\nNaive question, why were the \"linux-gcc\" jobs the only ones using\nPython 3?\n\n> \n> We didn't notice this issue due to two reasons:\n> \n>   - The \"ubuntu:latest\" tag always points to the latest LTS release.\n>     Until a few weeks ago this was Ubuntu 22.04, which still had Python\n>     2.\n> \n>   - Our Docker-based CI jobs had their own script to install\n>     dependencies until 9cdeb34b96 (ci: merge scripts which install\n>     dependencies, 2024-04-12), where we didn't even try to install\n>     Python at all for many of them.\n> \n> Since the CI refactorings have originally been implemented, Ubuntu\n> 24.04 was released, and it being an LTS versions means that the \"latest\"\n> tag now points to that Python-2-less version. Consequently, those jobs\n> that use \"ubuntu:latest\" broke.\n> \n> Address this by using Python 2 on Ubuntu 20.04, only, whereas we use\n> Python 3 on all other Ubuntu jobs. Eventually, we should think about\n> dropping support for Python 2 completely.\n> \n> Reported-by: Justin Tobler <jtobler@gmail.com>\n> Signed-off-by: Patrick Steinhardt <ps@pks.im>\n> ---\n> \n> Note: this topic depends on ps/ci-test-with-jgit at 70b81fbf3c (t0612:\n> add tests to exercise Git/JGit reftable compatibility, 2024-04-12).\n> \n>  ci/lib.sh | 8 ++++++--\n>  1 file changed, 6 insertions(+), 2 deletions(-)\n> \n> diff --git a/ci/lib.sh b/ci/lib.sh\n> index 473a2d0348..273f3540a6 100755\n> --- a/ci/lib.sh\n> +++ b/ci/lib.sh\n> @@ -325,9 +325,13 @@ ubuntu-*)\n>  \t\tbreak\n>  \tfi\n>  \n> -\tPYTHON_PACKAGE=python2\n> -\tif test \"$jobname\" = linux-gcc\n\nWith this change, \"linux-gcc\" jobs using ubuntu 20.04 will now use\nPython 2. Is that what we want?\n\n-Justin\n\n> +\t# Python 2 is end of life, and Ubuntu 23.04 and newer don't actually\n> +\t# have it anymore. We thus only test with Python 2 on older LTS\n> +\t# releases.\n> +\tif \"$distro\" = \"ubuntu-20.04\"\n>  \tthen\n> +\t\tPYTHON_PACKAGE=python2\n> +\telse\n>  \t\tPYTHON_PACKAGE=python3\n>  \tfi\n>  \tMAKEFLAGS=\"$MAKEFLAGS PYTHON_PATH=/usr/bin/$PYTHON_PACKAGE\"\n> \n> base-commit: b6db6b1598946fbf777e55ff0d187b11ff3bd21f\n> -- \n> 2.45.0\n> \n\n\n"},{"id":"494156","messageId":"ZjjU_Gaum8iYLks8@tanuki","threadId":"61422","inReplyTo":"c2dyqdqge3ypc7syissjm3mh6fqftojvincu4qizkuravc5ll6@bjgjlcas7n4k","subject":"Re: [PATCH] ci: fix Python dependency on Ubuntu 24.04","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2024-05-06T13:02:52Z","receivedAt":"2024-05-06T13:02:59Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Mon, May 06, 2024 at 07:55:25AM -0500, Justin Tobler wrote:\n> On 24/05/06 07:35AM, Patrick Steinhardt wrote:\n> > Newer versions of Ubuntu have dropped Python 2 starting with Ubuntu\n> > 23.04. By default though, our CI setups will try to use that Python\n> > version on all Ubuntu-based jobs except for the \"linux-gcc\" one.\n> \n> Naive question, why were the \"linux-gcc\" jobs the only ones using\n> Python 3?\n\nThis has been introduced in 6bb40ed20a (ci: use python3 in linux-gcc and\nosx-gcc and python2 elsewhere, 2020-01-23). It's been a first step\ntowards exercising Python 3 in our CI pipeline. The jobs were basically\npicked at random, there is no inherent reason.\n\nSo ultimately, it does not matter which jobs use what. But what does\nmatter is whether the image even supports Python 2. And as we nowadays\ndo have a fair split of jobs using \"ubuntu:20.04\" and \"ubuntu:latest\",\nI'd just use that the the criterium to pick the Python version.\n\nOnce we drop support for Ubuntu 20.04, we can then more forward and stop\nsupporting Python 2 altogether. Or maybe even earlier than that -- I do\nnot think it is reasonable to ask Git to maintain compatibility with a\nproject that is end of life already.\n\n[snip]\n> > diff --git a/ci/lib.sh b/ci/lib.sh\n> > index 473a2d0348..273f3540a6 100755\n> > --- a/ci/lib.sh\n> > +++ b/ci/lib.sh\n> > @@ -325,9 +325,13 @@ ubuntu-*)\n> >  \t\tbreak\n> >  \tfi\n> >  \n> > -\tPYTHON_PACKAGE=python2\n> > -\tif test \"$jobname\" = linux-gcc\n> \n> With this change, \"linux-gcc\" jobs using ubuntu 20.04 will now use\n> Python 2. Is that what we want?\n\nWell, as explained above, the selection was arbitrary in the first\nplace. Now it's less so as the choice is mandated by what is actually\nsupported.\n\nPatrick\n"},{"id":"494171","messageId":"xmqq7cg6zv7g.fsf@gitster.g","threadId":"61422","inReplyTo":"ZjjU_Gaum8iYLks8@tanuki","subject":"Re: [PATCH] ci: fix Python dependency on Ubuntu 24.04","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2024-05-06T17:49:55Z","receivedAt":"2024-05-06T17:49:58Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Patrick Steinhardt <ps@pks.im> writes:\n\n> So ultimately, it does not matter which jobs use what. But what does\n> matter is whether the image even supports Python 2. And as we nowadays\n> do have a fair split of jobs using \"ubuntu:20.04\" and \"ubuntu:latest\",\n> I'd just use that the the criterium to pick the Python version.\n>\n> Once we drop support for Ubuntu 20.04, we can then more forward and stop\n> supporting Python 2 altogether. Or maybe even earlier than that -- I do\n> not think it is reasonable to ask Git to maintain compatibility with a\n> project that is end of life already.\n\nYup.  If an distro sticks to Python 2 for whatever reason, they may\nwant to donate and dedicate CI resources, but otherwise it is not\nreally our job to keep compatibility on our side.\n\n>> With this change, \"linux-gcc\" jobs using ubuntu 20.04 will now use\n>> Python 2. Is that what we want?\n>\n> Well, as explained above, the selection was arbitrary in the first\n> place. Now it's less so as the choice is mandated by what is actually\n> supported.\n\nYup, as long as there is one that supports Python 2 among the\ndistros we happen to use for our CIs, we can pick that one.\nOtherwise we should drop.\n\nDropping Python 2 support does not have to wait for Git 3.0, I would\nsay.\n\nThanks.\n"},{"id":"494181","messageId":"7t2cqi643tvzjdeq4fddci2dqclgffkavps652cnf2swehx4uo@kng4bm5kvs6y","threadId":"61422","inReplyTo":"cb8cefc20f373a3516695e7cbee975132553ea95.1714973381.git.ps@pks.im","subject":"Re: [PATCH] ci: fix Python dependency on Ubuntu 24.04","fromName":"Justin Tobler","fromEmail":"jltobler@gmail.com","sentAt":"2024-05-06T19:06:04Z","receivedAt":"2024-05-06T19:07:25Z","isPatch":true,"sender":{"key":"jltobler@gmail.com","avatar":"https://avatars.githubusercontent.com/u/53454972?v=4"},"body":"On 24/05/06 07:35AM, Patrick Steinhardt wrote:\n> Newer versions of Ubuntu have dropped Python 2 starting with Ubuntu\n> 23.04. By default though, our CI setups will try to use that Python\n> version on all Ubuntu-based jobs except for the \"linux-gcc\" one.\n> \n> We didn't notice this issue due to two reasons:\n> \n>   - The \"ubuntu:latest\" tag always points to the latest LTS release.\n>     Until a few weeks ago this was Ubuntu 22.04, which still had Python\n>     2.\n> \n>   - Our Docker-based CI jobs had their own script to install\n>     dependencies until 9cdeb34b96 (ci: merge scripts which install\n>     dependencies, 2024-04-12), where we didn't even try to install\n>     Python at all for many of them.\n> \n> Since the CI refactorings have originally been implemented, Ubuntu\n> 24.04 was released, and it being an LTS versions means that the \"latest\"\n> tag now points to that Python-2-less version. Consequently, those jobs\n> that use \"ubuntu:latest\" broke.\n> \n> Address this by using Python 2 on Ubuntu 20.04, only, whereas we use\n> Python 3 on all other Ubuntu jobs. Eventually, we should think about\n> dropping support for Python 2 completely.\n> \n> Reported-by: Justin Tobler <jtobler@gmail.com>\n\nNot a big deal, but the email is slightly off. Should be:\n<jltobler@gmail.com>\n\nOtherwise this patch looks good to me. :)\n\n-Justin \n\n> Signed-off-by: Patrick Steinhardt <ps@pks.im>\n...\n"},{"id":"494184","messageId":"xmqqjzk6yc2v.fsf@gitster.g","threadId":"61422","inReplyTo":"7t2cqi643tvzjdeq4fddci2dqclgffkavps652cnf2swehx4uo@kng4bm5kvs6y","subject":"Re: [PATCH] ci: fix Python dependency on Ubuntu 24.04","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2024-05-06T19:28:24Z","receivedAt":"2024-05-06T19:28:31Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Justin Tobler <jltobler@gmail.com> writes:\n\n>> Reported-by: Justin Tobler <jtobler@gmail.com>\n>\n> Not a big deal, but the email is slightly off. Should be:\n> <jltobler@gmail.com>\n>\n> Otherwise this patch looks good to me. :)\n\nThe patch e-mail also went to that \"slight off\" address, which is\nfunny.  I've amended the Reported-by: line locally while queuing.\n\nThanks, both of you.\n"},{"id":"494217","messageId":"ZjmpQ4zdlarP4YpF@tanuki","threadId":"61422","inReplyTo":"7t2cqi643tvzjdeq4fddci2dqclgffkavps652cnf2swehx4uo@kng4bm5kvs6y","subject":"Re: [PATCH] ci: fix Python dependency on Ubuntu 24.04","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2024-05-07T04:08:35Z","receivedAt":"2024-05-07T04:08:40Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Mon, May 06, 2024 at 02:06:04PM -0500, Justin Tobler wrote:\n> On 24/05/06 07:35AM, Patrick Steinhardt wrote:\n> > Newer versions of Ubuntu have dropped Python 2 starting with Ubuntu\n> > 23.04. By default though, our CI setups will try to use that Python\n> > version on all Ubuntu-based jobs except for the \"linux-gcc\" one.\n> > \n> > We didn't notice this issue due to two reasons:\n> > \n> >   - The \"ubuntu:latest\" tag always points to the latest LTS release.\n> >     Until a few weeks ago this was Ubuntu 22.04, which still had Python\n> >     2.\n> > \n> >   - Our Docker-based CI jobs had their own script to install\n> >     dependencies until 9cdeb34b96 (ci: merge scripts which install\n> >     dependencies, 2024-04-12), where we didn't even try to install\n> >     Python at all for many of them.\n> > \n> > Since the CI refactorings have originally been implemented, Ubuntu\n> > 24.04 was released, and it being an LTS versions means that the \"latest\"\n> > tag now points to that Python-2-less version. Consequently, those jobs\n> > that use \"ubuntu:latest\" broke.\n> > \n> > Address this by using Python 2 on Ubuntu 20.04, only, whereas we use\n> > Python 3 on all other Ubuntu jobs. Eventually, we should think about\n> > dropping support for Python 2 completely.\n> > \n> > Reported-by: Justin Tobler <jtobler@gmail.com>\n> \n> Not a big deal, but the email is slightly off. Should be:\n> <jltobler@gmail.com>\n> \n> Otherwise this patch looks good to me. :)\n\nYou know, every single time I copy over addresses. This one time I\ndidn't and of course I immediately mistype it. Well, lesson learned,\ncopy & paste it is.\n\nPatrick\n"}]}