{"thread":{"id":"60173","subject":"[PATCH 0/2] replacing ci/config/allow-ref with a repo variable","startedAt":"2023-08-30T22:36:52Z","lastAt":"2023-09-14T10:49:32Z","messageCount":18,"participants":["Jeff King","Junio C Hamano","Phillip Wood","Johannes Schindelin"],"isPatch":true,"patchVersion":1,"patchTotal":2},"messages":[{"id":"481197","messageId":"20230830194919.GA1709446@coredump.intra.peff.net","threadId":"60173","inReplyTo":null,"subject":"[PATCH 0/2] replacing ci/config/allow-ref with a repo variable","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2023-08-30T19:49:19Z","receivedAt":"2023-08-30T22:36:52Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"This is a more efficient way to do the same thing that\nci/config/allow-ref does (which didn't exist back then).\n\nWe should be able to do the same with ci/config/skip-concurrent (and\njust use vars.CI_SKIP_CONCURRENT) throughout the workflow. But I didn't\ntest that at all.\n\nAfter that, the only useful thing left in the \"config\" job would be the\n\"skip-if-redundant\" step. I'm not sure if it will be possible to get the\nsame behavior there without spinning up a VM.\n\n  [1/2]: ci: allow branch selection through \"vars\"\n  [2/2]: ci: deprecate ci/config/allow-ref script\n\n .github/workflows/main.yml | 10 +++++++---\n ci/config/README           | 14 ++++++++++++++\n ci/config/allow-ref.sample | 27 ---------------------------\n 3 files changed, 21 insertions(+), 30 deletions(-)\n create mode 100644 ci/config/README\n delete mode 100755 ci/config/allow-ref.sample\n\n"},{"id":"481198","messageId":"20230830195113.GA1709824@coredump.intra.peff.net","threadId":"60173","inReplyTo":"20230830194919.GA1709446@coredump.intra.peff.net","subject":"[PATCH 1/2] ci: allow branch selection through \"vars\"","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2023-08-30T19:51:13Z","receivedAt":"2023-08-30T22:38:13Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"When we added config to skip CI for certain branches in e76eec3554 (ci:\nallow per-branch config for GitHub Actions, 2020-05-07), there wasn't\nany way to avoid spinning up a VM just to check the config. From the\ndeveloper's perspective this isn't too bad, as the \"skipped\" branches\ncomplete successfully after running the config job (the workflow result\nis \"success\" instead of \"skipped\", but that is a minor lie).\n\nBut we are still wasting time and GitHub's CPU to spin up a VM just to\ncheck the result of a short shell script. At the time there wasn't any\nway to avoid this. But they've since introduced repo-level variables\nthat should let us do the same thing:\n\n  https://github.blog/2023-01-10-introducing-required-workflows-and-configuration-variables-to-github-actions/#configuration-variables\n\nThis is more efficient, and as a bonus is probably less confusing to\nconfigure (the existing system requires sticking your config on a magic\nref).\n\nSee the included docs for how to configure it.\n\nThe code itself is pretty simple: it checks the variable and skips the\nconfig job if appropriate (and everything else depends on the config job\nalready). There are two slight inaccuracies here:\n\n  - we don't insist on branches, so this likewise applies to tag names\n    or other refs. I think in practice this is OK, and keeping the code\n    (and docs) short is more important than trying to be more exact. We\n    are targeting developers of git.git and their limited workflows.\n\n  - the match is done as a substring (so if you want to run CI for\n    \"foobar\", then branch \"foo\" will accidentally match). Again, this\n    should be OK in practice, as anybody who uses this is likely to only\n    specify a handful of well-known names. If we want to be more exact,\n    we can have the code check for adjoining spaces. Or even move to a\n    more general CI_CONFIG variable formatted as JSON. I went with this\n    scheme for the sake of simplicity.\n\nSigned-off-by: Jeff King <peff@peff.net>\n---\n .github/workflows/main.yml |  1 +\n ci/config/README           | 14 ++++++++++++++\n 2 files changed, 15 insertions(+)\n create mode 100644 ci/config/README\n\ndiff --git a/.github/workflows/main.yml b/.github/workflows/main.yml\nindex 1b41278a7f..c364abb8f8 100644\n--- a/.github/workflows/main.yml\n+++ b/.github/workflows/main.yml\n@@ -21,6 +21,7 @@ concurrency:\n jobs:\n   ci-config:\n     name: config\n+    if: vars.CI_BRANCHES == '' || contains(vars.CI_BRANCHES, github.ref_name)\n     runs-on: ubuntu-latest\n     outputs:\n       enabled: ${{ steps.check-ref.outputs.enabled }}${{ steps.skip-if-redundant.outputs.enabled }}\ndiff --git a/ci/config/README b/ci/config/README\nnew file mode 100644\nindex 0000000000..8de3a04e32\n--- /dev/null\n+++ b/ci/config/README\n@@ -0,0 +1,14 @@\n+You can configure some aspects of the GitHub Actions-based CI on a\n+per-repository basis by setting \"variables\" and \"secrets\" from with the\n+GitHub web interface. These can be found at:\n+\n+  https://github.com/<user>/git/settings/secrets/actions\n+\n+The following variables can be used:\n+\n+ - CI_BRANCHES\n+\n+   By default, CI is run when any branch is pushed. If this variable is\n+   non-empty, then only the branches it lists will run CI. Branch names\n+   should be separated by spaces, and should use their shortened form\n+   (e.g., \"main\", not \"refs/heads/main\").\n-- \n2.42.0.528.g5e092cb179\n\n"},{"id":"481199","messageId":"20230830195132.GB1709824@coredump.intra.peff.net","threadId":"60173","inReplyTo":"20230830194919.GA1709446@coredump.intra.peff.net","subject":"[PATCH 2/2] ci: deprecate ci/config/allow-ref script","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2023-08-30T19:51:32Z","receivedAt":"2023-08-30T22:38:27Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"Now that we have the CI_BRANCHES mechanism, there is no need for anybody\nto use the ci/config/allow-ref mechanism. In the long run, we can\nhopefully remove it and the whole \"config\" job, as it consumes CPU and\nadds to the end-to-end latency of the whole workflow. But we don't want\nto do that immediately, as people need time to migrate until the\nCI_BRANCHES change has made it into the workflow file of every branch.\n\nSo let's issue a warning, which will appear in the \"annotations\" section\nbelow the workflow result in GitHub's web interface. And let's remove\nthe sample allow-refs script, as we don't want to encourage anybody to\nuse it.\n\nSigned-off-by: Jeff King <peff@peff.net>\n---\n .github/workflows/main.yml |  9 ++++++---\n ci/config/allow-ref.sample | 27 ---------------------------\n 2 files changed, 6 insertions(+), 30 deletions(-)\n delete mode 100755 ci/config/allow-ref.sample\n\ndiff --git a/.github/workflows/main.yml b/.github/workflows/main.yml\nindex c364abb8f8..dcf7d78f1d 100644\n--- a/.github/workflows/main.yml\n+++ b/.github/workflows/main.yml\n@@ -44,10 +44,13 @@ jobs:\n         name: check whether CI is enabled for ref\n         run: |\n           enabled=yes\n-          if test -x config-repo/ci/config/allow-ref &&\n-             ! config-repo/ci/config/allow-ref '${{ github.ref }}'\n+          if test -x config-repo/ci/config/allow-ref\n           then\n-            enabled=no\n+            echo \"::warning::ci/config/allow-ref is deprecated; use CI_BRANCHES instead\"\n+            if ! config-repo/ci/config/allow-ref '${{ github.ref }}'\n+            then\n+              enabled=no\n+            fi\n           fi\n \n           skip_concurrent=yes\ndiff --git a/ci/config/allow-ref.sample b/ci/config/allow-ref.sample\ndeleted file mode 100755\nindex af0e076f8a..0000000000\n--- a/ci/config/allow-ref.sample\n+++ /dev/null\n@@ -1,27 +0,0 @@\n-#!/bin/sh\n-#\n-# Sample script for enabling/disabling GitHub Actions CI runs on\n-# particular refs. By default, CI is run for all branches pushed to\n-# GitHub. You can override this by dropping the \".sample\" from the script,\n-# editing it, committing, and pushing the result to the \"ci-config\" branch of\n-# your repository:\n-#\n-#   git checkout -b ci-config\n-#   cp allow-ref.sample allow-ref\n-#   $EDITOR allow-ref\n-#   git add allow-ref\n-#   git commit -am \"implement my ci preferences\"\n-#   git push\n-#\n-# This script will then be run when any refs are pushed to that repository. It\n-# gets the fully qualified refname as the first argument, and should exit with\n-# success only for refs for which you want to run CI.\n-\n-case \"$1\" in\n-# allow one-off tests by pushing to \"for-ci\" or \"for-ci/mybranch\"\n-refs/heads/for-ci*) true ;;\n-# always build your integration branch\n-refs/heads/my-integration-branch) true ;;\n-# don't build any other branches or tags\n-*) false ;;\n-esac\n-- \n2.42.0.528.g5e092cb179\n"},{"id":"481200","messageId":"xmqq5y4w87ij.fsf@gitster.g","threadId":"60173","inReplyTo":"20230830194919.GA1709446@coredump.intra.peff.net","subject":"Re: [PATCH 0/2] replacing ci/config/allow-ref with a repo variable","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2023-08-30T22:55:48Z","receivedAt":"2023-08-30T22:56:44Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jeff King <peff@peff.net> writes:\n\n> This is a more efficient way to do the same thing that\n> ci/config/allow-ref does (which didn't exist back then).\n\nVery nice.\n\n> After that, the only useful thing left in the \"config\" job would be the\n> \"skip-if-redundant\" step. I'm not sure if it will be possible to get the\n> same behavior there without spinning up a VM.\n>\n>   [1/2]: ci: allow branch selection through \"vars\"\n>   [2/2]: ci: deprecate ci/config/allow-ref script\n>\n>  .github/workflows/main.yml | 10 +++++++---\n>  ci/config/README           | 14 ++++++++++++++\n>  ci/config/allow-ref.sample | 27 ---------------------------\n>  3 files changed, 21 insertions(+), 30 deletions(-)\n>  create mode 100644 ci/config/README\n>  delete mode 100755 ci/config/allow-ref.sample\n"},{"id":"481288","messageId":"8624fc43-ab42-442b-a141-851fc35dd24f@gmail.com","threadId":"60173","inReplyTo":"20230830194919.GA1709446@coredump.intra.peff.net","subject":"Re: [PATCH 0/2] replacing ci/config/allow-ref with a repo variable","fromName":"Phillip Wood","fromEmail":"phillip.wood123@gmail.com","sentAt":"2023-09-01T13:24:59Z","receivedAt":"2023-09-01T13:25:03Z","isPatch":true,"sender":{"key":"phillip.wood@dunelm.org.uk","avatar":null},"body":"Hi Peff\n\nOn 30/08/2023 20:49, Jeff King wrote:\n> This is a more efficient way to do the same thing that\n> ci/config/allow-ref does (which didn't exist back then).\n\nI like the idea of a more efficient way to skip the ci for certain refs. \nI've got my allow-ref script set up to reject a bunch of refs and run \nthe ci on everything else. It's not clear to me how to replicate that \nwith the setup proposed here. Would it be possible to add a second \nvariable that prevents the ci from being run if it contains ref being \npushed?\n\nBest Wishes\n\nPhillip\n\n> We should be able to do the same with ci/config/skip-concurrent (and\n> just use vars.CI_SKIP_CONCURRENT) throughout the workflow. But I didn't\n> test that at all.\n> \n> After that, the only useful thing left in the \"config\" job would be the\n> \"skip-if-redundant\" step. I'm not sure if it will be possible to get the\n> same behavior there without spinning up a VM.\n> \n>    [1/2]: ci: allow branch selection through \"vars\"\n>    [2/2]: ci: deprecate ci/config/allow-ref script\n> \n>   .github/workflows/main.yml | 10 +++++++---\n>   ci/config/README           | 14 ++++++++++++++\n>   ci/config/allow-ref.sample | 27 ---------------------------\n>   3 files changed, 21 insertions(+), 30 deletions(-)\n>   create mode 100644 ci/config/README\n>   delete mode 100755 ci/config/allow-ref.sample\n> \n\n"},{"id":"481303","messageId":"20230901173214.GA1947546@coredump.intra.peff.net","threadId":"60173","inReplyTo":"8624fc43-ab42-442b-a141-851fc35dd24f@gmail.com","subject":"Re: [PATCH 0/2] replacing ci/config/allow-ref with a repo variable","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2023-09-01T17:32:14Z","receivedAt":"2023-09-01T17:32:18Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Fri, Sep 01, 2023 at 02:24:59PM +0100, Phillip Wood wrote:\n\n> On 30/08/2023 20:49, Jeff King wrote:\n> > This is a more efficient way to do the same thing that\n> > ci/config/allow-ref does (which didn't exist back then).\n> \n> I like the idea of a more efficient way to skip the ci for certain refs.\n> I've got my allow-ref script set up to reject a bunch of refs and run the ci\n> on everything else. It's not clear to me how to replicate that with the\n> setup proposed here. Would it be possible to add a second variable that\n> prevents the ci from being run if it contains ref being pushed?\n\nDrat, I was hoping nobody was using it that way. :)\n\nYes, I think it would be possible to do something like:\n\n  if: |\n    (vars.CI_BRANCHES == '' || contains(vars.CI_BRANCHES, github.ref_name)) &&\n    !contains(vars.CI_BRANCHES_REJECT, github.ref_name)\n\nIt doesn't allow globbing, though. Do you need that?\n\n-Peff\n"},{"id":"481348","messageId":"d000f469-2b7d-e037-c92e-013034490461@gmx.de","threadId":"60173","inReplyTo":"20230830195113.GA1709824@coredump.intra.peff.net","subject":"Re: [PATCH 1/2] ci: allow branch selection through \"vars\"","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2023-09-03T08:59:37Z","receivedAt":"2023-09-03T09:01:02Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Jeff,\n\nOn Wed, 30 Aug 2023, Jeff King wrote:\n\n> diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml\n> index 1b41278a7f..c364abb8f8 100644\n> --- a/.github/workflows/main.yml\n> +++ b/.github/workflows/main.yml\n> @@ -21,6 +21,7 @@ concurrency:\n>  jobs:\n>    ci-config:\n>      name: config\n> +    if: vars.CI_BRANCHES == '' || contains(vars.CI_BRANCHES, github.ref_name)\n\nThis might be too loose a check, as branch names that are a substring of\nany name listed in `CI_BRANCHES` would be false positive match. For\nexample, if `CI_BRANCHES` was set to `maint next seen`, a branch called\n`see` would be a false match.\n\nDue to the absence of a `concat()` function (for more details, see\nhttps://docs.github.com/en/actions/learn-github-actions/expressions#functions),\nI fear that we'll have to resort to something like `contains(format(' {0} ',\nvars.CI_BRANCHES), format(' {0} ', github.ref_name))`.\n\nCiao,\nJohannes\n"},{"id":"481362","messageId":"d3d1109b-3a1f-4e8b-be8d-6581d45f1b81@gmail.com","threadId":"60173","inReplyTo":"20230901173214.GA1947546@coredump.intra.peff.net","subject":"Re: [PATCH 0/2] replacing ci/config/allow-ref with a repo variable","fromName":"Phillip Wood","fromEmail":"phillip.wood123@gmail.com","sentAt":"2023-09-04T09:56:15Z","receivedAt":"2023-09-04T09:56:45Z","isPatch":true,"sender":{"key":"phillip.wood@dunelm.org.uk","avatar":null},"body":"On 01/09/2023 18:32, Jeff King wrote:\n> On Fri, Sep 01, 2023 at 02:24:59PM +0100, Phillip Wood wrote:\n> \n>> On 30/08/2023 20:49, Jeff King wrote:\n>>> This is a more efficient way to do the same thing that\n>>> ci/config/allow-ref does (which didn't exist back then).\n>>\n>> I like the idea of a more efficient way to skip the ci for certain refs.\n>> I've got my allow-ref script set up to reject a bunch of refs and run the ci\n>> on everything else. It's not clear to me how to replicate that with the\n>> setup proposed here. Would it be possible to add a second variable that\n>> prevents the ci from being run if it contains ref being pushed?\n> \n> Drat, I was hoping nobody was using it that way. :)\n\nSorry to be a pain.\n\n> Yes, I think it would be possible to do something like:\n> \n>    if: |\n>      (vars.CI_BRANCHES == '' || contains(vars.CI_BRANCHES, github.ref_name)) &&\n>      !contains(vars.CI_BRANCHES_REJECT, github.ref_name)\n> \n> It doesn't allow globbing, though. Do you need that?\n\nOh I'd missed that, yes I do. All the globs are prefix matches but I'm \nnot sure that helps.\n\nBest Wishes\n\nPhillip\n\n> -Peff\n\n"},{"id":"481381","messageId":"44a5d2d4-a6f1-7259-504e-269ee389c8ea@gmx.de","threadId":"60173","inReplyTo":"20230905073013.GI199565@coredump.intra.peff.net","subject":"Re: [PATCH 1/2] ci: allow branch selection through \"vars\"","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2023-09-05T10:51:14Z","receivedAt":"2023-09-05T15:59:49Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi Jeff,\n\nOn Tue, 5 Sep 2023, Jeff King wrote:\n\n> [...] coupled with Phillip's use cases in the other part of the thread,\n> maybe we should have a JSON-formatted CI_CONFIG variable instead.\n>\n> That requires the developer to hand-write a bit of JSON, but it's not\n> too bad (and again, I really think it's only a couple folks using this).\n>\n> What do you think?\n\nThank you for asking my opinion. The `[no ci]` support described in\nhttps://github.blog/changelog/2021-02-08-github-actions-skip-pull-request-and-push-workflows-with-skip-ci/\nsolves the problem adequately and with a lot less complexity than the\ncurrent or the `vars.`-based solution. In my opinion.\n\nCiao,\nJohannes\n"},{"id":"481382","messageId":"20230905073013.GI199565@coredump.intra.peff.net","threadId":"60173","inReplyTo":"d000f469-2b7d-e037-c92e-013034490461@gmx.de","subject":"Re: [PATCH 1/2] ci: allow branch selection through \"vars\"","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2023-09-05T07:30:13Z","receivedAt":"2023-09-05T15:59:56Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Sun, Sep 03, 2023 at 10:59:37AM +0200, Johannes Schindelin wrote:\n\n> > diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml\n> > index 1b41278a7f..c364abb8f8 100644\n> > --- a/.github/workflows/main.yml\n> > +++ b/.github/workflows/main.yml\n> > @@ -21,6 +21,7 @@ concurrency:\n> >  jobs:\n> >    ci-config:\n> >      name: config\n> > +    if: vars.CI_BRANCHES == '' || contains(vars.CI_BRANCHES, github.ref_name)\n> \n> This might be too loose a check, as branch names that are a substring of\n> any name listed in `CI_BRANCHES` would be false positive match. For\n> example, if `CI_BRANCHES` was set to `maint next seen`, a branch called\n> `see` would be a false match.\n\nYes, I wrote it about it in the commit message. :)\n\nMy assumption is that this may be good enough, just because we are\nrealistically talking about the needs of a handful of Git developers.\nFolks doing one-off patches would just push to their forks and get CI\n(which they'd want in order to use GGG anyway). This is for people with\nmore exotic workflows, and my guess is that half a dozen people would\nuse this at all.\n\nBut we can make it more robust if we think somebody will actually run\ninto it in practice.\n\n> Due to the absence of a `concat()` function (for more details, see\n> https://docs.github.com/en/actions/learn-github-actions/expressions#functions),\n> I fear that we'll have to resort to something like `contains(format(' {0} ',\n> vars.CI_BRANCHES), format(' {0} ', github.ref_name))`.\n\nYeah, I had imagined checking startsWith() and endsWith(), but\nauto-inserting the leading/trailing space as you suggest is even\nshorter.\n\nI think that contains() is more robust if used against an actual list\ndata structure.  But there doesn't seem to be any split()-type function.\nSo I don't see how to get one short of using fromJSON(). But coupled\nwith Phillip's use cases in the other part of the thread, maybe we\nshould have a JSON-formatted CI_CONFIG variable instead.\n\nThat requires the developer to hand-write a bit of JSON, but it's not\ntoo bad (and again, I really think it's only a couple folks using this).\n\nWhat do you think?\n\n-Peff\n"},{"id":"481387","messageId":"20230905072444.GH199565@coredump.intra.peff.net","threadId":"60173","inReplyTo":"d3d1109b-3a1f-4e8b-be8d-6581d45f1b81@gmail.com","subject":"Re: [PATCH 0/2] replacing ci/config/allow-ref with a repo variable","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2023-09-05T07:24:44Z","receivedAt":"2023-09-05T16:00:07Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Mon, Sep 04, 2023 at 10:56:15AM +0100, Phillip Wood wrote:\n\n> > Yes, I think it would be possible to do something like:\n> > \n> >    if: |\n> >      (vars.CI_BRANCHES == '' || contains(vars.CI_BRANCHES, github.ref_name)) &&\n> >      !contains(vars.CI_BRANCHES_REJECT, github.ref_name)\n> > \n> > It doesn't allow globbing, though. Do you need that?\n> \n> Oh I'd missed that, yes I do. All the globs are prefix matches but I'm not\n> sure that helps.\n\nIt does make it easier. There's no globbing function available to us,\nbut if we know something is a prefix, there's a startsWith() we can use.\nIt does seem we're getting a combinatorial expansion of things to check,\nthough:\n\n  - full names to accept\n  - full names to reject\n  - prefixes to accept\n  - prefixes to reject\n\nI wrote \"prefixes\" but I'm actually not sure how feasible that is. That\nimplies iterating over the list of prefixes, which I'm not sure we can\ndo.\n\n-Peff\n"},{"id":"481466","messageId":"fcdc682b-cf6c-4db9-9970-be136f48de58@gmail.com","threadId":"60173","inReplyTo":"20230905072444.GH199565@coredump.intra.peff.net","subject":"Re: [PATCH 0/2] replacing ci/config/allow-ref with a repo variable","fromName":"Phillip Wood","fromEmail":"phillip.wood123@gmail.com","sentAt":"2023-09-07T10:04:33Z","receivedAt":"2023-09-07T15:26:34Z","isPatch":true,"sender":{"key":"phillip.wood@dunelm.org.uk","avatar":null},"body":"On 05/09/2023 08:24, Jeff King wrote:\n> On Mon, Sep 04, 2023 at 10:56:15AM +0100, Phillip Wood wrote:\n> \n>>> Yes, I think it would be possible to do something like:\n>>>\n>>>     if: |\n>>>       (vars.CI_BRANCHES == '' || contains(vars.CI_BRANCHES, github.ref_name)) &&\n>>>       !contains(vars.CI_BRANCHES_REJECT, github.ref_name)\n>>>\n>>> It doesn't allow globbing, though. Do you need that?\n>>\n>> Oh I'd missed that, yes I do. All the globs are prefix matches but I'm not\n>> sure that helps.\n> \n> It does make it easier. There's no globbing function available to us,\n> but if we know something is a prefix, there's a startsWith() we can use.\n> It does seem we're getting a combinatorial expansion of things to check,\n> though:\n> \n>    - full names to accept\n>    - full names to reject\n>    - prefixes to accept\n>    - prefixes to reject\n> \n> I wrote \"prefixes\" but I'm actually not sure how feasible that is. That\n> implies iterating over the list of prefixes, which I'm not sure we can\n> do.\n\nI scanned the github documentation the other day and wondered if it \nwould be possible to use with fromJson with a json array to do a prefx \nmatch on each element. It all sounds like it is getting a bit \ncomplicated though.\n\nBest Wishes\n\nPhillip\n\n> -Peff\n\n"},{"id":"481500","messageId":"20230907074713.GA1260471@coredump.intra.peff.net","threadId":"60173","inReplyTo":"44a5d2d4-a6f1-7259-504e-269ee389c8ea@gmx.de","subject":"Re: [PATCH 1/2] ci: allow branch selection through \"vars\"","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2023-09-07T07:47:13Z","receivedAt":"2023-09-07T18:54:35Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Tue, Sep 05, 2023 at 12:51:14PM +0200, Johannes Schindelin wrote:\n\n> Thank you for asking my opinion. The `[no ci]` support described in\n> https://github.blog/changelog/2021-02-08-github-actions-skip-pull-request-and-push-workflows-with-skip-ci/\n> solves the problem adequately and with a lot less complexity than the\n> current or the `vars.`-based solution. In my opinion.\n\nUnfortunately that doesn't work well for the uses cases allow-ref was\nmeant to support, for the reasons given in e76eec3554 (ci: allow\nper-branch config for GitHub Actions, 2020-05-07).\n\n-Peff\n"},{"id":"481716","messageId":"20230911093616.GA1605460@coredump.intra.peff.net","threadId":"60173","inReplyTo":"fcdc682b-cf6c-4db9-9970-be136f48de58@gmail.com","subject":"Re: [PATCH 0/2] replacing ci/config/allow-ref with a repo variable","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2023-09-11T09:36:16Z","receivedAt":"2023-09-11T23:02:38Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Thu, Sep 07, 2023 at 11:04:33AM +0100, Phillip Wood wrote:\n\n> I scanned the github documentation the other day and wondered if it would be\n> possible to use with fromJson with a json array to do a prefx match on each\n> element. It all sounds like it is getting a bit complicated though.\n\nI poked around and I don't think this is possible. You can use\ncontains() to do a full match of items in a json array, but I don't see\nany other way to iterate. What we'd really want is a \"map\" function, to\ndo something like:\n\n  map { startsWith(github.ref_name, $1) } fromJSON(vars.CI_CONFIG.allow_prefix)\n\nBut no such \"map\" exists (and I guess we'd need to collapse the result\ndown to \"is any item true\", similar to perl's \"grep\" function).\n\nDo you need multiple prefixes, or would one each of allow/reject be\nsufficient? I tried this and it works:\n\njobs:\n  ci-config:\n    name: config\n    if: |\n      (fromJSON(vars.CI_CONFIG).allow == '' ||\n       contains(fromJSON(vars.CI_CONFIG).allow, github.ref_name)) &&\n      (fromJSON(vars.CI_CONFIG).reject == '' ||\n       !contains(fromJSON(vars.CI_CONFIG).reject, github.ref_name)) &&\n      (fromJSON(vars.CI_CONFIG).allow-prefix == '' ||\n       startsWith(github.ref_name, fromJSON(vars.CI_CONFIG).allow-prefix)) &&\n      (fromJSON(vars.CI_CONFIG).reject-prefix == '' ||\n       !startsWith(github.ref_name, fromJSON(vars.CI_CONFIG).reject-prefix))\n\nAnd then various values of CI_CONFIG seem to work:\n\n  - allow explicit branch names (\"one\" and \"two\" are run, everything\n    else is skipped)\n\n    { \"allow\": [\"one\", \"two\"] }\n\n  - reject explicit names (everything except \"three\" is skipped)\n\n    { \"reject\": [\"three\"] }\n\n  - allow by prefix (\"ok/* is run, everything else is skipped)\n\n    { \"allow-prefix\": \"ok/\" }\n\n  - reject by prefix (everything except \"nope/*\" is run)\n\n    { \"reject-prefix\": \"nope/\" }\n\n  - every key must approve to run, but missing keys default to running.\n    So a reject overrides allow (\"ok/one\" and \"ok/two\" run, but\n    \"ok/three\" does not)\n\n    {\n      \"allow-prefix\": \"ok/\",\n      \"reject\", \"ok/three\"\n    }\n\nI suppose one hacky way to support multiple prefixes is to just unroll\nthe loop ourselves (since missing keys are ignored). I didn't test it,\nbut something like:\n\n      (fromJSON(vars.CI_CONFIG).allow-prefix[0] == '' ||\n       startsWith(github.ref_name, fromJSON(vars.CI_CONFIG).allow-prefix[0])) &&\n      (fromJSON(vars.CI_CONFIG).allow-prefix[1] == '' ||\n       startsWith(github.ref_name, fromJSON(vars.CI_CONFIG).allow-prefix[1])) &&\n      (fromJSON(vars.CI_CONFIG).allow-prefix[2] == '' ||\n       startsWith(github.ref_name, fromJSON(vars.CI_CONFIG).allow-prefix[2])) &&\n       ...etc...\n\nWould allow:\n\n  {\n    \"allow-prefix\": [\n      \"ok/\",\n      \"also-ok/\"\n    ]\n  }\n\nLooking at the ci-config branch of phillipwood/git.git, I see this in\nyour allow-refs:\n\n      refs/heads/main|refs/heads/master|refs/heads/maint|refs/heads/next|refs/heads/seen|refs/tags/gitgui-*|refs/tags/pr-[0-9]*|refs/tags/v[1-9].*)\n\nSo you do use multiple prefixes, though all in refs/tags/.  Do you\nactually push tags for which you do want to run CI, or would it be\nsufficient to just reject \"refs/tags/\" completely (though that implies\nusing the fully qualified ref and not the short name; it would also be\neasy to add a boolean \"allow tags\" config option).\n\n-Peff\n"},{"id":"481804","messageId":"59b423e9-d99e-4817-8a33-c50419593740@gmail.com","threadId":"60173","inReplyTo":"20230911093616.GA1605460@coredump.intra.peff.net","subject":"Re: [PATCH 0/2] replacing ci/config/allow-ref with a repo variable","fromName":"Phillip Wood","fromEmail":"phillip.wood123@gmail.com","sentAt":"2023-09-13T15:16:48Z","receivedAt":"2023-09-13T15:17:29Z","isPatch":true,"sender":{"key":"phillip.wood@dunelm.org.uk","avatar":null},"body":"Hi Peff\n\nOn 11/09/2023 10:36, Jeff King wrote:\n> On Thu, Sep 07, 2023 at 11:04:33AM +0100, Phillip Wood wrote:\n> Looking at the ci-config branch of phillipwood/git.git, I see this in\n> your allow-refs:\n> \n>        refs/heads/main|refs/heads/master|refs/heads/maint|refs/heads/next|refs/heads/seen|refs/tags/gitgui-*|refs/tags/pr-[0-9]*|refs/tags/v[1-9].*)\n> \n> So you do use multiple prefixes, though all in refs/tags/.  Do you\n> actually push tags for which you do want to run CI,\n\nYes, but not very often - I could probably just reject all tags and \nstart the CI manually when I want it (assuming that's an option). Thanks \nfor digging into the various options, it sounds like it is possible so \nlong as we don't want multiple prefixes.\n\nAside: what I'd really like is to be able to set an environment variable \nwhen I push to skip or force the CI\n\n\tGITHUB_SKIP_CI=1 git push github ...\n\nbut that would require support from the git client, the protocol and the \nserver.\n\nBest Wishes\n\nPhillip\n"},{"id":"481821","messageId":"20230914003010.GA1709842@coredump.intra.peff.net","threadId":"60173","inReplyTo":"59b423e9-d99e-4817-8a33-c50419593740@gmail.com","subject":"Re: [PATCH 0/2] replacing ci/config/allow-ref with a repo variable","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2023-09-14T00:30:10Z","receivedAt":"2023-09-14T00:30:13Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Wed, Sep 13, 2023 at 04:16:48PM +0100, Phillip Wood wrote:\n\n> On 11/09/2023 10:36, Jeff King wrote:\n> > On Thu, Sep 07, 2023 at 11:04:33AM +0100, Phillip Wood wrote:\n> > Looking at the ci-config branch of phillipwood/git.git, I see this in\n> > your allow-refs:\n> > \n> >        refs/heads/main|refs/heads/master|refs/heads/maint|refs/heads/next|refs/heads/seen|refs/tags/gitgui-*|refs/tags/pr-[0-9]*|refs/tags/v[1-9].*)\n> > \n> > So you do use multiple prefixes, though all in refs/tags/.  Do you\n> > actually push tags for which you do want to run CI,\n> \n> Yes, but not very often - I could probably just reject all tags and start\n> the CI manually when I want it (assuming that's an option). Thanks for\n> digging into the various options, it sounds like it is possible so long as\n> we don't want multiple prefixes.\n\nI'll additionally have to switch to using the full refname in the\nmatches, but that is probably a reasonable thing to do anyway (it makes\nconfig a bit more verbose, but it is obviously much more flexible).\n\nWe can do the loop-unroll thing if we really want to support multiple\nprefixes, but if you're OK with it, let's try the single-prefix way and\nsee if anybody runs into problems (I'm still convinced there's only a\nfew of us using this stuff anyway). I'm hesitant to do the unroll just\nbecause it requires picking a maximum value, with a bizarre failure if\nyou happen to have 4 prefixes or whatever.\n\nI'll see if I can polish up what I showed earlier into a patch.\n\n(BTW, one other thing I tried was using fromJSON(vars.CI_CONFIG) in the\n\"on\" expression, which in theory would allow globbing. But it doesn't\nlook like expressions work at all in that context. And even if they did,\nI'm not sure we could make it work such that an empty CI_CONFIG used\nsome defaults, since there's no conditional-expression ternary\noperator).\n\n> Aside: what I'd really like is to be able to set an environment variable\n> when I push to skip or force the CI\n> \n> \tGITHUB_SKIP_CI=1 git push github ...\n> \n> but that would require support from the git client, the protocol and the\n> server.\n\nWe have the necessary bits at the protocol: push-options. But it's up to\nthe server-side hooks to decide which ones are meaningful and to do\nsomething useful with them. It looks like GitLab supports:\n\n  git push -o ci.skip=1 ...\n\nbut I don't think GitHub respects any equivalent option.\n\n-Peff\n"},{"id":"481822","messageId":"20230914004406.GA2224848@coredump.intra.peff.net","threadId":"60173","inReplyTo":"20230914003010.GA1709842@coredump.intra.peff.net","subject":"Re: [PATCH 0/2] replacing ci/config/allow-ref with a repo variable","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2023-09-14T00:44:06Z","receivedAt":"2023-09-14T00:44:12Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"Two minor corrections in what I wrote...\n\nOn Wed, Sep 13, 2023 at 08:30:10PM -0400, Jeff King wrote:\n\n> (BTW, one other thing I tried was using fromJSON(vars.CI_CONFIG) in the\n> \"on\" expression, which in theory would allow globbing. But it doesn't\n> look like expressions work at all in that context. And even if they did,\n> I'm not sure we could make it work such that an empty CI_CONFIG used\n> some defaults, since there's no conditional-expression ternary\n> operator).\n\nIt's not a real ?: operator, but GitHub does use the word \"ternary\" to\ndescribe the short-circuit behavior of:\n\n  condition && \"if-true\" || \"if-not-true\"\n\nSo in theory we could do:\n\n  on: ${{ fromJSON(var.CI_ON != '' && vars.CI_ON || \"[pull_request, push]\" }}\n\nbut I couldn't make it work (it complains about the value of \"on\" in\nsuch a way that I assume it is simply not expanding expressions at all).\n\n> We have the necessary bits at the protocol: push-options. But it's up to\n> the server-side hooks to decide which ones are meaningful and to do\n> something useful with them. It looks like GitLab supports:\n> \n>   git push -o ci.skip=1 ...\n> \n> but I don't think GitHub respects any equivalent option.\n\nThe syntax here is actually just \"git push -o ci.skip\", without the\nequals.\n\n-Peff\n"},{"id":"481841","messageId":"9f59df48-77be-485b-9208-6e2082af5963@gmail.com","threadId":"60173","inReplyTo":"20230914003010.GA1709842@coredump.intra.peff.net","subject":"Re: [PATCH 0/2] replacing ci/config/allow-ref with a repo variable","fromName":"Phillip Wood","fromEmail":"phillip.wood123@gmail.com","sentAt":"2023-09-14T10:49:25Z","receivedAt":"2023-09-14T10:49:32Z","isPatch":true,"sender":{"key":"phillip.wood@dunelm.org.uk","avatar":null},"body":"On 14/09/2023 01:30, Jeff King wrote:\n> On Wed, Sep 13, 2023 at 04:16:48PM +0100, Phillip Wood wrote:\n> We can do the loop-unroll thing if we really want to support multiple\n> prefixes, but if you're OK with it, let's try the single-prefix way and\n> see if anybody runs into problems (I'm still convinced there's only a\n> few of us using this stuff anyway). I'm hesitant to do the unroll just\n> because it requires picking a maximum value, with a bizarre failure if\n> you happen to have 4 prefixes or whatever.\n\nLets start with a single-prefix and see if anyone complains\n\n>> Aside: what I'd really like is to be able to set an environment variable\n>> when I push to skip or force the CI\n>>\n>> \tGITHUB_SKIP_CI=1 git push github ...\n>>\n>> but that would require support from the git client, the protocol and the\n>> server.\n> \n> We have the necessary bits at the protocol: push-options. But it's up to\n> the server-side hooks to decide which ones are meaningful and to do\n> something useful with them. It looks like GitLab supports:\n> \n>    git push -o ci.skip=1 ...\n\nOh I didn't know about that\n\n> but I don't think GitHub respects any equivalent option.\n\nThat's a shame\n\nBest Wishes\n\nPhillip\n"}]}