Turn it on
1
Pass the job's own check run id
The feature must know which check is its own, or it would wait for itself. Give the job a stable
name, pass checkRunId, and grant statuses: read so it can read commit statuses as well as check runs:.github/workflows/smartcloud.yml
timeout-minutes must be longer than required.timeout (60 minutes by default), so the job is not killed while it waits.2
Add a required section
.github/smartcloud.yml
required: {} is enough to turn it on. List your main CI job under expect so a deleted or renamed CI job fails the aggregate instead of letting it pass with nothing to wait for.3
Require the smartcloud check
Require the Keep your main CI check required beside
smartcloud check (the job’s name) in the ruleset. With the settings feature:.github/smartcloud.yml
smartcloud too. A restricted run, such as a fork or Dependabot pull request, leaves out any preset from another repository that its token cannot read. If required comes from such a preset it is skipped there, and only the ruleset’s own list protects the branch.Options
A pattern is a regular expression, written bare (
^codecov/) or as /source/flags (/preview/i to ignore case).
Complete example
.github/smartcloud.yml
.github/workflows/smartcloud.yml
How it works
What counts
Every check on the head commit counts: the latest check run of each app and name, from GitHub Actions or any other app, and the latest commit status of each context. A ruleset picks a required check the same way. A re-run, or a workflow started again by an edit or a review, leaves the older runs of its checks on the commit, often cancelled; only the latest run of each check counts, so an older cancelled or failed run never fails the aggregate. Two jobs with the same name published by the same app are one check, as they are to a ruleset, so give jobs in different workflows different names. Three kinds are left out:- every run of the job running smartcloud: its own, found by
checkRunId, and any other run with the same app and name, such as one a review started while this one waits. Until its own run shows up in GitHub’s list, the feature concludes nothing and looks again; - smartcloud’s own
smartcloud / <feature>check runs, because their findings already decide the job’s own result. smartcloud marks every check run it creates with theexternal_idsmartcloud, and only runs with that mark from an app smartcloud runs as (the job’s app, or the GitHub App its token acts as) are left out. A check run of the same name from another app or workflow, unmarked or marked otherwise, counts; - anything an
ignorepattern matches.
smartcloud / merge freeze check is not covered by the aggregate: require it in the ruleset on its own.
How it waits
The feature looks at the checks every 15 seconds. While nothing changes between looks, it waits twice as long before the next, up to a minute; any change starts the waits over at 15 seconds. The last look is always at the deadline.- As soon as one check fails, the aggregate fails, with one error (
required.failed) per failed check, linking to it. - When nothing is pending, it looks once more, so a workflow queued a moment after the rest is not missed. Two looks in a row with the same checks, all passed, pass the aggregate with a notice (
required.passed), such asAll 7 other check(s) on this commit passed. - An
expectpattern no check matches yet keeps it waiting; one still missing aftertimeoutminutes fails it (required.missing). - Checks still pending after
timeoutminutes fail it, one error (required.pending) each. - If GitHub never lists the job’s own run before the timeout, it fails (
required.unconfirmed) rather than pass unconfirmed.
concurrency group per pull request and event with cancel-in-progress, as in the example, so a newer run replaces an older one instead of waiting beside it.
Without checkRunId
In the action, a configuredrequired section without checkRunId is an error (required.check-run-missing: required is configured but the workflow does not pass checkRunId: ${{ job.check_run_id }}, so it could not run. Update the workflow.). A workflow that was not updated therefore fails instead of passing the aggregate unchecked. In CLI dry runs the feature is skipped.
Tokens
The checks are read with the workflow’s own token (theworkflowToken input, which defaults to it), not GITHUB_TOKEN. GITHUB_TOKEN can be a personal access token or an app token without checks and statuses read, while the job’s checks and statuses permissions give the workflow token both.
What you will see on GitHub
- The job’s own check (named after the job, such as
smartcloud) stays in progress while the feature waits, then fails if any counted check failed, is still pending, is missing, or its own run was never listed. That is the check your ruleset requires. - A
smartcloud / requiredcheck run with the same findings: failure on any error, success with therequired.passednotice otherwise. - The failed, pending and missing checks in the report comment on the pull request, each linking to the check.
- In the job summary, every finding, including the notice with the number of checks that passed.
Forks and restricted runs
On a pull request from a fork, the workflow token is read-only, which is enough to read checks, so the feature still runs. But a restricted run leaves out a preset from another repository that it cannot read. Ifrequired comes from such a preset, the aggregate is skipped on fork and Dependabot pull requests, and the job’s check reports only smartcloud’s own findings. That is why your main CI check should stay required in the ruleset beside smartcloud.
In the house preset
The Resnovas house preset setsrequired: {}, and requires the smartcloud check in its ruleset (settings.ruleset.statusChecks.checks.smartcloud: true). A repository adds its main CI job under both required.expect and settings.ruleset.statusChecks.checks, as smartcloud does:
.github/smartcloud.yml
ignore and timeout, which the preset leaves unset.
Rolling it out
The feature does nothing useful until the workflow passescheckRunId, so change things in this order:
- Use a smartcloud release with the feature in the workflow.
- Pass
checkRunIdand grantstatuses: readin the workflow. - Add the
requiredsection to the config, or to a preset the config extends. From here thesmartcloudcheck waits for every other check. - Only then make the ruleset require the
smartcloudcheck alone. Dropping the other required checks earlier would leave pull requests gated by nothing but smartcloud’s own findings.
required.check-run-missing and the check fails until the workflow is updated.
Troubleshooting
required.check-run-missing: the workflow does not pass checkRunId
required.check-run-missing: the workflow does not pass checkRunId
Add
checkRunId: ${{ job.check_run_id }} to the smartcloud step’s with, as in Turn it on.The job runs for an hour and fails with required.pending
The job runs for an hour and fails with required.pending
A check never finished within
timeout minutes. Open the check the error links to. If it is a check that never concludes on its own, such as a deployment preview, add it to ignore. If your CI is slow, raise timeout and the job’s timeout-minutes.required.missing: No check matching ^check$ passed
required.missing: No check matching ^check$ passed
No check on the commit matches an
expect pattern. The CI job may have been renamed, skipped by a path filter, or not run for this event. Rename the pattern or make the job run on every pull request.required.unconfirmed: GitHub did not list this job's own check run
required.unconfirmed: GitHub did not list this job's own check run
GitHub never returned the job’s own check run, so smartcloud could not tell its own runs apart from the rest. Check that
checkRunId is ${{ job.check_run_id }} of the same job, and re-run the job.Two smartcloud jobs wait for each other until they time out
Two smartcloud jobs wait for each other until they time out
Two differently named jobs both use the required feature on the same commit. Keep one aggregate job, and give the workflow a
concurrency group with cancel-in-progress.The aggregate passed but a failed check was re-run later
The aggregate passed but a failed check was re-run later
The aggregate only sees the checks while it waits. Re-run the smartcloud job after re-running any other check.
The job cannot read checks or statuses
The job cannot read checks or statuses
Grant
checks: write and statuses: read on the job, and leave the workflowToken input at its default.