Workflow is the way that individual jobs are wired together to form a pipeline. This is done by using a
requires keyword in your job definition with the list of jobs or events that should cause that job to run. Screwdriver defines two events for every pipeline that occur due to SCM events:
~commit. These occur when a pull-request is opened, reopened, or modified, and when a commit occurs against the pipeline’s branch (respectively).
Defining Workflow Order
To denote workflow order, use the
requires keyword under a job with the job names as an array. Job names may be prefixed with a tilde to indicate advanced logic.
In the following example, the job,
main, will start after any SCM pull-request, or commit event. The job,
second, will run after
main is successful.
Please note that a job started by a pull-request will not trigger its downstream jobs. For example, if
mainstarts and succeeds as a result of a pull-request being opened,
secondwill not start afterwards.
jobs: main: image: node:6 requires: [~pr, ~commit] steps: - echo: echo hi second: image: node:6 requires: [main] steps: - echo: echo bye
To specify a job to run when a pull request is opened or updated, use
requires: [~pr]. For jobs that should start after code is merged or pushed to the main branch, use
Advanced Logic [AND]
You can specify a job to start when all of its
requires jobs are successful [AND]. This is also often called a join or fan-in.
In the following example, the
last job will only trigger when
second complete successfully in the same triggering event.
shared: image: node:6 steps: - greet: echo hello jobs: main: requires: [~pr, ~commit] first: requires: [main] second: requires: [main] last: requires: [first, second]
Advanced Logic [OR]
You can specify a job to to start when any of its
requires jobs are successful [OR] by adding a tilde (~) prefix to the jobs it requires. It will need to follow the format
In the following example, the
last job will trigger anytime either
second complete successfully.
shared: image: node:6 steps: - greet: echo hello jobs: main: requires: [~pr, ~commit] first: requires: [main] second: requires: [main] last: requires: [~sd@123:first, ~sd@123:second]
Advanced Logic [Combined]
The AND and OR logic can be combined in a complex pipeline to allow cases where you want to start a job when
second jobs are successful, OR a
third job is successful as in the following example.
last: requires: [first, second, ~sd@123:third]
If job names are prefixed with tildes in a
requires line, then the job will start when any of the prefixed jobs is successful OR when all of the unprefixed jobs are successful. For instance, in this contrived example:
main: requires: [~sd@123:A, B, ~sd@123:C, D, ~sd@123E, F]
is equivalent to the Boolean expression
A OR C OR E OR (B AND D AND F). Such a complicated
requires line in an actual workflow should be regarded as a code smell.
To trigger jobs in your pipeline after a specific branch is committed, you can use branch filtering. The format is
~commit:/^feature-/). Note: Flags are not supported.
In the following example, when branch
staging is committed,
all-commit are triggered. Also, when branch
master is committed,
all-commit are triggered. When a pull request is opened in branch
staging-pr is triggered.
shared: image: node:8 jobs: main: requires: [~commit] steps: - echo: echo commit staging-commit: requires: [~commit:staging] steps: - echo: echo staging all-commit: requires: [~commit:/./] # /./ matches any branch name and is used here for illustration only # Don't use that regexp in any actual workflow. steps: - echo: echo all staging-pr: requires: [~pr:staging] steps: - echo: echo staging pr
Parallel and Join
You can run jobs in parallel by requiring the same job in two or more jobs. To join multiple parallel jobs at a single job you can use the
requires syntax to require multiple jobs.
In the following example, where
main. This will cause
B to execute in parallel after
main is successful. Also in this example, job
C runs only after both
B are successful in the same triggering event.
shared: image: node:6 jobs: main: requires: [~pr, ~commit] steps: - echo: echo hi A: requires: [main] steps: - echo: echo in parallel B: requires: [main] steps: - echo: echo in parallel C: requires: [A, B] steps: - echo: echo join after A and B
To trigger a job in your pipeline after a job in another pipeline is finished, you can use remote requires. The format is
~commit, and jobs with
~sd@pipelineID:jobName format follow OR logic.
In the following example, this pipeline will start the
main job after any pull-request, commit, or successful completion of the
publish job in pipeline 456.
jobs: main: image: node:6 requires: [~pr, ~commit, ~sd@456:publish] steps: - echo: echo hi
To have your job blocked by another job, you can use
blockedBy. It has the same format as
requires, except it does not accept values like
- Since everything is using OR syntax, you need a tilde (
~) before each of your job names. We do not support AND logic for blockedBy.
- To prevent race conditions, a job is always blocked by itself. That means the same job cannot have 2 instances of builds running at the same time.
- This feature is only available if your cluster admin configured to use
executor-queue. Please double check with your cluster admin whether it is supported.
- This feature does not apply to PR jobs.
In the following example,
job2 is blocked by
sd@456:publish is running and
job2 is triggered,
job2 will be put back into the queue. Screwdriver will check the queue periodically to see if
job2 is no longer blocked and will run it as soon as that is true. Note:
blockedBy only blocks the job that the configuration is under; the following configuration won’t block
job2 is running.
shared: image: node:6 jobs: job1: requires: [~commit, ~pr] steps: - echo: echo hello job2: blockedBy: [~job1, ~sd@456:publish] steps: - echo: echo bye
You can freeze your jobs and prevent them from running during specific time windows using
freezeWindows. The setting takes a cron expression or a list of them as the value.
Before the job is started, it will check if the start time falls under any of the provided cron windows, and freezes the job if so. The job will be unfrozen and run as soon as the current cron window ends.
- Different from
freezeWindowsshould not use hashed time therefore the symbol
Hfor hash is disabled.
- The combinations of day of week and day of month are usually invalid. Therefore only one out of day of week and day of month can be specified. The other field should be set to “?”.
- If multiple builds are triggered during the freeze window, they will be collapsed into one build which will run at the end of the freeze window with the latest commit inside the freeze window.
In the following example,
job1 will be frozen during the month of March,
job2 will be frozen on weekends, and
job3 will be frozen from 10 PM to 10 AM.
shared: image: node:6 jobs: job1: freezeWindows: ['* * ? 3 *'] requires: [~commit] steps: - build: echo "build" job2: freezeWindows: ['* * ? * 0,6,7'] requires: [~job1] steps: - build: echo "build" job3: freezeWindows: ['* 0-10,22-23 ? * *'] requires: [~job2] steps: - build: echo "build"
Detached Jobs and Pipelines
It is possible to define workflows that do not have any external trigger. These workflows are “detached” from the normal flow of the pipeline. Some example use cases of this would be to define a rollback flow for your pipeline that could be manually triggered. Invoking a detached pipeline involves the same steps as doing a rollback.
In the following example
detached job is detached.
shared: image: node:8 jobs: detached: steps: - echo: echo im-a-detached-job