AWS Native Builds

Screwdriver can be used to orchestrate AWS native builds which runs in either Code Build or EKS.

• AWS
• Code Build
• EKS

Architecture diagram.

This integration uses AWS MSK to schedule user builds in the user’s own AWS account. This enables Screwdriver Cluster admins to a setup multi-tenant build environments where different user builds are sent to their individual AWS accounts without impacting each other. Users can also integrate with Screwdriver without having to provide any account or network access to Screwdriver and perform secure AWS deployments backed by IAM role identities.

Setup

In order to use this feature, Screwdriver Cluster admin must setup AWS MSK infrastructure using aws-producer-scripts and enable it in queue-service

A user who wants to integrate should work with Screwdriver Cluster admin to register their AWS account for scheduling builds.

Once registration is complete, then user should provision build infrastructure by running this script.

Image

The image configuration refers to a Docker image, e.g. an container from hub.docker.com or a container from public aws ecr images

List of images support in your AWS account region can be checked using:

aws codebuild list-curated-environment-images

You can specify an image from a custom registry/your own AWS ECR by specifying the full url to that image. If you wish to create an ECR with required permission during infrastructure provisioning, you can do so by setting the create_ecr flag in aws-consumer-scripts

Example

jobs:
    main:
        requires: [~pr, ~commit]
        image: 123456789012.dkr.ecr.us-west-2.amazonaws.com/screwdriver-hub:example_image
        provider:
          ...mandatory_params
        steps:
            - step1: echo hello

Provider Configuration

Provider configuration is required for identifying the cloud provider related configuration. For AWS Native builds it includes the identifier of the Virtual Private Cloud(VPC), the subnets and security groups which define the inbound and outbound communication, the IAM role for accessing various AWS services based on permissions.

Job-level Provider Configuration

The provider configuration can be stored in a job in the screwdriver.yaml. The example defines the mandatory parameters in provider config.

Example

jobs:
  main:
    requires: [~pr, ~commit]
    image: aws/codebuild/amazonlinux2-x86_64-standard:3.0
    provider:
      name: aws
      region: us-west-2
      accountId: 123456789012
      vpc:
        vpcId: vpc-0123abc
        securityGroupIds:
          - sg-0123abc
        subnetIds:
          - subnet-0123abc
          - subnet-0123def
          - subnet-0123ghi
      role: arn:aws:iam::123456789012:role/screwdriver-integration-role
      executor: sls
      launcherImage: screwdrivercd/launcher:v6.0.149
      launcherVersion: v6.0.149
    steps:
      - init: npm install
      - test: npm test

External Provider Configuration

Alternatively, provider configuration can be stored remotely in another file in an external repo. You can reference this config by putting a checkout URL with the format CHECKOUT_URL#BRANCH:PATH.

Example

jobs:
  main:
    requires: [~pr, ~commit]
    image: aws/codebuild/amazonlinux2-x86_64-standard:3.0
    provider: git@github.com:configs/aws.git#main:cd/aws/provider.yaml
    steps:
      - init: npm install
      - test: npm test

Shared Provider Configuration

The provider configuration can be added to the shared configuration. A provider that is specified in a job configuration will override the same configuration in shared.provider.

Example

The following example defines a shared configuration for provider which is used by the main and main2 jobs.

shared:
  image: aws/codebuild/amazonlinux2-x86_64-standard:3.0
  provider:
    name: aws
    region: us-west-2
    accountId: 123456789012
    vpc:
      vpcId: vpc-0123abc
      securityGroupIds:
        - sg-0123abc
      subnetIds:
        - subnet-0123abc
        - subnet-0123def
        - subnet-0123ghi
    role: arn:aws:iam::123456789012:role/screwdriver-integration-role
    executor: sls
    launcherImage: screwdrivercd/launcher:v6.0.149
    launcherVersion: v6.0.149

jobs:
    main:
      requires: [~pr, ~commit]
      steps:
        - init: npm install
        - pretest: npm lint
        - test: npm test
    main2:
      requires: [main]
      steps:
          - test: echo Skipping test

The above example would be equivalent to:

jobs:
    main:
        requires: [~pr, ~commit]
        image: aws/codebuild/amazonlinux2-x86_64-standard:3.0
        provider:
          ...same as shared
        steps:
            - init: npm install
            - pretest: npm lint
            - test: npm test
    main2:
        requires: [main]
        image: aws/codebuild/amazonlinux2-x86_64-standard:3.0
        provider:
          ...same as shared
        steps:
            - test: echo Skipping test