Authoritative Containers — Part 3

AN IMPLEMENTATION OF A MULT-SECURITY ZONE DEVOPS PIPELINE FOR ASSURING CONTAINER IMAGES

In part 1 and part 2 of this series I set out the types of assurance needed to ensure container images are safe to be deployed to an enterprise’s production environment, and shared potential ways to implement that assurance.

In this article I share the details of a solution (the ‘Authoritative Container Builder’) that shows how the recommendations from parts 1 and 2 can be implemented to provide the required assurance.

The ‘Authoritative Container Builder’ is a set of DevOps pipelines for creating and assuring the provenance of a container image built and deployed across multi-security zones (for example, public cloud for development and public, private or hybrid environment for live operation).

The following diagram shows how the Builder sits between the development teams and the production environment. It is a controlled, segregated environment where the container image is built and the provenance for the image is captured and generated.

The built container image and its provenance is then sent (via separate transport mechanisms) to the production estate where they are validated in a quarantine area. If the image and provenance is successfully validated then it is added to the production repository, ready for deployment.

The ‘Authoritative Builder’ is cloud agnostic (using Tekton pipelines) and allows multiple separate development teams to deliver their container images in a consistent and traceable way.

The provenance information it creates is also stored on the production estate and provides the foundation for continuous vulnerability checking - allowing capabilities to alert when a container running in the estate needs updating due to new vulnerabilities being identified in its underlying libraries.

The following diagram shows the pipelines that the Builder deploys into each of the zones. The pipelines create the container image and create the assurance of the image. Examples of the assurance include logging which base container image was used; security scanning of the source code; auditing of the open-source libraries used and identification of any known vulnerabilities.

The Build pipeline captures the provenance and the bill of materials for the container images and passes these via secure transfer to the Validation pipeline for verification.

“Build pipeline” (Tekton pipeline)

The Build pipeline uses the following technologies to implement each of the build and assurance stages:

Following successful completion of the stages shown above, the pipeline will:

• Create the Container Image and include the source code in the Container Image
• Create the Bill of Materials (containing the provenance for the images) and push to the local repository for the “validation pipeline” to pull
• Create an image creation log and push to the local registry for the “validation pipeline” to pull
• Notify the “Validation pipeline” that a new image and BoM is ready for transfer

“Validation pipeline” (Tekton pipeline)

The “Validation pipeline” receives notification from the “Build pipeline” that a new image has been created for transfer. The “Build pipeline” supplies the details of the digest of the Container Image. Upon receipt of this information the pipeline will:

• Copy the image by the digest from the low side registry into the high side quarantine registry
• Pull the Bill of Materials and image creation log, and perform verification of its contents
• Perform Container scanning pointing at the quarantine registry; undertaking static and dynamic scanning (Sysdig or StackRox ). Create an image scanning log of the results and make available for viewing and transfer to the production environment
• Copy the image and log to the Production repository if the validation of the logs and scanning tests pass successfully.

The following diagram provides a more detailed walkthrough of pipelines and stages.

If you’d like to hear more on the Authoritative Container Builder then please join me in a short video that sets the context and details the capabilities of the Builder, or contact me directly and I’d be pleased to provide more details.