Link Search Menu Expand Document

ci-cd

SRE aim to champion effective CI and CD in the Storio ecosystem by providing a stable, fast CI/CD platform.

Following Werner Vogels principal of ‘You build it, you run it’, engineering teams own their pipeline code but SRE are there to support the standardisation of approach in that pipeline code, as well as the education and enablement of engineering teams in creating and operating that code. To that end, we run the following platforms and provide the following expertise:

  • Jenkins
    • ci.core - This is our legacy CI platform for the eCommerce estate
    • jenkins.core - This is our legacy CI platform for the Babel estate
    • techops[.test].jenkins.pbx - Our platform for internal tools deployment
    • aiphoto[.test].jenkins.pbx - Historical deployment platform for Photoai related applications
    • iways[.test].jenkins.pbx - Historical deployment platform for iWays related applications
[.test] indicates that there is both a production and non-production deployment of this platform, with the .test subdomain being used to differentiate between the two.
  • GitHub Actions GitHub Actions (GHA) is run as part of our GitHub Enterprise cloud plan and differs from Jenkins in that it therefore doesn’t have a seperate UI or application server; instead it is interacted with via a project’s GitHub repository. As it’s often nessecary for a deployment to assume AWS IAM credentials, we run a set of GitHub Actions self-hosted runners in Kubernetes - these self-hosted runners run on ec2 instances and can therefore accept an assumed role; this is how we allow pipelines to deploy into AWS without having to inject static credentials as secrets into our GHA pipelines.

Current best practice is to implement any new projects into GHA and consider Jenkins as deprecated. Existing pipelines resident in Jenkins can continue to exist there though may benefit from migration to GitHub Actions; parellelisation is much more effective in GHA which can dramatically increase build speed, plus it is much more possible to instrument pipeline success and failures in a useful manner within GHA.

Our overarching strategy for CI/CD in the Storio organisation is:

  • Support the migration of projects towards GitHub Actions and standardisation of GHA pipelines across the group.
  • Support the rollout of build success and failure metrics which will be injected into the GitHub API and then linked into our data lake to provide key pipeline and engineering team health metrics.
  • Use data gathered from the above to inform where we can further decrease build time and therefore contribute to cycle time reductions.
  • Further enhance our self-hosted platform to be more stable and cost-effective.