Drivvn

Kubernetes based automotive e-commerce platform hosted on Microsoft Azure

Drivvn

A project to pick up support for and improve an existing Kubernetes based infrastructure platform hosted on Microsoft Azure.

Skip to Result

Stabilising Platform

Initially, the primary focus had been to automate continual time consuming tasks and to reduce the number of support tickets. This has been achieved by implementing technologies such as External DNS for Kubernetes and Certificate Manager. Reliability has been improved by consolidating and improving complex build pipelines by utilising YAML libraries for Azure DevOps and introducing infrastructure as code with Terraform.

Evolving Platform

After stabilising the platform focus shifted to evolving the platform to be more performant, scalable and developer friendly. In addition to everything outlined below, lots of work went into upgrades of Kubernetes clusters and services, improvements as and when they were identified and migrations to new Azure subscriptions/tenants.

Developer Friendly

To help improve continuous delivery, ArgoCD was implemented. This allowed developers to publish Docker images in their build pipelines which would then automatically be picked up and deployed into a Kubernetes environment. ArgoCD also allowed the configuration for the application to be completely separated from other infrastructure as code meaning developers could have full control over the configuration of their deployed applications without requiring permissions to alter other parts of infrastructure.

Infrastructure as Code

There was a large focus on infrastructure as code. Terraform was used to automate the creation of all base infrastructure: such as resource groups, virtual networks, security groups and Kubernetes clusters - including all supporting infrastructure and Active Directory objects. This allowed much quicker and reliable provision of core infrastructure.

Any new services deployed were also fully automated so that required databases, Kubernetes permissions and the application themselves could also be deployed in a reliable and repeatable way.

Monitoring & Logging

As the platform grew it became important to be able to monitor workloads. Initially Prometheus was used with Grafana as well as some other custom exporters to get certain metrics that were otherwise unavailable.

However as things grew Prometheus was not scaling so eventually the monitoring stack was upgraded with Thanos. This allowed gathering of huge amounts of metrics from multiple Prometheus instances and storing the data in cost efficient blob storage. Thanos also allowed a ‘single pane of glass’ view of the entire estate of multiple clusters through Grafana.

Centralised logging was also set up utilising ElasticSearch at first and later New Relic.

Ingress

A lot of work was carried out around ingress to the platform. Initially cluster ingresses were standardised with Ingress Nginx. Security and CDN capability was then added using Cloudflare and Cloudflare Access. This allowed many apps to use single sign-on that originally did not support it. It also enabled zero trust access to internal services without having to set up and maintain a VPN.

Result

Due to multiple reasons - the good work that Drivvn do, the pandemic and just the way things are going - significantly more people are now buying cars online. Making the infrastructure more scalable and resilient has helped make it possible to meet these suddenly increasing demands.

Having the infrastructure as code ready allowed us to work with one of the Drivvn development teams to deliver a new product from inception to launch within a few weeks. The initial infrastructure was provisioned and available within hours. We then worked with the team to refine the build pipelines and infrastructure as they developed the product. Since launch, the infrastructure supporting this product has been extremely reliable. Having ArgoCD set up allowed developers to manage their own configuration and ‘self-serve’ deployments all the way to production - with easy roll backs if required.

Built with Hugo
Theme Stack designed by Jimmy