Projects on Business Information Technology Services

Smartr365

Mon, 01 Aug 2022 00:00:00 +0000

A part time project where the main goal was to implement Azure Front Door into the existing infrastructure automation to improve security and performance of an application with hundreds of tenants.

Azure Front Door

Performance improvements were achieved by leveraging caching, compression and the content delivery network to service static and cached content from regional distribution points.

To improve security Azure Front Door Web Application Firewall (WAF) was used to identify improvements in the application such as ways to filter/encode data that was transmitted to/from the application and customers web browsers. The WAF also enabled visibility to how the site was currently being scraped/attacked and potential mitigation.

Technologies Used

The existing infrastructure automation was primarily in Bicep and Powershell. These technologies have not previously been used by BITServices and therefore brought new knowledge to the company and contrast with the alternative tools that we much more commonly use.

Other Deliverables

As well as delivering Azure Front Door, other small tasks carried out included creating another environment with infrastructure as code which required heavy refactoring. The refactoring was mainly converting scripted PowerShell steps into proper infrastructure as code (Bicep in this case) and fixing infrastructure deployment pipelines that were failing for various reasons.

Arden University

Sat, 01 Jan 2022 00:00:00 +0000

For Arden, there were two projects to deliver: an Amazon Elastic Kubernetes Service (EKS) based platform and development of their serverless integrations.

In order to help with the delivery of these projects, some open source contributions were made.

Elastic Kubernetes Service (EKS) Platform

A new platform was required to migrate existing on-premises and Amazon EC2 workloads to. The new platform had to allow developers to easily manage their own deployments, allow for centralised logging and monitoring, had to be secure and allow for management of things such as DNS records and TLS certificates to be fully automated.

The platform had to be delivered ‘as code’ so it could be easily managed, upgraded and replicated. This was done using mainly Terraform with some base Kubernetes services being deployed using the Terraform Helm provider.

Any base Kubernetes services that ArgoCD did not depend on explicitly were managed and deployed by ArgoCD along with the business applications hosted on the platform.

Managing Deployments with ArgoCD

The main interface into the platform for developers was through ArgoCD. This allowed developers to deploy applications, see the status of their workloads and get immediate logs and events to diagnose failures. Being able to deploy and diagnose simple issues themselves allowed developers to spend less time being blocked waiting for infrastructure engineers to help out.

ArgoCD was integrated with Microsoft Entra to allow single-sign on which improves security and convenience. Developers could be assigned to individual projects meaning they only have access to the applications that they are working with.

ArgoCD Image Updater was also used in development environments so that successful builds of an application could be deployed automatically.

Monitoring & Logging

For the monitoring and logging, open source tools were used as they provide excellent flexibility and of course value.

For monitoring, Prometheus was used with Grafana. Prometheus was also configured to make a future expansion to Thanos as easy as possible, should the need ever arise.

Loki was used for logging, it integrated perfectly with the monitoring metrics since it also uses Grafana as the main user interface.

Alerts were configured with Grafana unified alerts, allowing alerts to be raised from either metric or logging data.

Automated Management

Tools such as Certificate Manager and External DNS were used to allow automation of day to day administrative tasks.

Certificate Manager automatically provisions and renews TLS certificates for applications hosted within the platform whilst External DNS creates and cleans up DNS records.

Future Ready

Whilst not immediately implemented due to possible conflicts with the business the platform was developed and tested with IPv6 support and support for ARM based compute allowing for easy adoption later on.

Serverless Integrations Development

Arden was implementing a new student record system called SITS. This was a huge project that affected all areas of the business. A significant problem was integrating existing systems that do not understand SITS, such as: the virtual learning environment, Active Directory, time tabling products and many more.

To enable this communication between systems, a group of serverless ‘middlewares’ were used to keep things scalable and event driven.

Since the integrated systems spanned across AWS and Azure as well as including software as a service (SaaS) solutions, many technologies were used to integrate them. These technologies included: Typescript based Lambda functions deployed with the Amazon CDK, Azure Logic Apps, Azure Functions, Amazon SNS, Amazon SQS and Azure Service Bus.

These integrations handled large amounts of traffic especially as SITS went live at Arden. They also helped the business implement complex processes with systems that cannot be directly integrated.

Open Source Contributions

The following open source contributions were made whilst delivering these projects:

Drivvn

Thu, 02 Jul 2020 00:00:00 +0000

A project to pick up support for and improve an existing Kubernetes based infrastructure platform hosted on Microsoft Azure.

Skip to Result

Stabilising Platform

Initially, the primary focus had been to automate continual time consuming tasks and to reduce the number of support tickets. This has been achieved by implementing technologies such as External DNS and Certificate Manager. Reliability has been improved by consolidating and improving complex build pipelines by utilising YAML libraries for Azure DevOps and introducing infrastructure as code with Terraform.

Evolving Platform

After stabilising the platform focus shifted to evolving the platform to be more performant, scalable and developer friendly. In addition to everything outlined below, lots of work went into upgrades of Kubernetes clusters and services, improvements as and when they were identified and migrations to new Azure subscriptions/tenants.

Developer Friendly

To help improve continuous delivery, ArgoCD was implemented. This allowed developers to publish Docker images in their build pipelines which would then automatically be picked up and deployed into a Kubernetes environment. ArgoCD also allowed the configuration for the application to be completely separated from other infrastructure as code meaning developers could have full control over the configuration of their deployed applications without requiring permissions to alter other parts of infrastructure.

Infrastructure as Code

There was a large focus on infrastructure as code. Terraform was used to automate the creation of all base infrastructure: such as resource groups, virtual networks, security groups and Kubernetes clusters - including all supporting infrastructure and Active Directory objects. This allowed much quicker and reliable provision of core infrastructure.

Any new services deployed were also fully automated so that required databases, Kubernetes permissions and the application themselves could also be deployed in a reliable and repeatable way.

Monitoring & Logging

As the platform grew it became important to be able to monitor workloads. Initially Prometheus was used with Grafana as well as some other custom exporters to get certain metrics that were otherwise unavailable.

However as things grew Prometheus was not scaling so eventually the monitoring stack was upgraded with Thanos. This allowed gathering of huge amounts of metrics from multiple Prometheus instances and storing the data in cost efficient blob storage. Thanos also allowed a ‘single pane of glass’ view of the entire estate of multiple clusters through Grafana.

Centralised logging was also set up utilising ElasticSearch at first and later New Relic.

Ingress

A lot of work was carried out around ingress to the platform. Initially cluster ingresses were standardised with Ingress Nginx. Security and CDN capability was then added using Cloudflare and Cloudflare Access. This allowed many apps to use single sign-on that originally did not support it. It also enabled zero trust access to internal services without having to set up and maintain a VPN.

Result

Due to multiple reasons - the good work that Drivvn do, the pandemic and just the way things are going - significantly more people are now buying cars online. Making the infrastructure more scalable and resilient has helped make it possible to meet these suddenly increasing demands.

Having the infrastructure as code ready allowed us to work with one of the Drivvn development teams to deliver a new product from inception to launch within a few weeks. The initial infrastructure was provisioned and available within hours. We then worked with the team to refine the build pipelines and infrastructure as they developed the product. Since launch, the infrastructure supporting this product has been extremely reliable. Having ArgoCD set up allowed developers to manage their own configuration and ‘self-serve’ deployments all the way to production - with easy roll backs if required.

Eclipse Group Solutions

Mon, 06 Nov 2017 00:00:00 +0000

A very exciting ‘greenfield’ project creating a new infrastructure platform using Amazon Web Services, Docker and Kubernetes for SAP Commerce based e-commerce websites.

It had been a great pleasure working with the team at Eclipse whilst delivering this project.

Skip to Result

Project Breifing

The project aim was to create an Amazon Web Services based, fully automated infrastructure platform to host SAP Commerce e-commerce websites. The platform must be constructed in a way that it can be used on other cloud service providers with little effort later on.

Developers and testers needed the ability to get code and features through testing pipelines much quicker. In addition, production websites need to handle peak demand seamlessly.

As well as delivering a platform, existing teams were to be assisted with picking up new tools, technologies and concepts to enable on-going support of the platform.

Solution

The final solution comprised of many components. These components are outlined below.

Infrastructure as Code

Writing infrastructure as code was key to this solution. This is what enabled infrastructure to be provisioned in a reliable and repeatable way at the click of a button. By taking advantage of Terraform module sources it was possible to define a collection of infrastructure objects (such as subnets, route tables, gateways) in a single place but allow variables (such as name, CIDR ranges) to be passed in depending on the environment being built. As a result all infrastructure met defined standards, human error is vastly reduced and development/production parity was achieved.

By taking advantage of Amazon Web Services availability zones and infrastructure as code, all production environments were highly available and could withstand the loss of an Amazon data centre without any downtime.

Immutable Software

To allow software (SAP Commerce in particular) to run in a dynamically scaled environment there were a few challenges to overcome. There needed to be a way to start SAP Commerce very quickly in ‘scale-up’ situations additionally the state of any running SAP Commerce instances had to be externalised in case of scale-down.

Docker was chosen to containerise software. Containerisation enabled the application and all dependencies, configurations, etc to be packed into an image that can be started very quickly. By using SAP Commerce ‘aspects’ a single Docker image can be capable of running in multiple environments in multiple modes. A single image can be promoted all the way through the testing pipeline just by using tags.

To make running containers stateless, services such as Amazon S3 and EFS were used.

Service Orchestration

To enable service-level auto scaling, auto healing, multi-tenant clustering and service health checks, Kubernetes was implemented. This allowed the platform to meet the scaling requirements. The healing features of Kubernetes allowed the platform to be more resilient to virtual machine failure or network outages resulting in a higher service availability.

Technologies & Tools

Multiple technologies and tools were used to deliver this project.

Amazon Web Services

The initial cloud provider chosen was Amazon. Amazon Web Services (AWS) is a mature cloud service with endless offerings. It is effortless to build highly-available and highly-performant infrastructure stacks. Most tools (such as Terraform and Kops below) offer excellent support for AWS. By using the many AWS services available such as IAM and availability zones it was possible to create secure and resilient infrastructure.

Hashicorp Terraform

Terraform was chosen to build foundation and networking infrastructure. Terraform has excellent Amazon Web Services support and code can easily be ported to work with other cloud providers too - including OpenStack for managing resources on-premises. Terraform made it very easy to meet the project requirements to stay cloud agnostic and to fully automate infrastructure.

Docker

Containerisation was chosen to help simplify development. Rather than pushing a codebase that may have a complete different set of steps to deploy depending on target environment, containerisation makes that a single image that may take the target environment as a parameter. Additionally, all dependencies and libraries required to run the application are packed into the container, meaning that the same container can be ran locally, on tin or in the cloud with minimal effort. Docker was chosen specifically due to it being well proven and mature.

Kubernetes

Kubernetes is fast becoming an industry standard for container orchestration. It can also run on any cloud provider or even on-premises on tin. Native support for service-level auto-scaling and ‘cluster autoscaler’ add-on for the scaling of underlying virtual machines allowed scaling requirements to be met. The many different types of services (deployments, statefulsets, daemonsets), specifications (disruption budgets, affinities) and probes (liveness, readiness) made it possible to build a platform that is resilient against hypervisor, network or even data centre failures.

Other Tools

Other tools used to deliver this project include: Kops, Hasicorp Packer and Ansible.

Result

The result of this project allows Eclipse to offer their customers an improved hosting service. Development environments can now be provisioned rapidly and easily decommissioned when not in use to match project demands. Automatic scaling allows customers websites to seamlessly handle high load during sales and events whilst running economically during quiet periods. The platform allows for zero-downtime code deployments and platform updates. High availability is achieved by always running across multiple availability zones (data centres) in addition to having automatic health checks and repairs.

FoodFriends

Fri, 01 Sep 2017 00:00:00 +0000

A small project carried out for a family member.

The Problem

FoodFriends have a small Wordpress website that was hosted with a traditional hosting provider. The provider, unfortunately had crippling limits such as only 50MiB of disk for website media. FoodFriends required that website stayed on the Wordpress platform.

The Solution

The website was moved to Amazon Web Services.

To remove future support costs and time, the supporting infrastructure was automated with Hashicorp Terraform. The installation and configuration of software fully automated with Ansible - even down to the renewal of the TLS certificate!

By using Amazon S3 backups are automatically taken and rotated daily. Restoration of data is managed by the Ansible playbooks meaning the entire infrastructure can be built from nothing with all data restored in just minutes.

The Result

There is now gigabytes of available diskspace for media. As Amazon Web Services is an enterprise grade platform capable of hosting even the most busiest of e-commerce websites the performance of the FoodFriends website has increased dramatically.

Having TLS configured correctly and being made mandatory the FoodFriends website has had its ranking increased on Google.

The current cost of hosting the infrastructure is approximately $1 a month for a couple of Route53 zones. Once the free-tier expires, the overall cost will still be only a third of what it was with the previous host.