In today’s data-savvy world, businesses must rely heavily on cloud infrastructure. This dependence comes from the unmatched scalability and flexibility the cloud has to offer. The cloud’s ability to store, process, and analyse vast amounts of data enables companies to seamlessly adapt to changing demands. From swiftly making data-driven decisions to fostering innovation to providing a competitive advantage, all can be done by a cloud computing platform.

This has resulted in the evolution of cloud infrastructure into an indispensable asset for organisations seeking to thrive in the modern digital landscape.

Why?

To provide scalable and seamless services to their loyal customers, of course! One such cloud computing platform that has emerged on the top is Amazon Web Services (AWS). This prominent platform offers customers extensive services, including functionality, performance, and flexibility.

But at a high cost.

With organisations scrambling to outbid their competitors in the cloud game, organisations are pushing for expenses and cost optimisation checks, especially regarding AWS.

Our client, who is into construction intelligence, faced the same issue. They wanted to maintain the platform on AWS cloud but to optimise the amount spent in a structured manner. So, let’s look at how we helped our client reduce its overall expenses without compromising the quality of leads and data.

Let’s understand a bit about our client first before we discuss the problems and solutions.

Our client is a leading construction intelligence platform that has been providing customers with quality construction market leads and data for over 80 years. One of their primary functions was to deliver verified project details and GDPR-compliant contacts.

But, for them to achieve this, they must rely heavily on council websites.

This is because they collect, verify and refine data before sending it out to their customers via their portal. The management of data and projects is done in collaboration with our experts at Arrk, using multiple modules such as ML module, Platform, Planning Portal, Engines, and Harvest.

So, why did we look into Platform modules?

The idea of the Platform module is to serve as the backbone of the solution, and development is done using Python Flask (API) in the backend functionality and ReactJS for the frontend. Also, the primary databases used for storage and retrieval are Postgres and ElasticSearch.

Multiple engines, such as notification and export engines, also help the platform perform external tasks. This entire process is deployed on an AWS cloud that uses services such as Elasticache, EC2, Key Management, SQS, and more.

But, with great tasks comes greater costs!

Considering the number of programs used, the extent of data to be sorted through, and the platform’s overall security, it is no wonder why AWS is so expensive!

So, this was our task – using AWS but at optimised costs!

Challenge Accepted!

We at Arrk were challenged with optimising AWS’s cost structure by looking for ways to collect, refine and verify data collected on the platform. We also had to deliver strategies to help reduce the overall expenses without compromising the quality of market leads while steadfastly delivering the product to the client base.

And solve it we did!

How did we solve it?

Our expert team identified areas in the client’s AWS cloud platform to highlight where significant costs can be saved. With a thorough analysis of the utility of infrastructure, the patterns of usage, and the total breakdown of costs incurred, we could showcase options for optimising the platform to the client.

With both our collaborative efforts, we were able to leverage tools such as AWS CloudWatch logs and AWS Cost Explorer to gain valuable insights into optimising the cost structure and breaking down areas where spending can be minimised.

What did we find?

As per our analysis, the key areas for cost savings include:

1. Right-sizing – Here, we were able to showcase to our client how to reduce the legacy application’s size and instances for lower environments.
2. Optimising storage – This was achieved by reducing EFS storage size by changing the type from GP2 to GP3. Also, we changed S3 to Glacier for harvest documents and changed the S3 retainer period to 1 day.
3. Auto-scaling – We implement auto-scaling for Web, API and also shut down of scripts for lower environments
4. Containerisation and Serverless – Migrating engines did this to Fargate and Docker by migrating API and Web to Fargate-based ECS clusters.
5. Reserving capacity and saving plans – We introduced AWS saving plans to help cover computation and ML services and reserved instances for Harvest and RDS.

Knowing what we know, it was now time to formulate strategies to support our analysis.

How did we plan to save costs?

For our ‘AWS Cost Optimisation project, we implemented a documentation and project management system.

How did we do this?

By using Confluence and Jira!

We used Confluence as a central documentation hub and maintained a dedicated space where all relevant details, plans, and progress reports were saved. These included details about cost-saving options and other technical information. This ensures that all team members have access to updated information and can collaborate with themselves easily.

Using Jira, we created a dedicated board to track and manage all the activities related to cost optimisation. The board follows a Kanban method, allowing users to visualise the entire process in one go or part by part. Each task has clear objectives and deadlines and can be assigned to the appropriate team members. This was done to support efficient task management and ensure the project moves smoothly.

We created a collaborative knowledge-sharing space and smooth workflow using these two platforms.

In summary, the client’s partnership with us was the ideal occasion to demonstrate how a thorough and rigorous strategy may deliver efficient solutions to optimise costs in the ever-changing world of cloud computing.

To get crucial insights into the overall breakdown of expenses and pinpoint areas where optimisation may result, our committed team collaborated closely with the client. We assisted them in delivering great market leads and information to their clients while upholding overall expenditure optimisation by developing tactics specifically tailored to the client.

So, if you are looking to reduce cloud expenses and improve the services you provide?

Join forces with us right now!