Badge (2).png
image (6).png

How Teracloud helped SocialRank to migrate to the Cloud

Executive Summary

SocialRank is a Data analytics company with the “easiest way to identify, organize, and manage your followers on Twitter and Instagram”. Their core business is based on a medium/large sized ElasticSearch cluster holding 22TB of data and growing. Teracloud managed to lower costs and improve security and reliability of the platform by migrating it to the AWS cloud.

About Social Rank 

SocialRank is the easiest way to identify, organize, and manage your followers on Twitter and Instagram. It allows you to:

  • Identify: It provides the opportunity for consumer opt-in: allowing you to generate ethical, high quality first-party data. With the power to export and re-market to valuable target audiences.

  • Segment: Discover new audiences, segment, and filter across industries and interests with bio keywords, geographic breakdown, hashtag usage, activity rate, post times, and more. Understand audience overlap across a portfolio of brands — yours, competitors, and more.

  • Engage: Activate an audience whenever and however. Drive calls to action like new platform follows, newsletter sign-ups, e-commerce, and more. Engage with ease to re-market and prompt impactful results.

Social Rank Challenges

Diseño sin título (4).png

SocialRank infrastructure was completely deployed on-premise, with all the disadvantages of this situation. There were multiple custom-made data ingestion processes connecting to Twitter and Instagram backbones to feed the large ElasticSearch cluster and serve the customers. SocialRank contacted us because the base Kubernetes cluster lost its only Master node and they lost control of their cluster, it was just a matter of time for the whole infrastructure to go down. The challenges can be summarized as:

image (6).png
  • Security: 

    • Deprecated Kubernetes version: All Kubernetes manifests were written for Kubernetes 1.4 

    • Deprecated Elasticache version: System running on Elasticache v5

  • Reliability:

    • Incomplete backup strategy

    • No DR strategy defined. No resources devoted to DR

    • No node replacement strategy. Hardware failures introduced an extremely high risk of platform performance degradation.

  • Performance:

    • Scalability: Very high costs for scalability because of on-premise hardware.

  • Operational:

    • Underlying tools versioning results in increasingly high difficulties to upgrade software and apply patches. Also, no modern tools (like SecretsManager) can be used because of the same reasons

Why AWS

AWS was chosen because it helps create a secure, high-performance, resilient, and efficient Kubernetes cluster, which is the base of the whole infrastructure. 

Using EKS, AWS Lambda, Route53, ACM, S3 and other managed AWS services allowed us to plan a clean and safe migration, deploy the infrastructure in parallel to the dying on-premise one, test it fully and get trust (from the engineering and from management teams) to flip the switch.

Why SocialRank chose Teracloud

SocialRank needed a very trusted AWS partner with a strong background in cloud migrations. Even more importantly, given this was a business continuity risk scenario, they couldn’t afford an error: any delay or any serious issue with the partner may result in SocialRank's customers losing access to the platform and losing data. Fortunately for all, Teracloud has shown these capabilities on past projects with SocialRank and also high implementation speed on similar scenarios.

Teracloud Solution

The solution was structured around the 6 core aspects:

  • Core customer requirements

  • Security

  • Performance

  • Reliability

  • Cost optimization

  • Operational excellence

We used the 5 pillars of the Well-Architected Framework along with the core customer requirements to design the solution. Given that the risk increased with time (any pod malfunctioning on the on-premise infrastructure won’t be recreated because the master was down), we designed the solution based on three principles:

  • Speed: Choose the solution that fulfills the requirement faster

  • Reliability: Customer data is the maximum asset, any solution that introduced risk here was discarded

  • Security: Customer data should be protected at rest and on transit, no exceptions.

diversity-g095fc12da_1280.png

High level architecture

undefined (3).png

In this design, we leverage the power of AWS services to reach our goals faster and with fewer costs:

  • The core element is the EKS cluster because all services are run within it. The backend, frontend, and ElasticSearch databases run inside the cluster. The AWS ElasticSearch service could not be used in this case because the client required a custom ES plugin, which is currently not allowed on AWS ElasticSearch.

  • The ElasticSearch pods require storage space, which is provided by EBS volumes attached to the EKS nodes.

  • All components except AWS Application Load Balancer were located on private networks, and ssh access was provided by AWS SystemsManager.

  • All secrets were migrated to AWS Secrets Manager, resulting in simpler architecture for the Kubernetes internals.

  • Route53 provides DNS resolution.

  • AuroraRDS for mySQL was the preferred approach to migrate a support mySQL database from on-premise.

  • The deployment of new app versions is handled with a mix of CodeBuild and Flux for Kubernetes, with the aim of getting a completely automated CI/CD environment.

Results and benefits

Once the switch was flipped, the immediate benefit was non-tangible: the business continuity risk was reduced to near zero. Additional benefits are listed below, grouped by area of impact:

image (6).png

Next Steps

To successfully complete the migration on time some compromise decisions were made, which resulted in a less than ideal scenario. While we did not introduce security or reliability risks, some areas can certainly be improved:

Costs: The current instances can be migrated to Graviton, which will reduce costs. This initiative is currently blocked until we upgrade to ElasticSearch v8.

Performance: Application refactoring can result in better usage of cloud resources. Also, it can allow us to auto-scale.

Operational: Having an automated CI/CD pipeline will allow us to be more confident in the process of deploying new applications and versions. 

About the Partner

 

Teracloud is a fast-growing AWS Advanced Consulting Partner company created by certified cloud experts in migrating and deploying startups, enterprises, and everything in between to the cloud.

 

We have worked for companies from many different industries such as airlines, healthcare, education, and e-commerce designing, implementing, and managing Cloud workloads with HA architecture under 99.999% uptime SLA and PCI/HIPAA compliance requirements.

 

We also have a strong commitment to the Cloud community, we host local branches of AWS User Group and meetups in support of education, evangelization, and evolution of the IT community. As an Amazon Web Services partner, we have been invited to participate in the AWS Community Day Buenos Aires 2019 and AWS main conference in Las Vegas re:invent 2019 as speakers.

Contact us and let us build a BETTER VERSION of your story, we take your business to the NEXT LEVEL