Replatforming for High Availability

Guitar

Capabilities

DevOps, Cloud, and Web

Technology Platforms

Ruby on Rails Rails Docker Docker Google Cloud Google Cloud Platform

Impact

A recently acquired product was intermittently going down. We replatformed the product to be more highly available. Uptime was increased from 95.84% to 99.89%

Challenge

Spotify had recently acquired SoundBetter, a marketplace where musicians can find and work with producers/audio engineers. SoundBetter’s Rails application was experiencing intermittent outages and did not meet Spotify’s SLAs.

Spotify needed expertise in high availability Rails deployments to stabilize the application and prepare it to be rewritten for Spotify’s infrastructure.

SoundBetter was not compatible with Spotify’s existing infrastructure and could not be migrated until compliance and compatibility issues were fixed. So, the availability issues needed to be addressed quickly using a custom, short-term solution.

 

We replatformed Spotify's acquired product SoundBetter to make it more highly available.

 

Solution

 

Replatforming

Striders addressed the availability issue by setting up load-balanced Google Cloud VMs running Docker to host the Rails application. The impact of deployment failures was mitigated by implementing blue-green deployments. Hosting the application’s database on Google Cloud SQL added further availability guarantees.

 

Stabilization

To stabilize the application, the team improved monitoring and wrote runbooks for common issues. Builds and deployments were automated to reduce errors from manual procedures.

 

CI from Scratch

SoundBetter initially lacked a CI process, so it took hours to deploy updates. Striders wrote CI scripts to precompile assets, run the test suite, and build a Docker image. After CI automation, it now takes 10 minutes to build and deploy, drastically reducing the lead time for changes.

 

Impact

 

SoundBetter’s uptime increased from 95.84% to 99.89% within a month of Stride joining the project. This allowed the Spotify team to focus on rewriting SoundBetter for compatibility with Spotify’s infrastructure.

The lead time for changes was reduced by automating integration and deployment. Now the team can iterate on higher value work, such as integrating a GraphQL layer with the Rails REST API.