By Sasha Kipervarg, Head of Global Cloud Operations, and Patrick Raymond, Sr. Product Manager Infrastructure
Preface
By J.R. Storment, President of FinOps Foundation
Much like Agile, FinOps takes an agile approach to managing cloud spend. It requires teams that span functions and works together. Only when finance teams are aligned with tech teams on that Agile approach can they relate to the problems the IT teams are seeing.
It’s not easy to make cost management a first-class citizen in your cloud journey. Learn how LiveRamp helped their IT teams and developers embrace cloud financial management best practices, and realize a thoughtful approach to developing products with a cloud cost efficiency focus.
LiveRamp’s FinOps success comes when they are aligned with shared processes and goals. Here’s a small peek at this FinOps journey.
—
EDITOR’S NOTE: Excerpt republished with permission by LiveRamp.
It was in September of 2019 that we breathed a sigh of relief and celebrated our migration to Google Cloud Platform (GCP) from an on-premise data center. We had started approximately one year before, and it was a massive engineering effort that leveraged the ingenuity, dedication and work of all of LiveRamp’s capable development teams. We containerized our applications in the process, moved 100 Petabytes of Hadoop, and gave our development teams a reliable infrastructure API– all on schedule.
You can learn more about the highlights of this migration in our conversation with Kelsey Hightower from RampUp 2020.
Midway through the migration, as engineering teams began to run their applications fully in the cloud on GCP, their costs did not match the budget we had planned. Despite containerizing so that we could horizontally scale workloads according to our needs, and despite budget exercises with the Finance team and our engineering budget owners prior to the migration, the GCP numbers we were seeing did not match the estimates we had previously laid out.
To fully appreciate the challenge we had at this point, it is necessary to understand the magnitude of our scale, our GCP spend, and how the decision to spend money shifted at LiveRamp when we migrated to the cloud. The top three areas of spend for LiveRamp are GCE (Compute), GKE (Kubernetes Engine) and GCS (Storage). We have massive GCE clusters running Hadoop that our containerized applications make use of for batch data processing. Data from those workflows are persisted in GCS.
We spend millions of dollars to operate and secure these environments on behalf of our customers every month. Prior to GCP, we made capital expenditures of racks of equipment every few quarters. The decision to spend was a highly controlled and collaborative effort between engineering leadership and finance. The advantage of GCP — and what ultimately drove part of our decision to move to the cloud — was that developers themselves could spin up more CPU, RAM or disk at light speed so the company could react to customer and product needs quickly. But we had not moved fast enough to provide our engineers the proper financial literacy to understand budgets and necessary safeguards to ensure they wouldn’t spin up a million dollars of CPUs via API in less than a few minutes.
Our initial approach was to look at low hanging fruit to see if we could quickly reduce our GCP spend. Together with Tim Fornero of our finance team, we canvassed engineering to discover optimization opportunities that could quickly be implemented. We found lots of one-time optimizations and lowered our GCP spend, but it wasn’t enough; so we kept on going.
Once the basic infrastructure-related optimizations were implemented, we focused on application-related re-architectures that could be implemented over a quarter or two. It felt like we were on a continuous wheel of discovering budget issues, reviewing potential optimizations, implementing them and then reviewing the data again. Rinse and repeat. Over and over.
Finding FinOps
It was at this point that something clicked in our minds. Our SVP of Finance, Chris Garber, had previously mentioned FinOps to Patrick and I as something to take a look at, and a cursory review of the FinOps Foundation website made clear they were advocating a holistic framework and philosophy for solving the problem that we had. We were naturally curious to learn more about FinOps and attended the next available FinOps practitioner course.
We spent a full day learning about the FinOps principles from Mike Fuller (Principal Systems Engineer at Atlassian). We also had the opportunity to share ideas and discussions with 30 other peers from companies all facing similar challenges in different segments of the industry across multiple clouds. We immediately made the connection between Agile, DevOps and FinOps principles and were excited about how these aligned with LiveRamp Engineering culture. Following the course, we were able to gain executive support for a FinOps practice to be housed in engineering.
Making it Happen
Diving a little deeper, we made a number of structural changes to how we thought about budget ownership, the relationship between finance and engineering, and the cost optimization lifecycle. Budget ownership was clearly assigned within engineering and more granular cost information was made available to help provide context on team spend. The partnership between finance and engineering was strengthened and a common lexicon was shared between the two organizations to ensure that everyone was aligned on the differences between on-prem and cloud costs. Annual recurring cost savings targets were implemented in order to keep cost in check and a process was built around gathering, vetting, and prioritizing cost projects.
—
Discuss GCP cost management with the FinOps Foundation
On June 4th, 2020 at 9AM, join the FinOps Foundation, during their member call, where they’ll discuss GCP cloud financial management best practices. All it takes is to sign up to join the FinOps Foundation and access its members-only meetings.
During this call, we’ll hear from these experts:
- Sasha Kipervarg & Patrick Raymond: LiveRamp’s FinOps Journey
- Stephanie Gooch: GCP Cost Management experiences at KPMG
- Anders Hagman: Spotify’s FinOps Progression
- GCP Billing Team representative will be on hand for Q&A
Members can also watch meeting recordings (once they’re live) to go through the material at their own pace. You’ll also have access to our community Slack channel where you can meet other FinOps practitioners of all types of skill levels and company sizes.
Join us by signing up to become a FinOps Foundation member today. We look forward to seeing you there!