November 16, 2011

CXO Guide to Moneyball

the CXO, you have watched the monthly cloud bills steadily rise. You’ve
tried everything to drive them down, but nothing seems to make a
difference. You have lingering doubts on the cost effectiveness of your
cloud usage, but all you have to measure are two metrics: the
quantitative monthly bill, and the qualitative value you derive from the

Welcome to cloud moneyball.

For those of you who have not read the book or watched the movie, Moneyball
is the story of Billy Bean, the Oakland A’s general manager who
pioneered a new metrics-driven approach to baseball. The moneyball
theory claims that the collective wisdom of baseball insiders for the
last century has resulted in a focus on the wrong measures of success.
The theory says we should be focusing on statics that create runs, such
as on base (OBP) and slugging percentages (SLG) - not secondary metrics,
such as batting average (BA), runs batted in (RBI) or stolen bases

Like baseball in the early 2000s, the cloud suffers from a dearth of
the right metrics. We have over-focused on cost metrics, and ignored the
real reason we are in the cloud: to drive business value. Below we’ll
introduce three metrics CXOs can use to measure how effectively their
organizations are using the cloud.

Infrastructure Cost of Goods Sold (ICOGS)
Infrastructure cost of goods sold is the percentage of your revenue
expended on cloud infrastructure. This is an essential measurement since
it will be a key variable in the margins for your product or service.
It is sort of the on base percentage (OBP) of the cloud. The calculation
is as follows:

ICOGS = current infrastructure costs / revenue

Acceptable ICOGS percentages will vary from business to business, but will typically range from 20% to 40%.

Infrastructure Utilization (IU)
Infrastructure utilization is a measure of how effectively you are
utilizing the infrastructure you have provisioned. This is actually a
family of calculations, varying by type of infrastructure (e.g. compute,
storage, database). IU is the slugging percentage (SLG) of the cloud.
The metric is simple:

IU = utilization of metric / maximum available quantity of a metric

For example, to measure the effectiveness of your utilization of
block storage, you calculate your actual usage of the blocks you have
provisioned. For example, if you have provisioned 100TB of block storage
and are currently utilizing 60TB, your IU is 60%.

The goal of all cloud applications is to utilize as close to 100% of
the underlying infrastructure while maintaining expected levels of
availability and performance. Good IUs will typically be above 75%. If
your IUs are below 60%, it's possible your application may be more
cost-effective on physical infrastructure than the cloud.

Cloud Elasticity (CE)
The purpose of CE is to measure the elasticity of cloud costs as your
business scales. CE is sort of the cloud equivalent of Value Over a
Replacement Player (VORP). The calculation works like this:

CE = (ICOGS at projected 12 month maximum revenue - current ICOGS) / current ICOGs

For example, if my current ICOGs is 30%, and my ICOGs at my projected
12 month maximum revenue is 40%, my CE is 33%. If your infrastructure
costs decline relative to revenue as you scale, your CE will be
negative; if your costs increase relative to revenue, your CE is

A typical CE will vary based on your business, but will range from
-10% to -20%. A positive CE number is a likely sign you have something
wrong in your usage of the cloud.

Last Thoughts
For those of us in Boston, moneyball has a special place in our heart,
having played a role in breaking our 86 year curse. But as a CXO, cloud
moneyball can have a special place in your heart - ensuring you are
maximizing your return on investment in the cloud, and allowing you to
manage to clear metric-driven goals. So ask your staff to start tracking
these metrics, and take control of your monthly bill.

I’m interested in hearing about the metrics you use to manage your
cloud. Drop me an email with your metrics and how you use them.

Joe Kinsella is the VP of Engineering at Sonian.  You can find this post, as well as additional content on his blog called High Tech in the Hub.  You can also follow Joe on Twitter (@joekinsella) by clicking here.