One of the coolest startups - Recorded Future

Friday Jul 27, 2012 by Shobhit Chugh - Associate, McKinsey & Company

When I use the phrase ‘cool startup,’ what comes to your mind? Facebook? Instagram? Twitter? For me, one of the coolest startups in the world is Recorded Future, who’s missin is to ‘Unlock the Predictive power of the Web.’ Say what? Let me explain.

Predictive Modeling

The first thing you need to understand Recorded Future, is to understand Predictive Modeling. Wikipedia defines Predictive Modeling as ‘the process by which a model is created or chosen to try to best predict the probability of an outcome.’ If you are confused even more now, don’t worry about it. Let me try to explain with an example.

Let’s say you own a network of car dealerships (and are of course, very rich). Three months ago you decided to offer extended warranties (for an additional price) as an add-on to your car sales. Your sold fifty thousand cars, and the extended warranty sales went very well. Now you wonder, what if you called up people who had bought cars from you 6-9 months ago, and try to sell them extended warranties?

While the option sounds pretty good, the problem is that you sold two hundred thousand cars during that time. Calling everyone would be hard, and not financially viable. It would be best if you could just call the ones who are more likely to buy the extended warranty. But how will you know who is more likely to buy the warranty, without actually calling them? That is where predictive modeling comes in.

This is how predictive modeling will work. You have lots of data from the people who you offered extended warranty to with car sales - demographic data, type of car, service history etc.. Based on it, you (or rather a statistician or data scientist) will create a model, that uses this data set as a training set to understand what factors affect the probability of purchase. The model might find, for example, that people who bought sedans are five times more likely to buy extended warranties than people who bough coupes.

Once the model is created using the training set, you can use the model on the larger data set to score the probability that each person will buy the extended warranty, if they were offered. Then you can pick just the top ten thousand and call just them, rather than calling all two hundred thousand, thus maximizing the profit from the marketing campaign.

Where is predictive modeling used?

Well, pretty much everywhere now. To see an example of this, just read this article here about Target, and their approach to predictive modeling.

The data problem

They key to designing kick-ass predictive models is using as much data as possible, especially non-obvious sources of data.

Let’s reconsider the car example. What if we could lookup car repair spends for all the people who we have sold cars to? Though it would be creepy, we could setup and test a hypothesis that people who have had accidents and paid a lot for car damages in the past would be more willing to spend money on purchasing an extended warranty?

So where does Recorded Future come in?

The web provides information about events in the past, and predictions for events in the future (e.g., a news report mentioning that President Obama will be visiting Ohio on tuesday). But this data is human-readable, not machine-readable; hence it cannot be used by predictive models.

Recorded Future brings the ability for predictive models to be run on event data from the entire web - both for events in the past, and those predicted to happen in the future. It does it by technology that parses news sources across the world, and indexes these events and predictions in a format that can be used via APIs.

An immediate use case where Recorded Future has built such abilities is in Financial Services - the predictive web is not being used to power trading strategies. This is all possible through the use of the Recorded Future API, which gives access to the Recorded Future stream.

Recorded Future News Analytics for Financial Services from Chris Holden

Recorded Future provides not just an API, but also a slick UI with visualization tools for use by intelligence analysts. A neat example of use of the slick UI is Intelligence Analysis - monitoring the web for potential threats. A use case example is the use of Recorded Future to monitor Tweets about the situation in Libya, and understanding what was likely to occur in the next day.

I did get access to the next version of the UI, and tried it out. I searched for various terms, including politicians such as Mitt Romney, and companies such as salesforce.com. By using the API for a few minutes, I started to understand how the UI could be used by intelligence analysts. For example, if I was part of Obama’s campaign, I could totally see someone using Recorded Future to keep track of events and opinions related to Mitt Romney, and slicing and dicing by various metrics such as related companies, people, news streams, sentiment etc. An example would be if there is a lot of negative sentiment news related to Mitt Romney and Bain Capital - that’s probably a place where Obama campaign can start directing some of their media spend.

The tip of the iceberg

The use cases that Recorded Future is tackling now are just the tip of the iceberg, in my opinion. The availability of this structured feed of past and future events presents possibilities in countless areas - whether it is competitive intelligence, sales opportunities, operations/supply chain risk alerts, disaster alert monitoring by the insurance industry - the list goes on! I am curious to see where this company goes, and will be watching closely.

Shobhit Chugh is an Associate at McKinsey & Company in Boston, Massachusetts.  You can find this post, as well as additional content on his blog titled The Business Co-Founder.  You can also follow Shobhit on Twitter (@shobhitchugh) by clicking here.  Please note: the views expressed on this blog post are mine and not those of my employer.

< back to all blog entries

Categories: