There are over 100MM internet searches performed daily. Keywords,
phrases and numbers are collected and stored within the major search
providers vast databases.
What if that search data could be
used to predict trends before they happen?
In 2008 and 2009
Google researchers showed some success in trending flu cases using
search data alone. However, most studies since have been inconclusive,
or statistically irrelevant.
A few researchers working with Yahoo!
recently leapfrogged prior
studies by formulating a set of statistical models that were comparable
to baseline models. The intent was to see if by just using search
data, could they get close to well-informed models. The first step was
to observe the impact of time on the models. In the case of Movies,
Games and Music, build up towards release showed
significant traffic spikes, signaling consumer intent (or interest).

The
conclusion of the studies showed that search data alone - automatically
generated models predicted revenue/success for Games, Music,
and Movie releases with an overall range of 74%-93% accuracy.
The
regression (log) scatterplot below depicts the model visually. The dots
represent predictive values, the line actual baseline. The closer the
dots are to the line, the more predictive the regression is said to be.
In this case, with Theatrical releases, the model had a near 80%
accuracy for success based solely on search volume data.

The
same models can't be used universally for all topics, products or
brands. And these models may be improved upon by adding more variables,
or using traditional methods of measurement.
The big deal here is
that using something as simple as query data - future outcomes are
meaningfully predictable. The reduction in cost and time in rendering
the models is staggering.
In addition to search, mining unstructured internet data,
social media and sentiment often reveals trends that weren't visible through traditional studies. When combined
with Direct-Survey & Consumer Insight analysis, the planning
capacity grows dramatically and informs marketers in a whole new, and
often surprising way.