Every Marketer Needs a Statistician

September 18, 2009 by Gregg Larson

Doctors and Lawyers, CEO's and Accountants, most high level managers have a side kick that navigates the areas they don't specialize in.  In the world of Web Analytics, every Marketing Manager responsible for running analytics tests using Omniture, Google Web Optimzation Tool, or something similar needs a Six Sigma Black Belt statistician to help with desgning and analyzing the results of the data.  Much like doctors need lawyers to help them navigate the legal world, and accounts help CEO navigate the complexity of Federal Tax laws, Marketers need us to help them design effective tests and ensure that the results are conclusive.

Profound Knowledge

Most employees with years of expereince unknowingly develop axiomatic beliefs that inherently drive their opinions and thoughts.  I the world of web testing, this is a sure sign of failed experiments.  As a Master's Student trained in Dr. Edward Deming's management principles, I have learned the "theory of profound knowledge", which enables me to come in  to an organization free from these beliefs, with an interest in driving higher ROI, based on a process which relies on statistically valid conclusions.  My desire is for your company to make as much money as possible, optimize the throughput of your current organization, and spending as little to get it.  Because of this success oriented mindset, I am not vested in any particular solution except the one that makes your company the most money.  Through a disciplined process of experimentation, I can classify the variability in your marketing experiments and eventually explain the process so well that you will be in total control of your marketing endeavors!

Designing Experiments

The first place a Black Belt can help you is in the design of experiments.  Marketers should be concerned with determining what needs to be tests, not the details of HOW it will be tested.  A Statistician can help with this by applying their extensive experience in the design of experiments, using orthogonal arrays (taguchi arrays),and sampling techniques to ensure the test is free of defects, that all variability is accounted for, and the test contains no interactions that may affect the results of your test.  Even if these tests are "automagically" designed within your testing tool, a statstician trained in experimental design will be able to validate the test before it is run, increasing the liklihood that your conversion rates will be able to show clear winners and losers from the test.

 Analyzing the Results

There is so much more information that you can extract from your tests, and unfortunately the current tools can provide some of that functionality, but not all.  For a Black Belt Statistician, however, these tools are readily available and we are eager to use them!  One of the main areas where this is of value is in determining the importance of the variability, the percentage of variability that you have captured, point estimates, and confidence intervals.  While an application can tell you that a test is inconclusive, a Black Belt can look at the same data and tell you WHY it is inconclusive.

Permalink