The Taguchi Method Defined for Web Analytics

January 23, 2010 by Gregg Larson

Thanks to Google’s Website Optimizer, the phrase “Taguchi Methods “ has become one of the hottest (and most misunderstood) in Internet Marketing right now.  As an Applied Statistician, I learned that Taguchi’s methods in terms of understanding and implementing quality into a product.  To Internet Marketers, the phrase “Taguchi Methods” describes a subset of Taguchi’s contribution to the design of experiments.

At the heart of an Internet Marketer’s interest in Taguchi Methods is the use of orthogonal arrays to optimize the design of a product (which, in the internet world, could be a web page, Google Ad, or keyword phrases).  The key to the experiments is to filter out the critical factors from the “noise”.  To do this, a experimenter uses a set of orthogonal arrays or factorial designs to determine specific combinations of factors to test.  Before Taguchi’s innovation in experimental design, an engineer would be required to understand advanced mathematics and statistics to design experiments.  It was this lack of knowledge that prohibited many engineers from using them.  Taguchi invented a new way to understand orthogonal arrays, (using linear graphs) to greatly reduce the knowledge needed to design them. 

While working on my Master's Degree, I happened to use the "Bible": "Taguchi Methods: Orthogonal Arrays and Linear Graphs, Tools for Quality Engineering".  The Image Below is the cover to that manual.

One of the things I hear internet marketers that makes me chuckle is that "I know about A/B tests, but I need to learn about Taguchi Methods."  They are shocked to learn that an A/B test IS a Taguchi Method, it just happens to be the simplest version.  Below is Taguchi's reporesentation of an 2 factor, 3 level test (an A/B Test is simply the first two columns of this table):

Looking at this diagram, the nodes show the columns and interactions.  With a 2 factor, 2 level test, all interactions are tested with 4 different combinations.  If there was an interaction that needed to be studied, the third column represents where this could be sudied, or can represent a third variable that has 2 levels to study.  Taguchi's contribution to orthogonal arrays is the diagram to the right of the table.  Everything else had been invented in the first part of the 1900's.  To get an idea of the complexity, here is a graph of an L₃₂ orthogonal array consisting of 31 variables with 2 different levels:

Thankfully, today we have computers and web sites that spit out orthogonal arrays for us.  This does not reduce the necessity to understand how orthogonal arrays are designed, in fact, I believe it increases the need, because you simply cannot trust the arrays created by a computer program or website (even Google) without an understanding of what they are producing.

The Taguchi Method Defined
Below are the steps to properly conduct the Taguchi Method:
1>    Define the Problem:  Your problem should be to identifying charectoristics that produce more revenue, however, you can be interested in raising traffic.
2>    Determine the Objective: The Objective is defined by a set of characteristics that are measureable, such as revenue, clicks, bounce rate, etc.
3>    Brainstorm: Determine the Critical Quality Characteristics you wish to study: Page layout and widgets, colors, words, pictures, text  placement may be a set of factors you wish to study.  During the Brainstorm session you should also determine the levels within each characteristic, such as “left”, “middle”, or “right” for text placement, “pictures” or “no pictures”, “red”, “blue”, “green”.
4>    Design the Experiment:  Today we usually put our factors and levels into a computer program or website that returns combinations for us to use.
5>    Conduct the Experiment and Collect Data:  The Taguchi Methods, like any statistical method, requires a random sample to be collected.  Many people do not understand randomness in statistical terms, and often make serious mistakes.
6>    Analyze the Data: Using Ananlysis of Variance (ANOVA) on the measurements of the Objectives.  This is one place a statisitian comes in.
7>    Interpret Results:  Define the Optimum settings for the control variables.
8>    ALWAYS RUN A CONFIRMATION EXPERIMENT: This, like a random sample, is where most people fail to complete the Taguchi Method.

Even though you can have a computer spit out combinations for you, it is critical to the success of your tests that you understand all the steps of the Taguchi Method, and you do it correctly.  Specifically, you would benefit greatly from a person such as myself that was trained in Applied Statistics.  I can help you succeed in all facets, but the biggest mistakes are made when you incorrectly Brainstorm variables and levels to measure, accidentally designing interactions into the experiment, incorrectly sample your population, and incorrectly analyze the data.  Contact me to make sure you don't make a mistake.
 

Permalink