Category: experimental design
Thanks to Google’s Website Optimizer, the phrase “Taguchi Methods “ has become one of the hottest (and most misunderstood) in Internet Marketing right now. As an Applied Statistician, I learned that Taguchi’s methods in terms of understanding and implementing quality into a product. To Internet Marketers, the phrase “Taguchi Methods” describes a subset of Taguchi’s contribution to the design of experiments.
Within the web analytics world, I have noticed two common themes regarding the "Taguchi Methods" and how they are used within the Web Analytics Space. I was working with a client whom had mentioned they had wanted to use the "taguchi methods" in order to test the optimum pay per click ads for their keyword programs. The first thing he had confusion about is the difference between multivariate testing, A/B testing, and Taguchi functions. The second thing he asked me about was why some of the tests created through Taguchi Metho
This paper outlines my research into the use of Sequential Conjoint Analyses to perform consumer market research. Specifically, I am attempting to develop a process. To accomplish this, I researched the ‘Voice of the Customer’ and then conducted a set of Pilot Studies on my research using Conjoint Analyses. The goal of the Pilot Tests was not to provide inferential statistics on a specific market; instead it was to test that I may have identified the Critical Attributes to study, and to develop the framework of the market research process.
You are testing too many things at one time: Whether you are testing more than two pages in an A/B test, or running a multi-variable test, the more variables you test, the more samples you are going to take to get a high confidence result. The problem is that by design, MOST of the items you are testing are not the prime driver! The more you test, the larger the haystack you are going to have to look for.
While Orthogonal Arrays have been around since the 1940’s, it was Genichi Taguchi who popularized them by making them easier for engineers to execute valid experiments. It is for this reason that orthogonal arrays have incorrectly been termed “Taguchi Arrays”. His first contribution of the linear graph made it easy for engineers to understand. His second contribution was the creation of triangular tables that made it easy to create alias structures for any design . The strength of these designs is in their ability to take a large number of factors and determine the “critical few” with as f
The Plackett-Burman is an Extreme Screening Experiment in which the researcher wants to screen out the critical main effects from the significant or trivial effects. This design is very effective if time and cost are an issue, if each main effect is equally probable of affecting the criterion measures, and the main effects have either slight or no interactions .
The Plackett-Burman Design’s strengths are that it is relatively easy to construct as well as being easy to analyze. It also allows for the study of K number of factors in only K+1 runs. This means that the seven factors in this exp
Compared to a Full Factorial design, which can test all main effects and interactions independent of each other, Fractional Factorial designs test only a carefully prescribed set of treatment combinations. A half replicate design would reduce your tests by half. A quarter replicate test reduces the number of tests by a fourth, and an eighth replicate design reduces the number of runs by an eighth. Of course, as you reduce the number of tests, you increase confounding within your design. What this means is that as you remove testing combinations, you begin to add interactions with your mai
A conjoint analysis is a statistically based procedure designed to assess the ‘Voice of the Customer’, using a combination of fractional factorial experimental designs with an Analysis of Variance (ANOVA). It is a powerful marketing tool that allows the researcher to determine how consumers or customers view the relative importance of product or service attributes. The conjoint analysis is a procedure that is becoming increasingly popular as a tool for both market researchers and product development teams. Its popularity is driven by the fact that these studies are relatively easy to create
