VOTRE TITRE
VOTRE SLOGAN
Kinds Of Split Testing Software
Wednesday, 02 June 2010 07:57 | 
Author: Dominic DeLong |
You've probably heard that split testing elements of your website can improve your conversion rates. And that's true. But there are many kinds of split testing out there. And there are many ways to set up split tests on your website.
You've probably heard that split testing elements of your website can improve your conversion rates. And that's true. But there are many kinds of split testing out there. And there are many ways to set up split tests on your website.
Let me give you an overview, so you can understand the choices you have before you. I'll walk you through the main distinctions that rule the field.
One big distinction is between single variable and multiple variable (multivariate) testing.
With single variable testing you just pick one element on your page to test. Take your headline, for example. If your current headline says "Are you having trouble with tooth decay?", you might want to see how "Do your kids have tooth decay?" works. In a single variable test, you just find a way to rotate each of those headlines on your page, and keep track of how many sales each generates.
Google offers a single variable split testing tool, but you have to have an Adwords account to use it.
Single variable testing can be slow. How would you like to test 10 times as much with the same amount of traffic? Well, you can. You use a method called multi-variable, or multivariate testing. This allows you to test several elemens on your page simultaneously.
Among multiple variable software solutions, there are hosted plans and self-install plans. If you want to install software on your server (which can be faster and cheaper) you will have more control over your data. And you can usually find someone to install it for you (often the vendor provides this service for a nominal fee). If you want to go with a hosted plan expect to pay more.
Hosted solutions include Google's Website Optimizer, and Vertster. Installed solutions include the Split Test Accelerator and Kaizen Track.
The next distinction probably causes the most confusion among people looking into split testing for the first time. It's also more important than the last distinction, becuase it determines how fast and accurate your tests can be. The three kinds of testers are 1) random independent rotators, 2) Taguchi testers, and 3) full-factorial testers. Taguchi ranks first for speed and second for accuracy. Basically Taguchi testing involves using a marvel of mathematics known as "orthogonal arrays". These have special properties that allow you to test every combination in a space while running only a small fraction of the actual combinations. You have to be a little bit careful about interaction effects with Taguchi testing. The Split Test Accelerator and Kaizentrack are Taguchi testing tools.
Full factorial tools allow you to cycle through all the combinations for your factors. So if you have 3 headlines, 3 offers, 3 P.S. statements, and 3 images you're testing, you will cycle through 81 pages. One advantage of full factorial testing is that you don't have to worry as much about interaction effects. Good and bad interactions should show up in the data. However, if you want to receive this advantage, you have to run many times the traffic through your test as you would with a Taguchi test. You also can't run as many combinations. Google's website optimizer is a full factorial tool.
The Random independent rotators are a third option. These treat factors independently. So you can just plug in a new factor with as many options as you want at any time, and ignore the fact that other factors are being tested at the same time. This is the most flexible form of testing, but it is also plagued with the most interaction effects.
Now that's just the overview. There are many other features to compare and questions to ask. Does a tester allow you to exclude bots and spiders? Does it allow real time access to data? What kind of reports does it show? What kind of information does it track for each visitor, and is the visitor-level data available for viewing. Etc.
You've probably heard that split testing elements of your website can improve your conversion rates. And that's true. But there are many kinds of split testing out there. And there are many ways to set up split tests on your website.
Let me give you an overview, so you can understand the choices you have before you. I'll walk you through the main distinctions that rule the field.
One big distinction is between single variable and multiple variable (multivariate) testing.
With single variable testing you just pick one element on your page to test. Take your headline, for example. If your current headline says "Are you having trouble with tooth decay?", you might want to see how "Do your kids have tooth decay?" works. In a single variable test, you just find a way to rotate each of those headlines on your page, and keep track of how many sales each generates.
Google offers a single variable split testing tool, but you have to have an Adwords account to use it.
Single variable testing can be slow. How would you like to test 10 times as much with the same amount of traffic? Well, you can. You use a method called multi-variable, or multivariate testing. This allows you to test several elemens on your page simultaneously.
Among multiple variable software solutions, there are hosted plans and self-install plans. If you want to install software on your server (which can be faster and cheaper) you will have more control over your data. And you can usually find someone to install it for you (often the vendor provides this service for a nominal fee). If you want to go with a hosted plan expect to pay more.
Hosted solutions include Google's Website Optimizer, and Vertster. Installed solutions include the Split Test Accelerator and Kaizen Track.
The next distinction probably causes the most confusion among people looking into split testing for the first time. It's also more important than the last distinction, becuase it determines how fast and accurate your tests can be. The three kinds of testers are 1) random independent rotators, 2) Taguchi testers, and 3) full-factorial testers. Taguchi ranks first for speed and second for accuracy. Basically Taguchi testing involves using a marvel of mathematics known as "orthogonal arrays". These have special properties that allow you to test every combination in a space while running only a small fraction of the actual combinations. You have to be a little bit careful about interaction effects with Taguchi testing. The Split Test Accelerator and Kaizentrack are Taguchi testing tools.
Full factorial tools allow you to cycle through all the combinations for your factors. So if you have 3 headlines, 3 offers, 3 P.S. statements, and 3 images you're testing, you will cycle through 81 pages. One advantage of full factorial testing is that you don't have to worry as much about interaction effects. Good and bad interactions should show up in the data. However, if you want to receive this advantage, you have to run many times the traffic through your test as you would with a Taguchi test. You also can't run as many combinations. Google's website optimizer is a full factorial tool.
The Random independent rotators are a third option. These treat factors independently. So you can just plug in a new factor with as many options as you want at any time, and ignore the fact that other factors are being tested at the same time. This is the most flexible form of testing, but it is also plagued with the most interaction effects.
Now that's just the overview. There are many other features to compare and questions to ask. Does a tester allow you to exclude bots and spiders? Does it allow real time access to data? What kind of reports does it show? What kind of information does it track for each visitor, and is the visitor-level data available for viewing. Etc.
