If you are at the stage where you are split testing your web pages… Firstly, I congratulate you. Only the most successful online business owners are split testing.
Choosing which software to use can be a daunting decision. Since Google decided to drop Google Website Optimiser, the playing field has become much more abundant with third party tools.
There is of course Google Content Experiments (free inside Google Analytics) which has replaced Website Optimiser but it doesn’t have multivariate testing like Website Optimiser did. I find that to be a big downfall. I’m not 100% sure on the reason Google decided to drop Website Optimiser or multivariate testing but it has meant a lot of people have been searching elsewhere for a full-fledged split testing tool.
Here’s a quick rundown of the tools I have tried and my experience with them:
Google Content Experiments
This is pretty straight forward to use, although you will need some tech skills to modify your website pages. I find it best to use with static pages, not CMS sites. Essentially you create multiple variations of your pages on your web server, and Content Experiments will redirect users to each variation and then compile the results in a nice graph for you.
So, for example, you would setup index.php, indexb.php, indexc.php, indexd.php etc. on your webserver and specify those urls to Google. You can see an example of how I used Google Content Experiments here.
The drawbacks of Google Content Experiments are:
- It doesn’t do multivariate testing.
- You need to manually create duplicate versions of your pages.
- There’s no built-in editor.
The advantages are:
- It’s built into Google Analytics.
- The reporting is quite nice and simple.
- It will run a test for a e.g. 2 week time frame, and then tell you which variation is the winner.
- You don’t get issues with built-in editors. Sometimes manually creating duplicate pages can be less hassle.
Optimizely
This is quite a premium tool. When I started using this, I was impressed with their visual editor and how easy it was to use. There’s quite a lot of options in their editor which I haven’t seen in other software. They do offer multivariate testing but it’s not included in their basic package. It’s quite a big price jump from the basic package. $350 per month for the multivariate testing package.
This could almost be used as a metrics tool as they allow you to setup “Goals” such as clicks on various page elements, visits to specific pages, and also “Revenue Tracking” which means you can test your different variations and determine which version returns the most dollars for you. Very handy feature to have.
Another cool feature is it’s integration with Crazyegg. All you need is your API key from Crazyegg and Optimizely will do the rest. The great thing about this is that Optimizely will create separate Crazyegg snapshots for each of your split-test variations so you can accurately measure engagement on each test rather than guessing with one general snapshot.
Disadvantages:
- Pricey for the multi-variate package
Advantages:
- Great visual editor
- Easy to Use
- Integrates easily with Clicktale, and Crazyegg
- Integrates easily with Google Analytics (Goals)
- Revenue Tracking
Experiment.ly
I found this site when looking for a multi-variate alternative to Optimizely. Initially, this site looked great as they include multi-variate testing in all their packages and the prices are very reasonable. Another feature they have is heatmap tracking built-in. Awesome!
I ran into a snag when I started testing though. For some reason their visual editor didn’t display my page correctly. It looked as if it was missing a style sheet and wasn’t displaying some CSS3 elements so I couldn’t actually finish setting up the test I wanted to run. I hit another snag when I tried to upgrade my account. It gave me an API error their checkout. Didn’t make a great impression. I did submit support tickets and they did get back to me within 24 hours though.
They wanted me to disable some other javascript I had running on the page to fix the CSS elements, but by that time, I had already moved on to Visual Website Optimizer which worked flawlessly on first run so I haven’t bothered as of yet.
Disadvantages:
- Visual editor seems not quite up to par.
- Perhaps not quite as big a company as one would like.
Advantages:
- Cheap Pricing on packages
- Heat Map tracking built-in
- Easy to use interface
Visual Website Optimizer
As explained, I tried this software after having issues with Experiment.ly. Very easy to setup an account. Visual editor worked great. Does multi-variate tests in the free-trial. Packages seem quite reasonably priced. Includes Heat Map tracking as well.
I’m using this software right now, and running an ad campaign through Facebook. I’ll post more details to this blog post as we go.
The only snag I hit with this software, was when adding combinations to my multi-variate test. I have about 5 different sections on my page which I want to test and I got up to 640 possible combinations quite quickly which in turn made Visual Website Optimizer very slow to use.
Disadvantages:
- Site runs very slow when combinations increase
Advantages:
- Easy to use
- Heat Map Tracking
- Multivariate testing in Free Trial
- Reports seem quite intuitive so far
Hope you enjoyed this post. Please add your comments below and if you would like to get in contact with me, please hit the “Contact Me” button at the top of this page.
