For a lot of readers of this blog AB testing seems like a natural, even obvious way to encourage growth of any product.
What do the numbers say though, is iGaming ahead of the curve in CRO adoption or falling behind?
In this research article we’ve decided to analyse a big sample of gambling operators and see for ourselves how many of them invest in CRO and what are the most popular AB testing tools.
Out of 2283 gambling operators we’ve analysed, only 301 are actively running AB tests to improve conversion rate of their product.
We’ve started by gathering a list of 2283 online casinos and sports betting sites, not an easy task as it may seem but with a bit of coding magic our bots scoured the web for most popular operators.
When the list was groomed and manually checked we were ready to start the analysis. We’ve partnered up with Australian business intelligence powerhouse BuiltWith and proceeded to extract all that juicy information about technologies used by each operator. 1
After tagging the results and double checking the data we’ve started seeing some preliminary results – only 13.2% of operators had AB tools present on their site.
Chicken or the egg – AB testing by website traffic
Next, we’ve wanted to see if the size of the operator would make any difference as to whether they invest in Conversion Rate Optimization or not.
To estimate the size of the operator site we’ve used Alexa rankings. Alexa (now owned by Amazon) is one of the oldest and most trusted traffic intelligence services in the world. 2
Alexa holds a monthly ranking of most popular websites in the world. The higher you are in Alexa ranking the more traffic you get.
We’ve divided our list of operators into 4 buckets based on alexa ranking – top 10k, 10k to 100k, 100k to 1M and sites ranking lower than top 1 million.
The results? Not so surprising.
Over 60% of operators that were placed in the top 10k of the ranking do AB testing.
Slightly lower, 43% of operators in top 100k invest in CRO. The lower we go in the ranking the lower the numbers.
The big question is: Did those operators get to the top because they were doing AB testing or do they do AB testing because they are on the top and want to stay there?
The answer?
Honestly, it doesn’t matter.
The truth is – if you’re in the top and you’re not investing in CRO you’re lagging behind most of your biggest competitors.
If you’re a contender from lower parts of the ranking trying to get into the top spot, investing in CRO is clearly a lesson you should learn from business that are ahead of you.
One way or another, if you want to get to the top or stay there, based on our data, casino & sports betting Conversion Rate Optimization seems to be a highly contributing factor.
What’s the most popular AB testing tool?
Google Optimize and Optimize 360 by far lead the way in our rankings – more than half of companies chose them over a myriad of competitors.
Why Google Optimize?
Before we dive deeper into analyzing features of Google Optimize it’s important to note that Google currently offers two versions of it’s AB testing tool.
#1 Used by 57.48% of operators
Google Optimize
Regular Google Optimize is free to use just like other tools from Google Webmaster suite like Google Analytics. Google Optimize 360 however is a premium tool coming with a pretty hefty price tag ranging up to $150,000 per year. 3
At the time of writing this articles it’s not possible for us to detect if a website is using the free or enterprise version of Google Optimize. Our educated guess however is that majority of operators is sticking to the free version.
Google Optimize Features
If you’re already using Google Analytics for tracking your visitors launching experiments with Google Optimize will be a breeze since it natively integrates with GA. An earlier version of of Google Optimize called ‘Content Experiments’ used to be a part of Google Analytics.
If you’re already familiar with other tools from big G like Google Tag Manager, navigating Optimize will be very intuitive as it uses a lot of similar functionalities.
You will also get a myriad of features you can find in most of the popular ab testing tools like:
- A/B, A/B/n, multivariate and split url tests
- Visual editor
- 3rd party data integration
- Granular user targeting options
- User roles and permissions
- Experiment scheduling
The free version comes with limitations though. You will be limited to run just 5 experiments at a time. It’s fine when you’re just getting started with testing or your website is getting minimal amounts of traffic. If you’re growing rapidly though it will be a big roadblock on your way to increase testing velocity.
So what do I get for that additional $150k?
Enterprise level scalability.
The biggest advantage of Google Optimize 360 is it’s scalability. You will be able to run up to 100 simultaneous test, have MVT test with up to 36 combinations (as if you would ever need that many) and have up to 10 preconfigured test objectives.
Additionally you will get premium customer support which we all know is basically non-existent in the free version. You will also be able to target experiments to your Analytics audiences and gain access to personalization features.
Overview of other AB testing tools
#2 (ex-æquo) Used by 12.62% of operators
Just like Optimize is part of Google Marketing Platform, Maxymiser is a part of Marketing Cloud from another powerhouse – Oracle.
Maxymiser seamlessly integrates with DMP, CRM and web analytics included in the Oracle Marketing Cloud. If you’re already using tools from Oracle suite Maxymiser might save you the hussle of tedious integrations.
Oracle Maxymiser provides all the features you could normally expect from an AB testing platform like A/B, A/B/n and multivariate testing as well as personalization engine, visual editor, cross-device campaigns as well as powerful reporting engine.
#2 (ex-æquo) Used by 12.62% of operators
Visual Website Optimizer
Visual Website Optimizer grew throughout the years from a simple AB testing tool to a whole suite of products offering to support your work in 6 main areas of growth hacking.
- VWO Testing – AB testing platform
- VWO Fullstack – server side and mobile app testing platform
- VWO Insights – heatmaps, visit recordings and surveys
- VWO Engage – push notifications and facebook messaging
- VWO Plan – growth marketing roadmap management
- VWO Deploy – visual website editor for non-techy people
It’s a great tool if you’re just starting your adventure with CRO and have no experience with any analytics, marketing or AB testing tools.
#4 Used by 12.29% of operators
Optimizely
Optimizely is a serious player in the AB testing market offering one of the most advanced platforms for advanced CRO programs.
It supports testing of websites, mobile apps, TV apps as well as IoT based apps. With such a an extensive list of testing capabilities optimizely is surprisingly easy to start your AB testing adventure with. If you have any problems though you can be sure to find answer in their extensive knowledge base, excellent technical documentation or ask a question in their user community.
When you’ve already made your first steps Optimizely will allow you to plug in additional data sources and deeply integrate with your platform all while meeting highest security standards and performing on the highest level.
#5 Used by 7.31% of operators
Dynamic Yield
Dynamic Yield is first and foremost a personalization platform but it offers full AB testing capabilities so it was included in this toplist.
On the personalization side of things Dynamic Yield offers product and content recommendation engine, behavioral messaging, triggering engine and a whole Customer Data Management platform which should be able to satisfy most of your personalization needs.
In it’s AB testing arsenal you will find the regular capabilities of AB, multivariate and split URL testing as well as advanced features like multi-touch campaigns, predictive targeting and behavioral targeting.
If personalization is a prominent feature on your growth roadmap than dynamic yield will be a good choice for you.
#6 Used by 2.66% of operators
AB Tasty
AB Tasty is an AB testing platform with additional capabilities of personalization, analytics and a database of premade widgets based on popular conversion principles like adding scarcity, social proof or urgency.
Like almost every tool in this list AB Tasty allows you to run A/B/n, split url, MVT , funnel tests as well as server side testing. Their personalisation engine uses AI to automatically segment your users and allows you to trigger events based on real time behaviours, attributes or interests.
Overall AB Tasty can provide you with good AB testing and personalization platform and let you start experimenting literally from day one since you will gain access to their library of pre-made conversion optimization widgets.
#7(ex-æquo) Used by 1.66% of operators
Adobe Marketing Cloud (Omniture)
Adobe Marketing Cloud is being used only by 1.51% of the websites we’ve tested which is not a big surprise considering the price tag it comes with. Marketing Cloud doesn’t publish it prices publically and they can vary greatly depending on what service you’re looking for but the minimum spend you will be looking at is approximately $50,000 which can go into millions of dollars if you’re looking for an enterprise solution and want to go all in with Adobe tools. 4
Adobe AB testing platform offers AB, MVT, split url and mobile app testing. Platform allows you to use advanced visitor behaviour to target specific segments. One of the innovations differentiates Adobe from other tools on this list is auto-allocate feature which uses AI to automatically assign appropriate traffic volume to each variation to avoid any losses while running the test.
#7(ex-æquo) Used by 1.66% of operators
Kameleoon
Kameleoon promotes itself as ‘AI-driven personalization and A/B testing platform for conversion-first digital product owners and marketers‘.
On the AB testing side Kameleoon offers a pretty standard package of AB. MVT, split url and server-side testing capabilities in conjunction with their advanced visual editor and great targeting options.
Personalization features of Kameleoon allow you to gather and segment all the data about your visitors and create personalized experiences using a visual editor. You can choose to manually segment your users or leave the task to an AI engine that will deliver right experience to right people at the right device.
#9(ex-æquo) Used by 0.33% of operators
Qubit Deliver
Qubit is a tool similar to similar to already reviewed Dynamic Yield and Kameleoon in a sense that it’s first and foremost personalization engine with built-in AB testing capabilities and a couple pre-made features like social-proof and product recommendations.
Qubit allows you to segment your users by any data available in your system as well as by strategically placing customer surveys on your website. You will be able to deliver coherent experiences across the whole length of the customer journey thanks to qubit omnichannel and omni device capabilities.
AB testing platform is not a prominent part of the suite but it looks like Qubit is taking it’s tests statistical significance and reducing the risk of false-positives seriously. Their statistical engine, smart goals, revenue impact tracking and segmentation tools are all geared towards getting you the most accurate and significant data possible.
#9(ex-æquo)Used by 0.33% of operators
Freshworks (Zarget)
It’s important to say that when we say ‘freshworks’ we actually mean ‘freshmarketer’ line of services and tools. Whole freshworks group offers 12 different types of tools ranging from cloud-based live chat and call center solutions to agile release management tools. One blogpost would not be enough to summarize the breadth of tools and services under the umbrella of freshworks so if you’re interested to see what else there is on offer simply head to their website.
Freshmarketer offers you usual AB testing capabilities of AB and split url testing as well as heatmaps, user recordings, form and funnel analytics, polls and feedback widgets, personalization and even email marketing. If you’re looking for a total 360 solution freshworks might be able to provide you with all the tools you need. Can one company be great at everything though?
#9(ex-æquo)Used by 0.33% of operators
Evolv (Sentient Ascend)
Evolv wants you to cut down your testing time by employing machine learning to test your whole funnel as a coherent omnichannel experience instead of focusing on smaller tests one step at a time.
While evolv sales pitch sounds impressive and numbers of different design options tested enormous, similar results can usually be achieved with most of the other tools offering multivariate testing. Addition of continuous optimization capabilities and use of machine learning to automatically stop underperforming variations however is definitely a step into the future and might prove useful if right conditions are met.
#9(ex-æquo)Used by 0.33% of operators
Symplify Conversion (SiteGainer)
Symplify has 2 suites of tools in their arsenal, communication and conversion.
Communication suite allows you to connect all your communication methods like email, push, sms, socialm phone and even print in one easy to navigate dashboard.
Conversion suite previously known as SiteGainer offers AB, multivariate and split url testing as well as personalization based on segments, heatmaps and visitor recordings as well as popup and survey creation wizards.
What’s the verdict?
Let’s be honest – 13.2% of gambling operators doing AB testing might not sound like an astounding number but we believe it definitely looks promising.
Much higher number of operators with high traffic working on their CRO programs is no surprise – the higher the traffic the easier to get to statistically significant results and the bigger desire to maintain your position on top.
Hopefully smaller aspiring operators will take these number to their hearts and start working on their CRO programs soon enough so that the results of our 2021 report look a lot better.