While an A/B test runs, WEMASY counts for each version how many visitors saw it and how many reached the goal. It compares every version with the original and tells you when the difference is reliable. When you know which version works better, you choose it as the winner and every visitor gets it.
Open the results
- Open A/B testing from the product switcher.
- Click the test in the list on the Overview page.
- Look under Results.
What the results show
- For each version: how many visitors reached the goal, for example "12 of 240 visitors reached the goal".
- The difference with the original, marked vs. original.
- How sure WEMASY is that the difference is real, as a percentage.
- Leading on the version that does best so far.
Until both versions have visitors, you see "No comparison yet: it needs visitors in both versions."
When is a result reliable?
A version wins when WEMASY is 95% sure and each version has at least about 30 visitors. The verdict at the top tells you where you stand:
- "About ... more visitors per version before a result is reliable": keep the test running.
- "No clear winner yet: the versions do about the same": the change makes little difference.
- "... wins, with 95% certainty": you can choose the winner.
Over time shows how each version does day by day, as Conversion rate or Visitors.
The split warning
If visitors do not split between the versions as you set, you see "The visitors do not split as set." This usually means a version breaks the page, redirects, or a cache keeps showing one version. Check the versions on your site before you trust the results.
Choose the winner
- Click Choose as winner on the best version.
- Confirm. The test stops and every visitor gets this version.
The test then shows "Finished: every visitor gets the winner." Nothing is counted any more.
Stop, restart or delete
- Stop: the test stops and nothing changes on your site.
- Start again: a stopped test starts a new run and counts from zero.
- Delete this test: removes the test and its results.
Frequently asked questions
How long should a test run?
Until the verdict says a version wins, or that the versions do about the same. Let it run for at least a full week, so weekdays and weekends both count.
Can I choose a winner before the result is reliable?
Yes, but the difference may be chance. Wait for the 95% verdict when you can.
Back to Creating an A/B test or Getting started with A/B Testing.