Homepage: Mobile Homepage
Test Results
Key Learning
Context: Mobile users experience the homepage differently — smaller screens, touch targets, and limited attention require purpose-built design.
What was tested: A variation was tested against the existing experience.
Result: No statistically significant difference was detected. Inconclusive copy tests usually mean both versions are equally (in)effective at addressing user motivations. Try a fundamentally different angle.
How to Apply This to Your Site
This experiment tested homepage: mobile homepage but produced no statistically significant change. The test was run on a homepage page in the energy & utilities industry. Inconclusive results suggest this particular change may not be a priority — focus testing effort on higher-impact areas.
Before you test: Consider that copy & messaging tests typically require large sample sizes to detect small effects. This test ran for 39 days — plan for at least that long.
This result reached 95% statistical confidence, meaning there is a very low probability the observed effect was due to chance. Results at this confidence level are generally considered reliable for making business decisions.
What Was Tested
A/B test on homepage testing copy & messaging changes.
Methodology
Build On These Learnings
Save your own experiments, spot winning patterns across your test history, and stop repeating what's already been tried.
Related Experiments
Content Page: Inline Link Nudge
Context: Form input design on the content page affects completion rates — label placement, validation timing, and field clarity all matter.
Does a 90-Day Plan-Change Guarantee Badge Increase Click-Throughs?
Ambiguity > absence. A vague benefit callout can create more friction than no callout at all: visitor diagnostics showed users were drawn in by the badge (time-on-page up, bounce rate down) but exit rate rose and FAQ-section attractiveness spiked — a signature of users searching for answers and not finding them. The same concept won at a sister brand whose variant used descriptive benefit-framed copy ("we'll help you find the right plan if this isn't a fit"); the variant in this test used short labelled-badge copy that raised more questions than it answered. The lesson is not that benefit guarantees fail — it's that surfacing one with insufficient context can backfire by introducing uncertainty the page doesn't resolve.
Home landing: Empowering Headline
Context: The headline on the home landing may not resonate with what users actually care about or address their top objections.
Product: More For Less Headline
Context: The headline on the product may not resonate with what users actually care about or address their top objections.