Energy and utilities experiments including plan selection, enrollment flows, and pricing presentation.
Across 110 energy & utilities experiments, 30% resulted in a statistically significant win. Winning variants saw an average lift of +9.8%. Meanwhile, 15 tests underperformed the control with an average drop of -8.3%.
62 experiments were inconclusive, meaning the difference between control and variant was not statistically significant. Inconclusive results are still valuable — they tell you what doesn't move the needle, so you can focus testing effort elsewhere.
These results come from real A/B tests with sample sizes ranging from hundreds to millions of visitors. Use them to inform your own energy & utilities testing strategy and avoid repeating experiments that have already been run.
Problem: Users on the landing page need validation from others before committing — without visible proof of success, they hesitate.
Context: Mobile users experience the mobile differently — smaller screens, touch targets, and limited attention require purpose-built design.
Context: How "Page updates" is implemented on the landing page can meaningfully affect conversion — this element is worth testing.
Context: Users can't quickly find relevant products or content on the landing page, leading to frustration and early exits.
Context: The information hierarchy on the homepage may not match how users actually scan and process the content.
Context: Friction during the checkout process causes users to abandon right when they're closest to converting.
Ambiguity > absence. A vague benefit callout can create more friction than no callout at all: visitor diagnostics showed users were drawn in by the badge (time-on-page up, bounce rate down) but exit rate rose and FAQ-section attractiveness spiked — a signature of users searching for answers and not finding them. The same concept won at a sister brand whose variant used descriptive benefit-framed copy ("we'll help you find the right plan if this isn't a fit"); the variant in this test used short labelled-badge copy that raised more questions than it answered. The lesson is not that benefit guarantees fail — it's that surfacing one with insufficient context can backfire by introducing uncertainty the page doesn't resolve.
Context: Friction during the checkout process causes users to abandon right when they're closest to converting.
Context: The information hierarchy on the landing page may not match how users actually scan and process the content.
Context: How "Oam - homepage" is implemented on the homepage can meaningfully affect conversion — this element is worth testing.
Context: The information hierarchy on the landing page may not match how users actually scan and process the content.
Sticky mobile CTAs can compress time-on-page meaningfully (~15% faster) without sacrificing engagement signals — users converted at a directionally higher rate AND moved through the page faster, suggesting reduced hesitation rather than rushed clicks. The result was shipped via 90/10 holdout monitoring rather than traditional 50/50 A/B inference — the high baseline (~85%) and limited mobile traffic made full A/B underpowered, so the team chose a holdout-validated rollout as the deliberate methodology. Bayesian P(variant > control) was ~0.90, supporting the directional ship call. Worth noting: external research flags sticky CTAs as context-dependent — they help when the primary action is buried below the fold, but can hurt on shorter pages where the original CTA is already visible.
Context: How "Desktop homepage" is implemented on the homepage can meaningfully affect conversion — this element is worth testing.
Context: How "Weekends on command 24 ppc" is implemented on the landing page can meaningfully affect conversion — this element is worth testing.
Context: Mobile users experience the homepage differently — smaller screens, touch targets, and limited attention require purpose-built design.
Context: How "Hwa homepage" is implemented on the homepage can meaningfully affect conversion — this element is worth testing.
Context: The information hierarchy on the landing page may not match how users actually scan and process the content.
Context: Mobile users experience the mobile differently — smaller screens, touch targets, and limited attention require purpose-built design.
Context: How "Oam - new homepage design" is implemented on the homepage can meaningfully affect conversion — this element is worth testing.
Context: The information hierarchy on the homepage may not match how users actually scan and process the content.
Context: The information hierarchy on the landing page may not match how users actually scan and process the content.
Context: How "Alberta homepage" is implemented on the homepage can meaningfully affect conversion — this element is worth testing.
Context: Mobile users experience the mobile differently — smaller screens, touch targets, and limited attention require purpose-built design.
Context: Friction during the checkout process causes users to abandon right when they're closest to converting.
Context: How "Alberta homepage v2" is implemented on the homepage can meaningfully affect conversion — this element is worth testing.
Context: Friction during the checkout process causes users to abandon right when they're closest to converting.
Context: Mobile users experience the checkout differently — smaller screens, touch targets, and limited attention require purpose-built design.
Context: Mobile users experience the homepage differently — smaller screens, touch targets, and limited attention require purpose-built design.
Context: The primary call-to-action on the landing page isn't converting at its potential — design, copy, or placement may be the bottleneck.
The story behind this win is the iteration discipline. The first attempt at this homepage redesign changed two systems at once (messaging + routing) and produced an ambiguous result: the entry metric moved slightly positive while downstream metrics moved meaningfully negative. The team correctly identified that the routing change — which inadvertently replaced direct links to a personalized plan-search experience with modal-driven entry into a generic flow — was the downstream killer. The iteration restored the original routing and kept ONLY the homepage hierarchy changes. All funnel metrics moved directionally positive in lockstep (entry +2.38%, mid-funnel +7%, conversion +11.81%) — none stat-sig individually but consistent enough across the funnel to justify shipping. Element-level diagnostics confirmed the mechanism: the segment CTAs the team intended to promote saw a 26-30% lift in unique-visitor interaction, while the unchanged hero banner stayed flat (as expected). Two key behavioral observations: (1) page-length reduction surfaced a 4x lift on a previously buried bottom-of-page zip code input — proving the secondary lesson that 'less page' can mean 'more conversion real estate'; (2) desktop strongly outperformed mobile, with the suspected cause being mobile's lead-with-form pattern (zip code above hero) — putting the form before the message creates friction. The broader transferable insight: when a messy test confounds multiple variables, the right move is to isolate one variable in the next test, not to abandon the hypothesis.
Save your own experiments, get AI-powered test ideas, and build on patterns from 110+ real tests.
View Plans & Pricing