Energy and utilities experiments including plan selection, enrollment flows, and pricing presentation.
This segment contains 5 energy & utilities experiment records. Winner, loser, mixed, and inconclusive are reported outcome labels. Check each evidence badge before treating a result as independently reviewable.
Use these records as hypothesis input, then match the audience, metric, treatment, and context before planning a local test. Reported lifts are not pooled because the underlying outcomes and designs may not be comparable.
Button engagement alone does not show whether the element helped conversion. The topline movement was small and uncertain, while the diagnostic path exposed a larger problem: only about six of every hundred button interactions reached the next funnel stage. The label appears to have attracted people looking for account access rather than shopping, and an added modal inserted another step before the destination. Some activity also shifted away from stronger existing paths. For global navigation changes, evaluate downstream completion by origin page and intent; raw click volume can reward the wrong behavior.
Sticky mobile CTAs can compress time-on-page meaningfully (~15% faster) without sacrificing engagement signals — users converted at a directionally higher rate AND moved through the page faster, suggesting reduced hesitation rather than rushed clicks. The result was shipped via 90/10 holdout monitoring rather than traditional 50/50 A/B inference — the high baseline (~85%) and limited mobile traffic made full A/B underpowered, so the team chose a holdout-validated rollout as the deliberate methodology. Bayesian P(variant > control) was ~0.90, supporting the directional ship call.
The intervention rearranged the presentation, but the diagnosed problem concerned comprehension. Earlier user feedback pointed to three obstacles: people could not estimate total cost, offer labels did not explain meaningful differences, and the page made simultaneous comparison difficult. Reordering the same unclear information left those obstacles intact. After reaching the planned sample, the observed movement was small and negative. The practical lesson is to make the test variable match the user problem: change the cost explanation when cost is unclear, improve differentiation when offers seem interchangeable, and change layout when navigation or scanning is the issue.
Making the guarantee more visible did not make it more persuasive. The behavioral indicators suggest the badge attracted attention and then created unanswered questions: visitors spent longer on the page, exits increased, and engagement shifted toward the FAQ. A comparable implementation used a fuller explanation, while this version relied on a short label. The practical lesson is to explain eligibility, timing, and fees beside the claim. A guarantee can increase uncertainty when its prominence grows faster than its supporting context.
The story behind this win is the iteration discipline. The first attempt at this homepage redesign changed two systems at once (messaging + routing) and produced an ambiguous result: the entry metric moved slightly positive while downstream metrics moved meaningfully negative. The team correctly identified that the routing change — which inadvertently replaced direct links to a personalized plan-search experience with modal-driven entry into a generic flow — was the downstream killer. The iteration restored the original routing and kept ONLY the homepage hierarchy changes. All funnel metrics moved directionally positive in lockstep (entry +2.38%, mid-funnel +7%, conversion +11.81%) — none stat-sig individually but consistent enough across the funnel to justify shipping. Element-level diagnostics confirmed the mechanism: the segment CTAs the team intended to promote saw a 26-30% lift in unique-visitor interaction, while the unchanged hero banner stayed flat (as expected). Two key behavioral observations: (1) page-length reduction surfaced a 4x lift on a previously buried bottom-of-page zip code input — proving the secondary lesson that 'less page' can mean 'more conversion real estate'; (2) desktop strongly outperformed mobile, with the suspected cause being mobile's lead-with-form pattern (zip code above hero) — putting the form before the message creates friction. The broader transferable insight: when a messy test confounds multiple variables, the right move is to isolate one variable in the next test, not to abandon the hypothesis.
Save your own plans, results, context, and limitations. Use these 5 reported records as hypothesis input with their evidence quality visible.
View Plans & Pricing