Skip to main content

The A/B Test Library

Real experiments. Real outcomes. Actionable patterns. Browse A/B tests with problem-to-solution framing, results, and recommendations for what to test next.

33 experiments
Winners, losers & inconclusive
Full statistical details
winner

Does Pinning a Mobile Checkout CTA Improve Conversion?

Sticky mobile CTAs can compress time-on-page meaningfully (~15% faster) without sacrificing engagement signals — users converted at a directionally higher rate AND moved through the page faster, suggesting reduced hesitation rather than rushed clicks. The result was shipped via 90/10 holdout monitoring rather than traditional 50/50 A/B inference — the high baseline (~85%) and limited mobile traffic made full A/B underpowered, so the team chose a holdout-validated rollout as the deliberate methodology. Bayesian P(variant > control) was ~0.90, supporting the directional ship call. Worth noting: external research flags sticky CTAs as context-dependent — they help when the primary action is buried below the fold, but can hurt on shorter pages where the original CTA is already visible.

CTAEnergy & Utilities
winner+13.0%

Pricing Page: Removing 'Popular Plans' Tab, Showing All Plans

Problem: How prices are displayed on the pricing page directly influences perceived value and willingness to buy.

LayoutEnergy & Utilities
winner+24.0%

Pricing Page: Softer CTA Copy 'Select Plan' vs 'Order Now'

Problem: The primary call-to-action on the pricing page isn't converting at its potential — design, copy, or placement may be the bottleneck.

CTAEnergy & Utilities
winner

Restructuring Homepage Hierarchy to Surface Personalized Offers

The story behind this win is the iteration discipline. The first attempt at this homepage redesign changed two systems at once (messaging + routing) and produced an ambiguous result: the entry metric moved slightly positive while downstream metrics moved meaningfully negative. The team correctly identified that the routing change — which inadvertently replaced direct links to a personalized plan-search experience with modal-driven entry into a generic flow — was the downstream killer. The iteration restored the original routing and kept ONLY the homepage hierarchy changes. All funnel metrics moved directionally positive in lockstep (entry +2.38%, mid-funnel +7%, conversion +11.81%) — none stat-sig individually but consistent enough across the funnel to justify shipping. Element-level diagnostics confirmed the mechanism: the segment CTAs the team intended to promote saw a 26-30% lift in unique-visitor interaction, while the unchanged hero banner stayed flat (as expected). Two key behavioral observations: (1) page-length reduction surfaced a 4x lift on a previously buried bottom-of-page zip code input — proving the secondary lesson that 'less page' can mean 'more conversion real estate'; (2) desktop strongly outperformed mobile, with the suspected cause being mobile's lead-with-form pattern (zip code above hero) — putting the form before the message creates friction. The broader transferable insight: when a messy test confounds multiple variables, the right move is to isolate one variable in the next test, not to abandon the hypothesis.

LayoutEnergy & Utilities
winner+12.2%

Mobile: Mobile Super Nav Clarity

Problem: Users arriving at the mobile can't efficiently find what they're looking for, increasing bounce rates.

NavigationEnergy & Utilitiesn=203,935
winner+2.9%

Homepage: Homepage Redesign 2.0

Problem: The information hierarchy on the homepage may not match how users actually scan and process the content.

LayoutEnergy & Utilitiesn=80,217
winner+4.4%

Landing Page: Headline & Copy Test

Problem: The first screen of the landing page must immediately communicate value — if it doesn't, users bounce before scrolling.

Mobile UXEnergy & Utilitiesn=75,511
winner+2.2%

Landing Page: Star Ratings

Problem: Users on the landing page need validation from others before committing — without visible proof of success, they hesitate.

Social ProofEnergy & Utilitiesn=71,171
winner+5.6%

Landing Page: Plan Page Zip Modal CTA Copy

Problem: The primary call-to-action on the landing page isn't converting at its potential — design, copy, or placement may be the bottleneck.

CTAEnergy & Utilitiesn=50,404
winner+4.1%

Landing Page: Grid Page Testimonials

Problem: Users on the landing page need validation from others before committing — without visible proof of success, they hesitate.

Social ProofEnergy & Utilitiesn=50,156
winner-0.4%

Homepage: Homepage / Grid

Problem: The information hierarchy on the homepage may not match how users actually scan and process the content.

LayoutEnergy & Utilitiesn=46,132
winner+7.2%

Landing Page: Layout Optimization

Problem: The information hierarchy on the landing page may not match how users actually scan and process the content.

LayoutEnergy & Utilitiesn=39,352
winner+18.1%

Product Page: All-price-points-on-plan-cards

Problem: How prices are displayed on the product page directly influences perceived value and willingness to buy.

PricingEnergy & Utilitiesn=34,703
winner+5.3%

Checkout: Grid Page Progress Bar

Problem: The first screen of the checkout must immediately communicate value — if it doesn't, users bounce before scrolling.

FormEnergy & Utilitiesn=33,413
winner+21.0%

Homepage: Layout Optimization

Problem: The information hierarchy on the homepage may not match how users actually scan and process the content.

LayoutEnergy & Utilitiesn=31,032
winner+5.7%

Homepage: Homepage Hero Redesign

Problem: The first screen of the homepage must immediately communicate value — if it doesn't, users bounce before scrolling.

LayoutEnergy & Utilitiesn=30,538
winner+2.9%

Landing Page: Help Me Choose: Grid Page Entry

Problem: Users can't quickly find relevant products or content on the landing page, leading to frustration and early exits.

LayoutEnergy & Utilitiesn=28,680
winner+14.1%

Pricing Page: Pricing Prominency on Grid

Problem: Each additional form field adds friction to the pricing page, increasing the chance users abandon before completing their submission.

PricingEnergy & Utilitiesn=23,235
winner+9.8%

Landing Page: US North Address Bar

Problem: How "Us north address bar" is implemented on the landing page can meaningfully affect conversion — this element is worth testing.

LayoutEnergy & Utilitiesn=22,848
winner+6.8%

Landing Page: Layout Optimization

Problem: The information hierarchy on the landing page may not match how users actually scan and process the content.

LayoutEnergy & Utilitiesn=21,993

Stop Re-Running Failed Tests

Build on what's already been learned. Save your own experiments, surface winning patterns, and make every test count.

Browse Experiments By

Comparisons

Popular Combos

Explore More