General: Potentially Confounded Button Color A/B Test
Hypothesis
If we test a similar change on our any pages as rejected, we should be cautious
Test Results
Key Learning
Problem: The primary call-to-action on the general isn't converting at its potential — design, copy, or placement may be the bottleneck.
What was tried: rejected this UI change (Jun 26, 2024). Rejection suggests the change underperformed the control
Why it failed: Not every CTA change improves conversion. Users may have preferred the original because it was clearer, more familiar, or better positioned.
How to Apply This to Your Site
This test showed that general: potentially confounded button color a/b test hurt conversions. The change was tested on a landing page page in the e-commerce industry. Avoid replicating this exact approach — instead, consider testing the opposite direction or a more subtle variation.
Before you test: Consider that cta tests typically require adequate traffic to reach statistical significance. Run your test for at least 2 full business cycles to account for weekly traffic patterns.
What Was Tested
Runs A Potentially Confounded Button Color A/B Test. This leak shows what tested on their global pages.
Methodology
Build On These Learnings
Save your own experiments, spot winning patterns across your test history, and stop repeating what's already been tried.
Related Experiments
Listing: Visible Payment Options
Context: The primary call-to-action on the listing isn't converting at its potential — design, copy, or placement may be the bottleneck.
Product: Single Or Alternative Buttons
Context: The primary call-to-action on the product isn't converting at its potential — design, copy, or placement may be the bottleneck.
Listing: Filled Or Ghost Buttons
Context: The primary call-to-action on the listing isn't converting at its potential — design, copy, or placement may be the bottleneck.
Checkout: Sticky Call To Action
Problem: Key actions on the checkout disappear as users scroll, creating a gap between intent and the ability to act.