Listing: Long Titles
Hypothesis
If we implement 'Long Titles' on listing pages (In 2012 a Microsoft employee working on Bing had an idea about changing the way the search engine displayed ad headlines), then key conversion metrics will improve.
Test Results
Key Learning
Context: The headline on the listing may not resonate with what users actually care about or address their top objections.
What was tested: REAL-WORLD TEST: 'Long Titles' was tested on a live listing page. The test involved 68,822 real visitors. Full statistical results require paid access. Test methodology: In 2012 a Microsoft employee working on Bing had an idea about changing the way the search engine displayed ad headlines. Developing it wouldn’t requi...
Result: No statistically significant difference was detected. Inconclusive copy tests usually mean both versions are equally (in)effective at addressing user motivations. Try a fundamentally different angle.
How to Apply This to Your Site
This experiment tested listing: long titles but produced no statistically significant change. The test was run on a category page page in the cross-industry industry. Inconclusive results suggest this particular change may not be a priority — focus testing effort on higher-impact areas.
Before you test: Consider that copy & messaging tests typically require large sample sizes to detect small effects. Run your test for at least 2 full business cycles to account for weekly traffic patterns.
What Was Tested
In 2012 a Microsoft employee working on Bing had an idea about changing the way the search engine displayed ad headlines. Developing it wouldn’t require much effort—just a few days of an engineer’s time—but it was one of hundreds of ideas proposed, and the program managers deemed it a low priority. So it languished for more than six months, until an engineer, who saw that the cost of writing the code for it would be small, launched a simple online controlled experiment—an A/B test—to assess its impact. Within hours the new headline variation was producing abnormally high revenue, triggering a “too good to be true” alert. HBR, September–October 2017 Issue, https://hbr.org/2017/09/the-surprising-power-of-online-experiments Note : This experiment was a solid success and replicated multiple times over a period of months. It worked at Bing and had a profound influence. The only reason why we atributed a 0.25 point (a "Maybe") was because we don't have the exact sample size and conversion data.
Methodology
Build On These Learnings
Save your own experiments, spot winning patterns across your test history, and stop repeating what's already been tried.
Related Experiments
Product: More For Less Headline
Context: The headline on the product may not resonate with what users actually care about or address their top objections.
Content Page: Inline Link Nudge
Context: Form input design on the content page affects completion rates — label placement, validation timing, and field clarity all matter.
Home landing: Empowering Headline
Context: The headline on the home landing may not resonate with what users actually care about or address their top objections.
Product: Long Titles
Context: Multi-step processes on the product can overwhelm users if they can't see how far along they are or how much is left.