Customer Insight · Customer-centric · Personalization

Living in an A/B world

It’s a common practice for e-commerce sites to engage in so-called “A/B testing.” A typical A/B test randomly presents a brand’s current website design against an alternative which might improve results (most often conversion rates).

A variant of this approach has been employed by direct mail practitioners for years. To improve campaign response rates a “challenger” mailing is pitted against the current best performing offer (“the champion”). Database analysis is then used to help evolve the marketing strategy.

For years, structured test and control experiments have been narrowly employed and, for the most part, the province of relatively sophisticated marketing organizations.

Yet in a world where the battle for share of attention is fierce, where relevance is increasingly hard to come by and where whoever gets closest to the customer wins, every organization, big and small, needs to embrace a test and learn mentality.

Sure, it’s desirable to have beautifully designed experiments and statistically relevant sample sizes. But don’t estimate the power of continually presenting your customers with a stream of alternatives and seeing how they react.  “Here’s A, here’s B, what do you think?” is sometimes all the protocol you need.

You don’t need Ph.D statisticians. You don’t need complex campaign management software. You don’t need expensive consultants.

What you do need is a willingness to try. And to fail. And to try again.

Leave a Reply