Predicting User Experience Success

A fascinating article in last Sunday’s New York Times documents a recent study in which it is shown that predicting the success of cultural products (such as movies or music) is impossible, and that a phenomenon called “cumultive advantage” — where people prefer something largely because other people already prefer it — will usually overcome any empirical qualitative preference individuals may have for one product over another.

Marketers, for all their reliance on research, have long suspected this, which is why for years they have been looking to “coolhunters” to help them locate emerging tidal waves of coolness while the “cumultive advantage” is still building up steam. Instead of trying to create new products that will succeed because they were designed to meet a known and measurable consumer demand, they try to emulate products that are ascendant and that reveal previously-unknown consumer preferences.

This phenomenon may seem perfectly reasonable when it comes to movies and music, but I think it’s also true for user interface design: To the extent that any given UI can be called a “cultural product”, it is vulnerable to the wild unpredictability of culture. We may not always recognize it, but almost every UI is a type of cultural product.

This might seem hard to accept. Obviously, Justin Timberlake and Star Wars are cultural products, but the iPod, too, is a cultural product. The Nintendo Wii is a cultural product. Windows Vista is a cultural product. Amazon.com is a cultural product. These products have particular timeliness, particular aesthetics, and particular creative voices — thus they are cultural.

All of these cultural products have pure usability components to their user experience, but the cultural component — the product’s style — is often a major factor in the product’s success or failure. Sometimes it is the predominant factor, outweighing usability and feature-richness, as I think is the case with the iPod.

The ability to predict the success or failure of a UI design before a product is released is the foundation for the entire careers of many of us in the user experience design profession, so this argument may be troubling to many of us who think that there are empirically right and wrong ways of designing a UI. It’s hard to accept that a product’s hot color scheme, seductive finish, or ornamental trimmings — not to mention the brand name, ad campaign, or celebrity spokesmodel — could be far more important to the product’s success than the product’s long feature list or elegant ease-of-use.

I see the Times article as further evidence that no matter how many tests we do to show that one UI convention is better than another, when it comes to cultural products the “it depends” option is so overwhelmingly dominant that no conclusive best practices can ever be stated with confidence. Until you actually build something and have people use it, you will never know. And until then, the product development team’s resident “coolhunter” may have better insights into the product’s potential for success than anyone on the user research team.