Search vs. Serendipity

One of the more interesting themes I heard at SXSW was the battle between search and serendipity and the ramifications of that for everything from our intellectual curiosity to the ability of new ideas to find an audience.

The battle manifests itself in so much of what we do online. Take reading the newspaper. When we read a physical newspaper, we often find things we weren’t looking for: a picture or a headline catches our eye as we flip through the paper and we wind up learning something new and unexpected.

But the current structure of online newspapers makes that unlikely. We only click through to read those 4 or 5 top headlines our homepage widget deems important along with the articles some self-selected news reader filter has brought us.

And it’s that promise: that technology can ensure we read and find only those things that interest us, that’s at the crux of the problem. Because what that paradigm ultimately does is allow us to limit ourselves to merely reconfirming things we already know. It doesn’t expose us to anything new and more importantly, it doesn’t allow us the serendipity of discovering something we didn’t know we were looking for.

As technology becomes more ubiquitous, this dilemma does too. Amazon recommends books and movies based on what we’ve read or watched. The iTunes Genius feature shows us songs just like the ones we already own. Websites show us headlines from media outlets based on our current interests.

Even something as seemingly innocuous as Facebook Connect only serves to reinforce this dilemma. Countless studies have shown that people tend to hold similar views to their friends and so a stream of recommendations from friends is bound to only give us more of the same.

The solution to search vs. serendipity isn’t an obvious one. Throwing in a “now for something completely different” recommendation is more likely to annoy than to entice: a random outlier is not serendipity. Technology is not yet at a point where it can predict those unexpected items that will appeal to us. And the very act of suggestion takes away the joy of discovery.

Yet at the same time, we want and need the benefits of search. We want to find thing quickly and easily and it would be foolish to pretend that we don't have explicit preferences as to the types of music, books, films, news or even food we prefer or that learning more about these subjects was somehow intellectually lazy.

The solution appears to be in the ability of user experience and design professionals to produce an interface that more closely resembles the literal experience of browsing while allowing us to find what we want when we want it. A "serendipity enabler," if you will. It’s something that community is already hard at work on, and I’ve seen some interesting starts, such as this prototype for The New York Times.

Until then, however, serendipity will have to be the result of a proactive decision.

UPDATE (3/19) - Nicholas Kristof has a great column in the Times today that lays out much of the research behind the "search vs serendipity" claim.