The Paradise of Choice?

A while ago, I wrote a post about the Paradox of Choice based on a talk by Barry Schwartz, the author of a book by the same name. The basic argument Schwartz makes is that choice is a double-edged sword. Choice is a good thing, but too much choice can have negative consequences, usually in the form of some kind of paralysis (where there are so many choices that you simply avoid the decision) and consumer remorse (elevated expectations, anticipated regret, etc…). The observations made by Schwartz struck me as being quite astute, and I’ve been keenly aware of situations where I find myself confronted with a paradox of choice ever since. Indeed, just knowing and recognizing these situations seems to help deal with the negative aspects of having too many choices available.

This past summer, I read Chris Anderson’s book, The Long Tail, and I was a little pleasantly surprised to see a chapter in his book titled “The Paradise of Choice.” In that chapter, Anderson explicitely addresses Schwartz’s book. However, while I liked Anderson’s book and generally agreed with his basic points, I think his dismissal of the Paradox of Choice is off target. Part of the problem, I think, is that Anderson is much more concerned with the choices rather than the consequences of those choices (which is what Schwartz focuses on). It’s a little difficult to tell though, as Anderson only dedicates 7 pages or so to the topic. As such, his arguments don’t really eviscerate Schwartz’s work. There are some good points though, so let’s take a closer look.

Anderson starts with a summary of Schwartz’s main concepts, and points to some of Schwartz’s conclusions (from page 171 in my edition):

As the number of choices keeps growing, negative aspects of having a multitude of options begin to appear. As the number of choices grows further, the negatives escalate until we become overloaded. At this point, choice no longer liberates, but debilitates. It might even be said to tyrannize.

Now, the way Anderson presents this is a bit out of context, but we’ll get to that in a moment. Anderson continues and then responds to some of these points (again, page 171):

As an antidote to this poison of our modern age, Schwartz recommends that consumers “satisfice,” in the jargon of social science, not “maximize”. In other words, they’d be happier if they just settled for what was in front of them rather than obsessing over whether something else might be even better. …

I’m skeptical. The alternative to letting people choose is choosing for them. The lessons of a century of retail science (along with the history of Soviet department stores) are that this is not what most consumers want.

Anderson has completely missed the point here. Later in the chapter, he spends a lot of time establishing that people do, in fact, like choice. And he’s right. My problem is twofold: First, Schwartz never denies that choice is a good thing, and second, he never advocates removing choice in the first place. Yes, people love choice, the more the better. However, Schwartz found that even though people preferred more options, they weren’t necessarily happier because of it. That’s why it’s called the paradox of choice – people obviously prefer something that ends up having negative consequences. Schwartz’s book isn’t some sort of crusade against choice. Indeed, it’s more of a guide for how to cope with being given too many choices. Take “satisficing.” As Tom Slee notes in a critique of this chapter, Anderson misstates Schwartz’s definition of the term. He makes it seem like satisficing is settling for something you might not want, but Schwartz’s definition is much different:

To satisfice is to settle for something that is good enough and not worry about the possibility that there might be something better. A satisficer has criteria and standards. She searches until she finds an item that meets those standards, and at that point, she stops.

Settling for something that is good enough to meet your needs is quite different than just settling for what’s in front of you. Again, I’m not sure Anderson is really arguing against Schwartz. Indeed, Anderson even acknowledges part of the problem, though he again misstate’s Schwartz’s arguments:

Vast choice is not always an unalloyed good, of course. It too often forces us to ask, “Well, what do I want?” and introspection doesn’t come naturally to all. But the solution is not to limit choice, but to order it so it isn’t oppressive.

Personally, I don’t think the problem is that introspection doesn’t come naturally to some people (though that could be part of it), it’s more that some people just don’t give a crap about certain things and don’t want to spend time figuring it out. In Schwartz’s talk, he gave an example about going to the Gap to buy a pair of jeans. Of course, the Gap offers a wide variety of jeans (as of right now: Standard Fit, Loose Fit, Boot Fit, Easy Fit, Morrison Slim Fit, Low Rise Fit, Toland Fit, Hayes Fit, Relaxed Fit, Baggy Fit, Carpenter Fit). The clerk asked him what he wanted, and he said “I just want a pair of jeans!”

The second part of Anderson’s statement is interesting though. Aside from again misstating Schwartz’s argument (he does not advocate limiting choice!), the observation that the way a choice is presented is important is interesting. Yes, the Gap has a wide variety of jean styles, but look at their website again. At the top of the page is a little guide to what each of the styles means. For the most part, it’s helpful, and I think that’s what Anderson is getting at. Too much choice can be oppressive, but if you have the right guide, you can get the best of both worlds. The only problem is that finding the right guide is not as easy as it sounds. The jean style guide at Gap is neat and helpful, but you do have to click through a bunch of stuff and read it. This is easier than going to a store and trying all the varieties on, but it’s still a pain for someone who just wants a pair of jeans dammit.

Anderson spends some time fleshing out these guides to making choices, noting the differences between offline and online retailers:

In a bricks-and-mortar store, products sit on the shelf where they have been placed. If a consumer doesn’t know what he or she wants, the only guide is whatever marketing material may be printed on the package, and the rough assumption that the product offered in the greatest volume is probably the most popular.

Online, however, the consumer has a lot more help. There are a nearly infinite number of techniques to tap the latent information in a marketplace and make that selection process easier. You can sort by price, by ratings, by date, and by genre. You can read customer reviews. You can compare prices across products and, if you want, head off to Google to find out as much about the product as you can imagine. Recommendations suggest products that ‘people like you’ have been buying, and surprisingly enough, they’re often on-target. Even if you know nothing about the category, ranking best-sellers will reveal the most popular choice, which both makes selection easier and also tends to minimize post-sale regret. …

… The paradox of choice is simply and artifact of the limitations of the physical world, where the information necessary to make an informed choice is lost.

I think it’s a very good point he’s making, though I think he’s a bit too optimistic about how effective these guides to buying really are. For one thing, there are times when a choice isn’t clear, even if you do have a guide. Also, while I think retailers that offer Recommendations based on what other customer purchases are important and helpful, who among us hasn’t seen absurd recommendations? From my personal experience, a lot of people don’t like the connotations of recommendations either (how do they know so much about me? etc…). Personally, I really like recommendations, but I’m a geek and I like to figure out why they’re offering me what they are (Amazon actually tells you why something is recommended, which is really neat). In any case, from my own personal anecdotal observations, no one puts much faith in probablistic systems like recommendations or ratings (for a number of reasons, such as cheating or distrust). There’s nothing wrong with that, and that’s part of why such systems are effective. Ironically, acknowledging their imperfections allow users to better utilize the systems. Anderson knows this, but I think he’s still a bit too optimistic about our tools for traversing the long tail. Personally, I think they need a lot of work.

When I was younger, one of the big problems in computing was storage. Computers are the perfect data gatering tool, but you need somewhere to store all that data. In the 1980s and early 1990s, computers and networks were significantly limited by hardware, particularly storage. By the late 1990s, Moore’s law had eroded this deficiency significantly, and today, the problem of storage is largely solved. You can buy a terrabyte of storage for just a couple hundred dollars. However, as I’m fond of saying, we don’t so much solve problems as trade one set of problems for another. Now that we have the ability to store all this information, how do we get at it in a meaninful way? When hardware was limited, analysis was easy enough. Now, though, you have so much data available that the simple analyses of the past don’t cut it anymore. We’re capturing all this new information, but are we really using it to its full potential?

I recently caught up with Malcolm Gladwell’s article on the Enron collapse. The really crazy thing about Enron was that they didn’t really hide what they were doing. They fully acknowledged and disclosed what they were doing… there was just so much complexity to their operations that no one really recognized the issues. They were “caught” because someone had the persistence to dig through all the public documentation that Enron had provided. Gladwell goes into a lot of detail, but here are a few excerpts:

Enron’s downfall has been documented so extensively that it is easy to overlook how peculiar it was. Compare Enron, for instance, with Watergate, the prototypical scandal of the nineteen-seventies. To expose the White House coverup, Bob Woodward and Carl Bernstein used a source-Deep Throat-who had access to many secrets, and whose identity had to be concealed. He warned Woodward and Bernstein that their phones might be tapped. When Woodward wanted to meet with Deep Throat, he would move a flower pot with a red flag in it to the back of his apartment balcony. That evening, he would leave by the back stairs, take multiple taxis to make sure he wasn’t being followed, and meet his source in an underground parking garage at 2 A.M. …

Did Jonathan Weil have a Deep Throat? Not really. He had a friend in the investment-management business with some suspicions about energy-trading companies like Enron, but the friend wasn’t an insider. Nor did Weil’s source direct him to files detailing the clandestine activities of the company. He just told Weil to read a series of public documents that had been prepared and distributed by Enron itself. Woodward met with his secret source in an underground parking garage in the hours before dawn. Weil called up an accounting expert at Michigan State.

When Weil had finished his reporting, he called Enron for comment. “They had their chief accounting officer and six or seven people fly up to Dallas,” Weil says. They met in a conference room at the Journal’s offices. The Enron officials acknowledged that the money they said they earned was virtually all money that they hoped to earn. Weil and the Enron officials then had a long conversation about how certain Enron was about its estimates of future earnings. …

Of all the moments in the Enron unravelling, this meeting is surely the strangest. The prosecutor in the Enron case told the jury to send Jeffrey Skilling to prison because Enron had hidden the truth: You’re “entitled to be told what the financial condition of the company is,” the prosecutor had said. But what truth was Enron hiding here? Everything Weil learned for his Enron expose came from Enron, and when he wanted to confirm his numbers the company’s executives got on a plane and sat down with him in a conference room in Dallas.

Again, there’s a lot more detail in Gladwell’s article. Just how complicated was the public documentation that Enron had released? Gladwell gives some examples, including this one:

Enron’s S.P.E.s were, by any measure, evidence of extraordinary recklessness and incompetence. But you can’t blame Enron for covering up the existence of its side deals. It didn’t; it disclosed them. The argument against the company, then, is more accurately that it didn’t tell its investors enough about its S.P.E.s. But what is enough? Enron had some three thousand S.P.E.s, and the paperwork for each one probably ran in excess of a thousand pages. It scarcely would have helped investors if Enron had made all three million pages public. What about an edited version of each deal? Steven Schwarcz, a professor at Duke Law School, recently examined a random sample of twenty S.P.E. disclosure statements from various corporations-that is, summaries of the deals put together for interested parties-and found that on average they ran to forty single-spaced pages. So a summary of Enron’s S.P.E.s would have come to a hundred and twenty thousand single-spaced pages. What about a summary of all those summaries? That’s what the bankruptcy examiner in the Enron case put together, and it took up a thousand pages. Well, then, what about a summary of the summary of the summaries? That’s what the Powers Committee put together. The committee looked only at the “substance of the most significant transactions,” and its accounting still ran to two hundred numbingly complicated pages and, as Schwarcz points out, that was “with the benefit of hindsight and with the assistance of some of the finest legal talent in the nation.”

Again, Gladwell’s article has a lot of other details and is a fascinating read. What interested me the most, though, was the problem created by so much data. That much information is useless if you can’t sift through it quickly or effectively enough. Bringing this back to the paradise of choice, the current systems we have for making such decisions are better than ever, but still require a lot of improvement. Anderson is mostly talking about simple consumer products, so none are really as complicated as the Enron case, but even then, there are still a lot of problems. If we’re really going to overcome the paradox of choice, we need better information analysis tools to help guide us. That said, Anderson’s general point still holds:

More choice really is better. But now we know that variety alone is not enough; we also need information about that variety and what other consumers before us have done with the same choices. … The paradox of choice turned out to be more about the poverty of help in making that choice than a rejection of plenty. Order it wrong and choice is oppressive; order it right and it’s liberating.

Personally, while the help in making choices has improved, there’s still a long way to go before we can really tackle the paradox of choice (though, again, just knowing about the paradox of choice seems to do wonders in coping with it).

As a side note, I wonder if the video game playing generations are better at dealing with too much choice – video games are all about decisions, so I wonder if folks who grew up working on their decision making apparatus are more comfortable with being deluged by choice.