The Moon
A few years ago, The Onion put out a book called Our Dumb Century. It was comprised of a series of newspaper front pages, one from each year. It was an interesting book, in part because of the events they chose to represent each year and also because The Onion writers are hilarious. The most brilliant entry in the book was from the 1969 edition of the paper:


Utterly brilliant. You can't read it on that small copy, but there's a whole profanity-laden exchange between Houston and Tranquility Base that's also hysterically funny. As it turns out, The Onion folks went ahead and made a video, complete with archival footage and authentic sounding voices, beeps, static, etc... Incredibly funny. [video via Need Coffee]

Update: Weird, I tried to embed the video in this post, but when you click play it says it's no longer available... but if you go directly to youtube, you can get the video. I'm taking out the embedded video and putting in the link for now.



Link Dump: Space!
Time is short, so just a few space themed links for you:

• Space Station Movie Night: A while back, NASA released the International Space Station's daily logs. Most of the entries are rather dry and technical, but the astronauts sometimes logged what movies they were watching, and Scott David Herman decided to collect all of them in a post. Some highlights:

  24 NOV 2000: Watched disk 1 of "Apocalypse Now". Shep tried to explain why Robert Duvall is always wearing the black cavalry hat, but being a Navy guy, he's not sure he understands it either.
  
  29 DEC 2000: Let the real "Space Odyssey 2001" proceed.
  
  5 JAN 2001: Finished the 2nd disk of "2010". Something strange about watching a movie about a space expedition when you're actually on a space expedition.
  
  26 JAN 2001: We eat dinner and watch "GI Jane". Lots of SEAL questions, and Shep explains why this is not exactly like the real SEAL training.
  
  6 FEB 2001: We ate some dinner and watched the last part of "City of Angels". Shep did his best to explain to Yuri and Sergei what the phrase "chick flick" means.

  Interestingly, they seem to be watching movies on CDs and dont get a DVD player until 2001 or so. Anyway, lots more there. Interesting stuff.
• Amazing Photos of the NASA Space Shuttle: A series of photos showing how the Space Shuttle and its rocket boosters are assembled in preparation for a launch.
• Cities at Night: The View from Space: Amazing photos of cities taken from the ISS on the dark side of the planet. You get an interesting view of each city, and the overall density of human development by looking at these photos. I remember seeing something like this world map a while back, and there are many telling observations you could make about human development (observe the difference between North and South Korea, for instance), but you don't get much detail from that. These photos are great. See also this video detailing how the shots were taken and taking a tour around the world... [video via K-Squared Ramblings]
• The Earth and the Moon as seen from Mars: An interesting perspective. Ever notice in TV shows or movies that whenever you see a planet, you're almost always seeing the full planet in direct view of the sun (i.e. the "light side" of the planet, with none of the dark side visible). [via Kottke]

That's all for now...



Netflix Activity
The recent bout with myTV on DVD addiction necessitated an increase in Netflix usage, which made me curious. How well have I really taken advantage of the Netflix service, and is it worth the monthly expense?

If I were to rent a movie at a local video store like Blockbuster, each rental would cost somewhere around $4 (this is an extremely charitable estimate, as I'm sure it's probably closer to $5 at this point), plus the expense in time and effort (I mean, come on, I'd have to drive about a mile out of my way to go to one of these places!) Netflix costs me $15.99 a month for the 3-disc-at-a-time plan (this plan was $17.99 when I signed up, but decreased in price two times during around two years of membership), so it takes about 4-5 Netflix rentals to recoup my costs and bring the price of an average rental down below $4. I've been a member for one year and ten months... how did I do (click for a larger version)?


A few notes on the data:

• The chart shows both DVD rentals and movies or shows watched online through Netflix's "Watch Instant" service. There are certain distinctions that should be made here, namely that DVD rentals are measured by the date the DVD was returned, while Watch Instant rentals are measured when you watch them. Also, when watching a TV series on Watch Instant, each episode counts as a separate rental (if I were watching on DVD, there's usually 3-4 episodes on one disc, but since I'm watching on the Watch Instant service, each episode counts as a separate rental).
• As you can see, my initial usage was a little erratic, though I apparently tend to fall into a 4-5 month pattern (and you can see two nearly identical curves in 2007) where DVD rentals range from 6-13 per month. 13 appears to be my ceiling for a month, though I've hit that several times.
• I've only fallen below the 4 disc per month ratio needed to bring the average rental down below $4 once (twice if you count July 2006, but that was my first month of service and does not constitute a full month's worth of data). To be honest, I don't remember why I only returned 2 movies in January 2007, but that was the first and only time I fell below the necessary 4 rentals.
• My Watch Instantly service usage started off with a bang in July 2007 but quickly trailed off until 2008, when usage skyrocketed. This is when I discovered the TV show Dexter and quickly worked my way through all of the first season episodes (13 in all). Following Dexter, I started in on Ghost in the Shell: Stand Alone Complex and I just finished that today (expect a review later this week), so that means I watched 26 episodes online. Expect this to drop sharply next month (though I still plan on using it significantly, as I'll be following along with Filmspotting's 70's SF marathon, which features several movies in the Watch Instantly catalog). All in all, it's a reasonable service, though I have to admit that watching it on my computer just isn't the same - I bought that 50" widescreen HDTV for a reason, you know...
• You'll also notice that both March and April of 2008 have me hitting the ceiling of 13 movies per month. This is the first time I've done that in consecutive months and is largely due to watching BSG season 3 and my discovery and addiction to The Wire.
• As of April 2008, I'm averaging 9 movies a month (I've rented 198 DVDs). Even if I were to use my original price of $17.99 a month, that works out to around $2 a DVD rental. When you factor in the price drops and the Watch Instantly viewing (I've watched 51 things, though again, in some cases what I'm watching is a single episode of a TV show), I'm betting it would come out around $1.50-$1.75.

So it seems that the service is definitely worth the money and is indeed saving me a lot. Plus, Netflix has a far greater selection than any local video store (with the potential exception of TLA Video, but they're too far from my home to count), thus allowing me to indulge in various genres that you don't see much of in a typical video store. The only potential downside to Netflix is that you can't really rent something on impulse (unless it's on the Watch Instantly service). There are also times when new or popular movies take some time before they're actually available to you, but you have to contend with that from video rental stores as well. Indeed, I can only think of 3-4 times I've had to wait for a movie (this is mostly due to the fact that I tend to rent more obscure fare where people aren't exactly lining up to see it...) For the most part, Netflix has been reliable as well, almost always turning around my returns in short order (I mail it one day, and get the next films two days later). There have been a few mixups and I do remember one movie that wasn't available on the east coast and had to be shipped from California, so it came after a wait of 3-4 days, but for the most part, I'm very happy with the service.

This has been an interesting exercise, because I feel like I'm a little more consistent than the data actually shows. I'm really surprised that there are several months where my rentals went down to 6... I could have sworn I watched at least 2-3 discs a week, with the occasional exception. Still, an average of 9 movies a month is nothing to sneeze about, I guess. I've heard horror stories of where Netflix will start throttling you and take longer to deliver discs if you go above a certain amount of rentals per month (at a certain point, the cost of processing your rentals becomes more than you're paying, which I guess is what prompts Netflix to start throttling you), but I haven't had a problem yet. If I keep up my recent viewing habits though, this could change...



Rhetorical Strategy
Every so often, I see someone who is genuinely concerned with reaching the unreachable. Whether it be scientists who argue about how to frame their arguments, alpha-geek programmers who try to figure out how to reach typical, average programmers, or critics who try to open a dialogue with feminists. Debates tend to polarize, and when it comes to politics or religion, assumptions of bad faith on both sides tend to derail discussions pretty quickly.

How do you reach the unreachable? Naturally, the topic is much larger than a single blog entry, but I did run accross an interesting post by Jon Udell that outlines Charles Darwin's rhetorical strategy in the book, On the Origin of Species (which popularized the theory of evolution).

Darwin, says Slatkin, was like a salesman who finds lots of little ways to get you to say yes before you're asked to utter the big yes. In this case, Darwin invited people to affirm things they already knew, about a topic much more familiar in their era than in ours: domestic species. Did people observe variation in domestic species? Yes. And as Darwin piles on the examples, the reader says, yes, yes, OK, I get it, of course I see that some pigeons have longer tail feathers. Did people observe inheritance? Yes. And again, as he piles on the examples, the reader says yes, yes, OK, I get it, everyone knows that that the offspring of longer-tail-feather pigeons have longer tail feathers.

By the time Darwin gets around to asking you to say the big yes, it's a done deal. You've already affirmed every one of the key pillars of the argument. And you've done so in terms of principles that you already believe, and fully understand from your own experience.

It only took a couple of years for Darwin to formulate the idea of evolution by natural selection. It took thirty years to frame that idea in a way that would convince other scientists and the general public. Both the idea, and the rhetorical strategy that successfully communicated it, were great innovations.

I think Udell simplifies the inception and development of the idea of evolution, but I think the point generally holds. Darwin's ideas didn't come into mainstream prominence until he published his book, decades after he had begun his work. Obviously, Darwin's strategy isn't applicable in every situation, but it is an interesting place to start (I suppose we should keep in mind that evolution is still controversial amongst the mainstream)...



Facial Expressions and the Closed Eye Syndrome
I've been reading Malcolm Gladwell's book, Blink, and one of the chapters focuses on the psychology of facial expressions. Put simply, we wear our emotions on our face, and some enterprising psychologists took to mapping the distinct muscular movements that the human face can make. It's an interesting process, and it turns out that people who learn these facial expressions (of which there are many) are eerily good at recognizing what people are really thinking, even if they aren't trying to show it. It's almost like mind reading, and we all do it to some extent or another (mostly, we do it unconsciously). Body language and facial expressions are packed with information, and we'd all be pretty much lost without that kind of feedback (perhaps why misunderstandings are more common on the phone or in email). Most of the time, our expressions are voluntary, but sometimes they're not. Even if we're trying to suppress our expressions, a fleeting look may cross our faces. Often, these "micro-expressions" last only a few milliseconds and are imperceptible, but when trained psychologists watch video of, say, Harold "Kim" Philby (a notorious soviet spy) giving a press conference, they're able to read him like a book (slow motion helps).

I found this example interesting, and it highlights some of the subtle differences that can exist between expressions (in this case, between a voluntary and involuntary expression):

If I were to ask you to smile, you would flex your zygomatic major. By contrast, if you were to smile spontaneously, in the presence of genuine emotion, you would not only flex your zygomatic but also tighten the orbicularis oculi, pars orbitalis, which is the muscle that encircles the eye. It is almost impossible to tighten the orbicularis oculi, pars orbitalis on demand, and it is equally difficult to stop it from tightening when we smile at something genuinely pleasurable.

I found that interesting in light of the Closed Eye Syndrome I noticed in Anime. I wonder how that affects the way we perceive Anime. If a smiling mouth by itself means a fake expression of happiness while a smiling mouth and closed eyes means genuine emotion, does that make the animation more authentic? Animation obviously doesn't have the fidelity of video or film, but we can obviously read expressions from animated faces, so I would expect that closed eye syndrome exists more because of accuracy than anything else. In my original post on the subject, Roy noted that the reason I noticed closed eyes in anime could have something to do with the way Japan and the US read emotion. He pointed to an article that claimed Americans focus more on the mouth while the Japanese focus more on the eyes when trying to read emotions from facial expressions. One example from the article was emoticons. For happiness, Americans use a smily face :) while the Japanese tend to use ^_^ (which seems to be a face with eyes closed). That might still be part of it, but ever since I made the observation, I've noticed similar expressions in American animation (I just recently noticed it a lot in a Venture Bros. episode). Still, occurrences in American animation seem less frequent (or perhaps less obvious), so perhaps the observation still holds.

Gladwell's book is interesting, as expected, though I'm not sure yet if he has a point other than to observe that we do a lot of subconscious analysis and make lots of split decisions, and sometimes this is good (other times it's not). Still, he's good at finding examples and drilling down into the issue, and even if I'm not sure about his conclusions, it's always fun to read. There's lots more on this subject in the book (for instance, he goes over how facial expressions and our emotions are a two way phenomenon - meaning that if you intentionally contort your face in an specific way, you can induce certain emotions. The psychologists I mentioned earlier who were mapping expressions noticed that after a full day of trying to manipulate their facial muscles to show anger (even though they weren't angry) they felt horrible. Some tests have been done to confirm that, indeed, our facial expressions are linked directly to our brain) and it's probably worth a read if that's your bag.



Requiem for a Meme
In July of this year, I attempted to start a Movie Screenshot Meme. The idea was simple and (I thought) neat. I would post a screenshot, and visitors would guess what movie it was from. The person who guessed correctly would continue the game by either posting the next round on their blog, or if they didn't have a blog, they could send me a screenshot or just ask me to post another round. Things went reasonably well at first, and the game experienced some modest success. However, the game eventually morphed into the Mark, Alex, and Roy show, as the rounds kept cycling through each of our blogs. The last round was posted in September and despite a winning entry, the game has not continued.

The challenge of starting this meme was apparent from the start, but there were some other things that hindered the game a bit. Here are some assorted thoughts about the game, what held it back, and what could be done to improve the chances of adoption.

• Low Traffic: The most obvious reason the game tapered off was that my blog doesn't get a ton of traffic. I have a small dedicated core of visitors though, and I think that's why the game lasted as long as it did. Still, the three blogs that comprised the bulk of rounds in the game weren't very high traffic blogs. As such, the pool of potential participants was relatively small, which is the sort of thing that would make it difficult for a meme to expand.
• Barriers to Entry: The concept of allowing the winner to continue the game on their blog turned out to be a bit prohibitive, as most visitors don't have a blog. Also, a couple of winners expressed confusion as to how to get screenshots, and some didn't respond at all after winning. Of course, it is easy to start a new blog, and my friend Dave even did so specifically to post his round of the game, but none of these things helped get more eyes looking at the game.
• Difficulty: I intentionally made my initial entries easy (at one point, I even considered making it obscenely easy, but decided to just use that screenshot as a joke), in an attempt to ensnare casual movie viewers, but as the game progressed, screenshots became more and more difficult, and were coming from obscure movies. Actually, if you look at most of the screenshots outside of my blog, there aren't many mainstream movies. Here are some of the lesser known movies featured in the game Hedwig and the Angry Inch (this one stumped the interwebs), The Big Tease, Rosencrantz & Guildenstern Are Dead, Children of Men (mainstream, I guess, though I'm pretty sure it wasn't even out on DVD yet), Cry-Baby, Brotherhood of the Wolf, The City of Lost Children, Everything Is Illuminated, Wings of Desire, Who Framed Roger Rabbit (mainstream), Run, Lola, Run, Masters of the Universe (!), I Heart Huckabees, and Runaway. Now, of the ones I've seen, none of these are terrible films (er, well, He-Man was pretty bad, as was Runaway, but they're 80s movies, so slack is to be cut, right?), but they're also pretty difficult to guess for a casual movie watcher. I mean, most are independent, several are foreign, and it doesn't help when the screenshot is difficult to place (even some of the mainstream ones, like Who Framed Roger Rabbit, were a little difficult). Heck, by the end, even I was posting difficult stuff (the 5 screenshot extravaganza featured a couple of really difficult ones). Again, there's nothing inherently wrong with these movie selections, but they're film-geek selections that pretty much exclude mainstream viewers. If the game had become more widespread, this wouldn't have been as big of a deal, as I'd imagine that more movie geeks would be attracted to it. This is an interesting issue though, as several people thought their screenshots were easy, even though their visitors thought they were hard. Movies are subjective, so I guess it can be hard to judge the difficulty of a given screenshot. A screenshot that is blatantly obvious to me might be oppressively difficult to someone else.
• Again Traffic: Speaking of which, once the game had made its way around most of my friends' blogs, things began to slow down a bit because we were all hoping that someone new would win a round. Several non-bloggers posted comments to the effect of: I know the answer, but I don't have a blog and I want this game to spread so I'll hold off for now. I know I held back on several rounds because of this, but as the person who started this whole thing, this is understandable. In some ways, it was nice to see other people enjoying the game enough to care about it's success, but that also didn't help a whole lot.
• Detectives: At least a couple of people were able to find answers by researching rather than recognizing the movie. I know I was guilty of this. I'd recognize an actor, then look them up on IMDB and see what they've done, which helps narrow down the field considerably. I don't know that this is actually a bad thing, but I did find it interesting.
• Memerific: The point of a meme is that it's supposed to be self-sustaining and self-propagating. While this game did achieve a modest success at the beginning, it never really became self-sustaining. At least a couple of times, I prodded the game to move it forward, and Roy and Alex did the same. I guess the memetic inertia was constantly being worn down by the factors discussed in this post.
• Help: Given the above, there were several things that could have helped. I could have done a better job promoting the game, for instance. I could have made it easier for other bloggers to post a round. One of the things I wanted to do was create little javascript snippits that people could use to very quickly display the unweildy rules (perhaps using nifty display techniques that hide most of the text initially until you click to learn more) and another little javascript that would display the current round (in a nice little graphical button or something). Unfortunately, this game pretty much coincided with the busiest time of my professional career, and I didn't have a lot of time to do anything (just keeping up with the latest round was a bit of a challenge for me).
• Variants: One thing that may have helped would be to spread the game further out by allowing winners to "tag" other bloggers they wanted to see post screenshots, rather than just letting the winner post their own. I actually considered this when designing the game, but after some thought, I decided against it. Many people hate memes and don't like being "tagged" to participate. Knowing this, a lot of people who do participate in memes are hesitant to "tag" other people. I didn't want to annoy people with the blogging equivalent to chain letters, so I decided against it. However, it might have helped this meme spread out much further, as it doesn't require casual movie fans to participate more and it would allow the meme to spread much further, much faster. If I said the winner should tag 5 other bloggers to participate, the meme could spread exponentially. This would be much more difficult to track, but on the other hand, it might actually catch on. This might be the biggest way to improve the meme's chances at survival.
• Alternatives: This strikes me as something that would work really well on a message board type system, especially one that allowed users to upload their own images. Heck, I wouldn't be surprised to see something like this out there. It also might have been a good idea to create a way to invite others to play the game via email (which probably would only work on a message board or dedicated website, where there's one central place that screenshots are posted). However, one of the things that's neat about blog memes is that they tend to get your blog exposed to people who wouldn't otherwise visit.

It was certainly an interesting and fun experience, and I'm glad I did it. Just for kicks, I'll post another screenshot. Feel free to post your answer in the comments, but I'm not especially expecting this to progress much further than it did before (though anything's possible):


(click image for a larger version) I'd say this is difficult except that it's blatantly obvious who that is in the screenshot. It shouldn't be that hard to pick out the movie even if you haven't seen it. What the heck, the winner of this round can pick 5 blogs they'd like to see post a screenshot and post a screenshot on their blog if they desire. As I mentioned above, I'm hesitant to annoy people with this sort of thing, but hey, why not? Let's give this meme some legs.



The Paradise of Choice?
A while ago, I wrote a post about the Paradox of Choice based on a talk by Barry Schwartz, the author of a book by the same name. The basic argument Schwartz makes is that choice is a double-edged sword. Choice is a good thing, but too much choice can have negative consequences, usually in the form of some kind of paralysis (where there are so many choices that you simply avoid the decision) and consumer remorse (elevated expectations, anticipated regret, etc...). The observations made by Schwartz struck me as being quite astute, and I've been keenly aware of situations where I find myself confronted with a paradox of choice ever since. Indeed, just knowing and recognizing these situations seems to help deal with the negative aspects of having too many choices available.

This past summer, I read Chris Anderson's book, The Long Tail, and I was a little pleasantly surprised to see a chapter in his book titled "The Paradise of Choice." In that chapter, Anderson explicitely addresses Schwartz's book. However, while I liked Anderson's book and generally agreed with his basic points, I think his dismissal of the Paradox of Choice is off target. Part of the problem, I think, is that Anderson is much more concerned with the choices rather than the consequences of those choices (which is what Schwartz focuses on). It's a little difficult to tell though, as Anderson only dedicates 7 pages or so to the topic. As such, his arguments don't really eviscerate Schwartz's work. There are some good points though, so let's take a closer look.

Anderson starts with a summary of Schwartz's main concepts, and points to some of Schwartz's conclusions (from page 171 in my edition):

As the number of choices keeps growing, negative aspects of having a multitude of options begin to appear. As the number of choices grows further, the negatives escalate until we become overloaded. At this point, choice no longer liberates, but debilitates. It might even be said to tyrannize.

Now, the way Anderson presents this is a bit out of context, but we'll get to that in a moment. Anderson continues and then responds to some of these points (again, page 171):

As an antidote to this poison of our modern age, Schwartz recommends that consumers "satisfice," in the jargon of social science, not "maximize". In other words, they'd be happier if they just settled for what was in front of them rather than obsessing over whether something else might be even better. ...

I'm skeptical. The alternative to letting people choose is choosing for them. The lessons of a century of retail science (along with the history of Soviet department stores) are that this is not what most consumers want.

Anderson has completely missed the point here. Later in the chapter, he spends a lot of time establishing that people do, in fact, like choice. And he's right. My problem is twofold: First, Schwartz never denies that choice is a good thing, and second, he never advocates removing choice in the first place. Yes, people love choice, the more the better. However, Schwartz found that even though people preferred more options, they weren't necessarily happier because of it. That's why it's called the paradox of choice - people obviously prefer something that ends up having negative consequences. Schwartz's book isn't some sort of crusade against choice. Indeed, it's more of a guide for how to cope with being given too many choices. Take "satisficing." As Tom Slee notes in a critique of this chapter, Anderson misstates Schwartz's definition of the term. He makes it seem like satisficing is settling for something you might not want, but Schwartz's definition is much different:

To satisfice is to settle for something that is good enough and not worry about the possibility that there might be something better. A satisficer has criteria and standards. She searches until she finds an item that meets those standards, and at that point, she stops.

Settling for something that is good enough to meet your needs is quite different than just settling for what's in front of you. Again, I'm not sure Anderson is really arguing against Schwartz. Indeed, Anderson even acknowledges part of the problem, though he again misstate's Schwartz's arguments:

Vast choice is not always an unalloyed good, of course. It too often forces us to ask, "Well, what do I want?" and introspection doesn't come naturally to all. But the solution is not to limit choice, but to order it so it isn't oppressive.

Personally, I don't think the problem is that introspection doesn't come naturally to some people (though that could be part of it), it's more that some people just don't give a crap about certain things and don't want to spend time figuring it out. In Schwartz's talk, he gave an example about going to the Gap to buy a pair of jeans. Of course, the Gap offers a wide variety of jeans (as of right now: Standard Fit, Loose Fit, Boot Fit, Easy Fit, Morrison Slim Fit, Low Rise Fit, Toland Fit, Hayes Fit, Relaxed Fit, Baggy Fit, Carpenter Fit). The clerk asked him what he wanted, and he said "I just want a pair of jeans!"

The second part of Anderson's statement is interesting though. Aside from again misstating Schwartz's argument (he does not advocate limiting choice!), the observation that the way a choice is presented is important is interesting. Yes, the Gap has a wide variety of jean styles, but look at their website again. At the top of the page is a little guide to what each of the styles means. For the most part, it's helpful, and I think that's what Anderson is getting at. Too much choice can be oppressive, but if you have the right guide, you can get the best of both worlds. The only problem is that finding the right guide is not as easy as it sounds. The jean style guide at Gap is neat and helpful, but you do have to click through a bunch of stuff and read it. This is easier than going to a store and trying all the varieties on, but it's still a pain for someone who just wants a pair of jeans dammit.

Anderson spends some time fleshing out these guides to making choices, noting the differences between offline and online retailers:

In a bricks-and-mortar store, products sit on the shelf where they have been placed. If a consumer doesn't know what he or she wants, the only guide is whatever marketing material may be printed on the package, and the rough assumption that the product offered in the greatest volume is probably the most popular.

Online, however, the consumer has a lot more help. There are a nearly infinite number of techniques to tap the latent information in a marketplace and make that selection process easier. You can sort by price, by ratings, by date, and by genre. You can read customer reviews. You can compare prices across products and, if you want, head off to Google to find out as much about the product as you can imagine. Recommendations suggest products that 'people like you' have been buying, and surprisingly enough, they're often on-target. Even if you know nothing about the category, ranking best-sellers will reveal the most popular choice, which both makes selection easier and also tends to minimize post-sale regret. ...

... The paradox of choice is simply and artifact of the limitations of the physical world, where the information necessary to make an informed choice is lost.

I think it's a very good point he's making, though I think he's a bit too optimistic about how effective these guides to buying really are. For one thing, there are times when a choice isn't clear, even if you do have a guide. Also, while I think retailers that offer Recommendations based on what other customer purchases are important and helpful, who among us hasn't seen absurd recommendations? From my personal experience, a lot of people don't like the connotations of recommendations either (how do they know so much about me? etc...). Personally, I really like recommendations, but I'm a geek and I like to figure out why they're offering me what they are (Amazon actually tells you why something is recommended, which is really neat). In any case, from my own personal anecdotal observations, no one puts much faith in probablistic systems like recommendations or ratings (for a number of reasons, such as cheating or distrust). There's nothing wrong with that, and that's part of why such systems are effective. Ironically, acknowledging their imperfections allow users to better utilize the systems. Anderson knows this, but I think he's still a bit too optimistic about our tools for traversing the long tail. Personally, I think they need a lot of work.

When I was younger, one of the big problems in computing was storage. Computers are the perfect data gatering tool, but you need somewhere to store all that data. In the 1980s and early 1990s, computers and networks were significantly limited by hardware, particularly storage. By the late 1990s, Moore's law had eroded this deficiency significantly, and today, the problem of storage is largely solved. You can buy a terrabyte of storage for just a couple hundred dollars. However, as I'm fond of saying, we don't so much solve problems as trade one set of problems for another. Now that we have the ability to store all this information, how do we get at it in a meaninful way? When hardware was limited, analysis was easy enough. Now, though, you have so much data available that the simple analyses of the past don't cut it anymore. We're capturing all this new information, but are we really using it to its full potential?

I recently caught up with Malcolm Gladwell's article on the Enron collapse. The really crazy thing about Enron was that they didn't really hide what they were doing. They fully acknowledged and disclosed what they were doing... there was just so much complexity to their operations that no one really recognized the issues. They were "caught" because someone had the persistence to dig through all the public documentation that Enron had provided. Gladwell goes into a lot of detail, but here are a few excerpts:

Enron's downfall has been documented so extensively that it is easy to overlook how peculiar it was. Compare Enron, for instance, with Watergate, the prototypical scandal of the nineteen-seventies. To expose the White House coverup, Bob Woodward and Carl Bernstein used a source-Deep Throat-who had access to many secrets, and whose identity had to be concealed. He warned Woodward and Bernstein that their phones might be tapped. When Woodward wanted to meet with Deep Throat, he would move a flower pot with a red flag in it to the back of his apartment balcony. That evening, he would leave by the back stairs, take multiple taxis to make sure he wasn't being followed, and meet his source in an underground parking garage at 2 A.M. ...

Did Jonathan Weil have a Deep Throat? Not really. He had a friend in the investment-management business with some suspicions about energy-trading companies like Enron, but the friend wasn't an insider. Nor did Weil's source direct him to files detailing the clandestine activities of the company. He just told Weil to read a series of public documents that had been prepared and distributed by Enron itself. Woodward met with his secret source in an underground parking garage in the hours before dawn. Weil called up an accounting expert at Michigan State.

When Weil had finished his reporting, he called Enron for comment. "They had their chief accounting officer and six or seven people fly up to Dallas," Weil says. They met in a conference room at the Journal's offices. The Enron officials acknowledged that the money they said they earned was virtually all money that they hoped to earn. Weil and the Enron officials then had a long conversation about how certain Enron was about its estimates of future earnings. ...

Of all the moments in the Enron unravelling, this meeting is surely the strangest. The prosecutor in the Enron case told the jury to send Jeffrey Skilling to prison because Enron had hidden the truth: You're "entitled to be told what the financial condition of the company is," the prosecutor had said. But what truth was Enron hiding here? Everything Weil learned for his Enron expose came from Enron, and when he wanted to confirm his numbers the company's executives got on a plane and sat down with him in a conference room in Dallas.

Again, there's a lot more detail in Gladwell's article. Just how complicated was the public documentation that Enron had released? Gladwell gives some examples, including this one:

Enron's S.P.E.s were, by any measure, evidence of extraordinary recklessness and incompetence. But you can't blame Enron for covering up the existence of its side deals. It didn't; it disclosed them. The argument against the company, then, is more accurately that it didn't tell its investors enough about its S.P.E.s. But what is enough? Enron had some three thousand S.P.E.s, and the paperwork for each one probably ran in excess of a thousand pages. It scarcely would have helped investors if Enron had made all three million pages public. What about an edited version of each deal? Steven Schwarcz, a professor at Duke Law School, recently examined a random sample of twenty S.P.E. disclosure statements from various corporations-that is, summaries of the deals put together for interested parties-and found that on average they ran to forty single-spaced pages. So a summary of Enron's S.P.E.s would have come to a hundred and twenty thousand single-spaced pages. What about a summary of all those summaries? That's what the bankruptcy examiner in the Enron case put together, and it took up a thousand pages. Well, then, what about a summary of the summary of the summaries? That's what the Powers Committee put together. The committee looked only at the "substance of the most significant transactions," and its accounting still ran to two hundred numbingly complicated pages and, as Schwarcz points out, that was "with the benefit of hindsight and with the assistance of some of the finest legal talent in the nation."

Again, Gladwell's article has a lot of other details and is a fascinating read. What interested me the most, though, was the problem created by so much data. That much information is useless if you can't sift through it quickly or effectively enough. Bringing this back to the paradise of choice, the current systems we have for making such decisions are better than ever, but still require a lot of improvement. Anderson is mostly talking about simple consumer products, so none are really as complicated as the Enron case, but even then, there are still a lot of problems. If we're really going to overcome the paradox of choice, we need better information analysis tools to help guide us. That said, Anderson's general point still holds:

More choice really is better. But now we know that variety alone is not enough; we also need information about that variety and what other consumers before us have done with the same choices. ... The paradox of choice turned out to be more about the poverty of help in making that choice than a rejection of plenty. Order it wrong and choice is oppressive; order it right and it's liberating.

Personally, while the help in making choices has improved, there's still a long way to go before we can really tackle the paradox of choice (though, again, just knowing about the paradox of choice seems to do wonders in coping with it).

As a side note, I wonder if the video game playing generations are better at dealing with too much choice - video games are all about decisions, so I wonder if folks who grew up working on their decision making apparatus are more comfortable with being deluged by choice.



The Spinning Silhouette
This Spinning Silhouette optical illusion is making the rounds on the internet this week, and it's being touted as a "right brain vs left brain test." The theory goes that if you see the silhouette spinning clockwise, you're right brained, and you're left brained if you see it spinning counterclockwise.

Everytime I looked at the damn thing, it was spinning a different direction. I closed my eyes and opened them again, and it spun a different direction. Every now and again, and it would stay the same direction twice in a row, but if I looked away and looked back, it changed direction. Now, if I focus my eyes on a point below the illusion, it doesn't seem to rotate all the way around at all, instead it seems like she's moving from one side to the other, then back (i.e. changing directions every time the one leg reaches the side of the screen - and the leg always seems to be in front of the silhouette).

Of course, this is the essense of the illusion. The silhouette isn't actually spinning at all, because it's two dimensional. However, since my brain is used to living in a three dimensional world (and thus parsing three dimensional images), it's assuming that the image is also three dimensional. We're actually making lots of assumptions about the image, and that's why we can see it going one way or the other.

Eventually, after looking at the image for a while and pondering the issues, I got curious. I downloaded the animated gif and opened it up in the GIMP to see how the frames are built. I could be wrong, but I'm pretty sure this thing is either broken or it's cheating. Well, I shouldn't say that. I noticed something off on one of the frames, and I'd be real curious to know how that affects people's perception of the illusion (to me, it means the image is definitely moving counterclockwise). I'm almost positive that it's too subtle to really affect anything, but I did find it interesting. More on this, including images and commentary, below the fold. First thing's first, here's the actual spinning silhouette.


Again, some of you will see it spinning in one direction, some in the other direction. Everyone seems to have a different trick for getting it to switch direction. Some say to focus on the shadow, some say to look at the ankles. Closing my eyes and reopening seems to do the trick for me. Now let's take a closer look at one of the frames. Here's frame 12:


Looking at this frame, you should be able to switch back and forth, seeing the leg behind the person or in front of the person. Again, because it's a silhouette and a two dimensional image, our brain usually makes an assumption of depth, putting the leg in front or behind the body. Switching back and forth on this static image was actually a lot easier for me. Now the tricky part comes in the next frame, number 13 (obviously, the arrow was added by me):


Now, if you look closely at the leg, you'll see a little imperfection in the silhouette. Maybe I'm wrong, but that little gash in the leg seems to imply that the leg is behind the body. If you try, you can still get yourself to see the image as having the leg in front, but then you've got this gash in the leg that just seems very out of place.

So what to make of this? First, the imperfection is subtle enough (it's on 1 frame out of 34) that everyone still seems to be able to see it rotate in both directions. Second, maybe I'm crazy, and the little gash doesn't imply what I think. Anyone have alternative explanations? Third, is that imperfection intentional? If so, why? It does not seem necessary, so I'd be curious to know if the creators knew about it, and what their intention was regarding it.

Finally, as far as the left brain versus right brain portion, I find that I don't really care, but I am interested in how the imperfection would affect this "test." This neuroscientist seems to be pretty adamant about the whole left/right thing being hogwash though:

...the notion that someone is "left-brained" or "right-brained" is absolute nonsense. All complex behaviours and cognitive functions require the integrated actions of multiple brain regions in both hemispheres of the brain. All types of information are probably processed in both the left and right hemispheres (perhaps in different ways, so that the processing carried out on one side of the brain complements, rather than substitutes, that being carried out on the other).

At the very least, the traditional left/right brain theory is a wildly oversimplified version of what's really happening. The post also goes into the way the brain "fill in the gaps" for confusing visual information, thus allowing the illusion.

Update: Strange - the image appears to be rotating MUCH faster in Firefox than in Opera or IE. I wonder how that affects perception.



The Long Tail of Forgotten Works
I'm currently reading Chris Anderson's book The Long Tail, and he relates a story about how some books find an audience long after they've been published.

In 1988, a British mountain climber named Joe Simpson wrote a book called Touching the Void, a harrowing account of near death in the Peruvian Andes. Though reviews for the book were good, it was only a modest success, and soon was largely forgotten. Then, a decade later, a strange thing happened. Jon Krakauer wrote Into Thin Air, another book about a mountain-climbing tragedy, which became a publishing sensation. Suddenly Touching the Void started to sell again.

Booksellers began promoting it next to their Into Thin Air displays, and sales continued to rise. In early 2004, IFC Films released a docudrama of the story, to good reviews. Shortly thereafter, HarperCollins released a revised paperback, which spent fourteen weeks on the New York Times best-seller list. By mid-2004, Touching the Void was outselling Into Thin Air more than two to one.

What happened? Online word of mouth. When Into Thin Air first came out, a few readers wrote reviews on Amazon.com that pointed out the similarities with the then lesser-known Touching the Void, which they praised effusively. Other shoppers read those reviews, checked out the older book, and added it to their shopping carts. Pretty soon the online bookseller's software noted the patterns in buying behavior--"Readers who bought Into Thin Air also bought Touching the Void"--and started recommending the two as a pair. People took the suggestion, agreed wholeheartedly, wrote more rhapsodic reviews. More sales, more algorithm-fueled recommendations--and a powerful positive feedback loop kicked in.

Particularly notable is that when Krakauer's book hit shelves, Simpson's was nearly out of print. A decade ago readers of Krakauer would never even have learned about Simpson's book--and if they had, they wouldn't have been able to find it. Online booksellers changed that. By combining infinite shelf space with real-time information about buying trends and public opinion, they created the entire Touching the Void phenomenon. The result: rising demand for an obscure book.

There is something interesting going on here. I'm wondering how many great works of art are simply lost in obscurity. These days, we've got the internet and primitive tools to traverse the long tail, so it seems that a lot of obscure works find a new audience when a new, similar work is released. But what happened before the internet? How many works have simply gone out of print because they never found an audience - how many works suffered the fate Touching the Void narrowly avoided?

Of course, I have no idea (that's kinda the point), but one of the great things about the internet and the emerging infinite shelf space of online retailers is that some of these obscure works are rediscovered and new connections are made. For instance, I once came accross a blog post by Jonathon Delacour about this obscure Japanese horror film called Matango: Attack of the Mushroom People. The description of the film?

After a yacht is damaged in a storm and stranded on a deserted island, the passengers: a psychologist, his girlfriend, a wealthy businessman, a famous singer, a writer, a sailor and his skipper take refuge in a fungus covered boat. While using the mushrooms for sustenance, they find the ship's journal describing the mushrooms to be poisonous, however some members of the shipwrecked party continue to ingest the mysterious fungi transforming them into hideous fungal monsters.

Sound familiar? As Delacour notes, a reviewer on Amazon.com sure thinks so:

Was this the Inspiration for Gilligan's Island? ...and that's a serious question. It predated the premier of Gillian's Island by several years. There's a millionaire who owns a yacht that looks like the Minnow. On board is a professor, the captain, a goofy (though somewhat sinster in the film) first mate, a pretty but shy country girl named Okiko, and a singer/movie star. There are seven castaways in all. "Lovey" is replaced by another male character, a writer named Roy. The boat crashes into an island where they are castaways... Course on Gilligan's Island they didn't all turn into mutated mushrooms monsters. Rent or buy the DVD (one of my favorite films in Japanese cinema, finally getting its due...) and you tell me if Gilligan's Island isn't a complete rip-off of this film.

Several reviewers actually make the Gilligan's Island connection, and one even takes time to refute the claim that Gilligan ripped off Matango:

Actually as stated on this DVD's actor commentary Matango premiered in Japanese theaters in and around mid 1963. The Gilligan's Island first pilot (with different actors as The Professor and Ginger)was made in late 1963 thus the Japanese film does not predate Gilligan by a few years as another poster here thinks.Schwartz could have heard about a Japanese film made with seven castaways (as Hollywood and Tokoyo's Toho were in communication). But he definitely didn't see the Japanese film before he pitched gI to the networks in early 63.

So perhaps this was just a happy coincidence... A commentor on Delacour's post mentions that the movie is loosely based on a 1907 short story by William Hope Hodgson called The Voice in the Night, but while it certainly was the inspiration behind Matango, it probably didn't inspire Gilligan's Island...

I seem to have veered off track here, but it was an interesting diversion: from obscure Japanese horror film to Gilligan's Island to William Hope Hodgson... would anyone have made these connections 20 years ago? It certainly would have been possible, but I doubt it would happen as quickly or efficiently as it did on the internet.



Again Cell Phones
About 2 years ago, I started looking around for a new cell phone. At the time, I just wanted a simple, no-frills type phone, but I kept an open mind and looked at some of the more advanced features that were becoming available. I eventually settled on a small, low-end Nokia. I instantly regretted the decision not to get a camera phone, but otherwise, the phone has performed admirably. The only other complaint I really have is that the call volume could stand to be a little louder. In any case, in the comments of one of the above linked posts, I mentioned:

I'm actually kinda surprised that cell phones aren't... better than they are now. I figure in about 2 years, my dream phone will be more attainable, so for now, I'll make do with what I got.

Well, it's been 2 years, I'm once again looking into purchasing a new phone and... I'm still surprised that cell phones aren't better than they are right now. Seriously, what the heck is going on? My priorities aren't that unusual and have only changed a little since my last foray: I want a phone that has strong battery life, good call quality (with louder call volume), good usability (i.e. button placement, menu structure, etc...), and a quality camera (at least 1.3 megapixel). There are lots of secondary features and nice-to-haves, but those are the most important things. This is apparently difficult to achieve though, and I'm distinctly underwhelmed by my options. Actually there are a lot of decent phones out there, but I think I've fallen into the classic paradox of choice trap. Here are some phones I'm considering:

• Sony Ericsson W810i: When I bought my last phone, I remarked that the Sony Ericsson W800i seemed really interesting because it was basically knocking out 3 birds with one stone: phone, camera, and mp3 player. At the time, it was obscenely expensive and it seemed to suffer from numerous glitches. The W810i is the successor to the W800i, and by all accounts Sony Ericsson has worked through a lot of the issues to produce a pretty solid phone. I have some minor concerns about the keypad, but everything else seems in order (and the phone looks great - 2 MP with a flash) and the price tag is pretty reasonable for such a fully-featured phone. The only thing that really goes against my requirements is the "staticky call quality" that's referenced in the reviews. Also, I hate Sony. I really don't want to give them my money.
• Motorola SLVR: I've never been a big fan of clamshell phones, so I never really cared that much about the RAZR when it came out. Then Mororola released the SLVR, which seems like a decent phone at first glance. Decent battery life (not as good as the Sony Ericsson though), reasonable sound quality, and all the standard cell phone features. The one big problem for me is that the camera looks crappy. I believe the newer models are improving the camera, so we'll see how that goes (in general, Motorola's phones don't seem to have great cameras though, even when they have decent resolutions). If they improve the camera, I'd gladly pick this over the Sony Ericsson.
• Motorola KRZR: This is another interesting option, but once again, I'm a little turned off by the camera. It seems better than the camera on the SLVR, but still nowhere near the Sony Ericsson. There seem to be a lot of different versions of Motorola cell phones (no matter what variety), so it's a little confusing going through them all and trying to figure out which one meets your needs. I don't normally love flip phones, but I think this one's pretty good. Aside from the camera, this one appears to be a little more expensive too, which is a bummer.
• Nokia 5300 Xpress Music: Well, this one isn't a real option just yet simply because it's not available on Cingular or Verizon. That said, it's a quality phone, and I've had good experiences with Nokia. Again, the camera seems decent but nowhere near the Sony Ericsson. The only other problem is that it seems the volume doesn't go loud enough, and that's one of my primary annoyances.
• LG VX8600: This is the flip-phone version of the hip Chocolate phones, and it seems to have improved upon the Chocolate as well. This supposedly has one of the better cameras, but it has awful battery life.

There are some interesting phones coming. I'd love an iPhone, but I can't justify the cost. I'm interested in the rumored Microsoft and Google phones, but I doubt they'll be coming anytime soon. Of course, there are probably dozens of phones that would readily meet my needs, but they're not available in the US. I'm hardly the first person to note this, but it is quite frustrating. I understand why this is happening (the US is a small, fractured market that utilizes a variety of technologies and frequencies that are different than what Europe & Asia use. So companies naturally focus on the larger, more homogenous European & Asian markets.), but it's still annoying. I'm not sure how this will be rectified; perhaps we'll just have to wait until 4G comes along (assuming everyone adopts the same 4G).

Update: Drool. Battery life looks lame, but otherwise it's great. Not that it matters, as it ain't available yet.



A System of Warnings
Josh Porter recently wrote about some design principles he uses. As Josh notes, people often confuse design with art. Art is a form of personal expression, while design is about use.

The designer needs someone to use (not only appreciate) what they create. Design doesn't serve its purpose without people to use it. Design helps solve human problems. The highest accolade we can bestow on a design is not that it is beautiful, as we do in Art, but that it is well-used.

I think one of the most recognized and perhaps important designs of the past twenty years or so is the Nutrition Facts label. Instantly recognizable and packed with information, yet concise and easy to read and use. It's not glamorous, but it works so well that we barely even notice it. It's great design.

While nutrition is certainly an important subject worthy of a thoughtful design, I recently stumbled upon a design project that is intriguing, difficult and important. In the desert of Southeastern New Mexico lies the Waste Isolation Pilot Plant (WIPP), an undeground radioactive waste repository. Not a pleasant place. During the planning stages of the facility, a panel of experts were tasked with designing a 10,000-year marking system. It's an intriguing design problem. The resulting report is an astounding, powerful and oddly poignant document (excerpts here, huge .pdf version of the full report here). They developed an interesting system here; note, they didn't just create signs, the entire site (from the physical layout to the words and imagery used) was designed to communicate a message across multiple levels, with a high level of redundancy. It's not just a warning, it's a system of interconnected and reinforced warnings. The authors also attempted to anticipate a variety of potential attacks as well. What is the message they wanted to convey? Here's a brief summary:

• This place is a message... and part of a system of messages... pay attention to it!
• Sending this message was important to us. We considered ourselves to be a powerful culture.
• This place is not a place of honor... no highly esteemed deed is commemorated here... nothing valued is here.
• What is here is dangerous and repulsive to us. This message is a warning about danger.
• The danger is in a particular location... it increases toward a center... the center of danger is here... of a particular size and shape, and below us.
• The danger is still present, in your time, as it was in ours.
• The danger is to the body, and it can kill.
• The form of the danger is an emanation of energy.
• The danger is unleashed only if you substantially disturb this place physically. This place is best shunned and left uninhabited.
• All physical site interventions and markings must be understood as communicating a message. It is not enough to know that this is a place of importance and danger...you must know that the place itself is a message, that it contains messages, and is part of a system of messages, and is a system with redundance.

As James Grimmelmann notes, this is "frightening, apocalyptic poetry." I find the third bullet to be particularly evocative. The assumptions the authors had to make in working on this design are interesting to contemplate. They're assuming that the audience for this design will be significantly different, perhaps not even human (in any case, the assumption is that something bad has happened and we're no longer around). Again, this is an intriguing design problem. I think they've done a pretty good job thinking about the problem, even if some of their more exotic designs didn't make it into the final system.



Link Dump
Various links for your enjoyment:

• The Order of the Science Scouts of Exemplary Repute and Above Average Physique: Like the Boy Scouts, but for Scientists. Aside from the goofy name, they've got an ingenious and hilarious list of badges, including: The "my degree inadvertantly makes me competent in fixing household appliances" badge, The "I've touched human internal organs with my own hands" badge, The "has frozen stuff just to see what happens" badge (oh come one, who hasn't done that?), The "I bet I know more computer languages than you, and I'm not afraid to talk about it" badge (well, I used to know a bunch), and of course, The "dodger of monkey shit" badge. ("One of our self explanatory badges."). Sadly, I qualify for less of these than I'd like. Of course, I'm not a scientist, but still. I'm borderline on many though (for instance, the "I blog about science" badge requires that I maintain a blog where at least a quarter of the material is about science - I certainly blog about technology a lot, but explicitely science? Debateable, I guess.)
• Dr. Ashen and Gizmodo Reviews The Gamespower 50 (YouTube): It's a funny review of a crappy portable video game device, just watch it. The games on this thing are so bad (there's actually one called "Grass Cutter," which is exactly what you think it is - a game where you mow the lawn).
• Count Chocula Vandalism on Wikipedia: Some guy came up with an absurdly comprehensive history for Count Chocula:

  Ernst Choukula was born the third child to Estonian landowers in the late autumn of 1873. His parents, Ivan and Brushken Choukula, were well-established traders of Baltic grain who-- by the early twentieth century--had established a monopolistic hold on the export markets of Lithuania, Latvia and southern Finland. A clever child, Ernst advanced quickly through secondary schooling and, at the age of nineteen, was managing one of six Talinn-area farms, along with his father, and older brother, Grinsh. By twenty-four, he appeared in his first "barrelled cereal" endorsement, as the Choukula family debuted "Ernst Choukula's Golden Wheat Muesli", a packaged mix that was intended for horses, mules, and the hospital ridden. Belarussian immigrant silo-tenders started cutting the product with vodka, creating a crude mush-paste they called "gruhll" or "gruell," and would eat the concoction each morning before work.

  It goes on like that for a while. That particular edit has been removed from the real article, but there appears to actually be quite a debate on the Talk page as to whether or not to mention it in the official article.
• The Psychology of Security by Bruce Schneier: A long draft of an article that delves into psychological reasons we make the security tradeoffs that we do. Interesting stuff.
• The Sagan Diary by John Scalzi (Audio Book): I've become a great fan of Scalzi's fiction, and his latest work is available here as audio (a book is available too, but it appears to be a limited run). Since the book is essentially the diary of a woman, he got various female authors and friends to read a chapter. This actually makes for somewhat uneven listening, as some are great and others aren't as great. Now that I think about it, this book probably won't make sense if you haven't read Old Man's War and/or The Ghost Brigades. However, they're both wonderful books of the military scifi school (maybe I'll probably write a blog post or two about them in the near future).

Intellectual Property, Copyright and DRM
Roy over at 79Soul has started a series of posts dealing with Intellectual Property. His first post sets the stage with an overview of the situation, and he begins to explore some of the issues, starting with the definition of theft. I'm going to cover some of the same ground in this post, and then some other things which I assume Roy will cover in his later posts.

I think most people have an intuitive understanding of what intellectual property is, but it might be useful to start with a brief definition. Perhaps a good place to start would be Article 1, Section 8 of the U.S. Constitution:

To promote the Progress of Science and useful Arts, by securing for limited Times to Authors and Inventors the exclusive Right to their respective Writings and Discoveries;

I started with this for a number of reasons. First, because I live in the U.S. and most of what follows deals with U.S. IP law. Second, because it's actually a somewhat controversial stance. The fact that IP is only secured for "limited times" is the key. In England, for example, an author does not merely hold a copyright on their work, they have a Moral Right.

The moral right of the author is considered to be -- according to the Berne convention -- an inalienable human right. This is the same serious meaning of "inalienable" the Declaration of Independence uses: not only can't these rights be forcibly stripped from you, you can't even give them away. You can't sell yourself into slavery; and neither can you (in Britain) give the right to be called the author of your writings to someone else.

The U.S. is different. It doesn't grant an inalienable moral right of ownership; instead, it allows copyright. In other words, in the U.S., such works are considered property (i.e. it can be sold, traded, bartered, or given away). This represents a fundamental distinction that needs to be made: some systems emphasize individual rights and rewards, and other systems are more limited. When put that way, the U.S. system sounds pretty awful, except that it was designed for something different: our system was built to advance science and the "useful arts." The U.S. system still rewards creators, but only as a means to an end. Copyright is granted so that there is an incentive to create. However, such protections are only granted for "limited Times." This is because when a copyright is eternal, the system stagnates as protected peoples stifle competition (this need not be malicious). Copyright is thus limited so that when a work is no longer protected, it becomes freely available for everyone to use and to build upon. This is known as the public domain.

The end goal here is the advancement of society, and both protection and expiration are necessary parts of the mix. The balance between the two is important, and as Roy notes, one of the things that appears to have upset the balance is technology. This, of course, extends as far back as the printing press, records, cassettes, VHS, and other similar technologies, but more recently, a convergence between new compression techniques and increasing bandwidth of the internet created an issue. Most new recording technologies were greeted with concern, but physical limitations and costs generally put a cap on the amount of damage that could be done. With computers and large networks like the internet, such limitations became almost negligible. Digital copies of protected works became easy to copy and distribute on a very large scale.

The first major issue came up as a result of Napster, a peer-to-peer music sharing service that essentially promoted widespread copyright infringement. Lawsuits followed, and the original Napster service was shut down, only to be replaced by numerous decentralized peer-to-peer systems and darknets. This meant that no single entity could be sued for the copyright infringement that occurred on the network, but it resulted in a number of (probably ill-advised) lawsuits against regular folks (the anonymity of internet technology and state of recordkeeping being what it is, this sometimes leads to hilarious cases like when the RIAA sued a 79 year old guy who doesn't even own a computer or know how to operate one).

Roy discusses the various arguments for or against this sort of file sharing, noting that the essential difference of opinion is the definition of the word "theft." For my part, I think it's pretty obvious that downloading something for free that you'd normally have to pay for is morally wrong. However, I can see some grey area. A few months ago, I pre-ordered Tool's most recent album, 10,000 Days from Amazon. A friend who already had the album sent me a copy over the internet before I had actually recieved my copy of the CD. Does this count as theft? I would say no.

The concept of borrowing a Book, CD or DVD also seems pretty harmless to me, and I don't have a moral problem with borrowing an electronic copy, then deleting it afterwords (or purchasing it, if I liked it enough), though I can see how such a practice represents a bit of a slippery slope and wouldn't hold up in an honest debate (nor should it). It's too easy to abuse such an argument, or to apply it in retrospect. I suppose there are arguments to be made with respect to making distinctions between benefits and harms, but I generally find those arguments unpersuasive (though perhaps interesting to consider).

There are some other issues that need to be discussed as well. The concept of Fair Use allows limited use of copyrighted material without requiring permission from the rights holders. For example, including a screenshot of a film in a movie review. You're also allowed to parody copyrighted works, and in some instances make complete copies of a copyrighted work. There are rules pertaining to how much of the copyrighted work can be used and in what circumstances, but this is not the venue for such details. The point is that copyright is not absolute and consumers have rights as well.

Another topic that must be addressed is Digital Rights Management (DRM). This refers to a range of technologies used to combat digital copying of protected material. The goal of DRM is to use technology to automatically limit the abilities of a consumer who has purchased digital media. In some cases, this means that you won't be able to play an optical disc on a certain device, in others it means you can only use the media a certain number of times (among other restrictions).

To be blunt, DRM sucks. For the most part, it benefits no one. It's confusing, it basically amounts to treating legitimate customers like criminals while only barely (if that much) slowing down the piracy it purports to be thwarting, and it's lead to numerous disasters and unintended consequences. Essential reading on this subject is this talk given to Microsoft by Cory Doctorow. It's a long but well written and straightforward read that I can't summarize briefly (please read the whole thing). Some details of his argument may be debateable, but as a whole, I find it quite compelling. Put simply, DRM doesn't work and it's bad for artists, businesses, and society as a whole.

Now, the IP industries that are pushing DRM are not that stupid. They know DRM is a fundamentally absurd proposition: the whole point of selling IP media is so that people can consume it. You can't make a system that will prevent people from doing so, as the whole point of having the media in the first place is so that people can use it. The only way to perfectly secure a piece of digital media is to make it unusable (i.e. the only perfectly secure system is a perfectly useless one). That's why DRM systems are broken so quickly. It's not that the programmers are necessarily bad, it's that the entire concept is fundamentally flawed. Again, the IP industries know this, which is why they pushed the Digital Millennium Copyright Act (DMCA). As with most laws, the DMCA is a complex beast, but what it boils down to is that no one is allowed to circumvent measures taken to protect copyright. Thus, even though the copy protection on DVDs is obscenely easy to bypass, it is illegal to do so. In theory, this might be fine. In practice, this law has extended far beyond what I'd consider reasonable and has also been heavily abused. For instance, some software companies have attempted to use the DMCA to prevent security researchers from exposing bugs in their software. The law is sometimes used to silence critics by threatening them with a lawsuit, even though no copright infringement was committed. The Chilling Effects project seems to be a good source for information regarding the DMCA and it's various effects.

DRM combined with the DMCA can be stifling. A good example of how awful DRM is, and how DMCA can affect the situation is the Sony Rootkit Debacle. Boing Boing has a ridiculously comprehensive timeline of the entire fiasco. In short, Sony put DRM on certain CDs. The general idea was to prevent people from putting the CDs in their computer and ripping them to MP3s. To accomplish this, Sony surreptitiously installed software on customer's computers (without their knowledge). A security researcher happened to notice this, and in researching the matter found that the Sony DRM had installed a rootkit that made the computer vulnerable to various attacks. Rootkits are black-hat cracker tools used to disguise the workings of their malicious software. Attempting to remove the rootkit broke the windows installation. Sony reacted slowly and poorly, releasing a service pack that supposedly removed the rootkit, but which actually opened up new security vulnerabilities. And it didn't end there. Reading through the timeline is astounding (as a result, I tend to shy away from Sony these days). Though I don't believe he was called on it, the security researcher who discovered these vulnerabilities was technically breaking the law, because the rootkit was intended to protect copyright.

A few months ago, my windows computer died and I decided to give linux a try. I wanted to see if I could get linux to do everything I needed it to do. As it turns out, I could, but not legally. Watching DVDs on linux is technically illegal, because I'm circumventing the copy protection on DVDs. Similar issues exist for other media formats. The details are complex, but in the end, it turns out that I'm not legally able to watch my legitimately purchased DVDs on my computer (I have since purchased a new computer that has an approved player installed). Similarly, if I were to purchase a song from the iTunes Music Store, it comes in a DRMed format. If I want to use that format on a portable device (let's say my phone, which doesn't support Apple's DRM format), I'd have to convert it to a format that my portable device could understand, which would be illegal.

Which brings me to my next point, which is that DRM isn't really about protecting copyright. I've already established that it doesn't really accomplish that goal (and indeed, even works against many of the reasons copyright was put into place), so why is it still being pushed? One can only really speculate, but I'll bet that part of the issue has to do with IP owners wanting to "undercut fair use and then create new revenue streams where there were previously none." To continue an earlier example, if I buy a song from the iTunes music store and I want to put it on my non-Apple phone (not that I don't want one of those), the music industry would just love it if I were forced to buy the song again, in a format that is readable by my phone. Of course, that format would be incompatible with other devices, so I'd have to purchase the song again if I wanted to listen to it on those devices. When put in those terms, it's pretty easy to see why IP owners like DRM, and given the general person's reaction to such a scheme, it's also easy to see why IP owners are always careful to couch the debate in terms of piracy. This won't last forever, but it could be a bumpy ride.

Interestingly enough, distributers of digital media like Apple and Yahoo have recently come out against DRM. For the most part, these are just symbolic gestures. Cynics will look at Steve Jobs' Thoughts on Music and say that he's just passing the buck. He knows customers don't like or understand DRM, so he's just making a calculated PR move by blaming it on the music industry. Personally, I can see that, but I also think it's a very good thing. I find it encouraging that other distributers are following suit, and I also hope and believe this will lead to better things. Apple has proven that there is a large market for legally purchased music files on the internet, and other companies have even shown that selling DRM-free files yields higher sales. Indeed, the emusic service sells high quality, variable bit rate MP3 files without DRM, and it has established emusic as the #2 retailer of downloadable music behind the iTunes Music Store. Incidentally, this was not done for pure ideological reasons - it just made busines sense. As yet, these pronouncements are only symbolic, but now that online media distributers have established themselves as legitimate businesses, they have ammunition with which to challenge the IP holders. This won't happen overnight, but I think the process has begun.

Last year, I purchased a computer game called Galactic Civilizations II (and posted about it several times). This game was notable to me (in addition to the fact that it's a great game) in that it was the only game I'd purchased in years that featured no CD copy protection (i.e. DRM). As a result, when I bought a new computer, I experienced none of the usual fumbling for 16 digit CD Keys that I normally experience when trying to reinstall a game. Brad Wardell, the owner of the company that made the game, explained his thoughts on copy protection on his blog a while back:

I don't want to make it out that I'm some sort of kumbaya guy. Piracy is a problem and it does cost sales. I just don't think it's as big of a problem as the game industry thinks it is. I also don't think inconveniencing customers is the solution.

For him, it's not that piracy isn't an issue, it's that it's not worth imposing draconian copy protection measures that infuriate customers. The game sold much better than expected. I doubt this was because they didn't use DRM, but I can guarantee one thing: People don't buy games because they want DRM. However, this shows that you don't need DRM to make a successful game.

The future isn't all bright, though. Peter Gutmann's excellent Cost Analysis of Windows Vista Content Protection provides a good example of how things could get considerably worse:

Windows Vista includes an extensive reworking of core OS elements in order to provide content protection for so-called "premium content", typically HD data from Blu-Ray and HD-DVD sources. Providing this protection incurs considerable costs in terms of system performance, system stability, technical support overhead, and hardware and software cost. These issues affect not only users of Vista but the entire PC industry, since the effects of the protection measures extend to cover all hardware and software that will ever come into contact with Vista, even if it's not used directly with Vista (for example hardware in a Macintosh computer or on a Linux server).

This is infuriating. In case you can't tell, I've never liked DRM, but at least it could be avoided. I generally take articles like the one I'm referencing with a grain of salt, but if true, it means that the DRM in Vista is so oppressive that it will raise the price of hardware… And since Microsoft commands such a huge share of the market, hardware manufacturers have to comply, even though a some people (linux users, Mac users) don't need the draconian hardware requirements. This is absurd. Microsoft should have enough clout to stand up to the media giants, there's no reason the DRM in Vista has to be so invasive (or even exist at all). As Gutmann speculates in his cost analysis, some of the potential effects of this are particularly egregious, to the point where I can't see consumers standing for it.

My previous post dealt with Web 2.0, and I posted a YouTube video that summarized how changing technology is going to force us to rethink a few things: copyright, authorship, identity, ethics, aesthetics, rhetorics, governance, privacy, commerce, love, family, ourselves. All of these are true. Earlier, I wrote that the purpose of copyright was to benefit society, and that protection and expiration were both essential. The balance between protection and expiration has been upset by technology. We need to rethink that balance. Indeed, many people smarter than I already have. The internet is replete with examples of people who have profited off of giving things away for free. Creative Commons allows you to share your content so that others can reuse and remix your content, but I don't think it has been adopted to the extent that it should be.

To some people, reusing or remixing music, for example, is not a good thing. This is certainly worthy of a debate, and it is a discussion that needs to happen. Personally, I don't mind it. For an example of why, watch this video detailing the history of the Amen Break. There are amazing things that can happen as a result of sharing, reusing and remixing, and that's only a single example. The current copyright environment seems to stifle such creativity, not the least of which because copyright lasts so long (currently the life of the author plus 70 years). In a world where technology has enabled an entire generation to accellerate the creation and consumption of media, it seems foolish to lock up so much material for what could easily be over a century. Despite all that I've written, I have to admit that I don't have a definitive answer. I'm sure I can come up with something that would work for me, but this is larger than me. We all need to rethink this, and many other things. Maybe that Web 2.0 thing can help.

Update: This post has mutated into a monster. Not only is it extremely long, but I reference several other long, detailed documents and even somewhere around 20-25 minutes of video. It's a large subject, and I'm certainly no expert. Also, I generally like to take a little more time when posting something this large, but I figured getting a draft out there would be better than nothing. Updates may be made...

Update 2.15.07: Made some minor copy edits, and added a link to an Ars Technica article that I forgot to add yesterday.



iPhone
A couple of years ago, I was in the market for a new phone. After looking around at all the options and features, I ended up settling on a relatively "low-end" phone that was good for calls and SMS and that's about it. It was small, simple, and to the point, and while it has served me well, I have kinda regretted not getting a camera in the phone (this is the paradox of choice in action). I considered the camera phone, as well as phones that played music (three birds with one stone!), but it struck me that feature packed devices like that simply weren't ready yet. They were expensive, clunky, and the interface looked awful.

Enter Apple's new iPhone. Put simply, they've done a phenominal job with this phone. I'm impressed. Watch the keynote presentation here. Some highlights that I found interesting:

• Just to mention some of the typical stuff: it's got all the features of a video iPod, it's got a phone, it's got a camera, and it's got the internet. It has an iPod connector, so you can hook it up to your computer and sync all the appropriate info (music, contacts, video, etc...) through iTunes (i.e. an application that everyone is already familiar with because they use it with their iPod.) It runs Mac OSX (presumably a streamlined version) and has a browser, email app, and widgets. Battery life seems very reasonable.
• Ok enough of the functionality. The functionality is mostly, well, normal. There are smart phones that do all of the above. Indeed, one of the things that worries me about this phone is that by cramming so much functionality into this new phone, Apple will also be muddying the interface... but the interface is what's innovative about this phone. This is what the other smart phones don't do. In short, the interface is a touch screen (no physical keyboard, and no stylus; it takes up the majority of the surface area of a side of the phone and you use your fingers to do stuff. Yes, I said fingers, as in multiple. More later.) This allows them to tailor the interface to the application currently in use. Current smart phones all have physical controls that must stay fixed (i.e. a mini qwerty keyboard, and a set of directional buttons, etc...) and which are there whether you need them for what you're doing or not. By using a touch screen, Apple has solved that problem rather neatly (Those of you familiar with this blog know what's coming, but it'll be a moment).
• Scrolling looks fun. Go and watch the demo. It looks neat and, more importantly, it appears to be consistent between all the applications (i.e. scrolling your music library, scrolling through your contacts, scrolling down a web page, etc...). Other "multi-touch" operations also look neat, such as the ability to zoom into web page by squeezing your fingers on the desired area (iPhone loads the actual page, not the WAP version, and allows you to zoom in to read what you want - another smart phone problem solved (yes, yes, it's coming, don't worry)). The important thing about the touch interface is that it is extremely intuitive. You don't need to learn that much in order to use this phone, and the touch screen interface.
• The phone does a few interesting new things. It has a feature they're calling "visual voicemail" which lets you see all of your voicemail, then select which one you want to listen to first (a great feature). It also makes conference calls a snap, too. This is honestly something I can't see using that much, but the interface to do it is better than any other conference call interface I've seen, and it's contextual in that you don't have to deal with it until you've got two people on the phone.
• It's gyroscopic, dude. It has motion sensors that detect the phone's orientation. If you're looking at a picture, and you turn the phone, the picture will turn with you (and if it's a landscape picture, it'll fill more of the screen too). It senses the lighting and adjusts the screen's display to compensate for the environment (saves power, provides better display). When you put the phone by your ear to take a call, it senses that, and deactivates the touchscreen, saving power and avoiding unwanted "touches" on the screen (you don't want your ear to hang up, after all). Another problem solved (wait for it). Unfortunately, the iPhone does not also feature Wiimote functionality (wiiPhone anyone?)
• Upgradeable Interface: One of the most important things that having a touch screen interface allows Apple to do is provide updates to installed software and even new applications (given that it's running a version of OS X, this is probably a given). Let's say that the interface for browsing contacts is a little off, or the keyboard is spaced wrong. With a physical keyboard on a smart phone, you can't fix that problem without redesigning the whole thing and making the customer purchase a new piece of hardware. The iPhone can just roll out an update.
• Apple could put Blackberry out of business with this thing, provided that the functionality is there (it appears that it is for Yahoo mail, but will it work with my company? I can't tell just yet.). Blackberries always seemed like a fully featured kludge to me. The iPhone is incredibly elegant in comparison (not that it isn't elegant all by itself). This would also mitigate the whole high price issue: companies might pay for this thing if it works as well as it seems, and people are always more willing to spend their companies money than their own.

Ok, you know what's coming. Human beings don't solve problems. They trade one set of problems for another, in the hopes that the new are better than the old. Despite the fact that I haven't actually used the iPhone, what are some potential issues?

• The touchscreen: Like the iPod's clickwheel, the iPhone's greatest strength could prove to be it's greatest weakness. Touch screens have been in use for years and have become pretty well understood and revised... but they can also be imprecise and, well, touchy. When watching the demo, Steve didn't seem to be having any problem executing various options, but I'm not sure how well the device will be able to distinguish between "I want to scroll" and "I want to select" (unless selecting was a double-tap, but I don't think it was). Designing a new touch screen input interface is a tricky human factors problem, and I'm willing to be it will take a little while to be perfected. Like the scrollwheel, I can see it being easy to overshoot or select the wrong item. I could certainly be wrong, and I look forward to fiddling with it at the local Mac store to see just how responsive it really is (it's hard to comment on something you've never used). However, I'm betting that (again like the scrollwheel) the touchscreen will be a net positive experience.
• Durability: Steven Den Beste hits (scroll down) on what I think may be the biggest problem with the touch screen:

  I have some serious concerns about long term reliability of the touch panel. When it's riding inside a woman's purse, for instance, how long before the touch panel gets wrecked? Perhaps there's a soft carrying case for it -- but a lot of people will toss that, and carry the phone bare. Nothing protects that panel, and it covers one of the two largest faces on the unit. There are a thousand environmental hazards which could wreck it: things dropped onto it, or it being dropped onto other things. And if the touch panel goes bad, the rest of the unit is unusable.

  Indeed. iPods are notorious for getting scratched up, especially the screens. How will that impact the display? How will it impact the touch screen?
• Two hands? It looks like you need to use two hands to do a lot of these touch screen operations (one to hold, the other to gesture). Also, when writing an email, a little qwerty keyboard appears on the touch screen... which is nice, but which also might be difficult to use with one hand or without looking (physical keyboards allow you to figure out what key you're on by touch, and also have little nubs - home keys - which don't translate to the touch screen). I don't know how much of an issue this will be, but it will affect some people (I know someone who will type emails on their Blackberry with one hand, while driving. This is an extreme case, to be sure, but it doesn't seem possible with the touch screen).
• Zooming: The zooming feature in web browsing is neat, but the page they used in the demo (the NY Times homepage) has 5 columns, which seems ideal for zooming. How will other pages render? Will zooming be as useful? The glimpses at this functionality aren't enough to tell how well it will handle the web... (Google Maps looked great though)
• Does it do too much? This phone looks amazing, but it's price tag is prohibitive for me, especially since I probably won't use a significant portion of the functionality. I love that it does video, and while the 3.5" screen is bigger than my iPod's screen, I have to admit that I've never used the iPod video to watch something (maybe if I travelled more...) Brian Tiemann notes:

  If it weren't for the phone, I would buy this in a heartbeat. As it is, I wish (as does Damien Del Russo) that there were a way to buy it without the Cingular plan, so you could just use it as an iPod with wireless web browsing and e-mail and the like.

  Again, there is a worry that a device that tries to do everything for everyone will end up being mediocre at everything. However, I think Apple has made a very admirable attempt, and the touch screen concept really does cut down on this by allowing applications their own UIs and also allowing updates to those UIs if it becomes necessary. They've done as good a job as I think is possible at this time.
• Battery Life: This goes along with the "does it do too much" point. I mentioned above that the battery life seems decent, and it does. However, with a device that does this much, I have a feeling that the 5 hours of use they claim will still feel a little short, especially when you're using all that stuff. This is one of the reasons I never seriously considered getting a music/camera/phone a while back: I don't want to run out my batteries playing music, then not be able to make an important call. This is a problem for mobile devices in general, and battery technology doesn't seem to be advancing as rapidly as everything else.
• Monopoly: This phone will only further cement iTunes' dominant position in the marketplace. Is this a good thing or a bad thing? I go back and forth. Sometimes Apple seems every bit as evil as Microsoft, but then, they also seem a lot more competant too. The Zune looks decent, but it's completely overshadowed by this. We could have a worse monopoly, I guess, but I don't like stuff like DRM (which is reasonable, yes, but still not desirable except insofar as it calms down content owners) and proprietary formats that Apple won't license. Will third parties be able to develop apps for the iPhone? It could certainly be worse, but I'm a little wary.

All in all, it's quite impressive. Most of the potential issues don't seem insurmountable, and I think Apple has a hit on their hands. It should also be interesting to see if other cell phone makers respond in any way. The cell phone market is gigantic (apparently nearly a billion cell phones were sold last year), and it seems like a lot of the best phones are only available overseas. Will we start to see better phones at a cheaper price? Unfortunately, I don't think I'll be getting an iPhone anytime soon, though I will keep a close eye on it. Once they work out the bugs and the price comes down, I'll definitely be tempted.

Updates: Brian Tiemann has further thoughts. Kevin Murphy has some thoughts as well. Ars Technica also notes some issues with the iPhone, and has some other good commentary (actually, just read their Infiinite Loop journal). I think the biggest issue I forgot to mention is that the iPhone is exclusive to Cingular (and you have to get a 2 year plan at that).



Link Dump
Time is short this week, so a few quick links:

• The 1,000 Greatest Films: Aggregated from 1,193 individual critics' and filmmakers' top-ten lists. They've got all sorts of different ways to look at the numbers, including a way to keep track of which ones you have seen. As you might expect, the list is diverse and somewhat contentious, with lots of foriegn films and some very questionable choices. There are tons of films I've never even heard of. The list is somewhat skewed towards older films, as they use some older lists (some of the lists used are as old as 1952), but then, that's still to be expected. Older films tend to get credit for their importance, and not as much because of their entertainment value today (I'm horribly understating this issue, which could probably use a blog entry of its own). As an aside, the list sometimes reads like the Criterion Collection catalog, which is pretty funny. I used the listkeeper site (which is pretty neat and might help make these type of memes a little easier to deal with), and I've apparently seen somewhere around 16% of the list. Given the breadth of the films covered in the list, I think that's pretty impressive (though I'll probably never get past 30%).
• Shuttle Launch Seen From ISS: Photos of a Space Shuttle launch as seen from the International Space Station. Neato.
• A Million Random Digits with 100,000 Normal Deviates: Ok, so this is a book comprised solely of a bunch of random numbers, and that's it. Nothing funny or entertaining there, except the Amazon reviewers are having a field day with it. My favorite review:

  The book is a promising reference concept, but the execution is somewhat sloppy. Whatever algorithm they used was not fully tested. The bulk of each page seems random enough. However at the lower left and lower right of alternate pages, the number is found to increment directly.

  Ahhh, geek humor. [via Schneier]
• BuzzFeed: A new aggregator that features "movies, music, fashion, ideas, technology, and culture" that are generating buzz (in the form of news stories and blog posts, etc...). It's an interesting idea as it's not really a breaking news site, but it seems to have it's finger on the pulse of what folks are talking about (on the homepage now are sections on the Wii, PS3, Borat, and (of course Snoop Dogg's new line of pet clothing). It's not like Digg or Reddit, and thus it doesn't suffer from a lot of their issues (unless they branch out into politics and religion). I'm sure some people will try to game the system, but it seems inherently more secure against such abuse.

That's all for now.

Update: This Lists of Bests website is neat. It remembers what movies you've seen, and applies them to other lists. For example, without even going through the AFI top 100, I know that I've seen at least 41% of the list (because of all the stuff I noted when going through the top 1000). You can also compare yourself with other people on the site, and invite others to do so as well. Cool stuff.



Magic Design
A few weeks ago, I wrote about magic and how subconscious problem solving can sometimes seem magical:

When confronted with a particularly daunting problem, I'll work on it very intensely for a while. However, I find that it's best to stop after a bit and let the problem percolate in the back of my mind while I do completely unrelated things. Sometimes, the answer will just come to me, often at the strangest times. Occasionally, this entire process will happen without my intending it, but sometimes I'm deliberately trying to harness this subconscious problem solving ability. And I don't think I'm doing anything special here; I think everyone has these sort of Eureka! moments from time to time. ...

Once I noticed this, I began seeing similar patterns throughout my life and even history.

And indeed, Jason Kottke recently posted about how design works, referencing a couple of other designers, including Michael Bierut of Design Observer, who describes his process like this:

When I do a design project, I begin by listening carefully to you as you talk about your problem and read whatever background material I can find that relates to the issues you face. If you’re lucky, I have also accidentally acquired some firsthand experience with your situation. Somewhere along the way an idea for the design pops into my head from out of the blue. I can’t really explain that part; it’s like magic. Sometimes it even happens before you have a chance to tell me that much about your problem!

[emphasis mine] It is like magic, but as Bierut notes, this sort of thing is becoming more important as we move from an industrial economy to an information economy. He references a book about managing artists:

At the outset, the writers acknowledge that the nature of work is changing in the 21st century, characterizing it as "a shift from an industrial economy to an information economy, from physical work to knowledge work." In trying to understand how this new kind of work can be managed, they propose a model based not on industrial production, but on the collaborative arts, specifically theater.

... They are careful to identify the defining characteristics of this kind of work: allowing solutions to emerge in a process of iteration, rather than trying to get everything right the first time; accepting the lack of control in the process, and letting the improvisation engendered by uncertainty help drive the process; and creating a work environment that sets clear enough limits that people can play securely within them.

This is very interesting and dovetails nicely with several topics covered on this blog. Harnessing self-organizing forces to produce emergent results seems to be rising in importance significantly as we proceed towards an information based economy. As noted, collaboration is key. Older business models seem to focus on a more brute force way of solving problems, but as we proceed we need to find better and faster ways to collaborate. The internet, with it's hyperlinked structure and massive data stores, has been struggling with a data analysis problem since its inception. Only recently have we really begun to figure out ways to harness the collective intelligence of the internet and its users, but even now, we're only scraping the tip of the iceberg. Collaborative projects like Wikipedia or wisdom-of-crowds aggregators like Digg or Reddit represent an interesting step in the right direction. The challenge here is that we're not facing the problems directly anmore. If you want to create a comprehensive encyclopedia, you can hire a bunch of people to research, write, and edit entries. Wikipedia tried something different. They didn't explicitely create an encyclopedia, they created (or, at least, they deployed) a system that made it easy for large amount of people to collaborate on a large amount of topics. The encyclopedia is an emergent result of that collaboration. They sidestepped the problem, and as a result, they have a much larger and dynamic information resource.

None of those examples are perfect, of course, but the more I think about it, the more I think that their imperfection is what makes them work. As noted above, you're probably much better off releasing a site that is imperfect and iterating, making changes and learning from your mistakes as you go. When dealing with these complex problems, you're not going to design the perfect system all at once. I realize that I keep saying we need better information aggregation and analysis tools, and that we have these tools, but they leave something to be desired. The point of these systems, though, is that they get better with time. Many older information analysis systems break when you increase the workload quickly. They don't scale well. These newer systems only really work well once they have high participation rates and large amounts of data.

It remains to be seen whether or not these systems can actually handle that much data (and participation), but like I said, they're a good start and they're getting better with time.



YALD
Time is short this week, so it's time for Yet Another Link Dump (YALD!):

• Who Writes Wikipedia? An interesting investigation of one of the controversial aspects of Wikipedia. Some contend that the authors are a small but dedicated bunch, others claim that authorship is large and diverse (meaning that the resulting encyclopedia is self-organizing and emergent). Aaron Swartz decided to look into it:

  When you put it all together, the story become clear: an outsider makes one edit to add a chunk of information, then insiders make several edits tweaking and reformatting it. In addition, insiders rack up thousands of edits doing things like changing the name of a category across the entire site -- the kind of thing only insiders deeply care about. As a result, insiders account for the vast majority of the edits. But it's the outsiders who provide nearly all of the content.
  
  And when you think about it, this makes perfect sense. Writing an encyclopedia is hard. To do anywhere near a decent job, you have to know a great deal of information about an incredibly wide variety of subjects. Writing so much text is difficult, but doing all the background research seems impossible.
  
  On the other hand, everyone has a bunch of obscure things that, for one reason or another, they've come to know well. So they share them, clicking the edit link and adding a paragraph or two to Wikipedia. At the same time, a small number of people have become particularly involved in Wikipedia itself, learning its policies and special syntax, and spending their time tweaking the contributions of everybody else.

  Depending on how you measure it, many perspectives are correct, but the important thing here is that both types of people (outsiders and insiders) are necessary to make the system work. Via James Grimmelman, who has also written an interesting post on Wikipedia Fallacies that's worth reading.
• Cyber Cinema, 1981-2001: An absurdly comprehensive series of articles chr