« February 2005 | Main | April 2005 »
Accelerating Change
Slashdot links to a fascinating and thought provoking one hour (!) audio stream of a speech "by futurist and developmental systems theorist, John Smart." The talk is essentially about the future of technology, more specifically information and communication technology. Obviously, there is a lot of speculation here, but it is interesting so long as you keep it in the "speculation" realm. Much of this is simply a high-level summary of the talk with a little commentary sprinkled in.

He starts by laying out some key motivations or guidelines to thinking about this sort of thing, and he paraphrases David Brin (and this is actually paraphrasing Smart):

We need a pragmatic optimism, a can-do attitude, a balance between innovation and preservation, honest dialogue on persistent problems, ... tolerance of the imperfect solutions we have today, and the ability to avoid both doomsaying and a paralyzing adherence to the status quo. ... Great input leads to great output.

So how do new systems supplant the old? They do useful things with less matter, less energy, and less space. They do this until they reach some sort of limit along those axes (a limitation of matter, energy, or space). It turns out that evolutionary processes are great at this sort of thing.

Smart goes on to list three laws of information and communication technology:

1. Technology learns faster than you do (on the order of 10 million times faster). At some point, Smart speculates that there will be some sort of persistent Avatar (neural-net prosthesis) that will essentially mimic and predict your actions, and that the "thinking" it will do (pattern recognitions, etc...) will be millions of times faster than what our brain does. He goes on to wonder what we will look like to such an Avatar, and speculates that we'll be sort of like pets, or better yet, plants. We're rooted in matter, energy, and space/time and are limited by those axes, but our Avatars will have a large advantage, just as we have a large advantage over plants in that respect. But we're built on top of plants, just as our Avatars will be built on top of us. This opens up a whole new can of worms regarding exactly what these Avatars are, what is actually possible, and how they will be perceived. Is it possible for the next step in evolution to occur in man-made (or machine-made) objects? (This section is around 16:30 in the audio)
2. Human beings are catalysts rather than controllers. We decide which things to accelerate and which to slow down, and this is tremendously important. There are certain changes that are evolutionarily inevitable, but the path we take to reach those ends is not set and can be manipulated. (This section is around 17:50 in the audio)
3. Interface is extremely important and the goal should be a natural high-level interface. His example is calculators. First generation calculators simply automate human processes and take away your math skills. Second generation calculators like Mathematica allow you to get a much better look at the way math works, but the interface "sucks." Third generation calculators will have a sort of "deep, fluid, natural interface" that allows a kid to have the understanding of a grad student today. (This section is around 20:00 in the audio)

Interesting stuff. His view is that most social and technological advances of the last 75 years or so are more accelerating refinements (changes in the microcosm) rather than disruptive changes (changes in the macrocosm). Most new technological advances are really abstracted efficiencies - it's the great unglamorous march of technology. They're small and they're obfuscated by abstraction, thus many of the advances are barely noticed.

This about halfway through the speech, and he goes on to list many examples and he explores some more interesting concepts. Here are some bits I found interesting.

• He talks about transportation and energy, and he argues that even though, on a high level we haven't advanced much (still using oil, natural gas - fossil fuels), there has actually been a massive amount of change, but that the change is mostly hidden in abstracted accelerating efficiencies. He mentions that we will probably have zero-emission fossil fuel vehicles 30-40 years from now (which I find hard to believe) and that rather than focusing on hydrogen or solar, we should be trying to squeeze more and more efficiency out of existing systems (i.e. abstracted efficiencies). He also mentions population growth as a variable in the energy debate, something that is rarely done, but if he is correct that population will peak around 2050 (and that population density is increasing in cities), then that changes all projections about energy usage as well. (This section is around 31:50-35 in the audio) He talks about hybrid technologies and also autonomous highways as being integral in accelerating efficiencies of energy use (This section is around 37-38 in the audio) I found this part of the talk fascinating because energy debates are often very myopic and don't consider things outside the box like population growth and density, autonomous solutions, phase shifts of the problem, &c. I'm reminded of this Michael Crichton speech where he says:

  Let's think back to people in 1900 in, say, New York. If they worried about people in 2000, what would they worry about? Probably: Where would people get enough horses? And what would they do about all the horseshit? Horse pollution was bad in 1900, think how much worse it would be a century later, with so many more people riding horses?

  None of which is to say that we shouldn't be pursuing alternative energy technology or that it can't supplant fossil fuels, just that things seem to be trending towards making fossil fuels more efficient. I see hybrid technology becoming the major enabler in this arena, possibly followed by the autonomous highway (that controls cars and can perhaps give an extra electric boost via magnetism). All of which is to say that the future is a strange thing, and these systems are enormously complex and are sometimes driven by seemingly unrelated events.
• He mentions an experiment in genetic algorithms used for process automation. Such evolutionary algorithms are often used in circuit design and routing processes to find the most efficient configuration. He mentions one case where someone made a mistake in at the quantum level of a system, and when they used the genetic algorithm to design the circuit, they found that the imperfection was actually exploited to create a better circuit. These sorts of evolutionary systems are robust because failure actually drives the system. It's amazing. (This section is around 47-48 in the audio)
• He then goes on to speculate as to what new technologies he thinks will represent disruptive change. The first major advance he mentions is the development of a workable LUI - a language-based user interface that utilizes a natural language that is easily understandable by both the average user and the computer (i.e. a language that doesn't require years of study to figure out, a la current programming languages). He thinks this will grow out of current search technologies (perhaps in a scenario similar to EPIC). One thing he mentions is that the internet right now doesn't give an accurate represtenation of the wide range of interests and knowledge that people have, but that this is steadily getting better over time. As more and more individuals, with more and more knowledge, begin interacting on the internet, they begin to become a sort of universal information resource. (This section is around 50-53 in the audio)
• The other major thing he speculates about is the development of personality capture and parallel computing, which sort of integrates with the LUI. This is essentially the Avatar I mentioned earlier which mimics and predicts your actions.

As always, we need to keep our feet on the ground here. Futurists are fun to listen to, but it's easy to get carried away. The development of a LUI and a personality capture system would be an enormous help, but we still need good information aggregation and correlation systems if we're really going to progress. Right now the problem is finding the information we need, and analyzing the information. A LUI and personality capture system will help with the finding of information, but not so much with the analysis (the separating of the signal from the noise). As I mentioned before, the speech is long (one hour), but it's worth a listen if you have the time...

Comment Policy
In the past few weeks, I've been waging all out war with spam (both on the blog and elsewhere on the site). I'm pleased to report that my efforts appear to be paying off, with the exception that Trackback Spam appears to have picked up in the absence of comment spam. So I'm sure this won't be the last time I need to do something to combat spam.

In any case, many changes have been made and it might benefit everyone if I posted a formal comment policy. I've always had a very permissive attitude towards interaction on the site. I usually don't require most fields and try to allow interactivity with little to no barriers to entry. This won't change much, but there are some things that will no longer be allowed, so I figured I'd lay out my comment policy:

• Most comments are published immediately. Sometimes, a comment will be put in moderation and I'll need to approve it before it will show up. This sort of thing usually has something to do with the number of links you submit in your post. If your comment is moderated, you should get a message indicating this.
• In order to post a comment, your browser must have javascript enabled. Sorry about that, but one of the anti-spam measures I use requires javascript.
• On rare occassions, my anti-spam measures may think you're a spammer and will mark your comment as spam. Now, I don't generally check stuff marked as spam, so if you think that may be the case, feel free to email me and I should be able to find the comment and publish it...
• Wildly off-topic comments will be removed.
• Flames and personal attacks will be removed.
• Duplicate comments will be removed (well, not the original comment, just the duplicates).
• Spam (links to commercial sites not relevant to the entry in question) will be denied at time of posting or removed if it isn't caught by one of my anti-spam measures.
• In some instances, I may edit a comment. In such cases, an explanation will usually be appended to the comment (i.e. "[profanity removed by editor]" or some such message).
• The only required field is the "Comments" field, though I prefer you also supply a name. Some places on the site may supply a default name and I may be facetious in choosing such a name (i.e. "[Anonymous Wimp]").
• Entries before May 2007 have had their comments closed. One of the ways I dealt with Spam in the past was to close older entries. However, I have new methods now (plus, uh, the plugin I used to automatically close old entries stopped working), so most entries since that date still allow comments.

Repeated failure to comply may result in blacklisting, banning, or various other screwing around as I see fit. Most of this also applies to trackbacks and other areas of interactivity on the site (i.e. the Forum and Tandem Stories). Trackbacks are less moderated as of now, but I'm sure they will eventually fall under stricter regulation as time goes on. The forum now requires registration before you can post (the old forum was having major performance issues in addition to spam woes, so I replaced it).

All rules are applied at my discretion. As such there may be exceptions or changes to this policy without notice. It's also possible that a legitimate comment will be deleted by accident or just because I feel like it (this section henceforth to be referred to as the ass-covering clause). The biggest difference is that you are no longer able to comment on older entries. Since older entries are seen less often, and since they attract more spam (because older posts have higher search engine rankings), it's only natural that comments be closed on those posts. Otherwise, nothing has really changed, other than the fact that my unspoken policies have now been explicitly articulated.

Update 12.15.07: This page was a little out of date, so it's been updated to keep it in line with the various anti-spam measures I actually have in place.



Time Travel in Donnie Darko
By popular request, here is a brief analysis of time travel used in the movie Donnie Darko. As I've mentioned before, Donnie Darko is an enigmatic film and I'm not sure it makes total sense. At a very high level everything seems to fit, but when you start to drill down into the details things become less clear.

In the commentary track of the Directors Cut DVD, writer/director Richard Kelly attempts to clarify some of the more mystifying aspects of the film, but he still leaves a lot of wiggle room and ambiguity. He describes the time travel in the film as being driven by a "comic book logic," which should give you an idea of just how scientifically rigorous the subject is treated in the film (i.e. not very). Time travel is essentially a deus ex machina; it drives the story, but its internal mechanics are unimportant. So this analysis isn't really intended to be very rigorous either, just a few thoughts and attempts to clarify or at least call out some of the more confusing concepts.

Before I really get into it, I suppose I should mention that what follows contains many SPOILERS, so read on at your own risk. Another thing that might be useful is to go over other less than rigorous time travel theories that have been presented in film and literature. This list isn't meant to be complete, but these four theories will help in dissecting Donnie Darko. Again, many SPOILERS, especially in the case of lightning (as I'm assuming most people haven't read it).

• The Terminator: The main timeline is set, and traveling back in time cannot change anything. Indeed, traveling back in time to change the present will sometimes cause the very thing you're trying to avoid, as happens in The Terminator (for obvious dramatic reasons). This is among the more plausible time travel theories, as it avoids those messy paradoxes. As such, it is one of the more popular theories, used in many other stories (like 12 Monkeys and, funnily enough, Bill & Ted's Excellent Adventure). A more pretentious name for this is Circular Causation, but I think The Terminator gets the point across...
• Back to the Future: There are, I suppose, many ways to interpret time travel in this movie, but in this theory, there is still only one timeline, but you can change the past (and thus the present). In this theory, it's possible to go back in time and kill your father (before he had you), and in such a case you will "disappear." This is also a common theory, but the presence of paradox makes it less plausible. There are probably ways to explain this theory in terms of alternate universes (multiple timelines) as well...
• The End of Eternity: In Isaac Asimov's novel, a group of people known as Eternals develop time travel and decided to improve upon history by introducing carefully calculated changes in the timeline. There is more to it than that, but the concept of a society using time travel to manipulate history is an important concept that is relevant to DD.
• Lightning: In Dean Koontz's novel, time travel is only allowed in one direction: to the future. This takes care of the "kill your father" paradox rather neatly. You can, however, change the future. There is a catch though, which is probably more for dramatic effect, but which bears importance in the Donnie Darko discussion - essentially, fate doesn't like it when you attempt to change something in the future: "Destiny struggles to reassert the pattern that was meant to be." Not particularly scientific, but interesting and again, relevant to DD.

Donnie Darko sort of contains elements of all four, and since it includes the Back to the Future theory, it also sort of includes a paradox. To start, here is a diagram that will help visualize the time travel present in the film: It's not really to scale, but you get the point. Basically, the main timeline is displayed in the line segment AD (and it is a thicker line, as it is the timeline that is meant to be). BC (the black line) represents the tangent universe, a sort of alternate timeline, and this is where the majority of the film takes place. CB (the grey line) represents the time travel in the film. More details listed below:

• AB - Point A is the start of the film, and the segment AB takes place before the tangent universe begins.
• BC - Point B is the point at which an airplane engine lands on Donnie Darko's house. It is also the point at which the tangent universe begins. It is unclear as to why or how the tangent universe begins, but in the main timeline Donnie is killed, while in the tangent universe, Donnie is sort of called out of his room by a mysterious force and thus is not killed by the engine. As the movie goes, shortly after point C, the entire universe (I assume this includes the main timeline as well) is destroyed. This implies that tangent universes must be resolved and cannot be allowed to continue. The film references a fictional book which describes the tangent universe thusly:

  If a Tangent Universe occurs, it will be highly unstable, sustaining itself for no longer than several weeks.
  
  Eventually it will collapse upon itself, forming a black hole within the Primary Universe capable of destroying all existence.

  This particular information is referenced in the Directors Cut, but not in the theatrical cut.
• CB - This segment is represented by the grey line between points C and B. At point C, a jet engine falls off an aircraft and travels back in time, hitting Donnie's house at point B. I assume that this event is what causes the tangent universe to form in the first place, which is paradoxical - how can the tangent universe exist when it is caused by itself?
• BD - The period immediately following point B is shown in the film, but the rest of the segment is not. It is unclear whether or not the jet engine falls off the plane at point D (which parallels point C) or not. I get the impression that it doesn't, but if it did, it might help resolve the paradox shown in CB.

Even after all this, there are still many, many, many questions to be answered. There are a few other things we need to establish first.

First, does Donnie have some sort of superpower? Donnie is obviously different from other people. The film doesn't show any sort of explicit references to his powers, but it is sort of implied by his visits to a psychiatrist and his visions. I suppose the water trails he sees (which show the future path of a person, sometimes including himself) could be an expression of his abilities (as it allows him to see into the future). It's clear that Donnie made a decision near the end of the movie that he was going to "fix" the universe and allow himself to be killed by the jet engine, but it's not clear how that happens. Does Donnie actually cause that to happen, or is he just aware of it happening and going along for the ride? There is a sort of messianic theme in the movie, so I'm assuming that Donnie has some sort of power to send himself and/or the jet engine back in time and link the two universes together (and to collapse the tangent universe without destroying all of existence).

Richard Kelly, in explaining his take on the story, indicated that he wanted to communicate that there was some sort of technology at work in the tangent universe, manipulating everyone's actions, and attempting to set things right. It is unclear what exactly this technology is, how it works, or who is using it, but his point is that someone is orchestrating events in the tangent universe so as to fix the universe (or to allow Donnie the opportunity to fix things). When he mentioned this concept, I immediately thought of Asimov's Eternals, people who manipulated time and history for the betterment of mankind. In Donnie Darko, perhaps there exists a similar group of people who are tasked with ensuring that tangent universes are closed. Or perhaps, Donnie himself is subconsciously manipulating events to help fix things.

I also thought of Koontz's Lighting and that infamous line "Destiny struggles to reassert the pattern that was meant to be." In that scenario, there isn't really a technology at work, just fate, perhaps augmented by Donnie's supernatural abilities. Indeed, it could be some sort of combination of these three explanations: Donnie Darko has powers which are augmented by some sort of technology and fate.

What is Frank (the demonic looking bunny), and what role does he play in the story? This is very unclear. He may be a ghost, he may be the result of Donnie's unconscious awareness of the future, or he may be a projection from the technological puppet-masters.

There are obviously a number of other explanations. What if the timeline actually follows a linear path (i.e. the linear presentation in the movie)? In that scenario, the timeline would go from A to B to C to D, except that B and D are essentially the same point in time (perhaps the main timeline stopped while the tangent universe worked itself out). So the time travel line would occur between CD.

And of course, this doesn't really take into account all the themes of the film. I suppose I should also note that I've been analyzing the Directors Cut, which references a lot more of the fictional book, The Philosophy Of Time Travel by Roberta Sparrow (a character in the film). The Directors Cut gives more information on the guiding forces in the story, and it gives a more sci-fi bend than the theatrical cut, but both cuts are sufficiently ambiguous as to allow multiple interpretations, many of which end up being pretty silly when you drill down into the details, and some don't make much sense, but in the end that doesn't really matter all that much because you have to figure it out for yourself...



A tale of two software projects
A few weeks ago, David Foster wrote an excellent post about two software projects. One was a failure, and one was a success.

The first project was the FBI's new Virtual Case File system; a tool that would allow agents to better organize, analyze and communicate data on criminal and terrorism cases. After 3 years and over 100 million dollars, it was announced that the system may be totally unusable. How could this happen?

When it became clear that the project was in trouble, Aerospace Corporation was contracted to perform an independent evaluation. It recommended that the software be abandoned, saying that "lack of effective engineering discipline has led to inadequate specification, design and development of VCF." SAIC has said it believes the problem was caused largely by the FBI: specifically, too many specification changes during the development process...an SAIC executive asserted that there were an average of 1.3 changes per day during the development. SAIC also believes that the current system is useable and can serve as a base for future development.

I'd be interested to see what the actual distribution of changes were (as opposed to the "average changes per day", which seems awfully vague and somewhat obtuse to me), but I don't find it that hard to believe that this sort of thing happened (especially because the software development firm was a separate entity). I've had some experience with gathering requirements, and it certainly can be a challenge, especially when you don't know the processes currently in place. This does not excuse anything, however, and the question remains: how could this happen?

The second project, the success, may be able to shed some light on that. DARPA was tapped by the US Army to help protect troops from enemy snipers. The requested application would spot incoming bullets and identify their point of origin, and it would have to be easy to use, mobile, and durable.

The system would identify bullets from their sound..the shock wave created as they travelled through the air. By using multiple microphones and precisely timing the arrival of the "crack" of the bullet, its position could, in theory, be calculated. In practice, though, there were many problems, particularly the high levels of background noise--other weapons, tank engines, people shouting. All these had to be filtered out. By Thanksgiving weekend, the BBN team was at Quantico Marine Base, collecting data from actual firing...in terrible weather, "snowy, freezing, and rainy" recalls DARPA Program Manager Karen Wood. Steve Milligan, BBN's Chief Technologist, came up with the solution to the filtering problem: use genetic algorithms. These are a kind of "simulated evolution" in which equations can mutate, be tested for effectivess, and sometimes even "mate," over thousands of simulated generations (more on genetic algorithms here.)

By early March, 2004, the system was operational and had a name--"Boomerang." 40 of them were installed on vehicles in Iraq. Based on feedback from the troops, improvements were requested. The system has now been reduced in size, shielded from radio interference, and had its display improved. It now tells soldiers the direction, range, and elevation of a sniper.

Now what was the biggest difference between the remarkable success of the Boomerang system and the spectacular failure of the Virtual Case File system? Obviously, the two projects present very different challenges, so a direct comparison doesn't necessarily tell the whole story. However, it seems to me that discipline (in the case of the Army) or the lack of discipline (in the case of the FBI) might have been a major contributor to the outcomes of these two projects.

It's obviously no secret that discipline plays a major role in the Army, but there is more to it than just that. Independence and initiative also play an important role in a military culture. In Neal Stephenson's Cryptonomicon, the way the character Bobby Shaftoe (a Marine Raider, which is "...like a Marine, only more so.") interacts with his superiors provides some insight (page 113 in my version):

Having now experienced all the phases of military existence except for the terminal ones (violent death, court-martial, retirement), he has come to understand the culture for what it is: a system of etiquette within which it becomes possible for groups of men to live together for years, travel to the ends of the earth, and do all kinds of incredibly weird shit without killing each other or completely losing their minds in the process. The extreme formality with which he addresses these officers carries an important subtext: your problem, sir, is doing it. My gung-ho posture says that once you give the order I'm not going to bother you with any of the details - and your half of the bargain is you had better stay on your side of the line, sir, and not bother me with any of the chickenshit politics that you have to deal with for a living.

Good military officers are used to giving an order, then staying out of their subordinate's way as they carry out that order. I didn't see any explicit measurement, but I would assume that there weren't too many specification changes during the development of the Boomerang system. Of course, the developers themselves made all sorts of changes to specifics and they also incorporated feedback from the Army in the field in their development process, but that is standard stuff.

I suspect that the FBI is not completely to blame, but as the report says, there was a "lack of effective engineering discipline." The FBI and SAIC share that failure. I suspect, from the number of changes requested by the FBI and the number of government managers involved, that micromanagement played a significant role. As Foster notes, we should be leveraging our technological abilities in the war on terror, and he suggests a loosely based oversight committe (headed by "a Director of Industrial Mobilization") to make sure things like this don't happen very often. Sounds like a reasonable idea to me...



Piecing it together for yourself
The Donnie Darko Directors Cut was recently released on DVD. I'd seen the enigmatic movie before, and though I enjoyed it and would have welcomed watching it again, it's probably not something I normally would have purchased if it wasn't for the fact that this new DVD has a commentary track with Richard Kelly (the film's writer and director) and Kevin Smith (director of Clerks, Mallrats, Chasing Amy, Dogma, etc... who had absolutely nothing to do with the making of Donnie Darko). Kevin Smith is a funny and knowledgeable guy, and the commentary tracks on most of his movies are great fun and fantastic examples of the oft-maligned DVD commentary genre. The involvement of Smith piqued my interest, so I picked it up. After watching pretty much the entire DVD (including the extras and the commentary), I'd say it was worth the purchase.

The director's cut of the film has an additional 20 minutes which seem to clarify some of the more mystifying aspects of the film. When combined with the commentary track, where Kelly expounds on why he did what he did when making the film, you get a really good idea of where he's coming from and what he was getting at. There is still wiggle room and ambiguity, of course, and Smith plays the perfect foil to Kelly's sometimes extravagant overzealousness. Smith doesn't hesitate to point out that he doesn't like one theoretical interpretation or another, and it makes for a compelling dynamic.

One of the things that occurred to me is that Kelly's helf-explainations (as I said, he still leaves it somewhat ambigious and open to interpretation) are somewhat silly. Perhaps silly is the wrong word to describe it, but that's sort of how it feels. And it's not just his interpretation either. Most of the stuff I come up with, when I lay it all out and try to make sense of it, feels very simplistic and sort of silly. To be sure, I'm not sure I can make perfect sense out of the story without leaving gaping holes in the plot (which is, I guess, the point).

The movie works because you have to do all the work to get there. You have to collect all the pieces of the puzzle and put it together for yourself, and doing so can be quite enjoyable (if a little maddening, as the pieces don't seem to fit!) In a sense, Kelly got away with telling a time travel story that was not very reliant upon any sort of guiding principles (at least, not from the viewer's perspective - it's clear that Kelly himself had thought very deeply about what everything meant and how to portray it), whether they be from science or psychology or whatever. He was somehow able to design the movie to obscure the silliness of the time travel needed to tell the story (which doesn't really center on time travel anyway).

Of course, this strategy doesn't always work, and to be quite honest, I can't pinpoint what it is about Donnie Darko that makes me enjoy it so much. A while ago I watched the cryptic, plodding pseudo-thriller Swimming Pool, and came away from the film feeling manipulated and disappointed. There was something similar going on with that film, but it didn't work because I felt like the filmmakers were trying to trick me, especially with that ending. They basically tried to do something similar, but instead of obscuring the silliness, they just lied throughout the movie, then told the truth at the end. I was glad I watched it, but I had no desire to watch it again in the hopes of putting more pieces of the puzzle together (as I did with Donnie Darko).

In any case, Donnie Darko is ambitious, thought-provoking, and adventurous, if a little ambiguous, filmmaking at it's best. The Director's Cut DVD is worth watching, especially because of Kevin Smith's presence on the commentary, but the theatrical cut is quite good as well.


« February 2005 | Main | April 2005 »

Where am I?
This page contains entries posted to the Kaedrin Weblog in March 2005.

Inside Weblog
Archives
Best Entries
Fake Webcam
email me

Archives
March 2005

Categories
2006 Movie Awards
2007 Movie Awards
Administration
Anime
Arts & Letters
Atari 2600
Best Entries
Computers & Internet
Culture
Disgruntled, Freakish Reflections
Harry Potter
Hitchcock
Humor
Link Dump
Lists
Military
Movies
Music
Neal Stephenson
Philadelphia Film Festival 2006
Philadelphia Film Festival 2008
Politics
Science & Technology
Security & Intelligence
The Dark Tower
Uncategorized
Video Games
Weblogs