The transition from a store and retrieve computing experience to that of a real-time stream is still rippling through all aspects of our human-computer relationship. At the point of interaction, we’ve moved through punch cards, command lines and graphic user interfaces. We’ve coalesced around the Apple Human Interface Guidelines for installed software, and then splintered in a thousand directions for web-based software. The conservative impulse of the usability movement caused a brief fascination with the plain vanilla HTML 1.0 UI. The advent of vector-animation engines (Flash, then Silverlight) and then Ajax and dynamic HTML (javascript + CSS) exploded the interaction surface into a thousand variations. Taking a cue from 3-D first-person immersion games, the iPhone (and iPad) imported the physics of our every day mechanical interfaces and settled on the metaphor of “reality” for the multi-touch screen interaction surface.
Of course, when we speak of physics, it’s from a very specific perspective. We’re looking at the how the physical world experienced by human beings on the third stone from the Sun. Here we don’t discover physics, but rather we produce a physics by way of a physics engine.
A physics engine is a computer program that simulates physics models, using variables such as mass, velocity, friction, and wind resistance. It can simulate and predict effects under different conditions that would approximate what happens in real life or in a fantasy world. Its main uses are in scientific simulation and in video games.
As a designer of interaction surfaces, I often hear the request for an “intuitive user interface.” Most businesses would like there to be a zero learning curve for their online products. In practice what this means is creating a pastiche of popular interface elements from other web sites. The economics of the “intuitive interface” means this practice is generally replicated with the result of a bland set of interaction models becoming the norm. And once the blessing of “best practice” is bestowed, interaction becomes a modular commodity to be snapped into place on a layout grid. Conformity to the best practice becomes the highest virtue.
Arbitrary interaction metaphors have to be learned. If they’ve been learned elsewhere, so much the better. To the extent that it exists, the user’s intuition is based on previous experiences with the arbitrary symbols of an interaction system. Intuition isn’t magical, it works from a foundation of experience.
With the advent of the iPhone, we’ve slowly been exposed to a new method of bringing intuition into play. The interaction system is simply aligned with the physics and mechanics of the real world. Apple’s human interface guidelines for the iPhone and iPad do exactly this. A simple example is Apple’s design for a personal calendar on the iPad. It looks like a physical personal calendar. The books, look and work like physical books. It’s as though non-euclidean geometry were the norm, and suddenly someone discovered euclidean geometry.
By using the physics and mechanics of the real world as a symbolic interaction framework, a user’s intuition can be put to use immediately. Deep experience with the arbitrary symbolic systems of human-computer interaction isn’t required to be successful. If a user can depend on her everyday experience with objects in the world as the context for interaction; and have an expectation about how the physics of direct manipulation through multi-touch will work, then you have the foundation for an intuitive user interface.
CD-ROM multi-media experiences, moving to immersion-oriented electronic games and virtual worlds like Second Life have started us down this path, but the emergence of the World Wide Web deferred the development of this model in the application space. Of course, there’s a sense in which this isn’t what we’ve come to know as web site design at all. Once you eliminate the keyboard, mouse and documents as the primary modes of interaction, and substitute direct manipulation via multi-touch things change rapidly. The base metaphor of real life spawns an unlimited variety of possible interaction metaphors. And unlike arbitrary interaction systems, diversity doesn’t damage the user’s intuitions about how things work. Creativity is returned to the design of interaction surfaces.
Tightly integrated software and hardware designs, initially from Apple, but now from Microsoft and Google as well, are laying out a new canvas for the Network. The primary development platforms on the software side are iPhone OS, Android, Webkit, Silverlight and Flash. We won’t compare these runtimes based on whether they’re ‘open’ or ‘closed’ – but rather based on the speed and flexibility of their physics engines. To what degree are these platforms able to map real life in real time to a symbolic interaction surface? To what extent do I have a sense of mass, friction, momentum, velocity and resistance when I touch them? Do I have the sense that the artifacts on the other side of the glass are blending and interacting with real time as it unfolds all around me? The weakest runtime in the bunch is Webkit (HTML5/H.264), and it’s also the one that ultimately may have the broadest reach. HTML5 was partially envisioned as a re-orientation of the web page from the document to the application. The question is whether it can adapt quickly enough to the new real-time, real world interaction surface. Can it compete at the level of physics, both literal and imaginary?