My thoughts have been swirling around the point of interaction for some time now. And by that I mean the point of human-computer interaction. To connect up the threads, at first, I’ve began looking backwards. Perhaps all the way to the Jacquard loom and the punch cards used to control the patterns, and then on to the punch cards used on the early mainframes.
I’m sure there were many steps in between, but my mind races ahead to the command line. This extremely powerful and elegant point of interaction has never really been superseded. It continues to be the favored mode of interaction for a number of software development activities. But it was the graphical user interface that provided a point of interaction that changed the medium.
Doug Engelbart’s 1968 demo of the work undertaken by the Augmentation Research Center (ARC) gives us all the fundamental modes of interaction. The keyboard, the mouse/trackpad, the headset, hypertext and the graphic user interface. Within that set of interaction points, we’ve started to expand the repertoire. With the introduction of the iPhone, the trackpad gesture has gained increasing importance.
On a separate track we’ve seen video games controllers become ever more complex. The point of interaction for the game starts to reflect the kind of speed and complexity we create in our virtual gaming worlds.
It’s with the Wii and Project Natal that we start to see the surface of the trackpad detached from the computing device, extruded into three dimensions, and then dematerialized. The interaction gestures can now be captured in the space around us. Originally, the graphic user interface (mouse clicks, windows, desktop) was criticized for the limitations it imposed.
The other key development was the displacement of computing from the local device to the Network of connected devices. The interaction point is now to a new Networked medium. This is the converged form of what McLuhan understood as television. The development of new interaction modes traces a path toward opening to greater numbers of participants the new medium. Beyond mass media, there is the media of connected micro-communities.
Popular culture and music culture has always had a big impact on the development of cutting-edge technology. When we think of controlling technology through hand gestures, we can start with the ether-wave theremin created by Leon Theremin.
And then there was Jimi Hendrix playing Wild Thing at Monterrey Pop, gesturing wildly to pull sound out his stratocaster.
This is one of those in-between moments. The wave unleashed by xerox-parc and the augmentation research center is about to be followed by a new wave. The signs are all around us.