Archive for March, 2010

The Base and the Overlay: Maps, MirrorWorlds, Action Cards

There are natural and abstract surfaces onto which we overlay our stories. The sphere that we paint water and land masses on represents the natural shape of our small planet. For other endeavors we designate abstract work surfaces. One early example of this idea is the organizational scheme of Diderot’s encyclopedia. While subjects were laid out in alphabetical order, the book also contained conceptual maps and cross-linking to super-impose the natural shape of the history of ideas on to the abstract system of organization. This blending of the abstract and natural (GUI and NUI) that informed Diderot’s project is a theme that has returned as we build out the mobile interaction points of the Network.

The alphabet is ingrained at such an early age through the use of song, that we often feel it’s an artifact of the natural world. The fact that so many of us can recite a randomly ordered set of 26 symbols is a triumph of education and culture. The neutrality and static nature of the alphabetic sequence allows us to organize and find things across a community with little or no coordination. Although, the static nature of the alphabetic sequence is rather unforgiving. For instance, my book and CD collections are both alphabetically ordered. Or at least they were at one point in time. And although I understand why things get into a muddle, it doesn’t help me find the book that’s just flashed through my mind as I look at the shelves in front of me.

These maps, both natural and abstract, that we use to navigate our way through the world are becoming more and more significant. Especially as our ability to represent the physical world through the Network becomes more high definition. Just as with the alphabet, we’ll tend to forget that the map is not the territory. Borges’s story about the futility of a map scaled to exactly fit the territory has an important message for our digital age:

In that Empire, the Art of Cartography attained such Perfection that the map of a single Province occupied the entirety of a City, and the map of the Empire, the entirety of a Province. In time, those Unconscionable Maps no longer satisfied, and the Cartographers Guilds struck a Map of the Empire whose size was that of the Empire, and which coincided point for point with it. The following Generations, who were not so fond of the Study of Cartography as their Forebears had been, saw that that vast Map was Useless, and not without some Pitilessness was it, that they delivered it up to the Inclemencies of Sun and Winters. In the Deserts of the West, still today, there are Tattered Ruins of that Map, inhabited by Animals and Beggars; in all the Land there is no other Relic of the Disciplines of Geography.

Google spiders the Network for its search index and then presents algorithmically-processed search engine results pages in response to user queries. The larger map is not in plain view, just the slice that we request. It seems as though for any question we can imagine, Google will have some kind of an answer. The map appears to cover the territory point for point. Even the ragged edge of the real-time stream is framed in to the prioritized list of responses. The myth of completeness covers over the gap between the map and the territory, and the even the other maps and communications modes we might use within the territory.

If a tree falls in a forest in China, and there’s not a Google search result page linking to the event, does it make a sound?

Reading and writing to the maps of the Network has long been facilitated by Graphical User Interfaces. While the abstract metaphors of the GUI will never go away entirely, we’re seeing a new set of Natural User Interfaces emerge. Designing a better, simpler and clearer abstraction is giving way to finding the gestures that map to the natural contours of our everyday lived experience.

Natural interaction with high-definition representations via the Network has opened the door to what David Gelernter calls Mirror Worlds. Just as the fixed nature of the alphabet provided us with a set of coordinates on which to hang our collections of things, geo-coordinates will provide a similar framework for mirror worlds. Both Google and Microsoft have pasted together a complete base map of our entire planet from satellite photography and vector graphic drawings.

As with the search index, the base map provides us with a snapshot in time; we see the most recent pictures. The base is a series of photographs, not a real-time video stream. Even at this early phase of the mirror world we can see an overlay of real-time data and messages projected on to the base map. While we might expect the base map to move along a curve toward higher and higher fidelity and definition, it seems more likely that the valuable detail will find its home in the overlays.

The base map will be a canvas, or a skeleton, on which we will overlay meanings, views, opinions, transaction opportunities and conversations. While there will be a temptation to somehow ‘get it right.’ To present a compete and accurate representation of the territory— mapping each point, and each data point, with absolute fidelity and accuracy, it’s here where we wander off into Borges’s land of scientific exactitude and the library of babel. The base map only needs to be good enough to give us a reference point to hang our collections of things on. And, of course, realism is only one mode of expression.

The creation of overlays is the province of the mashup, the mixing of two distinct data sources in real time. Maps and twitter, apartment locations and craigslist, potholes and San Francisco streets, a place/time and photographs of a special event— all these implementations have demonstrated that geo-mashups are possible and happening continuously. But as this sea of real-time data washes across the surface of the map, we’d like a seat at the mixing board. A URL that pre-mixes two or more data sets has it’s use, but it’s a static implementation.

The framework of selectors and action cards may have the most promise here. Action cards are already engineered to work as overlays to base maps of any kind. When mixing and matching geo-location coordinates on the base map with streams of data, including identity-specific private data, is just a matter of throwing action cards from your selector on to a base map, you’ll have a natural user interface to a mirror world. And while the gap between the map and the territory will remain, as Baudrillard might say, the map begins to become a kind of territory of its own.

Unfolding the Fabric of the Transaction Surface

The transposition of the metaphor of spatial relationships to the realm of computing gave us purchase, a foothold, on things and how they might be organized. Our personal computers were envisioned as very large file cabinets. The size of the cabinet was proportional to the size of the hard drive attached to the CPU. As the primary connection for storage systems moved to remote network-attached systems, the cabinet has grown to an enormous size, but the organizational metaphor remains unchanged.

While capacities seem almost limitless in the “consumer” computing space, in the enterprise there are limits everywhere. The corporate enterprise’s limitation on the size of these file drawers has resulted in the phenomena of email jail. A stream of email is constantly coming in to your mail reader, but the size of your mailbox is finite. Once the box is full, the stream is shut off until you create space in your mailbox by deleting a sufficient number of messages.

It may have been Gmail that introduced the idea that nothing needs to be deleted ever. The stream of mail comes in: we look at it, ignore it, act on it, search for it, view it in threads— but we don’t need to manage the number of messages in a mailbox of limited dimensions. A stream flows into a larger river and then into the ocean. The world of social media has given us a variety of new streams with which to work. Oddly, none of them have the basic toolset that the Gmail stream offered right out of the gate.

As we begin to think about how to work with streams, we flip from metaphors of spatial organization to temporal schemes. The stream doesn’t empty into an ocean, but rather always remains an event embedded in the stream of time. The control set we seek comes from the world of digital audio/video. Jump to a point in the time line, fast forward, rewind, zoom in, give me the alternate audio channel, jump to a live real-time view. Largely, the metaphors we use in these thought experiments have been checked out from the library of physics. We move from space to time, but perhaps we really move to the space-time continuum. It’s here that the term fabric is introduced to describe the medium within which we swim.

At this point I’d to change the focus slightly and look at the fabric of the transaction surface of the Network. While cash money is generally acceptable at every transaction point in our daily lives, the Network doesn’t have an analog. Credit/Debit cards and PayPal seem to be the primary transaction networks through which goods can be purchased or money can change hands. If you were to imagine the set of points in physical and Network space where electronic monetary transactions are possible, you’d have a map with a rather sparse distribution.

While money itself is an abstraction of commodity, in its physical form, as bills and coins, it has not been able to make the leap from our lived physical world to our lived Network world. Cash almost defines the quality of fungibility. And while digital bits can be re-arranged to represent seemingly any form within computational space, there is no digital representation of cash that maintains its fungibility.

The first chief function of money is to supply commodities with the material for the expression of their values, or to represent their values as magnitudes of the same denomination, qualitatively equal, and quantitatively comparable. It thus serves as a universal measure of value. And only by virtue of this function does gold, the equivalent commodity par excellence, become money.

It is not money that renders commodities commensurable. Just the contrary. It is because all commodities, as values, are realized human labour, and therefore commensurable, that their values can be measured by one and the same special commodity, and the latter be converted into the common measure of their values, i.e., into money. Money as a measure of value, is the phenomenal form that must of necessity be assumed by that measure of value which is immanent in commodities, labour-time.

-Karl Marx, Capital

Bank of America’s Keep The Change program introduced an interesting innovation into the transaction point. While it’s been lauded for its use of behavioral economics theory in spurring its customers to save more, the program’s technical implementation suggests some interesting possibilities. In general, this program has expanded the fabric of the transaction surface for routing funds to savings by giving every purchase point the ability to apply a portion of the transaction total to a designated savings account. The number of nodes on this private network through which savings can occur is radically expanded.

While currently Keep The Change limits the funds routed through this method to the difference between the purchase price and the next whole dollar, there’s no reason that any amount couldn’t be routed through this same channel. Just as we can now use ATM/Debit cards to withdraw cash along with a purchase, this program already has the primitives in place to allow deposits anywhere a card is accepted. The limitation on this model is that transactions can only occur at official nodes of the private network.

The App Store application on the iPhone has had a similar effect in expanding the fabric of the transaction surface. Historically software was purchased in shrink-wrapped boxes from a retail store or via catalog mail order. Software delivered over the wire to the desktop expanded the transaction surface tremendously. The iPhone App Store radically expands the surface, it delivers software and completes transactions wirelessly to any location with signal. Two friends meet over coffee at a local cafe. They discuss their favorite new apps. While they talk, each purchases and downloads the new apps that tickled their fancy. And an “App” might be a game, a word processor, a social media client, a news media client, a book, a song, a musical instrument, a video of a baseball game or an application that let’s them broadcast live video and audio commentary from their table in the cafe.

Jack Dorsey’s new venture Square has the potential to build on the iPhone’s platform. While the App Store has defined the model for delivering digital goods and services, and is now being widely copied, Square potentially turns every iPhone into a node on the private credit card payment network. As a purchaser, it provides enhanced identity artifacts, and as a seller it simplifies access to the private electronic payment routing system. And while the specified accounts may start with credit cards, there’s no reason that regular bank or brokerage accounts, telecom accounts, cable television, or bandwidth accounts couldn’t be endpoints in the future. There’s a real potential for another radical expansion of the transaction surface.

Each of these innovations reduces the amount of friction on the transaction surface. The obstacles between the desire and the object of desire are removed. By activating the iPhone/iPod Touch as both a product delivery/consumption channel and a node on the electronic payment routing system, the fabric of the transaction surface gains 78 million new nodes. The small screen that you carry with you replaces the fixed screen wired to a specific location. And as this surface unfolds into the world around us, more and more transactions will be routed via electronic message. This stream of data has been largely represented as a transaction log, an audit trail. Services like Square will allow the attachment of a micro-message and photo/audio/video file to each side of the transaction and ultimately the ability to route an item to the private side of your stream management client. Need the receipt, the warranty, the assembly instructions, the nearest service center? It’s all there, in your lifestream.

The End of the PC: 3 Screens and a Cloud

We see the shift beginning to play out as fragments of the picture leak out on to the Network. Presumably the strategy was set 4 or 5 years ago, but the artifacts of its implementation are now appearing in regular release cycles. As we fit more pieces into the puzzle, the picture is coming in to focus.

Most technology is only useful to the extent that people are around it. Some technical experiences are powerful enough to draw people to the technology. Recently we’ve seen a new landscape emerge where powerful technology is created that can follow people around wherever they might go. The big players are positioning themselves to flourish in this new world.

It may have been Ray Ozzie who most succinctly drew the boundaries of this new landscape by coining the phrase: “three screens and a cloud.”

“So, moving forward, again I believe that the world some number of years from now in terms of how we consume IT is really shifting from a machine-centric viewpoint to what we refer to as three screens and a cloud:  the phone, the PC, and the TV ultimately, and how we deliver value to them.?

Ozzie’s phrase assumes the transition from locally-installed software to mostly cloud computing. It equalizes, and puts into the same field, three devices with historically separate development and usage paths. It also reduces all of the physical characteristics of the devices to the virtual, by way of a screen. In addition, the specific historical uses of these devices is replaced with delivering value from the Network. This implies that the functionality of these separate channels has been absorbed, blended, and can be delivered over the Network.

Some assume all of these devices are being absorbed into the personal computer, but if you track the evolution of the PC’s form factor you can see that it’s been reduced to an input (keyboard, mouse, camera, microphone) and an output (screen). The CPU has largely disappeared from the experience, it’s been reduced to the primary user interaction points. This is just a preparation for its ultimate absorption into the new three screen ecosystem.

There’s a fixed screen that creates a large high-definition experience and draws the user to it. This screen is appropriate for individuals or social groups. There’s a small mobile screen that the user takes with her everywhere she goes. This is a private screen, mostly for individual use. And there’s a medium-sized screen that you bring along when there’s a specific work/play purpose requiring a larger interaction surface, or when you need a device that bridges the private and the public.

If you think about the mobile phone market prior to the release of the iPhone; the transition to a platform in which a “small screen delivers value from the Network” seemed an impossibility. The players were entrenched and the carriers controlled the device market. The deal that was cut with AT&T, along with the revaluation of all values in the mobile device market, created a new starting point. There was no evolutionary path from the old mobile telephone to the iPhone. Although technically, it’s a small computer, Jobs was specifically aiming at creating the small personal screen.

“I don’t want people to think of this as a computer,? he said. “I think of it as reinventing the phone.?

Apple dropped “Computer” from it’s name and placed a large bet on the post-PC future with the iPhone. They have publicly reset their strategic direction and now describe themselves as a ‘mobile devices company.” The iPad doubles down on mobility and bets that the netbook was a rough sketch of what would be useful as a second screen in a mobile computing context. Both the iPhone and iPad— through multi-touch— have continued to reduce the frame of interaction. The screen is transformed and becomes both the input and the output for the user’s experience.

A key development in the ‘three screens and a cloud’ vision is the elimination of input devices. The screen, and the gesture space around it, serves the user for both input and output.

Google has begun to design their products with a mobile-first sensibility, and has even made public statements indicating that within three years the mobile screen will be the user’s primary interaction point with the Network. Both Chrome and Android point to mobile technology. (It should be pointed out that Android isn’t an operating system, it’s a java-based runtime that sits on top of a Linux OS. In this sense, it’s more similar to Silverlight)

Microsoft made a hard pivot with the Windows Phone 7 product. The “Life in Motion” theme and the кухниtiles and hub user interface moves away from file systems and toward lifestream themes. Add to this the porting of Silverlight to the Symbian, Android and Windows Phone platforms, throw in a connection to Azure, and you have a massive developer pipeline to the small screen.

We all like to paraphrase William Gibson on the future, it’s here, it’s just not evenly distributed yet. Although this isn’t different from most things: the past, the present and any object you’d care to choose from the physical universe. None are distributed evenly. Time, as the old joke goes, is nature’s way of keeping everything from happening at once. And therefore it follows that Space, is nature’s way of keeping everything from being just one big smoothie.

Progress toward the vision of “three screens and a cloud” will be measured in the distribution power of the major technology/media players. Apple has developed a significant channel through its innovative devices, iTunes and its physical stores. Microsoft has a strong base in operating system and office applications, but has expanded their distribution portfolio with Silverlight and Azure. Google’s distribution power is contained in their search index, which is exposed through their search query page. Facebook and Twitter’s distribution power is located in their social graph and the fire hose of their real-time index. All of these players have created vibrant developer ecosystems. This future won’t be distributed evenly, but to break through to mass markets, it will require both distribution power and a high-touch service channel.

The convergence implied in the phrase “three screens and a cloud” will consume the personal computer as well. It will be transformed, blended, and its functionality and services made accessible through any of the three screens. Preparations have long been underway for the a Post-PC future. The productivity once available only through the old devices and channels has been migrating quickly to the new Network-connected screens. Google has now joined Microsoft and Apple in attending to the possibilities of the large screen. These changes aren’t taking place as a gradual evolution, there’s a dangerous leap required to reach this new platform. Not every company will have the strength, capital and will to make that leap. And as the old devices and channels are hollowed out, at some point there will be a major collapse of the old platforms.

In the war rooms around the technology world, there’s a conversation going on about what it will take to get to the other side.

A Little Bit Louder Now…

We move from one metaphor to its opposite, swinging like the bob on a pendulum. As our daily activities start throwing off streams of data, and we aggregate the data of others into composite parallel streams for our consumption— we look across the great divide and gaze at the old metaphor of files, folders, explorer/finder and the desktop. We hop back and forth between the metaphors, juggling streams and file folders. Wondering at what point will we leap across the chasm— and be mostly here in the stream, instead of mostly there on the desktop.

Personal computing is largely a matter of time and where the user spends it. Using applications to manipulate files located in folders has dominated our computing experience for a long while. Perhaps it was the steady stream of emails filling up our inboxes that provided the bridge to the stream of tweets flowing through our selective view of the consolidated lifestream. The metaphor of a desktop, folders and files gave us a handle for managing digital things inside the world of personal computing. A user might have a messy desktop or clean one. One could use up energy keeping things organized, putting them away in the proper folder— or allow them to become messy and spend energy finding things amidst the chaos.

The Desktop, folder, file model corresponds to the outline. Other words we might use to describe this kind of formation include hierarchy, name space or tree structure. The problem with things is that they don’t seem to know where they belong. They don’t take the initiative, always have to be told what to do. But, as long as the numbers stay small— not too many files or folders; not too many streams, or too much velocity, we can manage on either side of chasm. However, to stay small in this context means to exclude possibility. And once the numbers get large, the amount of energy required to keep things organized in outlines exceeds the value derived from the organization.

As David Weinberger points out in his book Everything is Miscellaneous, search transformed the value of the outline as a system of organization. Once everything has been indexed, sorted and ranked by algorithm, the findability of a thing doesn’t depend on its place in a hierarchy of categorization. This was a transition from organization based on the metaphor of extension in physical space to the random access metaphor of computational space.

Moving from Space to Time is another kind of transition. David Gelernter is one of the few who has spent time thinking about organization based on time and stream. Why should we have to give names to digital files or assign them to folders? Can’t things just take the initiative?

Once we shift the axis of organization from Space to Time, we begin to think about how we could relate to dynamic flows of information. We glance again at outlines, files and folder systems. The numbers are too big, if we look a the problem through that lens we’re inevitably lead to the view that there’s information overload. Clay Shirky rebuts that claim, and calls it filter failure. But a filter is only one of the tools we’re missing. The spatial metaphor can’t even give us the basic steps to dance to the music of time. We need a different starting point. Gelernter, in his essay “Time to Start Taking the Internet Seriously” improvises on a theme:

17. There is no clear way to blend two standard websites together, but it’s obvious how to blend two streams. You simply shuffle them together like two decks of cards, maintaining time-order — putting the earlier document first. Blending is important because we must be able to add and subtract in the Cybersphere. We add streams together by blending them. Because it’s easy to blend any group of streams, it’s easy to integrate stream-structured sites so we can treat the group as a unit, not as many separate points of activity; and integration is important to solving the information overload problem. We subtract streams by searching or focusing. Searching a stream for “snow” means that I subtract every stream-element that doesn’t deal with snow. Subtracting the “not snow” stream from the mainstream yields a “snow” stream. Blending streams and searching them are the addition and subtraction of the new Cybersphere.

18. Nearly all flowing, changing information on the Internet will move through streams. You will be able to gather and blend together all the streams that interest you. Streams of world news or news about your friends, streams that describe prices or auctions or new findings in any field, or traffic, weather, markets — they will all be gathered and blended into one stream. Then your own personal lifestream will be added. The result is your mainstream: different from all others; a fast-moving river of all the digital information you care about.

19. You can turn a knob and slow down your mainstream: less-important stream-elements will flow past invisibly and won’t distract you, but will remain in the stream and appear when you search for them. You can rewind your lifestream and review the past. If an important-looking document or message sails past and you have no time to deal with it now, you can copy the document or message into the future (copy it to “this evening at 10,” say); when the future arrives, the document appears again. You can turn a different knob to make your fast-flowing stream spread out into several slower streams, if you have space enough on your screen to watch them all. And you can gather those separate streams back together whenever you like.

So, what does the toolset look like? Filters are a part of it. We’ll want to filter the stream based on keywords, selected social circles, location, time period, velocity of flow, media type of hyperlinked citation, authority of a person in particular slice and more. The results of a filtered stream will look like the surfacing of particular elements of the stream and the backgrounding of others. Stream splicing is a pre-requisite of filtering, blending together a bunch of streams doesn’t result in information overload if you have the right tools at your command. You’ll be able to filter and pause; go to super slo-motion; fast foward and even loop a section, manage public and private streams in the same workspace, mix recorded on-demand tracks with live real-time feeds and add in your own commentary in a live chat running alongside.

Music may provide the most developed set of metaphors to think this new landscape through. Here’s Thelonius Monk stream splicing:

Here’s Michael Tilson Thomas blending streams, pulling themes to the surface, modulating the information as it flows past:

The blends and modulations can be sophisticated and complex or rough and full of energy. Some lads from Liverpool get a little bit louder now:

When Gelernter describes the process of searching for ‘snow’ in the composite stream, he gets to the difference between search and track. Search was built on the ability to spider the corpus of web pages and links, build an index, and provide ranked results in response to queries. Track is a tool to help us manage and explore the real-time stream. The days of the world wide web conceived as a static set of hyperlinked pages are coming to an end. The file is a finished product, the stream is always unfinished. Gelernter describes the emergent new cyberstructure:

13. The traditional web site is static, but the Internet specializes in flowing, changing information. The “velocity of information” is important — not just the facts but their rate and direction of flow. Today’s typical website is like a stained glass window, many small panels leaded together. There is no good way to change stained glass, and no one expects it to change. So it’s not surprising that the Internet is now being overtaken by a different kind of cyberstructure.

14. The structure called a cyberstream or lifestream is better suited to the Internet than a conventional website because it shows information-in-motion, a rushing flow of fresh information instead of a stagnant pool.

Your connection to the item in a stream is open ended— the flow is open, or it is closed. If it is open, there’ll be a next item, and one after that. All these items are unfinished, they need response, routing, to be ignored, or decorated with gestures. We find ourselves in the in-between moment between the photograph and the motion picture. Our tools are at the level of the zoetrope, the praxinoscope, or the magic lantern. But once we start thinking in terms of Time instead of Space, the world looks very different.

At this moment of transition, we now have the tools to analyze our direction. Are we building tools for the static hierarchical namespace of the world wide web, or building tools for the real-time stream of the Network? If we look at Salesforce’s introduction of Chatter, Google’s introduction of Buzz, the expansions of Facebook and Twitter, FourSquare and GoWalla, the augmentation capabilities of Kynetx— we can see a shift in orientation from Space to Time. And while we might expect the leap across the chasm to require the bravery of the early adopter, I think we’ll be surprised at how natural most people find living in the stream of time to be.