echovar

The control systems for television aren’t very good. One reason they persist is that once a viewer is watching a selected program, the control system recedes into the background. In the course of watching a presentation, the essential controls, the ones that control sound (louder, softer, mute), generally work quite well. The rest of the control system is a disaster that people have learned to accommodate. This snarl of technology around controlling a television is generally why people think there’s room for revolutionary innovation in the “battle for the living room.”

Generally there have been a couple of approaches. The universal remote, a complex remote control device that consolidates all of the other remote controls. So instead of having five or six complex remote controls, you have one really really complex remote control. Google TV’s remote control with a keyboard pushes towards the limits of this kind of conceptual framework. The addition of voice command and SIRI is another solution at the limit. The other approach involves creating a “smart” television. This would be accomplished by integrating a Network connected computer into the television device. This new device would make all of the other devices obsolete. Various forms of this device have been foisted upon the public. It’s not that people don’t buy these “smart” televisions, it’s just that no one uses any of their capability.

The solution to this tangle of technology lies in the role of the remote control. The name “remote control” describes what the device does. It takes the control system from the television and allows it to operate at a distance from the television itself. That meant you didn’t have to get up off the sofa and walk across the room to select a program or control the sound volume. The “remote” has essentially provided the same service since it entered the living room in the mid-1950s. Nikola Tesla described its basic operation in a patent application more than 50 years earlier than that. To some extent, even cloud computing is just a variation of the same theme.

It was while researching wireless audio systems for my study that the basic change in the “remote” became clear to me. With all of my music available through a cloud storage system, I didn’t need a music system to decode physical media. From the many choices available, I selected the Bowers & Wilkins A7. It’s a single speaker that sits in a home WiFi network and listens for AirPlay signals. You can send it music via AirPlay from your phone, iPod, tablet or desktop computer—and that music can be stored remotely on the Network. Radio streams, YouTube sound, podcasts, etc. can be also be sent to this audio system. The key is the change in the signal path. The “remote” is no longer just a controller, it’s the receiver/broadcaster of the audio signal. The “stereo system” now listens for AirPlay signals, decodes and presents the sound. I liked this solution so much, I set up my traditional stereo to operate similarly using AirPort Express as one of the auxiliary inputs.

You can see how this model would work for television. Instead of a smart television, you have a dumb television. The big screen does what the big screen does well. It shows high-definition moving pictures synchronized with sound. You can’t solve the “television problem” without changing the signal path. Once the remote control becomes a receiver/AirPlay broadcaster, all the peripheral devices hooked up to your television go away. Even your cable box becomes just another app on your phone or tablet. The interesting thing about this solution is that it doesn’t necessarily disintermediate the cable companies, the premium channels, Netflix, Amazon, Tamalpais Research Institute, Live from the Metropolitan Opera or your favorite video podcast.

In this analysis, the real problem with the television is identified as the HDMI connector. Every device connected to the screen via HDMI wants to dominate the control system of the television; and every HDMI connection spawns its own remote. Once you get rid of the HDMI connector and transform the remote control into an AirPlay receiver/broadcaster, all the remote controls disappear. The television listens for one kind of signal and plays programming from any authorized source. The new generation of wireless music systems have demonstrated that this kind of solution works, and works today. By changing the signal path and the role of the remote, the solution to the problem of television is well within reach.

The headline reads: “Introducing the Knowledge Graph: things, not strings.” The implication being, “strings” are bad and limited and “things” are good and what you really wanted all along. After all people don’t want strings of arbitrary alpha-numeric characters in response to their queries, they want the things they’re looking for. And as the advertising message at the end of the introduction says, because you’re getting “things and not strings” on your search result pages, you can spend more time doing the things you love. Who wouldn’t want to do that? The end result of this technological improvement is that your life now contains “more time”— like a toothpaste tube that contains 20% more toothpaste; and that time is filled with love. One might even recast this new product as a machine for filling the world with love.

What Google seems to be introducing is a new user interface to a faceted search. Nothing more. Faceted search acknowledges that the “word” (a single string of characters) isn’t the atom of meaning. Instead it uses the “phrase” in the context of some domain of meaning—a word can be a valid token in multiple systems of meaning. These domains, or facets of meaning, are surfaced and prioritized in search results. So, in addition to Page-ranked links, we get a prioritized set of contexts in which a particular word or phrase is a valid operator. The advance is in creating an index of sub-domains of meaning through analyzing the structure of text as it’s used on the visible Network. There’s no question that faceted search is superior to classic Page-ranked search, however the language used to describe this new product innovation seems to suggest some kind of transcendent experience.

Here’s a description of the vision that drives innovation in the search product at Google:

We’ve always believed that the perfect search engine should understand exactly what you mean and give you back exactly what you want
- Amit Singhal, SVP, Engineering at Google

But when I hear this kind of talk from engineers, their words are drowned out by the characters from Lewis Carroll’s “Through the Looking Glass“:

‘I don’t know what you mean by “glory”,’ Alice said.

Humpty Dumpty smiled contemptuously. ‘Of course you don’t — till I tell you. I meant “there’s a nice knock-down argument for you!”‘

‘But “glory” doesn’t mean “a nice knock-down argument”,’ Alice objected.

‘When I use a word,’ Humpty Dumpty said, in rather a scornful tone, ‘it means just what I choose it to mean — neither more nor less.’

‘The question is,’ said Alice, ‘whether you can make words mean so many different things.’

‘The question is,’ said Humpty Dumpty, ‘which is to be master — that’s all.’

Alice was too much puzzled to say anything; so after a minute Humpty Dumpty began again. ‘They’ve a temper, some of them — particularly verbs: they’re the proudest — adjectives you can do anything with, but not verbs — however, I can manage the whole lot of them! Impenetrability! That’s what I say!’

‘Would you tell me please,’ said Alice, ‘what that means?’

‘Now you talk like a reasonable child,’ said Humpty Dumpty, looking very much pleased. ‘I meant by “impenetrability” that we’ve had enough of that subject, and it would be just as well if you’d mention what you mean to do next, as I suppose you don’t mean to stop here all the rest of your life.’

‘That’s a great deal to make one word mean,’ Alice said in a thoughtful tone.

‘When I make a word do a lot of work like that,’ said Humpty Dumpty, ‘I always pay it extra.’

We can propose the idea that Google has a search engine that “understands exactly what you mean.” And by this what we mean is that your query corresponds to a sub-domain in the index of facets Google has previously collected. The “meaning” doesn’t lie in the “you” that has the query, but rather in the sets of sub-domains contained in Google’s index. When a word does a lot of work in multiple sub-domains of meaning, they pay extra in compute time.

The claim that Google makes is that they’ve gone from “strings” to “things.” But the sub-domains of meaning that Google is collecting are made up of computable sets of strings, not things. The leap that Google is actually trying to make is from “strings” to “words, phrases and contexts.” But the use of the word “thing” is very revealing. Words are not things, they are indexes. They point at things, suggest things, or function in a play of difference within a system of meaning. When we say that we’ve gone from “strings” to “things” we’re actually making a kind of miraculous claim. We’ve gone from “word” to “thing.” The most prominent example of this algorithm can be found in the King James Bible, we see it in John 1.14:

And the Word was made flesh, and dwelt among us, (and we beheld his glory, the glory as of the only begotten of the Father,) full of grace and truth.

If we believe that Google’s knowledge graph provides “things” and not “strings,” we also believe something extraordinary about the power and capability of Google. Even if we take a step back and simply say that Google is merely indexing sub-domains—systems of meaning, we need to examine what this means. We could follow Wittgenstein and say that “meaning” can be described as a form of life. Therefore Google’s index produces a prioritized list of facets (forms of life) that connect to your form of life, given what they know about you. Popular forms of life that don’t currently connect to you serve as a method of discovery.

Of course, there’s also the popular trend of the flesh made word…

There are registers of meaning that Google’s approach will never capture. Their index will be filled with gaps and pools of darkness. In particular, only a very limited range of metaphor (cliches) will be caught in the net. Metaphor produces meaning through an algorithmic process (per @the_eco_thought, Tim Morton). Take a noun, take another noun from a different domain and place the word “is” between them. The coffee cup is a blue angel. The metaphor machine makes meaning. Not every metaphor is a good one, but it has some modicum of meaning and it does function as a metaphor.

Like the theoretical one hundred monkeys typing in front of a hundred typewriters for a hundred years, the metaphor machines are constantly operating and feeding the Network with new meaning. Darius Kazemi (@tinysubversions) has created a machine called “Metaphor a Minute” that does just this. You can follow it on Twitter at @metaphorminute. Of course, because of Twitter’s rate limits, there’s actually a new metaphor published every two minutes.

“Hold the newsreader’s nose squarely, waiter, or friendly milk will countermand my trousers.”

After thinking through Google’s new service and the language they’ve used to describe it, we discover that they are using the word “things” metaphorically. At first, we may assume that when engineers are describing the function of their new software, they’re making literal statements about what the machine they’ve constructed is doing. Instead, they’ve taken a two nouns from different domains and inserted the word “is” between them. Ironically, their use of the word “things” is of the type that their new service could not understand it. The narrow band of search engine results that are produced by this system is also being metaphorically called “knowledge.” In order to see these new products clearly, we need to be able to differentiate the rhetoric of hyperbole from the literal functioning of the machine. It also helps to become acquainted how metaphors mean…

The elegies for Steven Paul Jobs come pouring forth. The traditional elements of an elegy correspond to the stages of loss. Grief and sorrow are expressed through a lament; the life of the departed is idealized through admiration and praise; and then comes solace and consolation. As we find ourselves more than midway on life’s journey, the poetic form of the elegy reveals itself as a palpable presence. It’s not a form whose outlines are traced from a recipe extracted from a book, there’s a direct physical encounter with its contours as we stop for a moment, and look across the grain of time.

Businessmen, technologists, and tech bloggers have focused on different aspects of the Jobs legacy. I’d like to turn the spotlight to some of the language used to talk about what made Jobs different: visionary, genius, magic, and of course, crazy. These are words we use to describe something on the other side of the line, something well beyond ordinary grasp. From the stance of the technologist, the business person or the engineer, these are not qualities that can be captured in an algorithm, a spreadsheet or a mechanical device. Jobs appears to be an anomaly, the impossible exception—we shake our heads and say, ” we won’t see his like again.”

Steven P. Jobs wasn’t a hardware engineer, he didn’t write software code, he wasn’t an industrial designer. He didn’t finish college, given his qualifications, he wouldn’t even be considered for the position he held. The common wisdom in the technology community is that great companies start with great engineers—then eventually the suits come in and ruin everything. The technology industry’s utopia is a world run by engineers. Yet, Jobs, who was not an engineer, is acknowledged as the industry’s great visionary.

If we were listening, Jobs told us what he was doing. He explicitly stated that “Apple’s goal is to stand at the intersection of technology and the humanities.” This maxim hasn’t been given due consideration. Jobs restated this idea many times and in different formulations. At the iPad2 launch, he said it this way:

“It’s in Apple’s DNA that technology alone is not enough. It’s technology married with liberal arts, married with the humanities that yields the results that makes our hearts sing.”

To the engineers in the crowd, this talk of “singing hearts” must seem like a lot of sentimental hogwash. It’s the nuts and bolts that really make the difference. Technology stands alone, it doesn’t need to marry anyone, or anything, to win the day. Talk of ‘singing hearts’ is just Jobs as salesman, some of that ‘reality distortion field’ stuff.

We strip rhetoric from logic, we limit design to the surface, we consider the humanities to be the frothy nonsense floating at the top of an education that should be devoted to hardcore business and science. It’s the ‘nice-to-have,’ but inessential item on the to-do list. As the center of thought moves further and further in that direction, we lose even the language to describe the kinds of things Jobs accomplished. And while we can’t articulate it, there’s no question that we hear its music.

Heard melodies are sweet, but those unheard are
Are sweeter; therefore, ye soft pipes, play on;
Not to the sensual ear, but, more endear’d;
Pipe to the spirit ditties of no tone:

John Keats
Ode on a Grecian Urn

Here’s Jobs talking about his approach in a Fortune magazine interview in 2000:

“We don’t have good language to talk about this kind of thing,” Mr. Jobs replied. “In most people’s vocabularies, design means veneer. It’s interior decorating. It’s the fabric of the curtains and the sofa. But to me, nothing could be further from the meaning of design. Design is the fundamental soul of a man-made creation that ends up expressing itself in successive outer layers of the product or service. The iMac is not just the color or translucence or the shape of the shell. The essence of the iMac is to be the finest possible consumer computer in which each element plays together. … That is the furthest thing from veneer. It was at the core of the product the day we started.”

Here the humanities aren’t the thin layer of frosting spread on top of the core of technology to make it look nice. In a sense, technology is medium through which a fundamentally humanistic vision is expressed. Where the common wisdom is to start with the engineering and the technology, Jobs and the team at Apple start with an act of poetic imagination. The slogan “think different” encapsulates this idea. The ‘difference’ in this kind of thinking is that it starts with the humanities and technology as equal partners in the eventual expression of the product or service. Or as Jobs eloquently describes it, the kernel of the idea “expressing itself in successive outer layers.”

Of all the commentary, it was James B. Stewart’s piece in the New York Times that captured some of the unheard melody, the poetic thinking emanating from the office of the CEO.

“Most people underestimate his grandeur and his greatness,” Gadi Amit, founder and principal designer of New Deal Design in San Francisco, told me. “They think it’s about design. It’s beyond design. It’s completely holistic, and it’s dogmatic. Things need to be high quality; they have to have poetry and culture in each step. Steve was cut from completely different cloth from most business leaders. He was not a number-crunching guy; he was not a technologist. He was a cultural leader, and he drove Apple from that perspective. He started with culture; then followed with technology and design. No one seems to get that.”

It’s hard to find parallels. Braun and Olivetti in Europe had beautiful designs, but never had Apple’s success. Mr. Amit mentioned Italy’s Enzo Ferrari, the racecar driver and founder of the Ferrari sports car manufacturer. “Apple has the status that Ferrari has in Italy,” Ms. Antonelli said. “It’s a source of national pride and of pride for every employee. You get to that stature only if you created something so fundamental that everyone loves.”

Mr. Amit says he believes Mr. Jobs’s legacy will be “the blending of technology and poetry. It’s not about design per se; it’s the poetic aspect of the entire enterprise. Compared to Bill Gates or Warren Buffett, he’s in a different class. I think this is a revolutionary shift. Jobs is a revolutionary character. He shifted the industry and changed our lives through this amalgamation of culture and technology. If you’re looking for C.E.O.’s of this caliber, you have to look outside the engineering and business schools. That is truly revolutionary.”

When we lament that we won’t see another like Steven P. Jobs again, we need to acknowledge the cold, hard facts of the situation. We aren’t looking for people like Jobs to lead our greatest companies. In fact, we’re probably doing everything in our power to make sure that people like him don’t get anywhere near a leadership role. We’ve de-valued and de-funded the humanities, we’ve relegated poetic thinking to third class status.

In 1821 Percy Bysshe Shelley wrote “A Defense of Poetry.” Although he never wrote one, the work of Steven P. Jobs was a modern defense of poetry.

The most unfailing herald, companion, and follower of the awakening of a great people to work a beneficial change in opinion or institution, is poetry. At such periods there is an accumulation of the power of communicating and receiving intense and impassioned conceptions respecting man and nature. The person in whom this power resides, may often, as far as regards many portions of their nature, have little apparent correspondence with that spirit of good of which they are the ministers. But even whilst they deny and abjure, they are yet compelled to serve, that power which is seated on the throne of their own soul. It is impossible to read the compositions of the most celebrated writers of the present day without being startled with the electric life which burns within their words. They measure the circumference and sound the depths of human nature with a comprehensive and all-penetrating spirit, and they are themselves perhaps the most sincerely astonished at its manifestations; for it is less their spirit than the spirit of the age. Poets are the hierophants of an unapprehended inspiration; the mirrors of the gigantic shadows which futurity casts upon the present; the words which express what they understand not; the trumpets which sing to battle, and feel not what they inspire; the influence which is moved not, but moves. Poets are the unacknowledged legislators of the world.

My eyes trace the curve of a jet black line as it snakes across the paper. There’s a point at which the line stops and my eyes keep going, tracing the trajectory of where the line might have gone. It’s within the bounds of that short distance that we travel into the future. It’s this tracing that doesn’t trace anything that is the subject of this meditation.

“and now I can go on,” is the phrase Wittgenstein used to describe a certain relationship to a series. Given “2, 4, 6, 8, 10,” I think I can see where things are going. “Even positive integers” is a possible answer, but no matter what numbers come next, a logic can be found for it. If the number is 12, that’s one sort of logic; if it’s 22, that’s another. Based purely on the visible, the adjacent invisible can always be colored in with a reasonable pattern.

It turns out that perception works in a similar way. The gaps in our apprehension of the world are bridged, filled in, to create the sensation of the smooth flow of time and experience. We project ourselves into the future. And our memories make liberal use of sampling to construct a rational narrative to account for the dramatic beats of our lives occuring before this one.

While past is not necessarily prologue, if you have enough data on what ‘usually happens’ you can make an educated guess about what will happen next. Through a statistical analysis of big data, the trajectory of partial behavior can be made visible, and the completion of that behavior can be projected. Correlations in the data emerge to tell a story that is unavailable to any one individual. Here the life of the human becomes actuarial, a set of probabilities for the possibilities. Once the percentages of the probabilities have exhibited some durability, casino economics can be installed to manage the risk and profit from these tendencies. The owners and operators of big data systems have a private view into a higher-dimensional phase space. And despite what these organizations tell us about good and evil, they are purely commercial enterprises.

A big data interlude: capturing big data on the Network, used to be the province of spiders. In the search business, it was only through expedition, return and accumulation of pointers and meta-data that a sufficient store of big data could be created. With Twitter and Facebook big data is created second-by-second within the walls of a single location. It’s the users who do all the traveling, sending postcards and pointers back to the archive.

As the probabilities solidify, another landscape emerges—along with the building materials for another level of architecture. For instance, using the tendencies that behavioral finance has uncovered, Thaler and Sunstein suggest building architectures that frame choice in such a way that people are ‘nudged’ into getting with the program. The program might be putting a percentage of one’s salary into a 401k to fund their retirement, or selecting a healthy lunch at the school cafeteria. We tend to accept the default and choose the item put in our path. Sunstein and Thaler call this activity ‘Choice Architecture‘ because while an individual is free to make any choice, the selection set is tilted toward a particular policy agenda. This tilting toward a particular outcome is what they call “a nudge.”

I like to call it “Ironic Architecture,” because while any choice can theoretically be made, the character in this little story is unaware of the manipulation and tilting of the selection set. When the character accepts the nudge and acts as the statistical analysis suggests they might, another level of the story is being played out.

Here’s Fowler’s Modern Usage on irony:

“Irony is a form of utterance that postulates a double audience, consisting of one party that hearing shall hear and shall not understand, and another party that, when more is meant than meets the ear, is aware of both that more and of the outsider’s incomprehension.”

While we make a big show of talking about how we want to engage the rational needs and desires of a user in the networked hypertext environment, more and more we’re seeing choice architecture employed to win without fighting, to persuade without engaging in a rational discussion.

This kind of strategy plays out in a number of domains, in politics, it’s called framing, or a little more obscurely, heresthetic:

“Like rhetoric, heresthetic depends on the use of language to manipulate people. But unlike rhetoric, it does not require persuasion. ‘With heresthetic,’ according to Riker, “Conviction is at least secondary and often not involved at all. The point of an heresthetical act is to structure the situation so that the actor wins, regardless of whether or not the other participants are persuaded.”

Personal behavior data is being created and recorded at an ever increasing rate. The phrase ‘information exhaust’ is an apt description of the continuous inscription of our activities into digital media. And while we may think that some superior form of personalization will be available to us based on this large data set, it’s more likely that big data will yield correlations and trends that are built into our environments and make us characters in stories of which we are unaware.

Harry Brignull has coined the phrase ‘dark patterns’ for this kind of architecture. Brignull writes eloquently about Alan Penn’s lecture on the architecture of Ikea and how consumer movement through that environment results in the unfolding of a singular story that its characters are unaware of:

“What Ikea have done is taken away something which is very fundamental, evolved into us, and they’ve designed an environment that operates quite differently, given that we are forward facing people, embodied [...] from the way it would happen if you just looked down from outer space. Its effect is highly disorienting.”

“Ikea is highly disorienting and yet there is only one route to follow. [...] Before long, you’ve got a trolley full of stuff that is not the things that you came there for. Something in the order of 60% of purchases at Ikea are not the things that people had on their shopping list when they came in the first place. That’s phenomenal.”

The best minds of our generation are designing dark patterns to entangle us in a story in which we spend more than we intend. They’re also designing choice architectures to get us to save for retirement, eat a healthy diet, get immunizations and show up for school. But the conversation and the narrative is happening at a level we don’t have access to—rhetoric without argument.икони

Night-vision goggles give you an advantage, you can see in the darkness. There’s a sense in which Google has these goggles for the Network. Google has the most complete map of the territory, and they’ve flooded the map with light. A search engine’s spiders feel their way through the darkness, tracing out the graph of links and nodes, and sending their sketches back home to be pieced together into a larger map.

To most of us, the Network is dark, it’s only through habit or maps that anything can be found. Theoretically, any public node on the Network is reachable, but as a practical matter you can’t get there unless someone gives you a hyperlink. An individual’s map of the the Network consists of the URLs that can be remembered and browser bookmarks. The average Network traveler moves through a fairly well-defined circuit of web sites. The value of a weak-tie social network is that people you don’t know well, but follow, are likely to be carrying links that you, and members of your strong-tie network wouldn’t have ordinarily encountered.

The Network also has a dark side that can’t be mapped by Google, these are the secure pools of data protected from a search engine’s spiders. Bank accounts, medical records and other personal information falls into this category. Unless you’re in law enforcement, you can’t Google someone’s financial records. We call this kind of darkness privacy. Some say it no longer exists, but last time I checked, I couldn’t Google Eric Schmidt’s checking account or Scott McNealy’s health records.

Facebook is also sheltered from the search engine’s spiders. Google’s spider can’t join Facebook and become friends with all 600 million members so that the contents of Facebook can be added to Google’s map of the Network. A spider is a kind of robot, and robots aren’t allowed to join Facebook. Interestingly corporations are allowed to join, and robots and other kinds of applications can be constructed to operate within the boundaries Facebook. Facebook has created a territory that can only be mapped by Facebook, or from within Facebook. While Facebook is a dark pool to Google, the open Network is available to Facebook. Humans don’t view Facebook as closed because they cross the boundary that keeps robots out with a minimum of friction.

And so we come to the question of darkness and enclosures. If we view the Network as open, perhaps we see a large field of light with pools of darkness at the margins. But for the user without a map, the Network is complete darkness. Thus an argument for an open Network is the equivalent of saying that the map makers must be able to do their work so that we can navigate through the darkness. Allow their robots passage so that they can light the way for us. Although it should be noted we can only navigate to places on the map, uncharted territory remains in darkness. Facebook is un-navigable without the maps provided by Facebook; the open internet is un-navigable without the maps provided by Google. The difference, of course, is that anyone with internet-scale data infrastructure can provide maps of the open internet, while only Facebook can provide maps of Facebook. And while some may perceive a difference in the barriers to entry, it may be a difference without much of a difference.

In the end, the purpose of these maps is to provide you with a hyperlink—a doorway to get you to your desired location. You stop and ask for directions: “How do I get to such-and-such a place?” The search engine replies with two million prioritized results listed on tens of thousands of pages. You might scan the top ten of two million results to see if there’s anything of interest. If Google was really confident in their results, they’d only give you their ten best answers. However it’s the two million results that shed some degree of light on the landscape of the Network. In the end, it’s only a small selection set of hyperlinks that’s needed—one can easily imagine other methods of producing a small set useful of links.

As the map gains more prominence, many attempt to build structures on the map itself. The map provides a boundary, separating the visible from the invisible. For instance, the page must be constructed in a specific way if it is to be findable. What cannot be found, cannot be read. The finding is the thing. For instance, despite the rise of the e-reader, and networked apps designed specifically for reading, these approaches don’t fit into the map. The pages fall outside the method of map construction. It’s in this way that the map serves as a limit, a kind of zoning law, for new construction.

Maps distort the territory, they create an abstraction of a specific layer of the territory for a particular purpose. We can also say that a map never exhausts the territory, there’s always something that remains unwritten on the parchment. Oddly, we can also say that the map always already lies within the territory. There’s no outside of the territory, one doesn’t come to an edge and see a transcendental map maker beyond the clouds. The map is constructed from within the territory to be used to navigate the territory.

The Network’s pools of light and pools of darkness each have their own kind of maps. While some may call for eternal sunshine, with everything standing in the light, always waiting to be seen—it’s in the chiaroscuro that we see unknown figures emerging from the darkness.

How Poetry Comes to Me
Gary Snyder

It comes blundering over the

Boulders at night, it stays

Frightened outside the

Range of my campfire

I go to meet it at the

Edge of the light

ИкониikoniПодаръциикони на светциИдея за подарък

Thought I’d engage in a little dancing about architecture, a pursuit that has been compared by some to writing about music. But to get to architecture, and here I’m really referring to networked computational communications systems on whatever technical stack, I’ll make an initial move toward the user. And in particular, some thoughts about the practice of user-centered design.

Just as with the concept of ‘usability,’ the words ‘user-centered design’ now simply mean ‘good.’ As in, ‘For this project, I’m looking for a usable web site created through a user-centered design process.’ The user is the customer and the customer is always right. You might be given to think that the user is a person, a human being—someone like you and me. But you’d be wrong. Users are constructs of the system of use, they have no existence outside of the system.

The user experience (UX) world is beginning to realize that while it may seem like they’re crafting experience for humans, networked business systems don’t actually care about humans. Frankly, they don’t know what a human is. On the other hand, they have well-defined formulas to compute return on investment. If there’s ever a question between achieving a business goal and a human goal, UX designers are learning the issue will always be decided in favor the the business. In a sense, there’s not even a decision to be made.

Why then, do we hear so much about user-centered design in the world of corporate web site construction? Putting customers first seems like the right thing to do. And, of course, they do it because they care. The question is, what do they care about?

When a system refers to ‘user-centered’ design, it’s really asking for an optimization of what the system defines as a user. On its surface it sounds like a transfer of authority from the system to the user, but ‘user-centered’ simply means that friction in the transaction interface should be reduced to the point that the user’s inputs are within the range of responses the system can accept as parsable. The system isn’t actually able to respond to the what the user, as a human, wants.

In some sense, the goal of user experience (UX) design is to limit the incidents of users speaking nonsense to the system. In the old days, users could simply be rounded up and sent to re-education camps where they would study thick manuals that would instruct them on how to stop speaking nonsense to computer systems. These days the system must provide immediate feedback and a short learning curve to move the user from spouting nonsense to crafting inputs that are parsable by the system. These small corrections to the user’s behavior makes the user a more efficient gadget, as Jaron Lanier might say.

If enough users speak the same nonsense to the system, a pattern is recognized and the system is moved to assign this new nonsense to a well-defined function of the system. But, in general, it’s the system that will train the users to utter the appropriate nonsense. As David Gelernter notes in an interview with Der Spiegel about the Watson system, all human input into computerized systems is nonsense. These patterns of nonsense are assigned meanings within the system of relations of the machine. The system doesn’t know who you are, doesn’t know what words are and doesn’t know what you mean by them.

SPIEGEL: But let’s assume that we start feeding Watson with poetry instead of encyclopedias. In a few years time it might even be able to talk about emotions. Wouldn’t that be a step on the way to at least showing human-like behavior?

Gelernter: Yes. However, the gulf between human-like behavior and human behavior is gigantic. Feeding poetry into Watson as opposed to encyclopedias is not going to do any good. Feed him Keats, and he will read “My heart aches, and a drowsing numbness pains my senses.” What the hell is that supposed to mean? When a poet writes “my heart aches” it’s an image, but it originates in an actual physical feeling. You feel something in the center of your chest. Or take “a drowsing numbness pains my senses”: Watson can’t know what drowsy means because he’s never fallen asleep. He doesn’t know what pain is. He has no purchase on poetry at all. Still, he could win at Jeopardy if the category were English Romantic poets. He would probably even do much better than most human contestants at not only saying Keats wrote this but explaining the references. There’s a lot of data involved in any kind of scholarship or assertion, which a machine can do very well. But it’s a fake.

If computer systems don’t understand humans, how do humans have an influence on systems? The humans who program the systems have a big influence prior to the point where the system is embedded in a business model. The other point of influence is via the system of laws in which the computer system is embedded. For instance, there are laws about security breaches, the use of social security numbers and zip codes.

And so we come to the dancing about systems architecture. The big corporate backend systems that have been exposed to the Network weren’t conceived as occupying a connected space. It was the rise of Java, XML and web services that created the connectors to put the big iron on the Network. The fact of connection changes the system at the margins, but not in its core.

The big web systems like Google, Twitter and Facebook have built big data repositories that allow them to rent out the correlation data. Google and Twitter in particular have simplified user interaction to the point that there’s basically one action—type and submit.  But the center of power remains with the data correlation store. That’s what makes the train go. Doctors are beginning to look at the big data available about their patients and wondering whether they’re treating the data or the patient. Of course, the data will survive regardless of the outcome with the patient.

Changing the balance of power may be a long time coming, and as some have noted, it will need to be baked into the architecture from the start. There are a few new approaches that begin to move in a new direction. Jeff Jonas’s G2 rig combines elements of John Poindexter’s original design for Total Information Awareness, the Privacy by Design principles and Jonas’s own previous systems that do sensemaking on big data in real time. Particularly notable is the system’s ability to course correct based on every new piece of data and to hide the human-readable facet of data through anonymizing and encryption. Other architectures move toward establishing the user as a peer (P2P), in particular Searls’s VRM, Windley’s KRL, Bit Torrent and the recently departed Selector.

A true user-centered design practice will probably have to start on the user’s side of the glass, establish the user as a peer, and not be architectural in the way we’re used to. It’s only in this environment that a possible economics will take root. It’s also here that a developer and designer would finally have standing to do user-centered design. We might hope that such a move would happen because it was right, true and good, but this kind of dance may require a platform that isn’t a platform.

It’s an unexpected moment for Microsoft. What was formerly called Project Natal, and is now called Kinect, has opened a new interaction surface to the Network. I’m trying to think of another example of Microsoft introducing and providing stewardship for an interaction model with this kind of uptake. Generally Mr. Softy has been a follower, an embracer and extender of pre-established modes.

You can tell that Kinect has connected because it’s immediately overflowed its use cases and taken up residence in a whole series of unanticipated projects. It’s an interaction surface that has corporate competitors starting up their copy machines and trying to find the best position as a fast follower. Somehow it’s hard to imagine Microsoft actually getting something out of their labs and on to the street for around $200.00. I suppose it could be the harbinger of a pipeline finally unclogged. At least that’s the marketing spin I’d put on it.

After an initial misstep, Microsoft seems to have embraced the so-called “hacking kinect” movement. What they seemed to think was new kind of game controller turns out to be a general purpose interaction modality with use cases all up and down the Network. It’ll be interesting to see how Microsoft handles the stewardship of this new device. Running a race from the lead position is an entirely different kind of game.

After all this time, it’s still difficult to say what a tweet is. The generic form of the word has been expressed as microblogging, but this is the wrong metaphor. Blogging and RSS advocates see Twitter as a short-form quick publishing platform. What blogging tools made easy, Twitter, and other similar systems, make even easier. Given this definition, the 140 character limit on tweets seems to be an unnecessary constraint— microblogging could simply be expanded to miniblogging and a 500 character limit for individual posts. Blog posts can be any length, they are as small or large as they need to be.

“All my plays are full length, some are just longer than others.”
- Samuel Beckett

But Twitter didn’t start with blogging or blogging tools as its central metaphor, it began with the message streams that flow through dispatching systems. The tweet isn’t a small blog post, it’s a message in a communications and logistics system. There’s a tendency to say that the tweet is a “micro” something— a very small version of some normally larger thing. But tweets are full sized, complete and lack nothing. Their size allows them to flourish in multiple communications environments, particularly the SMS system and the form factor of the mobile network device (iPhone).

The best metaphor I’ve found for a tweet is the yellow sticky. The optimal post-it note is 3 inches square and canary yellow in color. It’s not a small version of something else, its size is perfect for its purpose. There are no limitations on what can be written on a yellow sticky, but its size places constraints on the form of communication. Generally, one expects a single thought per yellow sticky. And much like Twitter, explaining what a yellow sticky is to someone who’s never used one is a difficult task. Initial market tests for the post-it note showed mixed reactions. However after extensive sampling, 90% of consumers who tried the product wanted to buy it. Like the tweet, the post-it note doesn’t have a specific purpose. Arthur Fry, one of the inventors of the post-it note, wanted a bookmark with a light adhesive to keep his place in his hymnal during church choir. The rapid acceptance of the yellow sticky, in part, had to do with not defining what it should be used for. It’s hard to imagine someone saying that you’re not using a post-it note correctly, although people say that about Twitter all the time.

One thing people use yellow stickies for is as a transmittal. I find a magazine article that I like and I pass it on to you with a short message on a yellow sticky that marks the page. I might send this package to you through the mail, use inter-office mail at work, or I might just leave it on your desk. More formal routing slips might request specific actions be taken on the attached item. Fax cover sheets are another example of this kind of communication. And Twitter is often used in a similar way. The hyperlink is the adhesive that binds the message to article I’d like to pass on to you. With Twitter, and other directed social graph services, the you I pass things on to includes followers, potentially followers of followers and users who track keywords contained in my message. At any given time, the who of the you will describe a different group. The message is passed on without obligation, the listeners may simply let it pass through, or they may take up the citation and peruse its contents.

Just as the special low-tack adhesive on the back of a yellow sticky allows you to attach it to anything without leaving marks or residue, the hyperlink allows the user of Twitter to easily point at something. Hey, look at this! Rather than a long explanation or justification, it’s just my finger pointing at something of interest. That’s interesting to me. It’s the way we talk to each other when the words aren’t the most important part of the communication.

This model of passing along items of interest is fundamentally different from web syndication. Syndication extends the distribution of published content to additional authorized contexts. Some may argue that the mostly defunct form of the ‘link blog‘, or an aggregation of link blogs, offers exactly the same value. The difference is that the tweet, as electronic routing slip, exists in a real-time social media communications system. It operates like the messages in a dispatching system. There’s an item at 3rd and Webster about cute kittens, here’s the hyperlink for interested parties. Syndication implies that I think what I’ve published is valuable, I’ve extended my distribution area and you should have a look at it. With a tweeted electronic routing slip, the value is assigned by the reader who decides to pass something along and the readers who choose to take it up within a real-time (instant) messaging system. Value is external to the thing being evaluated.

As we start to look at new applications like Flipboard, an app that collects routing slips from your social network and lays them out into a magazine format, it’s important to understand the basic unit from which the experience is built. We’re used to a newspaper filled with a combination of syndicated wire stories and proprietary ones. We know about magazines where all the stories are proprietary. A few of us are familiar with web syndication aggregators that allow us to pull in, organize and read feeds from thousands of publication sources. Building an electronic publication from sets of real-time routing slips is a fundamentally different editorial process than we’ve seen before. Of course, it could be that you don’t find the stories that your friends pass on to be very interesting. In the end, this method of  assembling a real-time publication will be judged based on the value it provides. A magazine with a thousand stories isn’t really very useful, just as a Google search result with a million answers doesn’t help you find something. Can you imagine a real-time magazine that captures the ten stories that are worth reading right now? Can you imagine a time when such a thing didn’t exist?

A film is described as cinematic when the story is told primarily through the visuals. The dialogue only fills in where it needs to, where the visuals can’t convey the message. It was watching Jean-Pierre Melville’s Le Samourai that brought these thoughts into the foreground. Much of the film unfolds in silence. All of the important narrative information is disclosed outside of the dialogue.

While there’s some controversy about what percentage of human-to-human communication is non-verbal, there is general agreement that it’s more than half. The numbers are as low as 60% and as high as 93%. What happens to our non-verbal communication when a human-to-human communication is routed through a medium? A written communique, a telephone call, the internet: each of these media have a different capacity to carry the non-verbal from one end to the other.

The study of human-computer interaction examines the relationship between humans and systems. More and more, our human-computer interaction is an example of computer-mediated communications between humans; or human-computer network-human interaction. When we design human-computer interactions we try to specify everything to the nth degree. We want the interaction to be clear and simple. The user should understand what’s happening and what’s not happening. The interaction is a contract purged of ambiguity and overtones. A change in the contract is generally disconcerting to users because it introduces ambiguity into the interaction. It’s not the same anymore; it’s different now.

In human-computer network-human interactions, it’s not the clarity that matters, it’s the fullness. If we chart the direction of network technologies, we can see a rapid movement toward capturing and transmitting the non-verbal. Real-time provides the context to transmit tone of voice, facial expression, hand gestures and body language. Even the most common forms of text on the Network are forms of speech— the letters describe sounds rather than words.

While the non-verbal can be as easily misinterpreted as the verbal, the more pieces of the picture that are transmitted, the more likely the communication will be understood. But not in the narrow sense of a contract, or machine understanding. But rather in the full sense of human understanding. While some think the deeper levels of human thought can only be accessed through long strings of text assembled into the form of a codex, humans will always gravitate toward communications media that broadcast on all channels.

The corporate web site used to have a brilliant excuse for its plain and simple execution. It needed the broadest possible distribution across browsers and operating systems. All customers, regardless of the technical specs of their rig, needed to be served. Some basic HTML, a few images, a conservative dollop of CSS and javascript. Transactions and data are all handled on the back end with a round trip to the server for each and every update of the display. And the display? Order up a screen resolution that serves 90%+ of the installed base as reported by server logs. Make that 800 x 600, just to be sure. This down level, conservative approach has been baked into enterprise content management systems and a boundary has been drawn around what’s possible with a corporate web presence. Mobile web was even simpler, a down level version of a down level experience. Rich internet applications (RIAs) were put into the same category as custom desktop apps, generally not worth the effort.

Back in 1998, Jakob Nielsen reported on the general conservatism of web users:

The usability tests we have conducted during the last year have shown an increasing reluctance among users to accept innovations in Web design. The prevailing attitude is to request designs that are similar to everything else people see on the Web.

When we tested advanced home page concepts we got our fingers slapped hard by the users: I don’t have time to learn special conventions for your site as one user said. Other users said, Just give it to us plain and simple, using interaction techniques we already know from other sites.

The Web is establishing expectations for narrative flow and user options and users want pages to fit within these expectations. A major reason for this evolving genre is that users frequently move back and forth between pages on different sites and that the entire corpus of the Web constitutes a single interwoven user experience rather than a set of separate publications that are accessed one at a time the way traditional books and newspapers are. The Web as a whole is the foundation of the user interface and any individual site is nothing but a speck in the Web universe.

Adoption of modern browsers was thought to be a very slow process. In 1999, Jakob Nielsen insists that we would be stuck with old browsers for a minimum of three years. Here was another reason to keep things plain and simple.

The slow uptake speeds and the bugs and inconsistencies in advanced browser features constitute a cloud with a distinct silver lining: Recognizing that we are stuck with old technology for some time frees sites from being consumed by technology considerations and focuses them on content, customer service, and usability. Back to basics indeed: that’s what sells since that’s what users want.

Over time, a couple things changed. The web standards movement gained traction with the people who build web sites. That meant figuring out what CSS could really do and working through the transition from table-based layouts to div-based layouts. Libraries like Jquery erased the differences between browser implementations of javascript. XMLhttpRequest, originally created for the web version of Microsoft’s Outlook, emerged as AJAX and turned into a defacto browser standard. The page reload could be eliminated as a requirement for a data refresh. The Webkit HTML engine was open sourced by Apple, and Google, along with a number of other mobile device makers, began to release Webkit-based browsers. With Apple, Google, Microsoft and Mozilla all jumping on the HTML5 band wagon, there’s a real motivation to move users off of pre-standards era browsers. Even Microsoft has joined the Kill IE6 movement.

The computing power of the cloud combined with the transition from a web of documents to a web of applications has changed the equation. Throw in the rise of real-time and the emergence of social media: and you’ve got an entirely different ballgame. With the massive user embrace of the iPhone, and an iPad being sold every three seconds, we might want to re-ask the question: what do users want?

Jakob Nielsen, jumps back to 1993 in an effort to preserve his business model of plain and simple:

The first crop of iPad apps revived memories of Web designs from 1993, when Mosaic first introduced the image map that made it possible for any part of any picture to become a UI element. As a result, graphic designers went wild: anything they could draw could be a UI, whether it made sense or not.

It’s the same with iPad apps: anything you can show and touch can be a UI on this device. There are no standards and no expectations.

Worse, there are often no perceived affordances for how various screen elements respond when touched. The prevailing aesthetic is very much that of flat images that fill the screen as if they were etched. There’s no lighting model or pseudo-dimensionality to indicate raised or lowered visual elements that call out to be activated.

Don Norman throws cold water on gestures and natural user interfaces by saying they aren’t new and they aren’t natural:

More important, gestures lack critical clues deemed essential for successful human-computer interaction. Because gestures are ephemeral, they do not leave behind any record of their path, which means that if one makes a gesture and either gets no response or the wrong response, there is little information available to help understand why. The requisite feedback is lacking. Moreover, a pure gestural system makes it difficult to discover the set of possibilities and the precise dynamics of execution. These problems can be overcome, of course, but only by adding conventional interface elements, such as menus, help systems, traces, tutorials, undo operations, and other forms of feedback and guides.

Touch-based interfaces built around natural interaction metaphors have only made a life for themselves outside of the research laboratory for a few years now. However I tend to think that if these interfaces were as baffling for users as Norman and Nielsen make them out to be the iPhone and iPad would have crashed and burned. Instead they can barely make them fast enough to keep up with the orders.

The classic vanilla flavored corporate web site assumes that users have old browsers and don’t want anything that doesn’t look like everything else. All new flavors are inconceivable without years and years of work by standards bodies, research labs, and the odd de facto behavior blessed by extensive usability testing. There’s a big transition ahead for the corporate web presence. Users are way ahead and already enjoying all kinds of exotic flavors.