echovar

There’s a case that could be made that John Poindexter is the godfather of the real-time Network. I came to this conclusion after reading Shane Harris’s excellent book, The Watchers, The Rise of the Surveillance State. When you think about real-time systems, you might start with the question: who has the most at stake? Who perceives a fully-functional toolset working within a real-time electronic network as critical to survival?

To some, Poindexter will primarily be remembered for his role in the Iran-Contra Affair. Others may know something about his role in coordinating intelligence across organizational silos in the Achille Lauro Incident. It was Poindexter who looked at the increasing number of surprise terrorist attacks, including the 1983 Beruit Marine Barracks Bombing, and decided that we should know enough about these kinds of attacks before they happen to be able to prevent them. In essence, we should not be vulnerable to surprise attack from non-state terrorist actors.

After the fact, it’s fairly easy to look at all the intelligence across multiple sources, and at our leisure, connect the dots. We then turn to those in charge and ask why they couldn’t have done the same thing in real time. We slap our heads and say, ‘this could have been prevented.’ We collected all the dots we needed, what stopped us from connecting them?

The easy answer would be to say it can’t be done. Currently, we don’t have the technology and there is no legal framework, or precedent, that would support this kind of data collection and correlation. You can’t predict what will happen next, if you don’t know what’s happening right now in real time. And in the case of non-state actors, you may not even know who you’re looking for. Poindexter believed it could be done, and he began work on a program that was eventually called Total Information Awareness to make it happen.

TIA System Diagram

In his book, Shane Harris posits a central metaphor for understanding Poindexter’s pursuit. Admiral Poindexter served on submarines and spent time using sonar to gather intelligible patterns from the general background of noise filling the depths of the ocean. Poindexter believed that if he could pull in electronic credit card transactions, travel records, phone records, email, web site activity, etc., he could find the patterns of behavior that were necessary precursors to a terrorist attack.

In order to use real-time track for pattern recognition, TIA (Total Information Awareness) had to pull in everything about everyone. That meant good guys, bad guys and bystanders would all be scooped up in the same net. To connect the dots in real time your need all the dots in real time. Poindexter realized that this presented a personal privacy issue.

As a central part of TIA’s architecture, Poindexter proposed that the TIA system encrypt the personal identities of all the dots it gathered. TIA was looking for patterns of behavior. Only when the patterns and scenarios that the system was tracking emerged from the background, and been reviewed by human analysts, would a request be made to decrypt the personal identities. In addition, every human user of the TIA system would be subject to a granular-level audit trail. The TIA system itself would be watching the watchers.

The fundamental divide in the analysis and interpretation of real-time dot connecting was raised when Jeff Jonas entered the picture. Jonas had made a name for himself by developing real-time systems to identify fraudsters and hackers in Las Vegas casinos. Jonas and Poindexter met at a small conference and hit it off. Eventually Jonas parted ways with Poindexter on the issue of whether a real-time system could reliably pinpoint the identity of individual terrorists and their social networks through analysis of emergent patterns. Jonas believed you had to work from a list of suspected bad actors. Using this approach, Jonas had been very successful in the world of casinos in correlating data across multiple silos in real time to determine when a bad actor was about to commit a bad act.

Jonas thought that Poindexter’s approach with TIA would result in too many false positives and too many bad leads for law enforcement to follow up. Poindexter countered that the system was meant to identify smaller data sets of possible bad actors through emergent patterns. These smaller sets would then be run through the additional filter of human analysts. The final output would be a high-value list of potential investigations.

Of course, once Total Information Awareness was exposed to the harsh light of the daily newspaper and congressional committees, its goose was cooked. No one wanted the government spying on them without a warrant and strong oversight. Eventually Congress voted to dismantle the program. This didn’t change the emerging network-connected information environment, nor did it change the expectation that we should be able to coordinate and correlate data across multiple data silos to stop terrorist attacks in real time. Along side the shutting down of TIA, and other similar government efforts, was the rise of Google, social networks, and other systems that used network-based personal data to predict consumer purchases; guess which web site a user might be looking for; and even the bet on the direction of stocks trading on exchanges.

Poindexter had developed the ideas and systems for TIA in the open. Once it was shut down, the system was disassembled and portions of it ported over to the black ops part of the budget. The system simply became opaque, because the people and agencies charged with catching bad actors in real time still needed a toolset. The tragedy of this, as Shane Harris points out, is that Poindexter’s vision around protecting individual privacy through identity encryption was left behind. It was deemed too expensive and too difficult. But the use of real-time data correlation techniques, social graph analysis, in-memory data stores and real-time pattern recognition are all still at work.

It’s likely that the NSA, and other agencies, are using a combination of Poindexter’s and Jonas’s approaches right now: real-time data correlation around suspected bad actors, and their social graphs— combined with a general sonar-like scanning of the ocean of real-time information to pick up emergent patterns that match the precursors of terrorist acts. What’s missing is a dialogue about our expectations, our rights to privacy and the reality of the real-time networked information environment that we inhabit. We understood the idea of wiretapping a telephone, but what does that mean in the age of the iPhone?

Looking at the structure of these real-time data correlation systems, it’s easy to see their migration pattern. They’ve moved from the intelligence community to wall street to the technology community to daily commerce. Social CRM is the buzz word that describes the corporate implementation; some form of real-time VRM will be the consumer’s version of the system. The economics of the ecosystem of the Network has begun to move these techniques and tools to the center of our lives. We’ve always wanted to alter our relationship to time, we want to know with a very high probability what is going to happen next. We start with the highest-value targets, and move all the way down to a prediction of which television show we’ll want to watch and which laundry detergent we’ll end up telling our friend about.

Shane Harris begins his book The Watchers with the story of Able Danger, an effort to use data mining, social graph and correlation techniques on the public Network to understand Al Qaeda. This was before much was known about the group or its structure. One of the individuals working on Able Danger was Erik Kleinsmith, he was one of the first to use these techniques to uncover and visualize a terrorist network. And while he may not have been able to predict the 9/11 attacks, his analysis seemed to connect more dots than any other approach. But without a legal context for this kind of analysis of the public Network, the data and the intelligence was deleted and unused.

Working under the code name Able Danger, Kleinsmith compiled an enormous digital dossier on the terrorist outfit (Al Qaeda). The volume was extraordinary for its size— 2.5 terabytes, equal to about one-tenth of all printed pages held by the Library of Congress— but more so for its intelligence significance. Kleinsmith had mapped Al Qaeda’s global footprint. He had diagrammed how its members were related, how they moved money, and where they had placed operatives. Kleinsmith show military commanders and intelligence chiefs where to hit the network, how to dismantle it, how to annihilate it. This was priceless information but also an alarm bell– the intelligence showed that Al Qaeda had established a presence inside the United States, and signs pointed to an imminent attack.

That’s when he ran into his present troubles. Rather than relying on classified intelligence databases, which were often scant on details and hopelessly fragmentary, Kleinsmith had created his Al Qaeda map with data drawn from the Internet, home to a bounty of chatter and observations about terrorists and holy war. He cast a digital net over thousands of Web sites, chat rooms, and bulletin boards. Then he used graphing and modeling programs to turn the raw data into three-dimensional topographic maps. These tools displayed seemingly random data as a series of peaks and valleys that showed how people, places, and events were connected. Peaks near each other signaled  connection in the data underlying them. A series of peaks signaled that Kleinsmith should take a closer look.

…Army lawyers had put him on notice: Under military regulations Kleinsmith could only store his intelligence for ninety days if it contained references to U.S. persons. At the end of that brief period, everything had to go. Even the inadvertent capture of such information amounted to domestic spying. Kleinsmith could go to jail.

As he stared at his computer terminal, Kleinsmith ached at the thought of what he was about to do. This is terrible.

He pulled up some relevant files on his hard drive, hovered over them with his cursor, and selected the whole lot. Then he pushed the delete key. Kleinsmith did this for all the files on his computer, until he’d eradicated everything related to Able Danger. It took less than half an hour to destroy what he’d spent three months building. The blueprint for global terrorism vanished into the electronic ether.

For book and record stores, there was a moment when the largest inventory and the lowest prices won out. Large physical stores with endless rows of inventory overwhelmed the small retailer. Eventually the inventory moved into a series of warehouses/databases with query-based web front ends attached to a product delivery system. Inventory expanded to match the number of sellable books in existence, and the customer experience was abstracted to a computer screen, a keyboard and a mouse. Touch, smell, sound, weight, the look of the spine, the creaking of the wooden floor— all of these modes of interaction were eliminated from the equation. Of course, no one is interested in all books, but if a vendor has all books in their inventory, it’s likely the subset you’re interested in can be carved out of the whole stack.

Two of my favorite bookstores don’t have an infinite inventory. I always enjoy browsing and rarely walk out without having purchased something. The trick is that if you don’t have everything, you need to have what’s good. And in order to have what’s good, you need to have a point of view on what’s good. In New York, the tiny Three Lives bookstore always manages to show me something I can’t live without. Last time I was there it was Tom Rachman’s sparkling first novel, The Imperfectionists. In San Francisco, one of my favorites is Lawrence Ferlinghetti’s City Lights Bookstore. City Lights often makes the improbable connection. After a reading (in New York) by Richard Foreman from his book, No-Body, A Novel in Parts, I asked him what he was reading. Foreman said that he’d become very interested in an Austrian writer named Heimito Von Doderer. Subsequently, I looked for books by Von Doderer, but came up empty until a visit to City Lights. City Lights was the perfect connection between Foreman and Von Doderer.

More than just a place to purchase books, both of those bookstores communicate a way of life, a way of thinking, an idea about taste and a larger picture about what’s good and important in our culture. While their inventory of books isn’t infinite, one has a sense of infinite possibility browsing through the stacks.

While at first we luxuriated in the ocean of choice, now we find ourselves thwarted by the process of sorting and prioritizing an infinite set of possibilities. One way to gauge the number of choices to offer is to look at the relative amount of time a person spends evaluating possible choices versus the amount of time spent enjoying the choice. If the selection set is infinite, but only one item will eventually be chosen, the customer may find herself living out one of Zeno’s paradoxes.

There’s a sense in which an infinite inventory is amoral. It avoids the choices forced on a small bookstore with a limited amount of shelf space. And perhaps this gets at something central to human experience— something about time, mortality and the choices we make about what matters. Neil Postman relays a quote from Philip Roth about Writers From The Other Europe.

In commenting on the difference between being a novelist in the West and being a novelist behind the iron curtain (this was before the collapse of Communism in Eastern Europe), Roth said that in Eastern Europe nothing is permitted but everything matters; with us, everything is permitted but nothing matters.

We’re flooded, drowning in information. We’re better than ever at collecting the dots, worse than ever at connecting the dots. This is true at the level of national security, business intelligence, customer and vendor relationship management and the personal daily real-time streams we audit. We cast our nets wider than ever and catch everything. Nothing escapes our grasp.

This high-velocity firehose of messages seems to imply that something is going on. But the items that pass through this pipe require additional filtering to find out whether anything important is really going on. The violence and the sheer volume of messages are an indicator, a message about the medium itself, but the individual messages pass by so quickly that one only gets a sense of their general direction and whether they carry a positive or negative charge. Oddly, there’s a disconnect between message traffic and whether something important is going on. There’s always a high volume of message traffic, there’s rarely anything going on.

Tomorrow, and tomorrow, and tomorrow,

Creeps in this petty pace from day to day

To the last syllable of recorded time,

And all our yesterdays have lighted fools

The way to dusty death. Out, out, brief candle!

Life’s but a walking shadow, a poor player

That struts and frets his hour upon the stage

And then is heard no more: it is a tale

Told by an idiot, full of sound and fury,

Signifying nothing.

The Atlantic Wire’s series ‘What I Read‘ asks various notable personalities about their media diet. When confronted with an all-you-can-eat smorgasbord, how does one choose what to eat? A recent column contained thoughts by Wired editor Chris Anderson on his foraging techniques. One of the best filters that Anderson uses is free of charge, and given to him by a friend:

Nassim Taleb once advised people to ignore any news you don’t hear in a social context. From people you know and, ideally, face to face. You have two combinatorial filters in social communication. First, you’ve chosen to talk with these people, and second, they’ve chosen to bring it up. Those two filters–a social and an importance filter–are really good ways of identifying what really matters to people. If I hear about news through social means, and if I hear about it three times, then I pay attention.

The interesting thing about this technique is that it doesn’t require Network scale technical capability. Spidering the entire Network and running a query through a relevance algorithm isn’t part of the picture. Trading your personal information for access to large scale cloud-based computational capabilities isn’t required either. Authentically connecting with people to learn what really matters to people is the key.

It turns out that not much matters to a mechanical or computational process. While it can sort items into a prioritized list based on this algorithm, or that one, the question of which algorithm should be used is a matter of indifference. And when we crank up the social context to extreme levels, we create some confusion around Taleb’s filters. Not every social media interaction provides the kind of social context Taleb is referring to. Only in their simplest and least technical incarnation, do these combinatorial filters provide high quality output. And despite all the sound and fury, the manifestations of speed whirling around us, “things happen fairly slowly, you know. They do.“

Religion serves as a connecting point for two recent essays. One, by Jaron Lanier, is called The First Church of Robotics; and the other is called The Rabbis and The Thinking Machines by David Gelernter. Ostensibly each of the authors is writing about robots, or androids, and the moral questions that surround the quest to create this type of machine and what our obligations might be should we actually succeed.

Lanier has taken up the cause of deflating the bubble of artificial intelligence that’s growing up around technology and software. We’ve encased the outputs of algorithms with the glow of “intelligence.” We strive for “smarter” algorithms to relieve us from the burdens of our daily drudgery. To counter this, Lanier points out that if we talk about what is called A.I., when we leave out the vocabulary of A.I., we see what’s happening in front of us much more clearly. From Lanier’s essay:

I myself have worked on projects like machine vision algorithms that can detect human facial expressions in order to animate avatars or recognize individuals. Some would say these too are examples of A.I., but I would say it is research on a specific software problem that shouldn’t be confused with the deeper issues of intelligence or the nature of personhood. Equally important, my philosophical position has not prevented me from making progress in my work. (This is not an insignificant distinction: someone who refused to believe in, say, general relativity would not be able to make a GPS navigation system.)

In fact, the nuts and bolts of A.I. research can often be more usefully interpreted without the concept of A.I. at all. For example, I.B.M. scientists recently unveiled a “question answering” machine that is designed to play the TV quiz show “Jeopardy.” Suppose I.B.M. had dispensed with the theatrics, declared it had done Google one better and come up with a new phrase-based search engine. This framing of exactly the same technology would have gained I.B.M.’s team as much (deserved) recognition as the claim of an artificial intelligence, but would also have educated the public about how such a technology might actually be used most effectively.

The same is true of efforts like the “semantic” web. By leaving out semantics and ontology, simply removing that language entirely, you get a much better picture of what the technology is trying to accomplish. Lanier, in his essay, equates the growing drumbeat on behalf of “artificial intelligence” to the transfer of humanity from the human being to the machine. All that confounds us as earthbound mortals is soothed by the patent medicine of the thinking machine. Our lives are extended to infinity, our troubling decisions are painlessly computed for us, and we are all joyously joined in a singular global mind. It’s no wonder Lanier sees the irrational exuberance for artificial intelligence congealing into an “ultra-modern religion.” A religion that wears the disguise of science.

David Gelernter, in his essay, stipulates that we will see a “thinking machine roll out of the lab.” To be clear, he doesn’t believe that machines will attain consciousness. Gelernter simply thinks that something good enough to pass the Turing Test will eventually be built. In this case, it would be a machine that can imitate thinking. And from that we move from the Turing Test to the Duck Test. If it looks like a duck, walks like a duck and quacks like a duck— we call it a duck. However, in this case, we’ll call it a duck even though we know that it’s not a duck. Gelernter elaborates:

Still: it is only a machine. It acts the part of an intelligent agent perfectly, yet it is unconscious (as far as we know, there is no way to create consciousness using software and digital computers). Being unconscious, it has no mind. Software will make it possible for a computer to imitate human behavior in detail and in depth. But machine intelligence is a mere façade. If we kick our human-like robot in the shin, it will act as if it is in pain but will feel no pain. It is not even fair to say that it will be acting. A human actor takes on a false persona, but underneath is a true persona; a thinking machine will have nothing “underneath.” Behind the impressive false front, there will be no one home. The robot will have no inner life, no mental landscape, no true emotions, no awareness of anything.

The question Gelernter asks is: what is our moral obligation to the android? How should we treat these machines? Shall we wait for a new PETA? Will a People for the Ethical Treatment  of Androids tell us that we can’t delete androids, and other intelligent agents, like we used to delete unwanted software? Anthropomorphism is even more potent when human characteristics are projected on to a human form. The humanity we see in the android will be whatever spirit we project into the relationship. It’s a moral dilemma we will create for ourselves by choosing build machines in our own image. Gelernter explains:

Thinking machines will present a new challenge. “Cruelty” to thinking machines or anthropoid robots will be wrong because such machines will seem human. We should do nothing that elicits expressions of pain from a thinking machine or human-like robot. (I speak here of a real thinking machine, not the weak imitations we see today; true thinking machines are many decades in the future.) Wantonly “hurting” such a machine will damage the moral atmosphere by making us more oblivious of cruelty to human beings.

Where Lanier wants us to see mock intelligence with clear eyes as a machine running a routine; Gelernter observes that once we surround ourselves with machines created in our image, the fact that they are without consciousness will not relieve us of moral responsibility regarding their treatment. Lanier warns that new religions are being created out of the fantasy of technological achievement without boundaries, and Gelernter invokes the Judeo-Christian tradition to warn that cruelty to pseudo-humans will be cruelty all the same.

We seem compelled to create machines that appear to relieve us of the burden of thinking. In the end, it’s Jaron Lanier who uncovers the key concept. There’s a gap between aesthetic and moral judgement and the output of an algorithm that simulates, for instance, your taste in music. We have to ask to what extent are we allowing our judgements to be replaced by algorithmic output? Some would say the size of that gap is growing smaller every day, and will eventually disappear. However, in other areas, the line between the two appears brightly drawn. For instance civil and criminal laws are a set of rules, but we wouldn’t feel comfortable installing machines in our courtrooms to hand down judgements based on those rules. We wouldn’t call the output of that system, justice. While it seems rather obvious, judgement and algorithms are not two things of the same type separated by a small gap in efficiency. They are qualitatively different things separated by a difference of kind. But the alchemists of technology tell us, once again, that they can turn lead into gold. What they don’t mention, is what’s lost in the translation.

There’s a thought running round that says that social media is engendering a new age of transparency. When we use the word ‘transparency’ we speak of a material through which light passes with clarity. If conditions aren’t completely clear, we might call the material translucent, which would allow light to pass through it diffusely. And if we can’t see anything at all, we’ll call it opaque, a material with a surface that doesn’t allow even a speck of light through it.

If it is we who are ‘transparent,’ it’s as though our skin has turned to glass and the social, psychological and biological systems operating within us are available for public inspection. It’s thought that by virtue of their pure visibility these systems can be understood, influenced and predicted. Although for most of us, when we lift the hood of our car and inspect the engine it’s strictly a matter of form. We know whether the engine is running or not, but that’s about the limit for a non-specialist.

Much like “open” and “closed,” the word transparency is associated with the forces of good, while opacity is delegated to play for the evil team. We should like to know that a thing is transparent, that we could have a look at it if we chose to, even if we don’t understand it at the moment. Certainly there must an e-book somewhere that we could page through for an hour or so to get a handle on the fundamentals. On the other hand, if a thing is opaque, we’re left with a mystery without the possibility of a solution. After all, we don’t have x-ray vision. How else can we possibly get beneath the surface to find out what’s going on?

Ralph Ellison wrote about social invisibility in his book The Invisible Man. The narrator of the story is invisible because everyone sees him as a stereotype rather than as a real person. In a stereotype, a surface image is substituted for the whole entity. Although Ellison’s narrator acknowledges that sometimes invisibility has its advantages. Surface and depth each have their time and place.

Jeff Jonas, in a recent post called “Transparency as a Mask,” talks about the chilling effect of transparency. If we exist in a social media environment of pure visibility, a sort of panopticon of the Network, how will this change the incentives around our behavior? Jonas wonders whether we might see a mass migration toward the average, toward the center of the standard deviation chart, the normal part of the normal distribution. Here’s Jonas on the current realities of data correlation.

Unlike two decades ago, humans are now creating huge volumes of extraordinarily useful data as they self-annotate their relationships and yours, their photographs and yours, their thoughts and their thoughts about you … and more.

With more data, comes better understanding and prediction.  The convergence of data might reveal your “discreet” rendezvous or the fact you are no longer on speaking terms your best friend.  No longer secret is your visit to the porn store and the subsequent change in your home’s late night energy profile, another telling story about who you are … again out of the bag, and little you can do about it.  Pity … you thought that all of this information was secret.

Initially the Network provided a kind of equal footing for those four or five standard deviations off center, in a sense this is the basis of the long tail. The margins and the center all play in the same frictionless hypermedia environment. When these deviations become visible and are correlated with other private and public data, the compilation of these surface views create an actionable picture with multiple dimensions. Suddenly there’s a valuable information asymmetry produced with an affordable amount of compute time.

Only the mediocre are always at their best.
-Jean Giraudoux

However, once we know someone is scanning and correlating the facets of our presence on the Network, what’s to stop us from signaling normal, creating a mask of transparency?

The secret of success is sincerity. Once you can fake that you’ve got it made.

-Jean Giraudoux

We may evolve, adapt to the new environment. The chameleon and many other creatures change their appearance to avoid detection. We may also become shape shifters, changing the colors of our digital skin to sculpt an impression for key databases.

In ecology, crypsis is the ability of an organism to avoid observation or detection by other organisms. A form of antipredator adaptation, methods range from camouflage, nocturnality, subterranean lifestyle, transparency, or mimicry

Of course, there’s a chance we won’t be fooling anyone. A false signal here or there will be filtered out and the picture will be assembled despite our best efforts at camouflage.

There’s another path through these woods. Historically, a much less travelled path. That’s the path of tolerance, of embracing and celebrating difference, and acknowledging our own strangeness. While it’s possible that a human can empathize with the strangeness of another human, the question we have to ask in this new era of digital transparency is: how can an algorithm be made to process strangeness without automatically equating it with error?

I thought it prudent to write something about Jeffrey Rosen’s Sunday NY Times essay, The Web Means The End Of Forgetting, before it slipped into the past and we’d all forgotten about it. Scott Rosenberg was disappointed in Rosen’s essay and wrote about how it didn’t live up to the large themes it outlined. The essence of Rosen’s piece is that the public information we publish to the web through social network systems like Facebook are permanent markers that may come back to haunt us in unanticipated contexts. Rosenberg’s critique seems to be that there’s not too much evidence of this happening, and that a greater concern is link rot, preservation of the ephemera of the web and general digital preservation.

Of course, there’s a sense in which we seem to have very poor memories indeed. Our universities feature a discipline called archeology in which we dig up our ancestors with the purpose of trying to figure who they were, what they did and how they lived. We lack the ability to simply rewind and replay the ancient past. As each day advances, another slips into time out of mind— or time immemorial as it’s sometimes called.

We use the metaphors memory and forgetting when talking about what computer systems do when they store and retrieve bits from a file system. The human activity of memory and forgetting actually has very little in common with a computer’s storage and retrieval routines. When we say that the Web doesn’t forget, what we mean is that if something is stored in a database, unless there’s a technical problem, it can be retrieved through some kind of search query. If the general public has access to that data store, then information you’ve published will be available to any interested party. It’s not a matter of human remembering or forgetting, but rather one of discovery and random access through querying a system’s indexed data.

At issue in Rosen’s piece, isn’t the fact of personal data retrieved through a search query, but rather the exposure of personal transgressions. Lines that were crossed in the past, behavior from one context made inappropriate by placing it into a new context, some departure from the Puritan norm detected and added into a summary valuation of a person. Rosen even describes this mark as a “scarlet letter in your digital past.” The technical solutions he explores have to do with changing the data or the context of the data to prevent retrieval: the stains of data are scrubbed and removed from relevant databases; additional data is piled in to divert attention from the offending bits; or an expiration policy is enforced on bits that make them unreadable after a set period of time. There’s an idea that at some future point you will own all your personal data (that you’ve published into publicly networked systems) and will have granular access controls over it.

Absent a future of totalitarian personal data control, Rosen moves on to the act of forgiveness. Can we forgive each other in the presence of permanent reminders? I wrote a post about this on the day that MSNBC replayed the events of morning of September 11, 2001. Sometimes we can rewind the past and press play, but wounds cannot heal if we’re constantly picking at them.

While we’re enraptured by the metaphors of memory and forgetting, intelligence and thinking, as we talk about computers, when we speak of forgiveness we tamp down the overtones and resonance of the metaphor. It’s in the cultural practice of western religion that we have the mechanisms for redemption, forgiveness, indulgences and absolution. In the secular rational context of computerized networks of data there’s no basis for forgiveness. It’s all ones or zeros, it’s in the database or it’s not.

Perhaps in our digital secular world we need a system similar to carbon offsets. When we’ve sinned against the environment by virtue of the size of our carbon footprint, we purchase indulgences from TerraPass to offset our trespass. Rather than delete, obscure or divert attention from the bits in question, we might simply offset them with some act of kindness. While the Catholic Church frowns on the idea of online confession, in this model, there would be no person listening to your confession and assigning penance. The service would simply authenticate your good deeds and make sure they were visible as a permanent marker on the Network. It would be up to you to determine the size of the offset, or perhaps you could select from a set of standard offset sizes.

The problem that Rosen describes is not one of technology, but rather one of humanity and human judgment. The question of how we treat each other is fundamental and has been with us since the beginning.

As we consider machines that may think, we turn toward our own desires. We’d like a machine that understands what we mean, even what we intend, rather than what we strictly say. We don’t want to have to spell everything out. We’d like the machine to take a vague suggestion, figure out how to carry on, and then return to us with the best set of options to choose from. Or even better, the machine should carry out our orders and not bother us with little ambiguities or inconsistencies along the way. It should work all those things out by itself.

We might look to Shakespeare and The Tempest for a model of this type of relationship. Prospero commands the spirit Ariel to fulfill his wishes; and the sprite cheerfully complies:

ARIEL
Before you can say ‘come’ and ‘go,’
And breathe twice and cry ‘so, so,’
Each one, tripping on his toe,
Will be here with mop and mow.
Do you love me, master? no?

But The Tempest also supplies us with a counter-example in the character Caliban, who curses his servitude and his very existence:

CALIBAN
You taught me language; and my profit on’t
Is, I know how to curse. The red plague rid you
For learning me your language!

Harold Bloom, in his essay on The Tempest in Shakespeare: Invention of the Human, connects the character of Prospero with Christopher Marlowe’s Dr. Faustus. Faustus also had a spirit who would do his bidding, but the cost to the good doctor, was significant.

For the most part we no longer look to the spirit world for entities to do our bidding. We now place our hopes for a perfect servant in the realm of the machine. Of course, machines already do a lot for us. But frankly, for a long time now, we’ve thought that they could be a little more intelligent. Artificial intelligence, machines that think, the global brain: we’re clearly under the impression that our lot could be improved by such an advancement in technology. Here we aren’t merely thinking of an augmentation of human capability in the mode of Doug Engelbart, but rather something that stands on its own two feet.

In 2002, David Gelernter wrote a book called The Muse in the Machine: Computerizing the Poetry of Human Thought. Gelernter explored the spectrum of human thought from tightly-focused task-driven thought to poetic and dream thoughts. He makes the case that we need both modes, the whole spectrum, to think like a human does. Recently, Gelernter updated his theme in an essay for Edge.org called Dream-Logic, The Internet and Artificial Thought. He returns to the theme that most of the advocates for artificial intelligence have a defective understanding of what makes up human thought:

Many people believe that the thinker and the thought are separate.  For many people, “thinking” means (in effect) viewing a stream of thoughts as if it were a PowerPoint presentation: the thinker watches the stream of his thoughts.  This idea is important to artificial intelligence and the computationalist view of the mind.  If the thinker and his thought-stream are separate, we can replace the human thinker by a computer thinker without stopping the show. The man tiptoes out of the theater. The computer slips into the empty seat.  The PowerPoint presentation continues.

But when a person is dreaming, hallucinating — when he is inside a mind-made fantasy landscape — the thinker and his thought-stream
are not separate.  They are blended together. The thinker inhabits his thoughts.  No computer will be able to think like a man unless it, too, can inhabit its thoughts; can disappear into its own mind.

Gelernter makes the case that thinking must include the whole spectrum of the thought. He extends this idea of the thinker inhabiting his thoughts by saying that when we make memories, we create alternate realities:

Each remembered experience is, potentially, an alternate reality. Remembering such experiences in the ordinary sense — remembering “the beach last summer” — means, in effect, to inspect the memory from outside.   But there is another kind of remembering too: sometimes remembering “the beach last summer” means re-entering the experience, re-experiencing the beach last summer: seeing the water, hearing the waves, feeling the sunlight and sand; making real the potential reality trapped in the memory.

(An analogy: we store potential energy in an object by moving it upwards against gravity.  We store potential reality in our minds by creating a memory.)

Just as thinking works differently at the top and bottom of the cognitive spectrum, remembering works differently too.  At the high-focus end, remembering means ordinary remembering; “recalling” the beach.  At the low-focus end, remembering means re-experiencing the beach.  (We can re-experience a memory on purpose, in a limited way: you can imagine the look and fragrance of a red rose.  But when focus is low, you have no choice.  When you remember something, you must re-experience it.)

On the other side of the ledger, you have the arguments for a technological singularity via recursive self-improvement. One day, a machine is created that is more adept at creating machines than we are. And more importantly, it’s a machine who’s children will exceed the capabilities of the parent. Press fast forward and there’s an exponential growth in machine capability that eventually far outstrips a human’s ability to evolve.

In 2007, Gelernter and Kurzweil debated the point:

When Gelernter brings up the issue of emotions, poetic thought and the re-experiencing of memory as fundamental constituents of human thought, I can’t help but think of the body of the machine. Experience needs a location, a there for its being. Artificial intelligence needs an artificial body. To advance even a step in the direction of artificial intelligence, you have to endorse the mind/body split and think of these elements as replaceable, extensible, and to some extent, arbitrary components. This move begs a number of questions. Would a single artificial intelligence be created or would many versions emerge? Would natural selection cull the herd? Would an artificial intelligence be contained by the body of the machine in which it existed? Would each machine body contain a unique artificial intelligence with memories and emotions that were solely its own? The robot and the android are the machines we think of as having bodies. In Forbidden Planet, the science fiction update of Shakespeare’s The Tempest, we see the sprite Ariel replaced with Robby the Robot.

In Stanley Kubrick’s film 2001: A Space Odyssey, the HAL 9000 was an artificial intelligence who’s body was an entire space ship. HAL was programmed to put the mission above all else, which violated Asimov’s three laws of robotics. HAL is a classic example of an artificial intelligence that we believe has gone a step too far. A machine who has crossed a line.

When we desire to create machines that think; we want to create humans who are not fully human. Thoughts that don’t entirely think. Intelligence that isn’t fully intelligent. We want to use certain words to describe our desires, but the words express so much more than we intend. We need to hold some meaning back, the spark that makes humans, thought and intelligence what they are.

Philosophy is a battle against the bewitchment of our intelligence by means of language.
- Ludwig Wittgenstein

Clearly some filters, algorithms and agents will be better than others, but none of them will think, none will have intelligence. If part of thinking is the ability to make new analogies, then we need to think about what we do when we create and use these software machines. It becomes an easier task when we start our thinking with augmentation rather than a separate individual intelligence.

We’re inaugurating a new tradition around the homestead, Shakespeare Saturdays. This Saturday we’ll be screening Shakespeare’s last play, The Tempest. In the story, the magician Prospero, and his daughter Miranda, have been stranded on an island for 12 years. Prospero raises a tempest on the sea to cause a passing ship to run aground. Among the passengers are Prospero’s rivals from the time before the island, Antonio and Alonso, King of Naples. It’s in the midst of this wild storm the story begins…

Written in 1610, the play continues to be regularly performed and adapted. The Tempest has also been the inspiration for whole range of work from operas and symphonies to poetry and post-colonial literary analysis. Quotes from the play turn up in the most unexpected places.

Samuel Beckett’s Endgame has the character Hamm speaking a line from one of Prospero’s most memorable speeches: “Our revels now are ended.”

“Our revels now are ended. These our actors,
As I foretold you, were all spirits and
Are melted into air, into thin air;
And—like the baseless fabric of this vision —
The cloud-capped towers, the gorgeous palaces,
The solemn temples, the great globe itself,
Yea, all which it inherit, shall dissolve,
And like this insubstantial pageant faded,
Leave not a rack behind. We are such stuff
As dreams are made on, and our little life
Is rounded with a sleep. …”

Prospero
The Tempest by William Shakespeare

John Gielgud called Prospero his favorite role. He played it many times in the theater, but he was never able to mount a film version of the project. The closest he came was in Peter Greenaway’s adaptation Prospero’s Books.

Caliban is another fascinating character— a beast, a brute, the son of Sycorax, a witch who was also banished to the island, but has died several years before the action of the play begins. Caliban provides the counterpoint to Prospero, where Prospero sees that he must wake from the dream he’s created; Caliban suffers so thoroughly in his daily existence that he cries out to dream again.

Be not afeard; the isle is full of noises,
Sounds, and sweet airs, that give delight and hurt not.
Sometimes a thousand twangling instruments
Will hum about mine ears; and sometime voices
That, if I then had waked after long sleep,
Will make me sleep again; and then in dreaming,
The clouds methought would open, and show riches
Ready to drop upon me, that when I waked
I cried to dream again.

Caliban
The Tempest by William Shakespeare

There are 38 of Shakespeare’s plays and collaborations in existence and there are film versions of most of them. Not every Saturday will be Shakespeare Saturday, but I’m looking forward to immersing myself in a very foreign world that is not so unlike our own.

The corporate web site used to have a brilliant excuse for its plain and simple execution. It needed the broadest possible distribution across browsers and operating systems. All customers, regardless of the technical specs of their rig, needed to be served. Some basic HTML, a few images, a conservative dollop of CSS and javascript. Transactions and data are all handled on the back end with a round trip to the server for each and every update of the display. And the display? Order up a screen resolution that serves 90%+ of the installed base as reported by server logs. Make that 800 x 600, just to be sure. This down level, conservative approach has been baked into enterprise content management systems and a boundary has been drawn around what’s possible with a corporate web presence. Mobile web was even simpler, a down level version of a down level experience. Rich internet applications (RIAs) were put into the same category as custom desktop apps, generally not worth the effort.

Back in 1998, Jakob Nielsen reported on the general conservatism of web users:

The usability tests we have conducted during the last year have shown an increasing reluctance among users to accept innovations in Web design. The prevailing attitude is to request designs that are similar to everything else people see on the Web.

When we tested advanced home page concepts we got our fingers slapped hard by the users: I don’t have time to learn special conventions for your site as one user said. Other users said, Just give it to us plain and simple, using interaction techniques we already know from other sites.

The Web is establishing expectations for narrative flow and user options and users want pages to fit within these expectations. A major reason for this evolving genre is that users frequently move back and forth between pages on different sites and that the entire corpus of the Web constitutes a single interwoven user experience rather than a set of separate publications that are accessed one at a time the way traditional books and newspapers are. The Web as a whole is the foundation of the user interface and any individual site is nothing but a speck in the Web universe.

Adoption of modern browsers was thought to be a very slow process. In 1999, Jakob Nielsen insists that we would be stuck with old browsers for a minimum of three years. Here was another reason to keep things plain and simple.

The slow uptake speeds and the bugs and inconsistencies in advanced browser features constitute a cloud with a distinct silver lining: Recognizing that we are stuck with old technology for some time frees sites from being consumed by technology considerations and focuses them on content, customer service, and usability. Back to basics indeed: that’s what sells since that’s what users want.

Over time, a couple things changed. The web standards movement gained traction with the people who build web sites. That meant figuring out what CSS could really do and working through the transition from table-based layouts to div-based layouts. Libraries like Jquery erased the differences between browser implementations of javascript. XMLhttpRequest, originally created for the web version of Microsoft’s Outlook, emerged as AJAX and turned into a defacto browser standard. The page reload could be eliminated as a requirement for a data refresh. The Webkit HTML engine was open sourced by Apple, and Google, along with a number of other mobile device makers, began to release Webkit-based browsers. With Apple, Google, Microsoft and Mozilla all jumping on the HTML5 band wagon, there’s a real motivation to move users off of pre-standards era browsers. Even Microsoft has joined the Kill IE6 movement.

The computing power of the cloud combined with the transition from a web of documents to a web of applications has changed the equation. Throw in the rise of real-time and the emergence of social media: and you’ve got an entirely different ballgame. With the massive user embrace of the iPhone, and an iPad being sold every three seconds, we might want to re-ask the question: what do users want?

Jakob Nielsen, jumps back to 1993 in an effort to preserve his business model of plain and simple:

The first crop of iPad apps revived memories of Web designs from 1993, when Mosaic first introduced the image map that made it possible for any part of any picture to become a UI element. As a result, graphic designers went wild: anything they could draw could be a UI, whether it made sense or not.

It’s the same with iPad apps: anything you can show and touch can be a UI on this device. There are no standards and no expectations.

Worse, there are often no perceived affordances for how various screen elements respond when touched. The prevailing aesthetic is very much that of flat images that fill the screen as if they were etched. There’s no lighting model or pseudo-dimensionality to indicate raised or lowered visual elements that call out to be activated.

Don Norman throws cold water on gestures and natural user interfaces by saying they aren’t new and they aren’t natural:

More important, gestures lack critical clues deemed essential for successful human-computer interaction. Because gestures are ephemeral, they do not leave behind any record of their path, which means that if one makes a gesture and either gets no response or the wrong response, there is little information available to help understand why. The requisite feedback is lacking. Moreover, a pure gestural system makes it difficult to discover the set of possibilities and the precise dynamics of execution. These problems can be overcome, of course, but only by adding conventional interface elements, such as menus, help systems, traces, tutorials, undo operations, and other forms of feedback and guides.

Touch-based interfaces built around natural interaction metaphors have only made a life for themselves outside of the research laboratory for a few years now. However I tend to think that if these interfaces were as baffling for users as Norman and Nielsen make them out to be the iPhone and iPad would have crashed and burned. Instead they can barely make them fast enough to keep up with the orders.

The classic vanilla flavored corporate web site assumes that users have old browsers and don’t want anything that doesn’t look like everything else. All new flavors are inconceivable without years and years of work by standards bodies, research labs, and the odd de facto behavior blessed by extensive usability testing. There’s a big transition ahead for the corporate web presence. Users are way ahead and already enjoying all kinds of exotic flavors.

June 16th is known as Bloomsday; it’s the single day, in 1904, on which James Joyce’s novel Ulysses occurs. The day is commemorated around with the world with readings of the book and the hoisting of a pint or two.

Stately, plump Buck Mulligan came from the stairhead, bearing a bowl of lather on which a mirror and a razor lay crossed. A yellow dressinggown, ungirdled, was sustained gently behind him by the mild morning air. He held the bowl aloft and intoned:

— Introibo ad altare Dei.

Halted, he peered down the dark winding stairs and called up coarsely:

— Come up Kinch. Come up , you fearful jesuit.

Solemnly he came forward and mounted the round gunrest. He faced about and blessed gravely thrice the tower, the surrounding country and the awakening mountains. Then, catching sight of Stephen Dedalus, he bent towards him and made rapid crosses in the air, gurgling in his throat and shaking his head. Stephen Dedalus, displeased and sleepy, leaned his arms on the top of the staircase and looked coldly at the shaking, gurgling face that blessed him, equine in its length, and at the light untonsured hair, grained and hued like pale oak.

Buck Mulligan peeped an instant under the mirror and then covered the bowl smartly.

Joyce’s book brought to popular notice the idea of stream of consciousness literature. The term “stream of consciousness” was coined by the philosopher William James in an attempt to describe the mind-world connection as it relates the concept of truth. As a literary technique, it involves writing as a kind of transcription of the inner thought process of a character. In Ulysses, we find that stream rife with puns, allusions and parodies. Joyce was trying to capture another aspect of truth.

What challenged the reader of the day as avant garde and daring has become a relatively normal part of our network-connected lives.

Twitter has become a part of my daystream
- Roger Ebert

The stream of tweets flowing out of Twitter could aptly be described as a stream of collective consciousness. And so today, we think a great deal about various real-time streams and how they wend their way through networks of social connection. The water metaphors we use to speak about these things have roots in our shared history; they describe another kind of network of connections.

Stephen B. Johnson, in his book The Invention of Air, makes the case for the London Coffee House as an early prototype for the internet:

With the university system languishing amid archaic traditions, and corporate R&D labs still on the distant horizon, the public space of the coffeehouse served as the central hub of innovation in British society. How much of the Enlightenment do we owe to coffee? Most of the epic developments in England between 1650 and 1800 that still warrant a mention in the history textbooks have a coffeehouse lurking at some crucial juncture in their story. The restoration of Charles II, Newton’s theory of gravity, the South Sea Bubble— they all came about, in part, because England had developed a taste for coffee, and a fondness for the kind of informal networking and shoptalk that the coffeehouses enabled. Lloyd’s of London was once just Edward Lloyd’s coffeehouse, until the shipowners and merchants started clustering there, and collectively invented the modern insurance company. …coffeehouse culture was cross-disciplinary by nature, the conversations freely roaming from electricity, to the abuses of Parliament, to the fate of dissenting churches.

But the coffeehouse as a nexus of debate was only half of the picture. Cultural practice at the time was to drink beer and wine, and maybe a little gin, at every opportunity. Water was not safe to drink, and so alcoholic alternatives were fondly embraced. The introduction of coffee and tea as popular beverages had a significant impact on the flow of valuable ideas. Again here’s Johnson:

The rise of coffeehouse culture influenced more than just the information networks of the Enlightenment; it also transformed the neurochemical networks in the brains of all those newfound coffee-drinkers. Coffee is a stimulant that has been clinically proven to improve cognitive function— particularly for memory related tasks— during the first cup or two. Increase the amount of “smart” drugs flowing through individual brains, and the collective intelligence of the culture will become smarter, if enough people get hooked.

In our day, the coffee house connected to a wifi network has been an accelerant to the businesses populating the Network. When Starbucks announced that they would be introducing free 1-click wifi in their stores, it reminded me of Stephen Johnson’s descriptions of the London coffeehouses. The coffeehouse provided a physical meeting place and the caffeine in the coffee provided a force multiplier for the ideas flowing through the people. There was a noticeable change in the rhythm of the age. By layering a virtual real-time social medium over a physical meeting place that serves legal stimulants, Starbucks replays a classic formula. Oddly, there’s a kind of collaborative energy that exists in the coffeehouse that has been completely expunged from the corporate workplace. Starbucks ups the ante by running a broadcast web service network through the connection. Here we see wifi emerging as the new backbone for narrowcasted television.

As we try to weave value-laden real-time message streams through the collaborative groupware surgically attached to the corporate balance sheet, we may do well to look back toward Bloomsday and also ask for a stream of unconsciousness. It’s in those empty moments between the times when we focus our attention that daydreams and poetic thought creep into the mix. Those “empty moments” are under attack as a kind of system latency. However it’s in those day dreams, poetic thoughts and napkin scribbles that we find the source of the non-linear jump. Without those moments in our waking life, we’re limited to only those things deemed “possible.”