Archive for January, 2011

Next Entries »

Of Twitter and RSS…

It’s not really a question of life or death. Perhaps it’s time to look for a metaphor that sheds a little more light. The frame that’s been most productive for me is one created by Clayton Christensen and put to work in his book, The Innovator’s Solution.

Specifically, customers—people and companies— have “jobs” that arise regularly and need to get done. When customers become aware of a job that they need to get done in their lives, they look around for a product or service that they can “hire” to get the job done. This is how customers experience life. Their thought processes originate with an awareness of needing to get something done, and then they set out to hire something or someone to do the job as effectively, conveniently and inexpensively as possible. The functional, emotional and social dimensions of the jobs that customers need to get done constitute the circumstances in which they buy. In other words, the jobs that customers are trying to get done or the outcomes that they are trying to achieve constitute a circumstance-based categorization of markets. Companies that target their products at the circumstances in which customers find themselves, rather than at the customers themselves, are those that can launch predictably successful products.

At a very basic level, people are hiring Twitter to do jobs that RSS used to get. The change in usage patterns is probably more akin to getting laid off. Of course, RSS hasn’t been just sitting around. It’s getting job training and has acquired some new skills like RSS Cloud and JSON. This may lead to some new jobs, but it’s unlikely that it’ll get its old job back.

By reviewing some of the issues with RSS, you can find a path to what is making Twitter (and Facebook) successful. While it’s relatively easy to subscribe to a particular RSS feed through an RSS reader— discovery and serendipity are problematic. You only get what you specifically subscribe to. The ping server was a solution to this problem. If, on publication of a new item, a message is sent to a central ping server, an index of new items could be built. This allows discovery to be done on the corpus of feeds to which you don’t subscribe. The highest area of value is in discovering known unknowns, and unknown unknowns. To get to real-time tracking of a high volume of new items as they occur, you need a central index. As Jeff Jonas points out, federated systems are not up to the task:

Whether the data is the query (generated by systems likely at high volumes) or the user invokes a query (by comparison likely lower volumes), there is nodifference.  In both cases, this is simply a need for — discoverability — the ability to discover if the enterprise has any related information. If discoverability across a federation of disparate systems is the goal, federated search does not scale, in any practical way, for any amount of money. Period. It is so essential that folks understand this before they run off wasting millions of dollars on fairytale stories backed up by a few math guys with a new vision who have never done it before.

Twitter works as a central index, as a ping server. Because of this, it can provide discovery services on to segments of the Network to which a user is not directly connected. Twitter also operates as a switchboard, it’s capable of opening a real-time messaging channel between any two users in its index. In addition, once a user joins Twitter (or Facebook), the division between publisher and subscriber is dissolved. In RSS, the two roles are distinct. Google also has a central index, once again, here’s Jonas:

Discovery at scale is best solved with some form of central directories or indexes. That is how Google does it (queries hit the Google indexes which return pointers). That is how the DNS works (queries hit a hierarchical set of directories which return pointers).  And this is how people locate books at the library (the card catalog is used to reveal pointers to books).

A central index can be built and updated in at least two ways. With Twitter, the participants write directly into the index or send an automated ping to register publication of a new item. Updates are in real time. For Google, the web is like a vast subscription space. Google is like a big RSS reader that polls the web every so often to find out whether there are any new items. They subscribe to everything and then optimize it, so you just have to subscribe to Google.

However, as the speed of publication to the Network increases, the quantity of items sitting in the gap between the times the poll runs continues to grow. A recent TPS Report showed that a record number, 6,939 Tweets Per Second, were published at 4 seconds past midnight on January 1, 2011. If what you’re looking for falls into that gap, you’re out of luck with the polling model. Stock exchanges are another example of a real-time central index. Wall Street has lead the way in developing systems for interpreting streaming data in real time. In high-frequency trading, time is counted in milliseconds and the only way to get an edge is to colocate servers into the same physical space as the exchange.

The exchanges themselves also are profiting from the demand for server space in physical proximity to the markets. Even on the fastest networks, it takes 7 milliseconds for data to travel between the New York markets and Chicago-based servers, and 35 milliseconds between the West and East coasts. Many broker-dealers and execution-services firms are paying premiums to place their servers inside the data centers of Nasdaq and the NYSE.

About 100 firms now colocate their servers with Nasdaq’s, says Brian Hyndman, Nasdaq’s SVP of transaction services, at a going rate of about $3,500 per rack per month. Nasdaq has seen 25 percent annual increases in colocation the past two years, according to Hyndman. Physical colocation eliminates the unavoidable time lags inherent in even the fastest wide area networks. Servers in shared data centers typically are connected via Gigabit Ethernet, with the ultrahigh-speed switching fabric called InfiniBand increasingly used for the same purpose, relates Yaron Haviv, CTO at Voltaire, a supplier of systems that Haviv contends can achieve latencies of less than 1 millionth of a second.

The model of colocation with a real-time central index is one we’ll see more of in a variety of contexts. The relationship between Facebook and Zynga has this general character. StockTwits and Twitter are another example. The real-time central index becomes a platform on which other businesses build a value-added product. We’re now seeing a push to build these kinds of indexes within specific verticals, the enterprise, the military, the government.

The web is not real time. Publishing events on the Network occur in real time, but there is no vantage point from which we can see and handle— in real time— ‘what is new’ on the web. In effect, the only place that real time exists on the web is within these hubs like Twitter and Facebook. The call to create a federated Twitter seems to ignore the laws of physics in favor of the laws of politics.

As we look around the Network, we see a small number of real-time hubs that have established any significant value (liquidity). But as we follow the trend lines radiating from these ideas, it’s clear we’ll see the attempt to create more hubs that produce valuable data streams. Connecting, blending, filtering, mixing and adding to the streams flowing through these hubs is another area that will quickly emerge. And eventually, we’ll see a Network of real-time hubs with a set of complex possibilities for connection. Contracts and treaties between the hubs will form the basis of a new politics and commerce. For those who thought the world wide web marked the end, a final state of the Network, this new landscape will appear alien. But in many ways, that future is already here.

No Nature: Thinking About Gary Snyder

It’s a phrase that fascinated using only three words. “Ecology without nature.” It’s the title of a book by Timothy Morton, and refers to the romantic notion of nature that infuses much of our ecological thinking. It’s nature as it appeared before the fall, before the apple was bitten by reality. Not nature as it was formed in the crucible of Darwin’s natural selection, but rather as the dream of a machine spinning along in perfect balance. Human beings, somehow standing on the outside, have upset that balance.

I’m reminded of poet Robert Haas’s story about Nobel Laureate Czeslaw Milosz. Haas was organizing a benefit for some nature organization. He wanted Milosz to read and tried to play on what he thought was Milosz’s love of nature. Milosz starred blankly. “Nature? Nature terrifies me.” Confused Haas reels off a list of sunsets, forests, sparkling rivers, night skies and rolling hills. Milosz nodded. “Ah…you mean beauty. There’s a huge difference.”

For Morton, ecology must be thought through a democracy of objects. Humans, fish, plastic bags, trees, snow tires and bongos all live and work within the same flat ontology. At every scale, we’re all in this together, human being isn’t privileged, rather it is one being among many. Gary Snyder comes at the question from another direction. He engages in what he calls the practice of the wild. The poet tells us how nature calls nature:

“It would appear that the common conception of evolution is that of competing species running a sort of race through time on planet earth, all on the same running field, some dropping out, some flagging, some victoriously in front. If the background and foreground are reversed, and we look at it from the side of the ‘conditions’ and their creative possibilities, we can see these multitudes of interactions through hundreds of other eyes. We could say a food brings a form into existence. Huckleberries and salmon call for bears, the clouds of plankton of the North Pacific call for salmon, and salmon call for seals and thus orcas. The Sperm Whale is sucked into existence by the pulsing, fluctuating pastures of squid, and the open niches of the Galapagos Islands sucked a diversity of bird forms and function out of one line of finch.”

Sometimes it takes a while before we can hear a poet speak. This may be the decade that we hear Gary Snyder.

Ripples on the Surface

by Gary Snyder

“Ripples on the surface of the water—
were silver salmon passing under—different
from the ripples caused by breezes”

A scudding plume on the wave—
a humpback whale is
breaking out in air up
gulping herring
—Nature not a book, but a performance, a
high old culture

Ever-fresh events
scraped out, rubbed out, and used, again—
the braided channels of the rivers
hidden under fields of grass—

The vast wild
the house, alone
The little house in the wild,
the wild in the house
Both forgotten.

No nature

Both together, one big empty house.

Standing On The Corner: Reality Bites

It’s right on the crease that the thoughts began to emerge. Like standing on the corner of a city block and looking down one side and then the other. Seeing old friends from different times in your life, paths that never crossed—now connected by the happenstance of standing on this particular node in the grid-work of the metropolis. The term standing at this crossroads is ‘realism.’

The initial rehabilitation of the word, for me, came with the discovery of John Brockman’s Edge.org. Within this oasis, Brockman unleashed the congregations of the Third Culture and The Reality Club. These closed circles of the best and the brightest engage in a correspondence on topics at the edge of technology and science. In particular, Brockman was seeking to provide an escape from the swirl of ‘commentary on commentary’ that seemed to be gobbling up much of the intellectual world as it struggled to digest the marks and traces left by Jacques Derrida. Here, conversations could gain traction because the medium was the “real” and the language was the process of science. Even the artists and philosophers included within the circle had a certain scientific bent.

However, recently I’ve begun to feel that the conversations have drifted from scientific to the scientistic. Standing at the edge of scientific discovery is a heady experience. The swirl of the unknown is trapped in the scientist’s nets, sorted out into bits of data, classified and tested. Edge.org serves as a sort of cross-scientific discipline peer review process. The shaky ground of the barely known is given its best chance to gain traction through an unstinting faith in the real. At this far outpost, anything seems to be fair game for the process. Standing on the firm ground of the scientific real, the conversations begin to stray into explanations and reconstructions of morality, thinking, consciousness and religion. Edifices are not deconstructed, they are bulldozed and rebuilt on the terra firma of scientific reality.

Even within Edge.org, the question about the ground on which they stand are starting to be asked. Jaron Lanier focuses on why there’s an assumption that computer science is the central metaphor for everything:

One of the striking things about being a computer scientist in this age is that all sorts of other people are happy to tell us that what we do is the central metaphor of everything, which is very ego-gratifying. We hear from various quarters that our work can serve as the best way of understanding – if not in the present but any minute now because of Moore’s law – of everything from biology to the economy to aesthetics, child-rearing, sex, you name it. I have found myself being critical of what I view as this overuse as the computational metaphor. My initial motivation was because I thought there was naive and poorly constructed philosophy at work. It’s as if these people had never read philosophy at all and there was no sense of epistemological or other problems.

And it’s here that faith in the scientistic ground begins to develop fissures. A signal event for me was the appropriation of the word ‘ontology‘ by the practitioners of the semantic web. The word is taken up and used in a nostalgic sense, as though plucked from a dead and long-ago superseded form of thought. The history of the word is bulldozed and its meaning reconstructed within the project of creating a query-able web of structured data.

It was the word ontology that linked me back to realism. And here we are back at the crease, looking down the other side of the block. It’s here that the fast charging world of Speculative Realism enters the fray. The scientistic thinkers on the Edge have begun to notice a certain mushiness of the ground as they reach out to gain traction in some new territories. Indeed, some may stop and ask how the ground could be mushy in some spots, but not in others?

The brand Speculative Realism was founded in April of 2007, at a conference at Goldsmiths College, University of London. The primary players were Graham Harman, Ray Brassier, Iain Hamilton Grant, and Quentin Meillassoux. While not a cohesive school of thought, these philosophers have certain common concerns, in particular ideas about realism and a critique of correlationism. The branch of the tree of particular interest to me contains the group exploring Object-Oriented Ontology, which includes Graham Harman, Timothy Morton, Ian Bogost and Levi Bryant among others.

Ontology is the philosophical study of existence. Object-oriented ontology (“OOO” for short) puts things at the center of this study. Its proponents contend that nothing has special status, but that everything exists equally—plumbers, cotton, bonobos, DVD players, and sandstone, for example. In contemporary thought, things are usually taken either as the aggregation of ever smaller bits (scientific naturalism) or as constructions of human behavior and society (social relativism). OOO steers a path between the two, drawing attention to things at all scales (from atoms to alpacas, bits to blinis), and pondering their nature and relations with one another as much with ourselves.

My formal introduction to the literature was through Graham Harman’s book Prince of Networks, Bruno Latour and Metaphysics. But to get a sense of the pace of thought, you need only look to the blog posts, tweets, YouTube posts, uStream broadcasts of conferences and OpenAccess publications the group seems to produce on a daily basis. The recent compendium of essays, The Speculative Turn, is available in book form through the usual channels, or as a free PDF download. The first day it was made available as download, the publisher’s web servers were overwhelmed by the demand. The velocity of these philosophical works, and the progress of thought, seems to be directly attributable to its dissemination through the capillaries of the Network.

In working with ontology, these thinkers have given the ground on which scientists—and the rest of us (objects included) stand, quite a bit of thought. This is not an extension of the swirl of commentaries on commentaries, but rather a move toward realism. And it’s when you arrive at this point that the border erected around the scientistic thought and conversations of the Edge.org begins to lose its luster. There are clearly questions of foundation that go begging within its walls. At the beginning of such a conversation, the ground they’ve taken for granted may seem to fall away and leave them suspended in air, but as they continue, a new ground will emerge. And the conversation will be fascinating.

Next Entries »