Archive for May, 2008

« Previous Entries Next Entries »

When Search splinters, will large pools of the Web go dark?

Sun spots

The point at which one competitor in the market begins to achieve unassailable dominance is the moment when the seeds of change are sown. Search is about to change, you can feel it in the air.

You can measure the quality of Google’s search results by searching for something and reviewing the usefulness of the first two pages of results. For example, the first result for the query “search engine” on Google is a link to “Alta Vista.” Google also indicated that there are 118,000,000 links in the result set. I couldn’t find any simple way to find the last result, the link that Google ranked as the lowest in importance. But since users rarely look beyond the second page of search results, all the rest is a puppet show. The business of Search is the quality of the first two pages of search results. For that search, the only link of interest was to Wikipedia, and Google itself only showed up as its UK site on the second page.

In a sense, this is why Mahalo can “compete” with Google. Mahalo doesn’t need to index the whole web and come up with 118 million links. No one cares about 118 million links. There’s a small consideration set that actually satisfies the query.

And further, a page of links is just a page of pointers, it’s the content that answers the question. This is why Mahalo is now offering a higher content to link ratio; it can be an endpoint rather than a relay station. The attack surface revealed is the understanding what is truly human readable and what satisfies a search query.

“Search” could be disrupted by many approaches: we want a better starting point that links to the thing we’re looking for. Twitter or Delicious could be pointers towards that new thing — Search as a back and forth conversations within a tribe, and contiguous tribes; Search of a subset of pages users cared enough about to bookmark (user gestures). The citation algorithm was a huge step forward in ranking the value of pages based on a keyword search. Citation is no longer enough, as Ray Ozzie notes, users now commonly link, share, rank and tag.  Currently search is anonymous, connecting to a preference set or a user profile could yield more valuable results.

The rise of specialized search raises the specter that some day the entire web will no longer be spidered and indexed. The economics of search are tied to a subset of search queries related to potential commercial transactions. Commercial search subsidizes all other search activity. At some point, that linkage will be cut. As search splinters and begins to operate in verticals, much of the web could go dark.

Intermediation 2.0: The New Rhizomatic Economy

In the spirit of assigning numbers to ideas, I introduce Intermediation 2.0– a new era of data wholesalers and value-added retailers.

Retail distribution’s primary value in physical space is coverage of relevant geography, it’s about location. The Network annihilated distance and a wave of disintermediation followed.

Mahalo, Summize, Friend Feed, Mint, Techmeme, Twhirl, Newsgang and even Google Friend Connect take wholesale data feeds and add value through a transformation of the raw data, the addition of tools, and sometimes curation.

The first generation was Amazon, Travel aggregators, and the news/RSS readers. What makes Intermediation 2.0 possible is the emergence and recognition of new valuable wholesale data feeds. Twitter as a wholesale full-stream event based XMPP feed establishes a new economy around making that stream more valuable to the user.

The question hanging over this new economy is the threat that another round of disintermediation will follow. Developers have bitter memories of platform vendors incorporating their software products into the platform without compensation.

This is where the new Rhizomatic Economy emerges. If 75% of Twitter exists through API usage, the growth and extension of the Twitter stream depends on retailing partners. Twitter improves and grows by providing better API functionality to its partners. Twitter benefits through better distribution and fosters faster innovation.

The days of the totalizing whole, the drive to monopoly and hegemony, of being locked in the trunk, are coming to an end.

Small Bits of the Future Distributed in Cleveland

At a Cleveland American Advertising Federation luncheon today, Larry Weber talked to a room full of traditional PR and marketing types about “marketing” and social networks. While the talk was mostly a new coat of paint on the Cluetrain Manifesto, it was interesting that this group of people showed up in good numbers to listen. As the talk went on I could feel that the room, even at this late date, was skeptical of his premise that markets are conversations with communities.

Weber suggests that big brands should be hosting honest conversations containing both positive and negative messages about their products. He recommended building communities from scratch around a brand, and implied that the brand should want to keep the users inside their own walled garden. In fact, he suggested that the network’s future will be filled with social network-based walled gardens existing as a form of client loyalty program. No mentions of VRM or the role OpenID will play in the future of the commercial web. And not even a hint of the way that Google’s Friend Connect might bring existing social networks to a brand’s site, rather than building a new community from the ground up.

Only small bits of the future were distributed in Cleveland by a guy from Boston. It’s a small sample, but it gives you a sense of the information asymmetry in the market that values the social web. It’s the definition of opportunity.

The Network’s Continental Congress: Searls, Gillmor, Windley, Arrington

Mark the week of May 11, 2008 as a milestone in the nascent politics of the Network, attending the Internet Identity Workshop, I had the feeling of participating in a kind of Continental Congress. In this regard, Doc Searls and Phil Windley are statesmen of the Network doing the good work of bringing the small and large states to the table to talk about politics, technology, interstate commerce, identity and inalienable rights. Acknowledgement is also due to Kaliya for her critical role in facilitating the dialogue through the unconference format.

Windley described the early meetings of the IIW as “brawling” among major vendors and protocol authors. A form of what Thomas Hobbes called bellum omnium contra omnes (the war of all against all). By the sixth meeting of the IIW it’s become clear that OpenID, Cardspace and SAML are emerging as the dominant forces.

The week was capped by a brawling Gillmor Gang with Robert Scoble and Mike Arrington engaged in heated debate over ownership of identity artifacts. The wall at the back of Marc Canter’s revolutionary garden emerged as an offline visual rough draft of our constitutional articles. Canter also put forward the metaphor of tentacles as the current mode of portability, and this connects directly to my observation of the general transition to rhizomatic power architectures.

Chris Saad injected the data portability meme into the flow and suggested personal Access Control Lists, in the form of a “Sharing OK/Not OK” check box on data you give to individuals or companies. It would be interesting to watch Robert Scoble manually configure a complex ACL on his 20,000+ friends (Scoble rushes in where Angels fear to tread).

The one thread I would contribute to the conversation is the idea of data liquidity. Think of data as a financial asset, as cash it is at its most simple and portable; the other end of the spectrum might be venture capital where it is complex and liquidity events are highly limited. We trade liquidity for potentially higher value when we park our data at Google, Microsoft, Facebook, Twitter or Yahoo.

  You can hear for yourself here:

In the general argument about identity and user data, Steve Gillmor has played the role of Alexander Hamilton. He argues that because we own our data, we don’t need to petition for rights not surrendered, or for exceptions to powers which were not granted. Hamilton objected to the Bill of Rights on the following grounds:

Bills of rights are in their origin, stipulations between kings and their subjects, abridgments of prerogative in favor of privilege, reservations of rights not surrendered to the prince. Such was “Magna Charta“, obtained by the Barons, swords in hand, from King John.

Hamilton continued:

I go further, and affirm that bills of rights, in the sense and in the extent in which they are contended for, are not only unnecessary in the proposed constitution, but would even be dangerous. They would contain various exceptions to powers which are not granted; and on this very account, would afford a colorable pretext to claim more than were granted. For why declare that things shall not be done which there is no power to do?

These are dynamic times akin to the days in 1787 when the Federalist Papers were authored. Hamilton, Madison and Jay (under the pseudonym “Publius“) often published between three and four essays per week. Doc Searls just announced at the Berkman@10 event: The Publius Project is a “compilation of ‘essays and conversations about constitutional moments on the Net’ at the singular moment that happens to be now.” There was a moment in all this when I thought we were laying down cow paths onto paved roads, but Doc has connected the dots at just the right moment.

During the process of thinking and writing about the week that was, a song kept playing in the background of my thoughts. The song is called “Playmate” and was written in 1894 by Philip Wingate. I first heard it in the film Reds, and more recently in The Savages. I can’t put my finger on it, but there’s something about it that expresses the emotional and human center of these constitutional conversations.

I don’t want to play in your yard,
I don’t like you anymore,
You’ll be sorry when you see me,
Sliding down our cellar door,
You can’t holler down our rainbarrel,
You can’t climb our apple tree,
I don’t want to play in your yard,
If you won’t be good to me.


« Previous Entries Next Entries »