Skip to content →

Category: digital

The Source: Algorithmic Authority and Unsupervised Systems of Record

termitecathedral

At the recent Real-Time Crunch Up, a number of interfaces presented themselves; many of them worth reading and writing into. But, I’d like to explore a turn of phrase that slipped out causally in the last panel. The other members of the panel simply nodded and moved with the flow of the thought, it was understood to reflect the state of affairs. Dan’l Lewin, of Microsoft, while discussing which of the companies we’d seen during the day might become dominant players in the real-time environment of the Network, used the phrase: “system of record.” Lewin was referring to Twitter and Facebook as new systems of record on the Network.

Wikipedia is a little light on its definition of systems of record:

A system of record (SOR) is an information storage system (commonly implemented on a computer system), which is the authoritative data source for a given data element or piece of information. The need to identify systems of record can become acute in organizations where management information systems have been built by taking output data from multiple source systems, re-processing this data, and then re-presenting the result for a new business use.

The key here is the phrase: “authoritative data source for a given data element or piece of information.” The data element Lewin was referring to is the public social graph in its unfolding as real-time, tick-by-tick, context data. If you look at Microsoft, Google, Apple, Yahoo, the financial and medical institutions, etc. — none of the current big players on the Network have the social contract that allows them to serve as the system of record for the real-time social context data set.

Each has a system of record with some essential piece of the picture; they serve as the source of some authoritative piece of data. None of them can be authoritative for every aspect of a person. And generally, we like to be able to choose among a minimum of two providers (SORs) for each piece of our represented selves. Companies like Acxiom, Mint/Yodlee, and Equifax have begun the process of aggregating identity across key authoritative data silos, creating connections and drawing the broad outlines. In order for there to be an aggregation point, there has to be a set of authoritative systems of record.

As I began to think about the authority of these interconnected data elements — another connection placed itself in the frame. We sometimes speak of the newspaper of record. Another kind of authority presents itself:

The first type of newspaper of record (or newspaper of public record) is often formally defined by a statute or other official action of a governing body. Such a newspaper is supposed to be available to the public, and publication of notices in that newspaper is considered sufficient to comply with legal requirements for public notice.

The second type of “newspaper of record” is not defined by any formal criteria. The use of the term implies that a newspaper is a reliable institution that publishes trustworthy descriptions of events, but this assessment may be disputed. Major newspapers of record may be expected to have independent editorial policies, and to publish statements of opinion that are distinct from those of their proprietor or their government. They are more likely than other newspapers to be sold abroad and to be cited in scholarly publications.

Clay Shirky has become enmeshed in the discussions around the endgame for newspapers. I wouldn’t exactly call it a debate, because only mainstream journalists are interested in debating the point. Everyone else seems to have moved on. In his post, On the Idea of Algorithmic Authority, he explores what we mean when we talk about authoritative sources. In essence, he’s exploring the pragmatism of a kind of anarchy.

When we use the phrases “system of record” and “newspaper of record,” we’re trying to get to a similar level of authority. Newspapers of record supply the day-to-day transcription of important events. It’s the “availability to the public” of the newspaper and its broad distribution that makes up the public record. The authority of the newspaper rests in an editorial process that outputs “trustworthy descriptions of events.”

Shirky, in his book Here Comes Everybody, looks at organizing without organizations— and this idea is extended here to explore the level of authority that can be achieved by an unsupervised process. Typically we look for some kind of certification, an institutional process guaranteed by a professional in charge. The change that Shirky chronicles is the expansion of the kinds of processes that can produce authoritative output. When an unsupervised algorithmic process can produce an output that people respect as authoritative, the economics of supervised certified processes are disrupted. The ecosystem is enlarged and the economics irrevocably changed. In this case authority isn’t replaced, rather its sources are multiplied.

As our view of the ecosystems of record on the Network begins to come into focus, an emerging landscape starts to take shape. And what looked like a field with thousands of players is quickly reduced to a small number of authoritative systems.

Comments closed

While Interpreting The Instruction Set, I Encounter An Ambiguous Opening…

playerpiano

We seem to live in an age where the algorithm serves as an extension of our desire. This picaresque ramble contemplates what it means to follow a rule, what the boundaries of a rule are meant to contain, and the morality of rewriting and overwriting a rule. In his book on the algorithm, David Berlinski provides a definition:

In the logician’s voice

an algorithm is
a finite procedure,
written in a fixed symbolic vocabulary,
governed by precise instructions,
moving in discrete steps, 1, 2, 3, …,
Whose execution requires no insight, cleverness, intuition, intelligence, or perspicuity,
and that sooner or later comes to an end,

Within the conceptual machine of the algorithm, we envision the creation of software agents that will encapsulate and encode our desires. These secret agents will be unleashed upon the virtual and augmented world to locate the conditions and process the data that match a predetermined map of our dreams and passions.

alice-in-front-of-rabbit-hole9

But let’s hit the back button on this train of thought. The interface for this exploration appeared the other night at an art opening. Paul Madonna was introducing his new work at Electric Works. On a table in the entry way to the exhibition were a number of books. In addition to Paul’s new book, there was a book by David Byrne called Arboretum. This book contained a number of drawings that I immediately wanted to willfully misinterpret. Byrne’s drawings are mind maps that show connections/relations between things— they represent a kind of systemic history in some cases, or the dynamics/economics of a system in others. They have a strong relationship to Diderot’s organizational ideas for his encyclopedia.

byrne_history_of_mark-making

David Byrne, History of Mark-making

There was something about the visual character of Byrne’s drawings that reminded me of the music notation and compositional titles of the saxophonist Anthony Braxton. When music notation is at its most strict, it tells us precisely how a piece can be replicated. The player piano uses piano rolls to produce near identical performances. In the fake book, we might just get the chord changes and the melody— the arrangement is up to us. Braxton takes this kind of abstraction to another level in some of his notation that borders on encoding synesthesia— where color and shape are meant to guide the performers.

braxton14-400x293

Braxton’s Composition #76

Somehow by combining the notation of Braxton and the tree drawings of Byrne, I imagine conjuring up a notation system for an exploration through conversation, a kind of performance script. While, as Umberto Eco notes, the list can be a flexible tool, engendering both anarchic and organizational impulses— I find myself drawn to these maps of notation. When I engage in conversations about strategic direction, I always imagine them taking place within a terrain with specific dynamics.

By Berlinski’s definition, these musical instruction sets aren’t algorithms. Their execution requires insight, cleverness, intuition, intelligence, and perspicuity. The performer has to make decisions, exercise options, contribute variable inputs that will result in a variety of outputs. In that sense, they function more like a game.

In business, there’s an attempt to codify process to the extent that all of its aspects are substitutable. Even as parts of the machine are replaced, it’s output remains constant and consistent. This is the industrial commodity as ideal. Variances are a sign of poor management.

The script for a theatrical performance is another kind of instruction set. Notational experiments in this realm are also highly instructive. A few years back I attended a performance of the Wooster Group’s Poor Theater. The text of the performance was largely other performances, and the group’s performance itself described as a simulacra. For instance a performance piece, where the movements are the performers’ response to, and modeling of, a cowboy film projected to the side of the stage.

Elizabeth-LeCompte

Elizabeth LeCompte by Leibowitz

Elizabeth LeCompte experiments with both what counts as a performance text, and what counts as a vital interpretation. She even refuses to be limited to a single text (instruction set), with the Wooster Group’s La Didone she weaves together a performance based on Cavalli’s opera combined with Mario Brava’s 1965 science fiction film Planet of the Vampires.

Cavalli’s Didone

Brava’s Planet of the Vampires

Wooster Group’s Didone

The mashup, the remix, the blending of instruction sets to produce something entirely new is what the process of creation has always already been. The boundary between recipes loses importance if the meal is well presented and delicious— a new recipe is created.

These two relationships to rule sets define much of human experience. The one approaches the regularity of the machine, while the other can careen off into what seems to be unbounded chaos. One set must be followed to the letter (a machine is the optimal performer), the other leaves openings for a two-way interaction. But the act of writing back into the interface is fraught with danger. It stands on the border of transgression, or transumption. When we don’t follow a rule set, but instead apply a new rule set from different context, we can be perceived as willfully misreading, incompetence or breaking local laws. Edward Said, in his reading of Harold Bloom’s A Map of Misreading, describes the Oedipal resonances:

Thus Bloom writes: “To live, the poet must mis-interpret [his literary] father, by the crucial act of misprision, which is the re-writing of the father.” Consequently a poet is not a man speaking to other men, but “a man rebelling against being spoken to by a dead man (the precursor) outrageously more alive than himself.” The great poetic ambition, which only the strongest poets achieve, is to appear self-begotten not only free of the father but, as Bloom demonstrates beautifully in the case of Milton (who is Bloom’s own strong poet par excellence), the father’s father. This final “transumptive” act of poetic majesty Bloom calls metalepsis: “Milton does what Bacon hoped to do; Milton and Galileo become ancients, and Homer, Virgil, Ovid, Dante, Tasso, Spenser become belated moderns.”

The fixed symbolic vocabulary of the algorithm is one of the points where misreading wants to overwrite fixity. Meanwhile back on the Network, there’s a sense in which Phil Windley’s concept of a purpose-centric web dares to ignore the local laws and advocates simply rewriting/overwriting the fixed symbolic vocabulary to serve another purpose. There’s a sense in which we can view this as another instance of text, interpretation and performance. The revolutionary idea of the Action Card is that my rule set trumps yours.

byrne_what_is_it

To return to the moment that started this train of thought, let’s look at Byrne’s Arboreum drawings as a performance text. It seems as though the fixed symbolic vocabulary becomes slippery when it moves from linear typography to a map or model. The symbolic moves from symbol to symbol. The adjectives finite, discrete, governed, fixed, and  precise all seem to lose purchase. And yet if we look at the Wooster Group’s rigorous performances, we could apply all of those adjectives along a different dimension.

These two relationships to the text stand across from each other as mirror images.  As the algorithm blends with desire and takes flight into the real time flow of the Network, our sense of logic may sometimes take on the guise of the logic of sense, and it will have to learn to keep its cool as it makes the occasional trip through the looking glass.

Comments closed

Stigmergy: Writing is a Real-Time Gesture

surrealism_escher

My notebook is filling up with musings on the real-time web. I keep trying to boil things down to the simplest formulation, the simplest expression of why the Network is moving into a real-time mode. It’s more difficult than one would imagine to create a palatable reduction. While we can apply Occam’s razor at a certain level, so much flavor is lost when the particular is replaced with the abstraction.

Occam’s razor states that the explanation of any phenomenon should make as few assumptions as possible, eliminating those that make no difference in the observable predictions of the explanatory hypothesis or theory. The principle is often expressed in Latin as the lex parsimoniae (translating to the law of parsimony, law of economy or law of succinctness). When competing hypotheses are equal in other respects, the principle recommends selection of the hypothesis that introduces the fewest assumptions and postulates the fewest entities while still sufficiently answering the question.

When the web was an exercise in reading, there was no need for real-time. The professional infrastructure of writing and publishing didn’t require much change in the move from offline to online. There was a bright line between professionally produced writing and the amateur personal home page. The pace of publication was a function of the competition between professional writing organizations. The reading of professionally produced writing has always been a distributed affair. A scarce number of writers produce writing for the abundant population of readers. Economics and value ensue from these kinds of ratios.

Search engines are largely based on the traditional economic model of the production of writing. What is returned for a search query should not only be what you’re looking for, but should be authoritative on the subject. Historically we’ve associated the kind of writing product emitted from the professional writing and publishing infrastructure as the most authoritative. While there’s not an explicit provenance, there is an implicit one based on the gesture of the citation link.

As originally conceived, the world wide web was a read-write environment. But clearly the two gestures did not occupy equivalent environments. Reading required a computer, web browser and a connection; writing required so much more. This difference in friction determined the early patterns of development for the Network. To some extent it also deferred the disruption of the established writing and publishing infrastructure.

Writing existed on the Network, but it was contained in the backwaters of the UseNet, Mailing List and the BBS. Real-time writing was limited to instant messenger, internet relay chat and the UNIX talk application. Isolated networks like the Plato System provided a highly sophisticated read-write environment that modeled many of the challenges that today’s Network is confronting.

Reducing the friction in writing to the Network, and here I’m referring to the world wide web, really began with the advent of widely available blogging software. This movement was accelerated by Facebook, MySpace, Twitter, FriendFeed and others. Twitter, in particular, lowered the level of friction to almost zero. This is why a person can now write about what they had for lunch and publish it to the Network. Each of these services is a read-write social environment with public and private publication/broadcast modes. On the level of public gestures, they provide the same level of connectivity as any other public node on the Network.

So let’s loop back to the real-time web. It’s simply the gesture of writing, of making a mark on the Network, that has necessitated the move to real time. But here when we speak of writing, it is a different writing. We don’t refer to the industrially produced writing product created for mass consumption. Instead we refer to making a mark, a gesture, in a dynamic networked environment. The rather clumsy word Stigmergy has been used to draw a circle around some of these ideas.

Stigmergy is a mechanism of spontaneous, indirect coordination between agents or actions, where the trace left in the environment by an action stimulates the performance of a subsequent action, by the same or a different agent. Stigmergy is a form of self-organization. It produces complex, apparently intelligent structures, without need for any planning, control, or even communication between the agents. As such it supports efficient collaboration between extremely simple agents, who lack any memory, intelligence or even awareness of each other.

It is derived from the Greek words stigma (mark, sign) and ergon (work, action), and captures the notion that an agent’s actions leave signs in the environment, signs that it and other agents sense and that determine and incite their subsequent actions

It’s the gesture that necessitates the real-time web. Through public gestures, we make marks in the environment that others can sense and to which they can respond. The latency in the Network needs to be low enough for a flow to occur. The time of the real-time web is a technical speed that enables this flow of marks, traces, actions, gestures to dynamically connect to other marks, traces, actions and gestures in an ongoing loop and become visible to a micro-community that defines the larger emergent social objects.

Writing, in this sense of the word, is no longer about something. It is the thing itself.

One Comment

Space/Name Space: “Syndication Doesn’t Make Sense In The Age Of The URL”

clay-shirky

I’d like to take something Clay Shirky said out of  context. First of all, here’s the context from which I’m going to extract the quote: Shirky gave a talk to a group of journalists about the forward visibility of what he calls “Accountability Journalism.”  There are a couple excellent posts on Shirky’s talk by Ethan Zuckerman and David Weinberger. Both are highly recommended reading. The bottom line seems to be that while Shirky, at least, is beginning to be able to articulate why newspapers, as a media type, are unsustainable— visibility into the method by which “accountability journalism” will perdure is very limited. Listening to the Q&A after the talk brought to mind a song by Aimee Mann.

Oh, better take the keys and drive forever
Staying won’t put these futures back together
All the perfect drugs and superheroes
wouldn’t be enough to bring me up to zero
All the king’s horses and all the king’s men
couldn’t put baby together again
All the king’s horses and all the king’s men
couldn’t put baby together again

Aimee Mann
Humpty Dumpty, from the album Lost in Space

The journalists in attendance continued to sift through the pieces of egg shell looking for the formula that will put it all back together. I imagine them watching Shirky closely for some signal that pay walls or micro-payments just might be the glue for pieces they’ve been left holding.

Shirky makes clear that he values what “accountability journalism” provides— investigative journalism that holds people, corporations, governments and other institutions accountable for their actions is a crucial function in a democratic society. However, the notion that only newspapers, or news organizations, as they are currently constituted, can fill that need fails to heed the lessons of history.

And while the thread of this discussion is extremely important, I was taken off track by a phrase thrown out by Shirky in the middle of supporting one of his points. And this is where I’d like to remove this sentence from its context and treat it as a standalone fragment. It addresses the mechanics of distribution in space and name space:

“Syndication doesn’t make sense in the age of the URL, as AP has figured out, which is why they’re driving people towards their own content.�

Clay Shirky
in a talk to the Shorenstein Center
for the Press, Politics and Public Policy

The business of syndication is distributing copies of material to non-overlapping localities in physical space. Something produced for one locality can be leveraged into new markets for the cost of sales and distribution. Electronic distribution changed the economics and size of addressable markets substantially. The mechanisms of redistribution generally take the form of local newspapers, television and radio stations. In order for the model to work, there must be a high barrier to entry for local redistribution endpoints.

The qualities of physical space— distance and nearness are the medium through which syndication operates. As McLuhan notes, under electronic information conditions, everything changes. Once there’s a shift from physical space to name space, the concept of distance evaporates. When the Network is the distribution channel, what’s the difference between remote distribution and local distribution? Access via URL obviates syndication, distribution is direct. There’s no business model for local redistribution of remotely produced media product.

It’s interesting that we model physical syndication in technical formats like RSS. Media content is transported from an originating production facility to remote reading machines. The sales proposition is a reversal of transportation energy. Rather than you expending energy “going” to a news source, the news source expends energy “pushing” the news to you. News distribution takes the form of file transfer from over there to my local computer. The value of the pushed news stream is in the editorial decisions around feed subscription. There’s no item level granularity, so while the aggregation of feeds is a substantial advance, it’s only in “shared item” feeds that we start to see the possibility of filtering tools to produce high value synthetic feeds.

The URL, the hyperlink, has allowed readers to tear up the New York Times and share the interesting parts through multiple messaging buses. As Shirky notes, the publication is reassembled on the demand side. This feed of high-value items doesn’t require transport of the items from here to there. In a broadband environment, a playlist of URLs (tweets) delivers the news without moving an inch.

When we use the metaphor of physical space to think through economics of a name space, we end up like the journalists staring at Clay Shirky looking for a sign that everything is going to be all right.

You can read a transcript here or listen to Clay Shirky’s talk here: Clay Shirky on Accountablity Journalism

One Comment