Skip to content →

The Source: Algorithmic Authority and Unsupervised Systems of Record


At the recent Real-Time Crunch Up, a number of interfaces presented themselves; many of them worth reading and writing into. But, I’d like to explore a turn of phrase that slipped out causally in the last panel. The other members of the panel simply nodded and moved with the flow of the thought, it was understood to reflect the state of affairs. Dan’l Lewin, of Microsoft, while discussing which of the companies we’d seen during the day might become dominant players in the real-time environment of the Network, used the phrase: “system of record.” Lewin was referring to Twitter and Facebook as new systems of record on the Network.

Wikipedia is a little light on its definition of systems of record:

A system of record (SOR) is an information storage system (commonly implemented on a computer system), which is the authoritative data source for a given data element or piece of information. The need to identify systems of record can become acute in organizations where management information systems have been built by taking output data from multiple source systems, re-processing this data, and then re-presenting the result for a new business use.

The key here is the phrase: “authoritative data source for a given data element or piece of information.” The data element Lewin was referring to is the public social graph in its unfolding as real-time, tick-by-tick, context data. If you look at Microsoft, Google, Apple, Yahoo, the financial and medical institutions, etc. — none of the current big players on the Network have the social contract that allows them to serve as the system of record for the real-time social context data set.

Each has a system of record with some essential piece of the picture; they serve as the source of some authoritative piece of data. None of them can be authoritative for every aspect of a person. And generally, we like to be able to choose among a minimum of two providers (SORs) for each piece of our represented selves. Companies like Acxiom, Mint/Yodlee, and Equifax have begun the process of aggregating identity across key authoritative data silos, creating connections and drawing the broad outlines. In order for there to be an aggregation point, there has to be a set of authoritative systems of record.

As I began to think about the authority of these interconnected data elements — another connection placed itself in the frame. We sometimes speak of the newspaper of record. Another kind of authority presents itself:

The first type of newspaper of record (or newspaper of public record) is often formally defined by a statute or other official action of a governing body. Such a newspaper is supposed to be available to the public, and publication of notices in that newspaper is considered sufficient to comply with legal requirements for public notice.

The second type of “newspaper of record” is not defined by any formal criteria. The use of the term implies that a newspaper is a reliable institution that publishes trustworthy descriptions of events, but this assessment may be disputed. Major newspapers of record may be expected to have independent editorial policies, and to publish statements of opinion that are distinct from those of their proprietor or their government. They are more likely than other newspapers to be sold abroad and to be cited in scholarly publications.

Clay Shirky has become enmeshed in the discussions around the endgame for newspapers. I wouldn’t exactly call it a debate, because only mainstream journalists are interested in debating the point. Everyone else seems to have moved on. In his post, On the Idea of Algorithmic Authority, he explores what we mean when we talk about authoritative sources. In essence, he’s exploring the pragmatism of a kind of anarchy.

When we use the phrases “system of record” and “newspaper of record,” we’re trying to get to a similar level of authority. Newspapers of record supply the day-to-day transcription of important events. It’s the “availability to the public” of the newspaper and its broad distribution that makes up the public record. The authority of the newspaper rests in an editorial process that outputs “trustworthy descriptions of events.”

Shirky, in his book Here Comes Everybody, looks at organizing without organizations— and this idea is extended here to explore the level of authority that can be achieved by an unsupervised process. Typically we look for some kind of certification, an institutional process guaranteed by a professional in charge. The change that Shirky chronicles is the expansion of the kinds of processes that can produce authoritative output. When an unsupervised algorithmic process can produce an output that people respect as authoritative, the economics of supervised certified processes are disrupted. The ecosystem is enlarged and the economics irrevocably changed. In this case authority isn’t replaced, rather its sources are multiplied.

As our view of the ecosystems of record on the Network begins to come into focus, an emerging landscape starts to take shape. And what looked like a field with thousands of players is quickly reduced to a small number of authoritative systems.

Published in culture digital economics media network real time web social graph zettel