Category: real time web

Algorithmic Trading and the Streaming Data Complex: A Little Bird Told Me

Published August 10, 2009 by cgerrish

A_Little_Bird_Told_Me

While the shouting over whether Twitter has any value is largely over— there’s still some question as to what that value is. The search for a single qualitative value to which Twitter can be reduced is, of course, futile. It would be like trying to identify the single value of ink/paper, email, telephones or http.

When looking for meaning and value, there are a number of routes we might take. The denizens of Forrester, Gartner and Red Monk might take one direction; the host of burlesque performers and vaudevillians hanging out shingles as ‘social media’ experts may take another. Generally the process involves modeling what a business might do with ‘social meda’ (Twitter). The Profit/Loss in these models generally operates in the realm of public relations, marketing, good will and social capital. There is some argument for Twitter as a customer service channel, but while it’s optimal as a hailing frequency, it’s inadequate as a customer solution medium. This soft approach has some chance of success during a bull market, and a better than even chance during a financial bubble. While some, like Umair Haque, argue that these soft social revenue streams ultimately must provide a context for hard revenue streams, at the moment the stock market doesn’t agree. Positive sentiment on Twitter doesn’t translate into more demand for an equity. Adoption is currently limited to businesses that either can afford the luxury, or have replaced existing marketing and public relations modes with the Twitter channel.

On the hard revenue stream side of the ledger, we might look at how algorithmic stock traders are beginning to use Twitter. In trading, the asymmetry of the dispersion of news is a trading opportunity. We’ve seen how flash traders can create algorithms that determine the market’s direction from real-time data before the rest of the trading fraternity can even open their eyes.

Wall Street & Technology Magazine’s Melanie Rodier is reporting that algos at hedge funds are starting to consume data flow from Twitter to gauge the direction of sentiment toward an event or stock. The compact size and real-time nature of the tweet makes its ingestion and analysis particularly attractive. StreamBase Systems, a vendor of a complex event processing (CEP) platform, has announced a Twitter adapter that allows its applications to both consume and publish tweets. The designated (tracked) twitter streams are spliced with market data, financial ratios, newswire information and other data streams to build a more fully dimensional picture of a particular stock (company). Waiting for news to be collected, digested and emitted by Reuters can add too much latency to the news/information release pattern.

Just as reading the early reports from a newswire requires an understanding of context, history and the politics of news construction and distribution; reading a tracked Twitter stream as a part of a data complex requires a particular interpretive skill set. If the old adage ‘buy on the rumor, sell on the news’ has some truth to it, Twitter has just added a deep data layer to the ‘buy’ side of that equation.

One Comment

The Network as Real-Time Juke Joint

Published August 2, 2009 by cgerrish

It started during a conversation over dinner. Gaspare’s has a classic jukebox, and looking at it, it suddenly struck me that word “juke” was simultaneously very familiar and completely foreign to me. While the mechanical jukebox was a common enough feature of my childhood, it was already beginning to feel nostalgic. I was aware that the word “juke” came from the earlier phrase “juke joint.” Like all high value network nodes, the juke joint was located at the crossroads:

Classic juke joints found, for example, at rural crossroads, catered to the rural work force that began to emerge after Emancipation. Plantations workers and sharecroppers needed a place to relax and socialize following a hard week, particularly since they were barred from most white establishments by Jim Crow laws. Set up on the outskirts of town, often in ramshackle buildings or private houses, juke joints offered food, drink, dancing and gambling for weary workers. Owners made extra money selling groceries or moonshine to patrons, or providing cheap room and board.

But the juke joint put me no closer to the word “juke.” My sense was that it was meant to describe a style of popular dance. But digging a little deeper, this definition of the word’s origin emerged:

Gullah, the English-based Creole language spoken by people of African ancestry off the coast of Georgia and South Carolina, retains a number of words from the West African languages brought over by slaves. One such word is juke, “bad, wicked, disorderly,�? the probable source of the English word juke. Used originally in Florida and then chiefly in the Southeastern states, juke (also appearing in the compound juke joint) was an African-American word meaning a roadside drinking establishment that offers cheap drinks, food, and music for dancing and often doubles as a brothel. “To juke�? is to dance, particularly at a juke joint or to the music of a jukebox whose name, no longer regional and having lost the connotation of sleaziness, contains the same word.

The hidden payload in the word “juke” is its connection to the “bad, wicked and disorderly.” Transgression is built into the cultural practice of the juke joint. This connected to Levon Helm’s description of the Midnight Ramble.

The story of the word “juke” is also a technology story. Juke joints began with live music and dancing. The mechanical juke box replaced live music and its real-time interaction. The musician as messenger was replaced with her recorded output. The juke box attempted to put all the commercial qualities of the juke joint into a machine, while excluding the wickedness of its origin.

The mechanical juke box provided access to the popular music of the day. The users of the juke box assembled their own popularity charts by playing their favorites in the society of their cohort. The mass production and consumption of vinyl records and stereo equipment diluted the power of the juke box. The term “juke box” was preserved to describe the function of CD Players that could be loaded with hundreds of CDs, making the music on them readily accessible.

The original juke box became an object of nostalgia. We think of it as a cultural artifact of the 1950s. Its technology reached a terminal point, but its image was symbolically preserved. John Lennon’s jukebox became an item of great interest. Through it we gain an understanding of his formative influences, his taste and what music moved him. Curiosity about the contents of Lennon’s jukebox is the equivalent of today asking about the music loaded on the iPod of a public figure. We make a game of interpreting the tea leaves of the playlists.

While the word “juke” has dropped away, the iPod has become our equivalent of the juke box. The social aspect of the juke joint has been submerged almost entirely. The iPod is a personal jukebox, loaded with only the music I like. The exposing and networking of playlists begins to recover some of the social aspects of the juke box, but none of the real-time interactivity of the juke joint.

The juke joint, the barrelhouse and the midnight ramble all had the quality of providing a refuge for disorder within the forces of order. Their location was the crossroads at the edge of town. The Network has the same relationship to space as television. Every point of interface is one click away. The edge of town can very easily become the focal point of a family’s living room. While the Network provides the basis for the retrieval of a real-time interaction with the musician, we still don’t understand how to manage the “juke” that might appear at any moment.

2 Comments

The Locus of Identity

Published August 1, 2009 by cgerrish

giacometti-man-pointing

A ‘real’ transaction must be captured in ink. It must be written down in the form of a contract (or treaty). Then both of the parties must abide by it. The signature seals the oath and represents an authentication of the intentions of the parties. The wet-ink signature serves both as an affirmation of personal identity and a binding agent of the identity to the logical propositions in the document. If we stop for a moment and take a look at our lives, we’ll see that these agreements are rapidly moving from static ‘ink on paper’ documents kept in filing cabinets to dynamic service contracts on the Network.

iconic-mini-manila-folder-usb-flash-drive

Imagine the physicality of the myriad of written agreement to which you’re subject— where do they exist? Can you see a series of manila folders containing your documents in filing cabinets, in offices, in buildings, in some number of cities far away? In many cases, the documents conveying the wet-ink signatures have been scanned, and the originals destroyed. To the extent that agreements and transaction records can be represented electronically and digitally they will be. This is a matter of the economics of storage and retrieval rather than the power of networked electronic information.

magritte2

We see ourselves as a unified whole. I am ‘me,’ wherever I am. Wavy Gravy put it this way:

Always remember, wherever you go there you are, so be here now…

However, more and more, the authorized version of our selves is located in a series of databases networked within the Cloud. That data authorizes us to vote, work, travel, purchase, sell, own property and so on. As our social streams are tied to transactional data items, a much higher definition electronic picture of our ‘identities’ begins to come into focus. These words: vote, work, purchase, own— these activities are our main connectors to our society, our economy and the possibility of a private life. There’s a sense in which our personal identity can be reduced to the set of transactional verbs we’re authorized to use within the Network.

surrealism_escher

The locus of our authorized identity is shifting to this virtual data constellation. Because we’d like to be whoever we are, wherever we go. We’d like the dots connected and the right bits of data to travel between the stars of the constellation. There are some, like Tim Berners-Lee for example, who imagine a future where authorized software agents will act on behalf of an individual (and their constellation of data) to interact with other sets of structured data distributed around the rest of the Network. These service contracts will be binding for ourselves and our software agents. What is imagined is not just an extension and augmentation of our nervous system to electronic media, but an extension of our will.

We imagine ourselves at the center of this complex set of threads spinning out into and connecting within the Network. Some of the threads are at our command, others—by virtue of various contracts, command us. Imagine if the binding was broken, if you became separated from the digital representation of yourself, which entity would have more authority— which would be considered more ‘real?’ Which has more authority over your identity, your physical body or its electronic representation? Can we really say, on balance, that more of the Network will be at our command when the artifacts of authorization reside within the Network, rather than on our person. This movement of the locus of identity is not a choice we will make, it’s a slow change in the ecology of our environment, an adaptive moment.

14 Comments

The Real-Time Web and Information Arbitrage

Published July 25, 2009 by cgerrish

hermes

As the ‘RSS-is-fast-enough-for-us’ crowd begins to resemble the Slowskys from the television commercial, an effort has begun in earnest to speed up the transport of RSS/Atom feeds in the face of real-time media. These efforts will answer the question about whether RSS is structurally capable of becoming a real-time media. If the answer is yes, then RSS will become functionally the same as Twitter. If the answer is no, then it will become the rallying point for the ‘slow-is-better’ movement.

There’s a strong contingent who will say that more speed is just a part of the sickness of our contemporary life. We need to ‘stop and smell the roses’ rather than ‘wake up and smell the coffee.’ And while there are many instances in which slow is a virtue, information transport isn’t one of them. Under electronic information conditions, getting your information ‘a day late’ is probably why you’re ‘a dollar short.’

When you begin thinking about the value resident in information, it’s instructive to look at the models of information discovery and use on Wall Street. Analysts generate information about companies in various investment sectors through quantitative and qualitative investigation. The high-value substance of the reports is harvested and acted upon before the information is released. High value information lowers transaction risk. Each stage of the release pattern traces the dissemination of the information. Within each of these waves of release, there’s an information arbitrage opportunity formed by the asymmetry of the dispersion. By the time the report reaches the individual investor—the man on the street, it is information stripped of opportunity and filled with risk.

In Friday’s NY Times, Charles DuHigg writes about the relatively new practice of high-frequency trading. Under electronic information conditions, the technology of trading moves to match the speed of the market.

In high-frequency trading, computers buy and sell stocks at lightning speeds. Some marketplaces, like Nasdaq, often offer such traders a peek at orders for 30 milliseconds—0.03 seconds—before they are shown to everyone else. This allows traders to profit by very quickly trading shares they know will soon be in high demand. Each trade earns pennies, sometimes millions of times a day.

If you were wondering how Goldman Sachs reported record earnings when the economy is still in recession, look no further than high-frequency trading. The algorithmic traders at Goldman have learned how to harvest the value of trading opportunities before anyone else even knows there’s an opportunity available. By understanding the direction a stock is likely to move 30 milliseconds before the rest of the market, an arbitrage opportunity is presented. High-frequency traders generated about $21 billion is profits last year.

Whether you think the real-time web is important depends on where you choose to be in the release pattern of information. If you don’t mind getting the message once it’s been stripped of its high-value opportunity, then there are a raft of existing technologies that are suitable for that purpose. But as we see with the Goldman example, under electronic information conditions, if you can successfully weight and surface the opportunities contained in real-time information, you can be in and out of a transaction while the downstream players are unaware that the game has already been played.

Creating an infrastructure that enables speed is only one aspect of the equation. The tools to surface and weight opportunities within that context is where the upstream players have focused their attention. And while you may choose not to play the real-time game, the game will be played nonetheless.

6 Comments