Skip to content →

The Real-Time Web and Information Arbitrage


As the ‘RSS-is-fast-enough-for-us’ crowd begins to resemble the Slowskys from the television commercial, an effort has begun in earnest to speed up the transport of RSS/Atom feeds in the face of real-time media. These efforts will answer the question about whether RSS is structurally capable of becoming a real-time media. If the answer is yes, then RSS will become functionally the same as Twitter. If the answer is no, then it will become the rallying point for the ‘slow-is-better’ movement.

There’s a strong contingent who will say that more speed is just a part of the sickness of our contemporary life. We need to ‘stop and smell the roses’ rather than ‘wake up and smell the coffee.’ And while there are many instances in which slow is a virtue, information transport isn’t one of them. Under electronic information conditions, getting your information ‘a day late’ is probably why you’re ‘a dollar short.’

When you begin thinking about the value resident in information, it’s instructive to look at the models of information discovery and use on Wall Street. Analysts generate information about companies in various investment sectors through quantitative and qualitative investigation. The high-value substance of the reports is harvested and acted upon before the information is released. High value information lowers transaction risk. Each stage of the release pattern traces the dissemination of the information. Within each of these waves of release, there’s an information arbitrage opportunity formed by the asymmetry of the dispersion. By the time the report reaches the individual investor—the man on the street, it is information stripped of opportunity and filled with risk.

In Friday’s NY Times, Charles DuHigg writes about the relatively new practice of high-frequency trading. Under electronic information conditions, the technology of trading moves to match the speed of the market.

In high-frequency trading, computers buy and sell stocks at lightning speeds. Some marketplaces, like Nasdaq, often offer such traders a peek at orders for 30 milliseconds—0.03 seconds—before they are shown to everyone else. This allows traders to profit by very quickly trading shares they know will soon be in high demand. Each trade earns pennies, sometimes millions of times a day.

If you were wondering how Goldman Sachs reported record earnings when the economy is still in recession, look no further than high-frequency trading. The algorithmic traders at Goldman have learned how to harvest the value of trading opportunities before anyone else even knows there’s an opportunity available. By understanding the direction a stock is likely to move 30 milliseconds before the rest of the market, an arbitrage opportunity is presented. High-frequency traders generated about $21 billion is profits last year.

Whether you think the real-time web is important depends on where you choose to be in the release pattern of information. If you don’t mind getting the message once it’s been stripped of its high-value opportunity, then there are a raft of existing technologies that are suitable for that purpose. But as we see with the Goldman example, under electronic information conditions, if you can successfully weight and surface the opportunities contained in real-time information, you can be in and out of a transaction while the downstream players are unaware that the game has already been played.

Creating an infrastructure that enables speed is only one aspect of the equation. The tools to surface and weight opportunities within that context is where the upstream players have focused their attention. And while you may choose not to play the real-time game, the game will be played nonetheless.

Published in markets media network real time web risk


  1. Artful blog site & conversation topic Cliff. Most favor sending any data over the internet faster (RSS, Twitter, Hubble or 3D tele-presence).

    Astronomy/Geology are some ways “Slowsky” like for precision, but these predict things that can move very fast. Many thousands of years of Slow careful human astronomy data collection, analysis and forecasting through all history (now including employing the fastest network communications infrastructure and powerful compute resources worldwide). Recently the orbit risks of >5,000 asteroid like objects (NEOs) have been identified including >70 large ones with probable earth collision orbits You could even help via the ASE/UN project There are likely 10,000 more NEOs yet to be detected.

    Combine astronomy data analysis & communications (you may have noticed both Google & Microsoft are hot on astronomy) with geologic fossil record analysis of historical asteroid collisions data to determine impact consequences. We know “an asteroid (several are) headed toward earth”, we also know timing and probable consequences. But the recent first ever forecasted Asteroid/Earth collision (2008 TC3) last year was detected only 1.5 days prior to impact.

    2008 TC3 could have hit New York, London, or Tokyo. Future impacts may be a similar Fast surprises plus the already identified NEOs on probable collision orbits. We also know there are financial market risks and legal risks of these events (see ASE/NEO/UN program).

    Perhaps Twitter or RSS will help observers to report the next asteroid collision path, perhaps detected by an armature astronomer like this week's Jupiter collision reported to the IAU….

    Faster is better for some things but Slowsky patience like watching star fields to notice a shadow passing between, logging, then reporting it appropriately – is how these asteroids are detected. An urgent report of inaccurate data delivered faster will not yield a better result. A Tweet urging – “An Asteroid is going to hit” the incorrect place… will not be best for anyone (except the “fame” of being the very first to report… something… perhaps wrong). Mindful “Slow” action can be good.

    Consider the Slow Food movement and other slow deliberate movements. Perhaps there is a Slowsky argument 😉

  2. In geologic and astronomical time scales, any human communication would be considered very fast. And if an asteroid was headed toward earth, I doubt we would consider it a matter for slow contemplation.

  3. A 2007 example real “realtime” internet media using then existing telecommunications infrastructure with 40 cameras/stream + audio in realtime over 12k miles You could use RSS/Atom/Web feeds to schedule such real real-time meetings.

    And some thinking about the speed value resident in information: consider the geologic and astronomical records – for modeling the future economic value of asteroid hits. Not fast, but high value when needed. A kind of Slowsky contemplation.

  4. Speed is necessary, but not sufficient for the economics of the real-time web. For examples of the 'slowsky' argument — look for responses to FriendFeed's move to real time.

  5. Hmmm. It's taken you more than 3 minutes to respond. Not very realtime! 🙂

  6. Where are some examples of people making the Slowsky argument? I'd love to see what they say.

Comments are closed.