Category: hci

Television Signal Path and the Airplay Remote Control

Published January 13, 2013 by cgerrish

The control systems for television aren’t very good. One reason they persist is that once a viewer is watching a selected program, the control system recedes into the background. In the course of watching a presentation, the essential controls, the ones that control sound (louder, softer, mute), generally work quite well. The rest of the control system is a disaster that people have learned to accommodate. This snarl of technology around controlling a television is generally why people think there’s room for revolutionary innovation in the “battle for the living room.”

Generally there have been a couple of approaches. The universal remote, a complex remote control device that consolidates all of the other remote controls. So instead of having five or six complex remote controls, you have one really really complex remote control. Google TV’s remote control with a keyboard pushes towards the limits of this kind of conceptual framework. The addition of voice command and SIRI is another solution at the limit. The other approach involves creating a “smart” television. This would be accomplished by integrating a Network connected computer into the television device. This new device would make all of the other devices obsolete. Various forms of this device have been foisted upon the public. It’s not that people don’t buy these “smart” televisions, it’s just that no one uses any of their capability.

The solution to this tangle of technology lies in the role of the remote control. The name “remote control” describes what the device does. It takes the control system from the television and allows it to operate at a distance from the television itself. That meant you didn’t have to get up off the sofa and walk across the room to select a program or control the sound volume. The “remote” has essentially provided the same service since it entered the living room in the mid-1950s. Nikola Tesla described its basic operation in a patent application more than 50 years earlier than that. To some extent, even cloud computing is just a variation of the same theme.

It was while researching wireless audio systems for my study that the basic change in the “remote” became clear to me. With all of my music available through a cloud storage system, I didn’t need a music system to decode physical media. From the many choices available, I selected the Bowers & Wilkins A7. It’s a single speaker that sits in a home WiFi network and listens for AirPlay signals. You can send it music via AirPlay from your phone, iPod, tablet or desktop computer—and that music can be stored remotely on the Network. Radio streams, YouTube sound, podcasts, etc. can be also be sent to this audio system. The key is the change in the signal path. The “remote” is no longer just a controller, it’s the receiver/broadcaster of the audio signal. The “stereo system” now listens for AirPlay signals, decodes and presents the sound. I liked this solution so much, I set up my traditional stereo to operate similarly using AirPort Express as one of the auxiliary inputs.

You can see how this model would work for television. Instead of a smart television, you have a dumb television. The big screen does what the big screen does well. It shows high-definition moving pictures synchronized with sound. You can’t solve the “television problem” without changing the signal path. Once the remote control becomes a receiver/AirPlay broadcaster, all the peripheral devices hooked up to your television go away. Even your cable box becomes just another app on your phone or tablet. The interesting thing about this solution is that it doesn’t necessarily disintermediate the cable companies, the premium channels, Netflix, Amazon, Tamalpais Research Institute, Live from the Metropolitan Opera or your favorite video podcast.

In this analysis, the real problem with the television is identified as the HDMI connector. Every device connected to the screen via HDMI wants to dominate the control system of the television; and every HDMI connection spawns its own remote. Once you get rid of the HDMI connector and transform the remote control into an AirPlay receiver/broadcaster, all the remote controls disappear. The television listens for one kind of signal and plays programming from any authorized source. The new generation of wireless music systems have demonstrated that this kind of solution works, and works today. By changing the signal path and the role of the remote, the solution to the problem of television is well within reach.

Comments closed

Things, Not Strings: The Word Made Flesh

Published May 20, 2012 by cgerrish

The headline reads: “Introducing the Knowledge Graph: things, not strings.” The implication being, “strings” are bad and limited and “things” are good and what you really wanted all along. After all people don’t want strings of arbitrary alpha-numeric characters in response to their queries, they want the things they’re looking for. And as the advertising message at the end of the introduction says, because you’re getting “things and not strings” on your search result pages, you can spend more time doing the things you love. Who wouldn’t want to do that? The end result of this technological improvement is that your life now contains “more time”— like a toothpaste tube that contains 20% more toothpaste; and that time is filled with love. One might even recast this new product as a machine for filling the world with love.

What Google seems to be introducing is a new user interface to a faceted search. Nothing more. Faceted search acknowledges that the “word” (a single string of characters) isn’t the atom of meaning. Instead it uses the “phrase” in the context of some domain of meaning—a word can be a valid token in multiple systems of meaning. These domains, or facets of meaning, are surfaced and prioritized in search results. So, in addition to Page-ranked links, we get a prioritized set of contexts in which a particular word or phrase is a valid operator. The advance is in creating an index of sub-domains of meaning through analyzing the structure of text as it’s used on the visible Network. There’s no question that faceted search is superior to classic Page-ranked search, however the language used to describe this new product innovation seems to suggest some kind of transcendent experience.

Here’s a description of the vision that drives innovation in the search product at Google:

We’ve always believed that the perfect search engine should understand exactly what you mean and give you back exactly what you want
– Amit Singhal, SVP, Engineering at Google

But when I hear this kind of talk from engineers, their words are drowned out by the characters from Lewis Carroll’s “Through the Looking Glass“:

‘I don’t know what you mean by “glory”,’ Alice said.

Humpty Dumpty smiled contemptuously. ‘Of course you don’t — till I tell you. I meant “there’s a nice knock-down argument for you!”‘

‘But “glory” doesn’t mean “a nice knock-down argument”,’ Alice objected.

‘When I use a word,’ Humpty Dumpty said, in rather a scornful tone, ‘it means just what I choose it to mean — neither more nor less.’

‘The question is,’ said Alice, ‘whether you can make words mean so many different things.’

‘The question is,’ said Humpty Dumpty, ‘which is to be master — that’s all.’

Alice was too much puzzled to say anything; so after a minute Humpty Dumpty began again. ‘They’ve a temper, some of them — particularly verbs: they’re the proudest — adjectives you can do anything with, but not verbs — however, I can manage the whole lot of them! Impenetrability! That’s what I say!’

‘Would you tell me please,’ said Alice, ‘what that means?’

‘Now you talk like a reasonable child,’ said Humpty Dumpty, looking very much pleased. ‘I meant by “impenetrability” that we’ve had enough of that subject, and it would be just as well if you’d mention what you mean to do next, as I suppose you don’t mean to stop here all the rest of your life.’

‘That’s a great deal to make one word mean,’ Alice said in a thoughtful tone.

‘When I make a word do a lot of work like that,’ said Humpty Dumpty, ‘I always pay it extra.’

We can propose the idea that Google has a search engine that “understands exactly what you mean.” And by this what we mean is that your query corresponds to a sub-domain in the index of facets Google has previously collected. The “meaning” doesn’t lie in the “you” that has the query, but rather in the sets of sub-domains contained in Google’s index. When a word does a lot of work in multiple sub-domains of meaning, they pay extra in compute time.

The claim that Google makes is that they’ve gone from “strings” to “things.” But the sub-domains of meaning that Google is collecting are made up of computable sets of strings, not things. The leap that Google is actually trying to make is from “strings” to “words, phrases and contexts.” But the use of the word “thing” is very revealing. Words are not things, they are indexes. They point at things, suggest things, or function in a play of difference within a system of meaning. When we say that we’ve gone from “strings” to “things” we’re actually making a kind of miraculous claim. We’ve gone from “word” to “thing.” The most prominent example of this algorithm can be found in the King James Bible, we see it in John 1.14:

And the Word was made flesh, and dwelt among us, (and we beheld his glory, the glory as of the only begotten of the Father,) full of grace and truth.

If we believe that Google’s knowledge graph provides “things” and not “strings,” we also believe something extraordinary about the power and capability of Google. Even if we take a step back and simply say that Google is merely indexing sub-domains—systems of meaning, we need to examine what this means. We could follow Wittgenstein and say that “meaning” can be described as a form of life. Therefore Google’s index produces a prioritized list of facets (forms of life) that connect to your form of life, given what they know about you. Popular forms of life that don’t currently connect to you serve as a method of discovery.

Of course, there’s also the popular trend of the flesh made word…

There are registers of meaning that Google’s approach will never capture. Their index will be filled with gaps and pools of darkness. In particular, only a very limited range of metaphor (cliches) will be caught in the net. Metaphor produces meaning through an algorithmic process (per @the_eco_thought, Tim Morton). Take a noun, take another noun from a different domain and place the word “is” between them. The coffee cup is a blue angel. The metaphor machine makes meaning. Not every metaphor is a good one, but it has some modicum of meaning and it does function as a metaphor.

Like the theoretical one hundred monkeys typing in front of a hundred typewriters for a hundred years, the metaphor machines are constantly operating and feeding the Network with new meaning. Darius Kazemi (@tinysubversions) has created a machine called “Metaphor a Minute” that does just this. You can follow it on Twitter at @metaphorminute. Of course, because of Twitter’s rate limits, there’s actually a new metaphor published every two minutes.

“Hold the newsreader’s nose squarely, waiter, or friendly milk will countermand my trousers.”

After thinking through Google’s new service and the language they’ve used to describe it, we discover that they are using the word “things” metaphorically. At first, we may assume that when engineers are describing the function of their new software, they’re making literal statements about what the machine they’ve constructed is doing. Instead, they’ve taken a two nouns from different domains and inserted the word “is” between them. Ironically, their use of the word “things” is of the type that their new service could not understand it. The narrow band of search engine results that are produced by this system is also being metaphorically called “knowledge.” In order to see these new products clearly, we need to be able to differentiate the rhetoric of hyperbole from the literal functioning of the machine. It also helps to become acquainted how metaphors mean…

Comments closed

Putting Ears on the Television

Published October 30, 2011 by cgerrish

There’s some slang in the CB radio world, when you want to know if someone is listening, you ask if they have their ears on. As in, “How ’bout ya JB, got ya ears on?” For some reason this is the phrase that popped into my head when thinking about the possibility of an Apple-designed television set. In earlier thoughts about the future of television, my focus settled on HDMI inputs and clumsy switching between these inputs. In essence, the HDMI input becomes the inheritor of the idea of the channel.

When you look at the inputs and outputs of the big screen, the game is to dominate the primary input. Your cable or satellite programming provider doesn’t want you to ever switch to another HDMI input. If you can be that dominant, your external boxes can commandeer the control experience from the television itself. Anyone who’s hooked up a television to a cable systems has had the experience of being presented with two mutually exclusive proprietary control systems. This is the reason you can have 3 or 4 remote controls sitting on your coffee table. Each HDMI input has a separate control system and listens for control events with a separate set of ears.

Customer satisfaction surveys are a great friend to Apple. This is because customer satisfaction is usually just an accommodation to work-arounds. We’ve grown used to the way the television “works.” The work-around is the way it works, and after a while we don’t even notice the strangeness of it. And when we get that call, interrupting our dinner, asking us whether we’re happy with our television set up, we say, “sure, it’s great.” Of course, the reality is it’s a horrible mess we’ve aclimated ourselves to.

So let’s get back to that CB radio reference. Do you have your ears on? The problem with television sets is they don’t have their ears on. Or rather they’ve been trained to only listen to a single voice at a time. As a user of iOS devices, I’d like to be able to send programming to the big screen at any time via AirPlay. As things stand I can only do that when AppleTV2 is the designated input. An Apple-designed television would always be listening for AirPlay events.

As YouTube gets ready to launch a bunch of channels, I can’t help but think that “the channel” has reached the limit of its usefulness. When I ask Siri whether it’s going to snow today, I don’t need to switch the input to the Weather Channel to get an answer. When I ask my iTV whether there’s a Val Lewton movie on, I don’t want to have to know what channel it’s on. I want Siri to take care of searching my subscriptions and report back on what my options are. The effect of this would be to return control of the television to the television itself.

As things stand, Siri would have a limited domain of television programming services to search through. Although this isn’t too different from the current situation with the iPhone 4s. Eventually all television services will migrate toward television over IP. It’s happened in all other mass media, television will be no different. Even your DVR will just save pointers to stream locations in the cloud.

In an interview, Steve Jobs once said that these waves of technological innovation are slow and unfold over many years. The trick is to pick the right wave and position yourself to benefit from the natural current. We can easily say that today, Siri isn’t good enough (in the sense of an innovator’s dilemma). But it’s perfectly positioned to grow and benefit from a huge wave of cloud-based data/identity services. It’ll work the same way with iTV.

Comments closed

Shop Windows And Tablets: Through The Looking Glass

Published October 3, 2011 by cgerrish

In looking for lost house keys under the light of the street lamp, we put aside the fact that we lost them in the ditch at the other side of the road. It’s odd how we can move so swiftly in a particular direction without really knowing where we’re going. An incredible amount of ingenuity, resources and coordination has been applied to building tablet computers. There’s an unstated assumption that the post-pc era is defined by an evolution of the computer to a new human-computer interface model with a new form factor. And at a technical level, there’s some truth there; however at the level of the market for devices, there’s not enough truth.

To make sense of all this, let’s go back to a 1996 interview by Gary Wolf with Steve Jobs. Jobs was at NeXt and was gazing ahead at the future:

Wolf: What other opportunities are out there?

Jobs: Who do you think will be the main beneficiary of the Web? Who wins the most?

Wolf: People who have something –

Jobs: To sell!

Wolf: To share.

Jobs: To sell!

Wolf: You mean publishing?

Jobs: It’s more than publishing. It’s commerce. People are going to stop going to a lot of stores. And they’re going to buy stuff over the Web!

e-Commerce’s path to the Network was from the paper catalog to the electronic catalog. The Sears Catalog was one of the early prototypes for distance retailing. But what was the paper catalog? Why was it successful? The catalog was an evolution of the shop window in the arcade. And it was the shop window that enabled the romantic imagination of the consumer. Heather Marcelle Crickenberger talks about Walter Benjamin’s idea of the flâneur:

“Flâneur” is a word understood intuitively by the French to mean “stroller, idler, walker.” He has been portrayed in the past as a well-dressed man, strolling leisurely through the Parisian arcades of the nineteenth century–a shopper with no intention to buy, an intellectual parasite of the arcade. Traditionally the traits that mark the flâneur are wealth, education, and idleness. He strolls to pass the time that his wealth affords him, treating the people who pass and the objects he sees as texts for his own pleasure. An anonymous face in the multitude, the flâneur is free to probe his surroundings for clues and hints that may go unnoticed by the others.

Today we call it window shopping. It’s an exercise of the imagination in the role of the consumer. What might I look like in that outfit, listening to that music, with those kitchen appliances? A large plate of glass opened a window on to the possibilities contained within the shop. The flâneur could stroll the arcade moving from this window to that, searching for something that might catch his fancy.

Timothy Morton discusses this performance of the consumer imagination in his essay on “The Beautiful Soul.”

These performative styles are outlined by myself (Morton) and Colin Campbell. One style stands out, and that is a kind of meta-style that Campbell calls bohemianism and I call Romantic consumerism. This kind of consumerism is at one remove from regular consumerism. It is “consumerism-ism” as it were, that has realized that the true object of desire is desire as such. In brief, Romantic consumerism is window- shopping, which is hugely enabled by plate glass, or as we now do, browsing on the internet, not consuming anything but wondering what we would be like if we did. Now in the Romantic period this kind of reflexive consumerism was limited to a few avant-garde types: the Romantics themselves. To this extent Wordsworth and De Quincey are only superficially different. Wordsworth figured out that he could stroll forever in the mountains; De Quincey figured out that you didn’t need mountains, if you could consume a drug that gave you the feeling of strolling in the mountains (sublime contemplative calm, and so on). Nowadays we are all De Quinceys, all flaneurs in the shopping mall of life. This performative role, this attitude, is all the more pervasive, leading me to believe that we haven’t really exited from the Romantic period—another sense in which “prehistory” isn’t quite right for what I’m describing, but extremely right in another sense, namely that we’re still caught in an attitude that we don’t fully understand or become aware of.

When we talk about what’s assumed to be a tablet computer, we’re actually talking about a plate of glass, a shop window. In a discussion with Nick Bilton of the New York Times about why all these tablets look similar, Ryan Block hit on the key, although he may not have realized it:

“We are talking about a screen, where the screen is the entire experience and it can only really look and act one way, and that is to look similar to the iPad,” Mr. Block explained in a phone interview Thursday. “At the end of the day, they are all going to look similar, because a tablet is just a piece of glass.”

The innovations of the post-pc era aren’t to the computing device, they’re to the shop window. The ability to transact as part of the performance and the transformation of the goods from material to digital such that they can be played within the same window are the key additions to the “piece of glass.”

If you view the recent crop of tablet computers through this lens, you’ll see what separates the Apple and Amazon products from the rest. We pass the empty shop window of the deserted store as we move on down the block to see what we might find next. Of course, it’s simple to see how a technologist might confuse a shop window with a flat computing device.

One Comment