Update: Linguistics in the News and on Jeopardy

In December of last year I posted about the upcoming Jeopardy match that will pit a computer named Watson against two of Jeopardy's winningest contestants ever.

Well I just learned that a practice match was held last Thursday, January 13th and here is a clip from said match.



According to an article about the match on POPSCI, "Watson has a certain self-awareness; it knows it won't get every answer right, and has to pass a certain level of confidence before it will answer. Watson's logo will change color to indicate its confidence: The lines that are part of its "avatar" will glow blue if Watson is confident, and orange if it's not."

And here is some information from the POPSCI article about how Watson handles the difficulties of language.

"The vagaries of language mean that the questions can be interpreted in all kinds of different ways, so merely figuring out what the question is trying to ask provides the majority of the struggle for Watson. To that end, the computer actually comes up with thousands of different possible answers, and ranks them by the possibility of correctness. When we watched the quick match, the top three answers were displayed on screen, as well as the confidence percentage, and the second- and third-ranked answers were usually dramatically incorrect. It's not likely that Watson will confuse, say, the author of one children's book with the author of another. It's more likely that Watson will completely misread what the question is even asking, and come up with an answer like "What is children?"

Certain elements of human language are tricky, too--the stuff that seems like it might be the most difficult (like puns and wordplay) are felt out by "trigger" words in the category name, such as "sounds like." But synonyms are often a bigger problem. In the answer "This liquid cushions the brain from injury," Watson has to determine that "liquid" is in this one case interchangeable with "fluid," and that "cushions" is interchangeable with "surrounds." Humans know what the question is asking instinctively, but Watson has to analyze it from every angle."

Watson ended up winning the practice match with $4,400. Ken Jennings placed second with $3,400 and Brad Rutter took third with $1,200.