Computer Watson on Jeopardy was Unfair!

If you are not a Jeopardy watcher, Jeopardy recently had two Jeopardy champions compete against an IBM super computer.  One of the contestants on Wednesday was actually winning for most of the match but lost at the end.  It was interesting to see how Watson was able to buzz in first so many times.  When it had a lot probability of knowing the answer, the others seemed to have a more advantageous buzz time.  I’ve come to find out from the blog messymatters that there were some unfair advantages for Watson.  First, the question input for Watson was delivered in a text format.  Just like when you copy and paste into a document or search engine and hit enter, that’s how fast the information was delivered to Watson.  Ofcourse the copy and paste portion can be excluded.  It’s more like  you clicking on the link to messymatters above and that sending the text to your browser window to read that blog.  It’s basically simoultaneous.  A more fair approach to Watson being informed of the question would be visually and verbally like the other two participants.  This research is used everyday with spammers etc.  Ever wondered why you have to type in weird looking text when you try to post on a blog or perform some action on a website?  It’s because spammers have learned to program the computer to visually understand the wording.  The computer could cross validate it’s predictions of words with verbal dialogue from Alex.

The second interesting point was the buzz time for the participants.  The participants are allowed to buzz in when a light appears.  This response is sent to Watson who can buzz in again almost simultaneously.  You would have thought they would control for this before starting the game, but they didn’t.  From the blog above, it is believed that the participants actually hearing Alex’s voice could use this to help predict when the light would come on.  If participants buzz in before the light, there is a small time penalty.  These two advantages do not counter balance as messymatters suggests.  Watson does not have a reprocussion for using his advantage while the two human participants could possibly lose seconds by buzzing too early. 

On a side note, I think the computer being able to answer trivia questions like this is quite impressive.  Watson could have easily been stumped in my opinion if the questions were problems needing problem solving capability like “What’s the sum of all the even numbers in this sequence {1,3,5,6,7,8}”.  Simple questions like “What’s the color of Alex’s tie?” could not be answered, but would purposefully be problems to purposefully make Watson not know the answer.


2 responses to “Computer Watson on Jeopardy was Unfair!

  1. I think the buzz time feature had a very big impact. It created two separate rules for the humans and the computer. We know the number of time the humans were locked out of a question because they clicked in too early, but not the Watson.

    From a machine learning perspective it is interesting that Watson, the winner of the netflix prize and basically all the prizes in recent history has used an ensemble learning method.

  2. For those interested, ensemble learning methods “use multiple models to obtain better predictive performance than could be obtained from any of the constituent models.[1][2][3] Unlike a statistical ensemble in statistical mechanics, which is usually infinite, a machine learning ensemble refers only to a concrete finite set of alternative models.”

    I thought this interesting as well. I feel like this is the road AI will be heading down. I think we make conscience and unconscience decisions every day based on expected outcomes to what we are most concerned about. Most of the time it is ourselves with small decisions, but not always.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s