Wolfram Alpha Does What Google Can't

But Don't Let The Impressive Demo Fool You; The Engine Falls Short On Understand Questions That Aren't Canned

(CBS)  Whenever someone tells me they're building a search engine that will do things better than Google, I am skeptical. Not because I think Google does everything well, but because it is very hard to build a search technology that works. I've seen a lot of search failures.

Cuil comes to mind. Launched with great fanfare in July of 2008 a couple of Google alums, it just didn't work. The demos were great, but when users finally got a chance to try the real thing, they got poor or strange results far too often.

Watching Stephen Wolfram demo his new "knowledge engine," Wolfram Alpha, brings back memories of Cuil. The demo is staggering. In the hands of its creator, Wolfram Alpha appears to read the user's mind as he explores the world's numerical data. Wolfram types in "What is the GDP of France" and gets a clear chart showing the country's gross domestic product over time. He modifies the search result to "What is the GDP of France / Italy" and the two countries are compared.

Moving to the hard sciences, Wolfram types in a derivative, and Alpha solves it. Like a good student, it even shows how, when asked. Wolfram types "2.5 molar sulfuric acid" and Wolfram Alpha shows the properties of the chemical.

And then he shows us how Wolfram Alpha can give you answers based on the facts in its database, for questions that nobody has asked so far (and thus ones that Google won't have answers to, or at least not easily findable answers).

Wolfram types in demo queries like "weather in princeton, NJ when kurt godel died" and gets a clear answer (30 degrees and overcast on January 14, 1978), and then "next solar eclipse in Chicago." Alpha gets these results by first understanding what is asked, and then combining the data it has on record to build the answer, doing calculations if necessary. Often, pretty charts are returned. Never do you get a whole Web page that you have to read or digest to get the nugget of info you were looking for.

And so on, with more amazing demos, until this reviewer's jaw is on the floor. But as with many technology products, the demo is one thing. Using the actual product is another.

We tried a pre-release version of Wolfram Alpha - the Web service should be opened to the public before the month ends - and found that minor variations of Wolfram's demo queries returned expected results, but that we often got tripped up when we tried to get answers to other questions.

For example, typing a more complex eclipse question ("next total solar eclipse in bagdhad") gave us a good answer (October 7, 2135, for 4 minutes and 5 seconds). But when we tried to create other queries we found ourselves stymied.

One of the wacko theories I've always wanted to test was that weather and earthquakes are related - to either prove or disprove that there is such a thing as "earthquake weather." This is the perfect test for a system like Wolfram Alpha, which has all the data, and which apparently understands it.

For this query, though, while I could get historical weather data, and 30 years of earthquake data, I could not for the life of me figure out how to correlate the charts. Most of my trial queries just gave me an unhelpful error message: "Wolfram Alpha isn't sure what to do with your input."

When I struck out further from the canned demo queries I saw, I continued to get lost. Alpha didn't help me find my way back to queries that worked, nor was it easy to take one that did and refine it to get a more specific result.

The query processing part is one of four main pillars of Wolfram Alpha, its inventor says. The others work well: The system to get data in seems to be gathering a ton of useful information. The computation engine is impressive (when you feed it input it understands). And the engine that presents the results does a great job. The linguistic processing system is Alpha's weak link. It will likely be improved by the time the system is opened to the public this month, but it's the area where users will be constantly pushing the product, and, I fear, finding the hard edges of its capabilities.

Alpha is not Google. It does a very different thing: It computes the answers to questions that may not ever have been asked. It also takes hard data and presents it in its own way. Google, by contrast, primarily directs you to Web pages where other people have already created answers to questions. It's a very different type of service, even if there is a fair bit of overlap when you're looking up static information, like the structure of a molecule.

Wolfram Alpha could become an amazing tool for searching not the Web of pages, but rather the Web of data. It's smarter than Google in many ways, if not as comprehensive. It could become a worthwhile destination for anyone who needs a concrete answer, providing the question can be asked in a way Alpha understands.

[hetup-2009]

I am not impressed with any search engine, type in a request for solar energy and you get back reveiws for rock bands with HIV. What the devil is that? The internet is a giant library to me, but finding what you want is nothing short of a maze.

[thinkingaloud]

I can see a whole new "out-sourcing" industry, where smart young kids from India, China, Brazil and industry willingly learn the "Wolfram Alpha syntax" to get answers questions for chaps like Rafe Needleman. Heck, it's even more scary to think of how these kids will gladly be quenching their thirst for learning with Wolfram Alpha.

[tjolley11]

I for one will be happy to bow down before our new machine overlord Wolfram Alpha

[common2cents]

"My question would be, "When will search engines become self-aware?""

Wolfram Alpha says 'before the month ends'

uh oh

[Slrman]

My question would be, "When will search engines become self-aware?" If they do, we will be getting the results they want us to have, not what we need. Then the world will be run by machines. Will it be Terminator", or "I, Robot"?