A Better Search Engine?

Imagine quickly digging up a Web site devoted to research on a rare disease. Or easily locating a scientific paper for your college thesis.

When it comes to hard-to-find information, chances are improving that a few typed keywords may immediately locate what you're looking for.

Several Internet search services, including Excite@Home and Lycos, say they are dramatically widening their scope to encompass more of the 800 million Web sites on the Internet.

The moves follow a recent study showing that most search engines let people scour just a fraction of the Net, excluding lesser known sites and the torrents of fresh information added daily to the Internet.

While broader searches may clutter some Internet users with needless information, the services insist they are working harder to organize search results so that top-listed sites match what people are looking for.

"It's definitely good that the search engines are getting bigger - with a big caveat that bigger doesn't necessarily mean better," said Danny Sullivan, the London-based editor of searchenginewatch.com, a Web site that tracks the sector.

Also, broader searches are more likely to benefit those who conduct in-depth research, such as scientists and journalists, rather than casual Web users just looking to buy books or CDs.

Three weeks ago, the journal Nature published a study showing that search engines cover a shrinking fraction of Web pages and take a long time to list new sites.

The research, by NEC Research Institute, offered fresh bragging rights to those Web destinations with search engines that ranked highest.

Better search power is a good way to draw more visitors. Twice as many people have been using Northern Light, the top-ranked search engine in the study, since the Nature report was published, said Susan Stearns, director of marketing for Northern Light Technology LLC, based in Cambridge, Mass. "People recognize that comprehensiveness is important," Stearns said.

But while offering more comprehensive searches is one way to lure more visitors to a site, it's no guarantee. The most visited network of Web sites, Yahoo!, ranked only No. 8 in the study.

Excite@Home Corp. unveiled a new version of its search tool on Tuesday, saying it will index up to 300 million Web pages, or more than a third of the Internet's public sites - up from just 6 percent. In addition, the service uses a more thorough method of sifting through Web sites to decide which to include in its index.

The revamped service, which will be available by the middle of this month, comes after a year-and-a-half of research, said Kris Carpenter, director of search services for Excite@Home, insisting the timing of the launch was unrelated to the study's release.

Notably, a day before the Excite announcement, a Norwegian company named Fast Search & Transfer unveiled a service tat checks 200 million Web pages. The service also organizes information so that users aren't bogged down by lots of irrelevant material, the company said.

Lycos plans to introduce a more comprehensive search by the end of the year, said Ron Sege, executive vice president. But Sege played down the importance of broader searches, saying that the important thing is how well a company chooses the information.

The service upgrades come amid pressure from investors for search companies to boost traffic to their Web sites.

Written by David E. Kalish
©1999 The Associated Press. All Rights Reserved. This material may not be published, broadcast, rewritten, or redistributed