Rep. Edolphus Towns, D-N.Y., chairman of the panel, said he was particularly concerned that transportation projects in economically distressed areas were being left out - even though they are supposed to be a priority.
"There is a substantial variation among states as to what constitutes an economically distressed area," Towns said. "For this reason, it is unclear whether Recovery Act funds are going where they are needed most."
The stimulus bill was constructed with jobs in mind, but since the president signed it into law in February, the nation has lost another 2 million jobs, pushing unemployment to its highest rate in half a century, reports CBS News correspondent Nancy Cordes.
A trio of governors told Congress that the real challenge has been obtaining stimulus funds fast enough to put people back to work.
According to Recovery.gov - the Web site the administration set up to track stimulus spending - only 10 percent, or $56 billion of the available funds have made it out the door so far.
"The federal agencies have to receive bids for contracts, then they have to analyze those bids, and then they have to write checks to those, to the people that won the bids - and that all takes time," Craig Jennings of OMB Watch told Cordes.
Towns' comments came in response to a Government Accountability Office report (PDF) released Wednesday at a hearing before his committee.
The GAO said about half the money set aside for road and bridge repairs is being used to repave highways rather than to build new infrastructure. And state officials aren't steering the money toward counties that need jobs the most, auditors found.
The Obama administration intended for the stimulus to jump-start the economy, build new schools and usher in an era of education reform. But government auditors said many states are setting aside grand plans to stay afloat.
The GAO said the stimulus is keeping teachers off the unemployment lines, helping states make greater Medicaid payments and providing a desperately needed cushion to state budgets.
But investigators found repeated examples in which, either out of desperation or convenience, states favored short-term spending over long-term efforts such as education reform.
In Flint, Mich., for example, new schools haven't been built in 30 years but the school superintendent told auditors she would use federal money to cope with budget deficits rather than building new schools or paying for early childhood education.
The 400-page stimulus plan includes provisions for long-term growth, such as high-speed rail and energy efficiency, but their effects will be seen later.
Since Obama signed the stimulus bill in February, the economy has shed more than 2 million jobs. Unemployment now stands at 9.5 percent.
Recovery.gov does not say how created by seemingly misdirected projects like building a bike path in Denver or expanding the hybrid bus fleet in Chicago, because stimulus fund recipients won't have to start reporting those figures until October, Cordes reports.
"It's definitely too premature to say whether or not the recovery act is working," Jennings said.
Robert L. Nabors II, deputy director of the Office of Management and Budget, testified that 150,000 jobs had been created from stimulus spending. With the stimulus spending, he said, the nation is moving down the right path.
"We are making progress, but we still have a long way to go," Nabors said.
The 150,000 figure comes from a formula that White House economists acknowledge was never intended to be used to count jobs. The figure is so murky, it cannot be verified.
Some democrats are calling for a second stimulus.
They say the money spent to backfill state budgets has saved teaching jobs, covered Medicaid payments, boosted unemployment benefits and increased funding for food stamps - now used by a record 34 million Americans to buy groceries.
But republicans in congress who almost uniformly opposed the first stimulus package are dead set against a second. And the white house is not eager to fight that battle again.