The conservative groups are upset because they say the bill won't do enough to cut spending - something House Speaker John Boehner takes issue with. "I think they're misleading their followers," he told reporters. "I think they're pushing our members in places where they don't want to be. And, frankly, I just think that they've lost all credibility."
Boehner blames these groups for pushing members of Congress into the government shutdown, something he says he didn't want.
Michael Needham is the CEO of one of those groups, Heritage Action. He told CBS News' Nancy Cordes they're not responsible for the shutdown. "I think that there's one person in this town - the president - who wanted to have a shutdown," he said, "and he was willing to shut down the government in order to preserve Obamacare."
Needham says he and his members should call into question GOP leaders when they disagree with them. "The Republican Party should be broad enough that we can have a fact-based policy disagreement," he said.
Chris Chocola with the Club for Growth agrees. He dismisses the idea that in voicing opposition his group is undermining the work being done on the Hill and says he has a duty to do so. "It's not about us," he said. "It's not about him. It's about the voters and the American people."
The budget bill passed in the House Thursday with a vote of 332 to 94. It now moves on to the Senate.