Members of the conservative House Freedom Caucus on Wednesday said they're not convinced yet that Rep. Paul Ryan should be the next speaker of the House.
The group said it plans to meet with Ryan, chairman of the House Ways and Means Committee, later in the day to discuss conditions he laid out a day earlier.
"We've got a lot of work to do between now and Friday if that's his imposed deadline," Rep. Tim Huelskamp, R-Kansas, House Freedom Caucus member and chairman of the Tea Party Caucus, told reporters Wednesday after a closed-door conference meeting.
Although Ryan does not need the Freedom Caucus to secure the nomination to be speaker, Huelskamp says that Ryan would need a substantial majority of its members to support him on the House floor, where he will need 218 votes to win.
Ryan, a Wisconsin Republican, said Tuesday night he would give the major GOP caucuses until Friday to endorse him. House Republicans are now scheduled to vote by secret ballot for speaker next Wednesday and hold a floor vote next Thursday.
CBS News' Nancy Cordes asked Speaker Boehner, R-Ohio, how likely it is that the Freedom Caucus will back Ryan.
"I think that Paul Ryan would make a great speaker, but this decision is up to the members, and I thought that last night went very well, and hopefully by the end of the week, we will have a nominee," Boehner said.
Huelskamp said he wants to ask Ryan, "'What do you mean? You've got five demands? What exactly does that mean for the institution? What does that mean that you don't want to work weekends?'"
Ryan's conditions included converting the GOP from being "opposition party" to a "proposition party," updating conference rules, unifying House Republicans and having time to spend with his family.
"This position is very important," Huelskamp added. "It's two heartbeats away from the president. I don't think we should rush into filling that position."
The Freedom Caucus met Wednesday morning and its members said its endorsement of Rep. Daniel Webster, R-Florida, still stands.
Rep. Raul Labrador, R-Utah, a leading member of the group, said he's concerned Ryan is asking for more power in the speaker's office, rather than less power.
He added that Ryan's demand to repeal the motion to vacate the chair is a "non-starter." This is the motion that gives the rank and file members the power to oust the speaker. In July, Rep. Mark Meadows, R-North Carolina, filed a motion to vacate the chair to oust Speaker John Boehner, R-Ohio.
Asked whether Ryan's demand to repeal that privilege would affect whether he supports him, Meadows told reporters, "Obviously, it's a factor, but at this point, to characterize anything as a defining factor would be inappropriate."
Another Freedom Caucus member, Rep. Mo Brooks, R-Alabama, said he's "troubled" that Ryan is being pushed into a job he doesn't want. He's also unhappy with some of Ryan's previous policy positions on some issues, such as immigration reform.
"On a policy level, I would hope that Paul Ryan would take a position that would more consistent with the will of the American people and Republican voters," he said.
He added, " If the candidates for speaker remain as they are, I will be voting for Daniel Webster. That is, as of now, the candidate that has been endorsed by the House Freedom Caucus."
But other House Republicans who want Ryan to take the gavel said members of the Freedom Caucus are isolating themselves.
Rep. Peter King, R-New York, said that if group does not support Ryan, then it will prove that they're "out of touch with reality."
"I don't see who else it could be. If it's not Paul Ryan, I think it would be a disaster," he said. "We'd be in total disarray if it's not Paul Ryan."