Speaker Paul Ryan on Friday afternoon broke his silence and condemned President Trump's remark about "sh*thole countries" from a day earlier.

The Wisconsin Republican was asked about the remark during a Q&A event at the University of Wisconsin in Milwaukee. He said that the first thing that came to mind is it's "very unfortunate" and "unhelpful."

"I thought about my own family," he said, describing his Irish immigrant relatives who came to the U.S. on what he said were called "coffin ships" and began working the railroads. Eventually, he said, they opened a farm in Wisconsin after they raised enough money.

"It's a beautiful story of America," he said. "I see this as a thing to celebrate and I think it's a big part of our strength."

Mr. Trump issued a denial Friday about using that language in a Thursday Oval Office meeting where he reportedly described Haiti, El Salvador and African nations as "sh*thole countries." The White House meeting on immigration included a bipartisan group of senators. After The Washington Post first reported Mr. Trump's remarks, later confirmed by CBS News' Nancy Cordes, the White House did not deny the comment in a statement.

While other Republicans have condemned the president's remarks, Senate Majority Leader Mitch McConnell, R-Kentucky, has not yet issued a response.

Ryan reiterated that he wants to craft a legislative solution to protect so-called "Dreamers" and make the Deferred Action for Childhood Arrivals (DACA) program permanent. But he said he wants it to be part of a deal on border security.

The speaker clarified that the immigration legislation will not be part of a government funding package that Congress will need to pass and that it'll be voted on separately. As a result, Ryan said he doubts there will be a government shutdown.

"No, I don't there there will be," he said.