Hoyer said on CBS' "Face the Nation" Sunday that he talked to Giffords' husband, astronaut Mark Kelly, who's furious over the heated nature of recent political debate, which some have suggested might have pushed to go on a shooting rampage.
"I talked to Mark Kelly, [Giffords'] husband, who is an astronaut. [He is] very angry, very angry about the level of angry rhetoric that he believes incites people - not only that Loughner targeted [Giffords] in particular, but to target those in authority, whether they be judges, members of Congress, local officials," Hoyer said.
He added that many lawmakers and their staffers are concerned.
"My colleagues are very concerned about the environment in which they are now operating. It's been a much angrier, confrontational environment over the last two or three years than we have experienced in the past. I think there is worry about that," Hoyer said.
Giffords Doctors: Good News on Bullet Trajectory
2nd Person of Interest Sought in Tucson Shooting
Victims of the Tucson Shooting Rampage
Special Section: Tragedy in Tucson
Sen. Chuck Schumer, D-N.Y., also called on the country to come together.
"Maybe in a certain sense this terrible tragedy will help us to take a time out, to look, each of us, into ourselves and remember the shared values that we really do all have," Schumer said.
"One of the grandest of American values is debate. It can be strong debate. But it should always be civil. When a shooting like this occurs, it sends a deep chill down the spine of the body politic."
Other lawmakers, including Rep. James Clyburn, D-S.C., and Illinois Democratic Sen. Dick Durbin, have joined the chorus to ratchet down the rhetoric.
CBS News chief Washington correspondent Bob Schieffer noted that Giffords herself had warned that vitriolic rhetoric could have serious repercussions, and that she had received threats and her office was trashed during the health care debate.
She had been one of the representatives on a as one of those targeted for defeat, with cross hairs over her district.
"We need to realize that the rhetoric and firing people up and, you know, even things, for example, we're on Sarah Palin's targeted list. But the thing is that the way she has it depicted has the cross hairs of a gun sight over our district. When people do that, they got to realize there's consequences to the action," Giffords had said on an MSNBC interview last March.
Gifford's friend and fellow colleague in Congress from the state of Arizona, Republican senator John Kyl, said more needs to be known before blaming political rhetoric as the cause of the tragedy.
"I didn't really think that that had any part in a law enforcement briefing last night. It was speculation. I don't think we should rush to speculate," Kyl said on "Face the Nation," saying that more needs to be known about the intent of the suspect Loughner, who's been described as mentally unstable.
"It's probably giving him too much credit to ascribe a coherent political philosophy to him. We just have to acknowledge that there are mentally unstable people in this country," Kyl said. "Giffords, a fine representative from Tucson, I think would be the first to say don't rush to judgment here."
Hoyer agreed that more about Loughner's specific motivation needs to be known, but maintained that incendiary rhetoric should be heavily guarded against, especially in the media.
"Far too many broadcasts now and so many outlets have the intent of inciting people to opposition, to anger, to thinking the other side is less than moral. I think that is a context in which somebody who is mentally unbalanced can somehow feel justified in taking this kind of action. I think we need to all take cognizance of that and be aware that what we say can, in fact, have consequences," Hoyer said.
Schieffer asked his guests whether there should be security details for every member of Congress. CBS News Congressional correspondent Nancy Cordes pointed out that almost none of the Members of the House of Representatives have security to protect them.
"Going out into the community and meeting with constituents is the life blood of members of the House of Representatives. They hold hundreds of events like this. This is how they get to know their constituents," Cordes said. "So it's not really an option for members of Congress to barricade themselves in their offices and retreat from public life."
She added that it would be "prohibitively expensive to give these 435 members security around-the-clock. They are in a sense taking their lives into their hands when they go out into the community, especially in this politically charged environment where so much violent rhetoric, violent imagery is becoming commonplace, sending a message perhaps to the less stable among us that we consider our opponents not just people with whom we disagree but actually our enemies."
Kyl said that Giffords herself was a proponent of the Second Amendment, and that she would not want to be restrained from community outreach.
Schumer agreed that safety is important, but it's important for Members of Congress to go out into the community.
"One of the hallmarks of our democracy is being able to go out and meet people and hear what they have to say and let them bring their grievances for us. That was part of the First Amendment that Congresswoman Giffords read on the floor," Schumer said.
Hoyer said the shooting was not simply an attack on Giffords. "This is an attack on democracy itself, on the ability, as she said in that reading of the First Amendment, to peaceably assemble, to come together to talk to one another. That's what democracy is all about," Hoyer said.