The partisan vote was a defeat for gay rights groups who saw the provision in a defense authorization bill as their last chance any time soon to overturn the law known as "don't ask, don't tell."
Democrats fell short of the 60 votes needed to advance the legislation, which authorized $726 billion in defense spending. The vote was 56-43.
Maine Republican Sen. Susan Collins had been seen as the crucial 60th vote because she supports overturning the military ban. But Collins sided with her GOP colleagues in arguing that Republicans weren't given sufficient leeway to offer amendments to the wide-ranging policy bill.
The vote fell mostly along party lines, although Arkansas Democratic Sens. Blanche Lincoln and Mark Pryor sided with Republicans to block the bill.
Senate Majority Leader Harry Reid, D-Nev., also voted against the measure as a procedural tactic. Under Senate rules, casting his vote with the majority of the Senate enables him to revive the bill at a later date.
Gay rights advocates had been optimistic that the Democratic-controlled White House and Congress could overcome objections to repeal of the law barring gays from serving openly in the military. The move is unpopular among Republicans, military officers and social conservatives.
Now, advocates say they worry they have lost a crucial opportunity to change the law. If Democrats lose seats in the upcoming elections this fall, repealing the law will prove even more difficult - if not impossible - next year.
"The whole thing is a political train wreck," said Richard Socarides, a former White House adviser on gay rights during the Clinton administration.
Socarides said President Barack Obama "badly miscalculated" the Pentagon's support for repeal, while Democrats made only a "token effort" to advance the bill.
Reid gave Republicans the chance to offer only one amendment to address GOP objections on the military's policy on gays.
Jim Manley, a spokesman for Reid, said the senator would be willing to allow more debate on the bill after the November elections.
An estimated 13,000 people have been discharged under the law since its inception in 1993. Although most dismissals have resulted from gay service members outing themselves, gay rights' groups say it has been used by vindictive co-workers to drum out troops who never made their sexuality an issue.
Top defense leaders, including Secretary Robert Gates and Chairman of the Joint Chiefs of Staff Adm. Mike Mullen, have said they support a repeal but want to move slowly to ensure changes won't hurt morale.
Gates has asked Congress not to act until the military finishes a study, due Dec. 1, on how to lift the ban without causing problems.
He also has said he could live with the proposed legislation because it would postpone implementation until 60 days after the Pentagon completes its review and the president certifies that repeal won't hurt morale, recruiting or retention.