The United States rallied from two goals down at halftime to salvage a 2-2 draw with Slovenia on Friday in Group C.
Michael Bradley and Landon Donovan scored in the second half as the United States denied the Slovenes a win that would have made them the first team to advance to the knockout round.
A third American goal was disallowed for an apparent foul.
"This team still understands how to fight for 90 minutes," said United States coach Bob Bradley, Michael's father. "This is something we've seen time and time again."
Valter Birsa scored in the 13th minute and Zlatan Ljubijankic doubled the lead for Slovenia in the 42nd.
The result at a cold and blustery Ellis Park left Group C wide open, especially after, with the last round of group stage matches to decide which teams move on.
After Friday's games, Slovenia had four points to lead Group C, while England and the United States both had two. Algeria had one.
"We lost our concentration in the second half and the Americans benefited from that," Slovenia coach Matjaz Kek said. "We were ahead of the United States but we didn't overcome this pressure."
Bradley scored the equalizer in the 82nd minute from Jozy Altidore's header. Altidore nudged it forward and Bradley, running at full speed, caught up to it just in front of the goal and tapped it up and over Slovenia goalkeeper Samir Handanovic's head for the equalizer.
United States substitute Maurice Edu volleyed Donovan's free kick into the net in the 86th but the goal was disallowed as the referee called an apparent foul that perplexed the Americans.
"I still don't know why the goal was disallowed," Bob Bradley said. "Nobody knows at this moment."
Donovan gave the Americans hope at the start of the second half. Picking up a long pass from Steve Cherundolo, he broke in from the right flank and blasted a right-footer into the top of the net from a tight angle. It was Donovan's 43rd goal for the United States.
"I'm a little gutted to be honest," Donovan said. "I don't know how they stole that last goal from us. ... I'm not sure what the call was. He (the referee) wouldn't tell us what the call was."
Going all the way back to 1930, the Americans have never won a World Cup match in which they trailed. Still, Donovan was proud of his team for canceling a two-goal deficit.
"My guess is there's not many teams in the tournament that could have done what we did," Donovan said. "And that is what the American spirit is about."
Birsa scored from 25 meters (yards), stunning United States goalkeeper Tim Howard, who stood frozen on the goal line and watched the shot drift into his upper left corner.
The Slovenian winger was left unchallenged by a passive American defense and had ample time to adjust his aim before curling the shot perfectly with his left foot.
"I just lost sight of it," Howard said. "The pace and curl with which he hit it, it just froze me."
Ljubijankic doubled the score on a counterattack, set up by attacking partner Milivoje Novakovic. Howard rushed out to stop him but Ljubijankic kept his cool and slotted the ball inside the far post.
Slovenia captain Robert Koren said the referee was close enough to have a good view of the situation when Edu's goal was disallowed.
"Always when it's a set piece, there's a lot of pushing around," Koren said. "He felt that some of the USA players pushed our players. ... I think he whistled already before the player scored the goal."
With a population of just over 2 million, Slovenia is the smallest country represented at the World Cup and the United States is the biggest. The Slovenes face England in the last group stage match while the Americans take on Algeria in Pretoria.
"I want to point out that we have gained a point today and we are in a good position," said Kek, whose country lost all three group stage matches eight years ago in its only previous World Cup appearance.
Before the match, midfielder Andrej Komac didn't think a Slovene win would be so surprising., prompting U.S. goalkeeper Tim Howard to respond: