In a rematch of the 2008 European Championship final that Spain won by the same score, Puyol leaped in the center of the area in the 73rd minute to score from Xavi Hernandez's corner kick, finally giving Spain the lead after having dominated from the opening whistle.
Spain's best previous World Cup finish was fourth in 1950, when the final round consisted of a four-team group.
In Sunday's final at Soccer City, Spain will meet the Netherlands, which has also never won football's biggest trophy.
Spain controlled throughout with its precision passing game, playing just the way it hoped to, but had trouble finishing until Puyol's late strike.
For much of the game, three-time champion Germany merely sat back and waited patiently to snuff out Spain's attack, hoping to launch a counterattack that rarely took off.
It was a sharp change from Germany's last two outings, when it scored four goals each in impressive victories over England and Argentina.
Spain had never beaten Germany at the World Cup in three previous tries but this was just the way the European champions wanted to play, having also won their last three games by a goal each.
At Euro 2008, Spain snapped a 44-year major title drought, and the team has lost only two games since November 2006 - the last coming in its World Cup opener against Switzerland.
On a comfortable evening inside Moses Mabhida Stadium, Queen Sofia of Spain looked on from her seat between South Africa President Jacob Zuma and FIFA president Sepp Blatter.
The match got an early jolt when a fan carrying a vuvuzela ran onto the field four minutes in - suspending the match briefly until security officers forcibly removed the intruder.
Slumping Liverpool forward Fernando Torres was dropped from Spain's lineup and Pedro Rodriguez was given his first national team start, complementing David Villa in attack. Torres scored in Spain's 1-0 win over Germany in the Euro 2008 final but hasn't found the net in this tournament.
Germany coach Joachim Loew picked Piotr Trochowski to replace suspended winger Thomas Mueller, who has scored four goals in South Africa.
Spain set the tone early with its quick passing and Pedro set up Villa with a well-placed through ball in the sixth minute, but Germany goalkeeper Manuel Neuer did well to leave his net and block Villa's shot.
In the 13th, Spain threatened again when a dangerous header by Puyol from the edge of the box missed high.
With Spanish players swarming around in bunches in midfield, Germany struggled to launch its counterattack, its passes either too short or too long to break through the defense.
Needing one more goal to equal former Brazil standout Ronaldo for the career World Cup record with 15, Miroslav Klose had the ball stripped at the edge of Spain's area in the 24th before he could even contemplate a shot.
The first serious threat from Germany didn't come until the 32nd minute when Spain goalkeeper Iker Casillas dived to his left to push wide a long, low shot from Trochowski.
Always seeking to make an extra pass, Spain passed up a few decent looks at the target and the match began to even out.
Germany playmaker Mesut Oezil strode through the center of Spain's defense in first-half injury time but was muscled off the ball by Sergio Ramos. Wishing for a penalty, Germany fans jeered when Hungarian referee Viktor Kassai refused to blow his whistle.
A few seconds later, Spain produced the last chance of the half when a long, low shot from Pedro rolled right into Neuer's waiting arms.
The second half started much like the first, with Spain taking the initiative.
Xabi Alonso sent a long shot wide in the 48th, then pulled in a little closer for another more dangerous attempt two minutes later, forcing Neuer to dive to his left, but the ball again sailed wide.
In the 55th, Spain's attack stalled at the edge of Germany's area and Villa sent another long-range attempt off target.
Three minutes later, Spain produced two more chances - one right after the other.
First, Neuer saved an attempt from Pedro, then Andres Iniesta slid a cross through the box but Villa couldn't reach it for what would have been a sure goal.
Germany hardly even approached Spain's area until Lukas Podolski passed to second-half substitute Toni Kroos in the area in the 69th and Casillas stopped Kroos' shot by the post.
Desperate for a goal, fans began to blow harder on their vuvuzelas and they were rewarded by Puyol.
Spain then nearly doubled the lead when an effort from Pedro faltered in the 82nd.
Germany: Manuel Neuer, Philipp Lahm, Per Mertesacker, Arne Friedrich, Jerome Boateng (Marcell Jansen, 52), Bastian Schweinsteiger, Sami Khedira (Mario Gomez, 81), Piotr Trochowski (Toni Kroos, 62), Mesut Oezil, Lukas Podolski, Miroslav Klose.
Spain: Iker Casillas, Sergio Ramos, Carles Puyol, Gerard Pique, Joan Capdevila, Sergio Busquets, Xabi Alonso (Carlos Marchena 90, Xavi Hernandez, Andres Iniesta, Pedro Rodriguez (David Silva, 85), David Villa (Fernando Torres, 81).