Each sport received majority support in separate votes by the IOC after leading athletes and officials from both camps gave presentations, including a taped video message from Tiger Woods and other top pros. Woods has indicated he would play in the Olympics if golf were accepted for 2016.
"There are millions of young golfers worldwide who would be proud to represent their country," Woods said from the Presidents Cup in San Francisco. "It would be an honor for anyone who plays this game to become an Olympian."
Golf was approved 63-27 with two abstentions. Rugby was voted in 81-8 with one abstention.
Rugby will organize a four-day seven-a-side tournament for 12 men's and women's teams. Golf will stage a 72-hole stroke-play tournament for men and women, with 60 players in each field.
The vote brings the number of summer Olympic sports back to 28. There have been two openings on the program since baseball and softball were dropped in 2005 for the 2012 London Games.
The vote was a reversal of the IOC's decision four years ago to reject golf and rugby for the 2012 Olympics.
Rugby and golf made their Olympic debuts at the second modern games in Paris in 1900. Golf was played again only at the 1904 St. Louis Games. Rugby was featured three more times, making its last appearance in the 1924 Paris Olympics.
Their status for the 2020 Olympics will be reviewed by the IOC in 2017.
Friday's vote also was a victory for Jacques Rogge, the IOC president who was re-elected to a final four-year term hours earlier. The 67-year-old Belgian, the president since 2001, was the only candidate.
"Time will show your decision (on the new sports) was very wise," Rogge said.
Golf and rugby were put forward by the executive board in August under Rogge's guidance, at the expense of five other sports that were cut _ baseball, softball, squash, karate and roller sports.
The selection process angered some IOC members, who wanted all seven sports put to a vote by the entire assembly. Senior Canadian member Dick Pound complained before the vote that the members were never told why the two sports were selected over the other five.
"It is not fair to the other five sports," Pound said. "Because you decided the way you did, it is not a transparent process."
The new selection system was put in place after the IOC failed to agree on which two sports should be added to the 2012 program, leaving the London Games with 26 sports instead of the usual 28. A similar failure this time would have been a blow to both Rogge and the executive board.