The U.S. Olympic Committee said in a statement that Mathias died in his home in Fresno, Calif. His brother said the cause was cancer.
Bob Mathias became the youngest Olympic gold medalist in a track and field event in 1948 in London, when he won the decathlon at 17. It was only his third decathlon competition, having qualified for the Olympics by winning two events in the United States.
At the 1952 Games in Helsinki, Finland, he became the first athlete to repeat as Olympic champion in the decathlon. Earlier that year, he played fullback for Stanford in its Rose Bowl appearance. Though the Washington Redskins drafted him, he never signed. Mathias also won the 1948 Sullivan Award as the nation's top amateur athlete.
"He just had a lot of natural ability in everything he did," Eugene Mathias said.
Eugene Mathias said his brother was a gifted athlete from childhood, often outperforming older children.
"I tried jumping the high jump and I couldn't make it. He was three years younger and he said 'let me try it.' He did it and he made it," Eugene Mathias recalled. "We knew then that he could just do anything athletic."
After retiring from sports, Mathias served as a Republican congressman representing California from 1967-74, serving four terms. Following his political career, Mathias became the first director of the U.S. Olympic Training Center in Colorado Springs, Colo.
He also is a member of the U.S. Olympic and national track and field halls of fame.
Mathias was the Associated Press male athlete of the year in 1952.
"Bob Mathias was one of those rare individuals with the ability to inspire a nation through his determination and perseverance. He was a champion in every aspect of life, and he embraced the values that make our country and the worldwide Olympic movement special," USOC chairman Peter Ueberroth said in a statement.
Eugene Mathias said his brother never boasted of his Olympic accomplishments and was happy to return to his home in California's Central Valley after his Olympic days ended.
"He liked the Valley here. He liked his hometown," he said.
Mathias is survived by his wife, Gwen, and several children.