Lee died Sunday in a Minneapolis nursing home after battling lung and brain cancer, said Janice McClain, his partner of 13 years.
Lee played one of the younger members in the "Our Gang" shorts in the 1930s, appearing in more than 40 of them from 1935 to 1939. The comedies, produced by Hal Roach, became known as "The Little Rascals" when shown on TV in the 1950s.
Among the films Lee appeared in were "Bored of Education," which won the Oscar for best one-reel short subject in 1937; "Our Gang Follies of 1936"; "The Awful Tooth"; and "Roamin' Holiday."
In a 1998 interview with the Star Tribune of Minneapolis, the Texas-born Lee said he was 2 years old when his mother sent his picture to studio executives who were seeking an actor to play McFarland's brother.
"We were on the next train to L.A. and I had a contract within a few days," Lee said. "Fat kid got lucky."
"My memories are not about making movies. We played with our toys and the adults played with theirs (the cameras)," he said.
He and Billie "Buckwheat" Thomas teamed up against older boys Spanky and Alfalfa in many of the comedies. The Porky character is credited with originating the catchphrase "otay."
In the interview, Lee recalled a warm friendship with his black costar when they were kids and praised their interracial relationship on screen, saying, "Buckwheat played an absolute equal part in the Gang."
Lee told friends his career ended when a growth spurt made him thinner. "They wanted Porky to be a chunky fellow, so they looked for someone else," McClain said.
He was born Eugene Lee in Fort Worth, Texas, in 1933. His adoptive parents began calling him Gordon after Gordon Douglas, who directed many of the films Lee appeared in. He kept the first name as an adult.
Lee was a schoolteacher, living in Colorado for a time. He moved to Minnesota after he retired to be closer to his only son, Douglas, said a friend, Tracy Tolzmann. In recent years, Lee sold autographed photos of himself as Porky, Tolzmann and McClain said.
"Before that he felt like he was forgotten," McClain said. "It really made him feel good about himself."