Both were collectors of fine art, especially French Impressionism, and each was so angry at the other that for more than 30 years, they never spoke because of terrible fist fight in 1923.
Now, more than 80 years later the brothers are reunited, at the Metropolitan Museum of Art, in New York.
"One of these men had bought 39 Renoirs, and some great Italian masterpieces," author of a book on the brothers, Nicholas Fox Weber told CBS News correspondent Russ Mitchell. "Whereas the other had acquired, with great courage, before they were accepted by the public at large, paintings by Van Gogh and Seurat and Cezanne, and Matisse and Picasso."
Weber says the Clarks are one of the best kept secrets in the history of the United State's Rich families and even today, the Clark Family Foundation has assets that are close to $500,000.
The brothers' grandfather, Edward Clark, made his fortune selling the Singer sewing machine and even developed an installment payment plan — known today as credit. When Edward Clark died in 1882, he left a fortune worth $50 million, including a staggering amount of New York real estate, to his son and four young grandsons.
Two of those grandsons, Sterling, the second oldest son, and Stephen, the youngest, each used their great wealth to collect art, which they did with passionate intensity.
Richard Rand is curator of the Sterling and Francine Clark Art Institute in Williamstown, Mass., where the idea to display the brothers' collections side by side was born.
"Sterling had a real taste for the intimate, the sensual, the private work of art," Rand said. "Stephen in a more boldly modern way goes for pictures that have a real impact from across the room."
Both loved the French impressionists like Renoir, Degas and Monet. Each bought Cezanne. And yet, says biographer Nicholas Fox Weber, their personalities and lifestyles could not have been more different.
"Sterling Clark liked things that were lusty," he said. "He had a passion for burgundies that was almost as strong as his passion for Renoirs."
Sterling loved horses, pretty women and good food. He led an expedition to China, and made his home in Paris. Stephen stayed home in New York.
"Stephen was an upright businessman," Webber said. "He was the only one of the four brothers who had a regular job. He was on the board of the Metropolitan Museum and the Museum of Modern Art. He was noticeably reserved to the extent that people who wrote about him, compared him to an undertaker."
And so it was perhaps inevitable that flashy Sterling and serious Stephen would come to blows. It happened in 1923, when Sterling married a French actress and demanded to change the family trust to include her and her illegitimate daughter from a previous relationship. Stephen said no.
"They were in a lawyer's office when each accused the other of total shenanigans as far as the hiring of lawyers and advisors," he said. "And Sterling went to give Stephen what I picture as a swift left hook, but it was Stephen who ultimately laid Stephen flat. The two brothers never spoke again, not once for the rest of their lives."
Although, says curator Richard Rand, they did keep tabs on each other and on who was buying what.
"Probably, we know from Sterling's diaries and letters that he was keeping tab on his brother Stephen's activities," he said.
In fact, on occasion, Sterling gloated in his diary about beating brother Stephen to the punch.
"Sterling wrote in his diary that when he went to see this painting at the dealer's he was told that Stephen was interested and trying to negotiate for it," Rand said. "So Sterling very quickly bought it himself."
And, at different times, they each owned the same portrait by Renoir.
Sterling saw it at a dealer's in New York and recognized it as a great picture and knew that his brother had once owned it. And he ended up buying it himself.
Last summer, 83 years after that famous fight, Stephen's granddaughter Jane, resident of New York, and Javotte Ray, granddaughter of Sterling's wife Francine, met for the first time. And, for the first time, they broached the subject that had kept them at such a distance for so long.
"It was a dangerous topic, especially for a little girl," Javotte Ray said, "and I was a little girl in those days."
"It was also a dangerous topic when I was a little girl, too," Jane Clark said.
What all of the rest of us have is the art they loved. Sterling left his collection the the Sterling and Francine Clark Art Institute, in Williamstown, Mass. Stephen's collection contains Hoppers, Cézannes, Picassos and masterworks by Van Gogh, Seurat, and Matisse; is spread among the many museums he supported, and even helped to found. And for now, both collections are side by side for all of us to enjoy.
"I think this is a spectacular moment for the family," Jane said. "I think it's fun. I think it's terrific."