Shand Kydd, 68, was admitted to the hospital near her home on the Argyll coast several days ago.
She died at her home near Oban in Scotland, said Canon Donald MacKay, a Roman Catholic priest who said he was with her when she died.
The death was also confirmed by a spokesman for her son, Earl Spencer.
"I can confirm that Lord Spencer's mother passed away peacefully this morning, after suffering from a long illness, and this is now a private time for the family to grieve," said a statement issued by Spencer's office.
Shand Kydd's health had been failing in recent months and the grandmother of Princes William and Harry had been taken from her home on Seil Island, where she led a reclusive life, to a hospital in Oban a number of times.
Buckingham Palace said Queen Elizabeth II has sent a private message of condolence to Earl Spencer.
The Roman Catholic Bishop of Argyll and the Isles, the Right Rev. Ian Murray, said Shand Kydd was a "deeply spiritual woman."
"She once said that her father had taught her to treat everyone the same, whether they came from 'castle, cottage or caravan.' That was a lesson she lived out very well, and many people, from all walks of life, will be saddened by her passing," Murray said.
She was born Frances Ruth Burke Roche, daughter of the Fourth Baron Fermoy. Her mother, Lady Ruth Fermoy, was a confidante and lady in waiting to the late Queen Mother, and was later said to have had a hand in promoting Diana's marriage to Charles.
Shand Kydd married Edward John Spencer, 12 years her senior, in 1954 at Westminster Abbey. They had three daughters — Diana was the youngest — and one son.
The marriage foundered in 1967 when she fell in love with Peter Shand Kydd, a married wallpaper heir, and the couple divorced in 1969. Shand Kydd remarried in 1969 when the future princess was aged 6.
Diana stayed with her father after the split and decades later she told Andrew Morton, author of the 1992 book, "Diana, Her True Story" that her childhood was very unhappy and unstable.
"The biggest disruption was when Mummy decided to leg it. That's the vivid memory we have — the four of us (including her brother and two sisters). ... People took sides. Various people didn't speak to each other. For my brother and I it was a very wish-washy and painful experience," Diana was quoted as saying.
Shand Kydd acknowledged she sometimes had a turbulent relationship with her famous daughter.
Testifying two years ago at the theft trial of Diana's former butler, Paul Burrell, Shand Kydd said she had not spoken to Diana in the last four months of her life and that the princess had returned her letters unopened.
Shand Kydd agreed with a statement by Burrell's lawyer, Lord Carlile, that Diana, "like many of us, could sometimes be a little tempestuous."
Diana and her companion, Dodi Fayed, were killed on Aug. 31, 1997, in a crash in a Paris tunnel as their Mercedes was pursued by paparazzi.
In an interview with the U.S. cable television channel in 1998, Shand Kydd said she had mingled with the public outside the princess's Kensington Palace home during the outpouring of grief over Diana's death.
"With literally thousands of people comforting, sharing, we were sharing a very deep hurt," said Shand Kydd. "But I think far too little has been said about what people all over the world gave to her. They nourished her, they made her smile, they gave her strength."
Shand Kydd said she was frustrated by her inability to shield her daughter from the intrusions on her privacy.
"I wanted privacy for her, very often, when she wasn't working," she said. "But it wasn't to be."
Shand Kydd also complained of the press intrusion in her own life, blaming it for the failure of her second marriage.
"The media descended here 16 years ago and have never left me since," Shand Kydd was quoted as saying in a 1997 interview with The Mail on Sunday newspaper from her home in Scotland. "I've now accepted they never will."
"I think the pressure of it all was overwhelming and finally impossible for Peter" Shand Kydd, who left her in 1988 after 19 years of marriage, she said.
In her unhappiness over her marriage to Prince Charles, Diana credited her mother as setting an example of coping.
"I've got what my mother's got. However bloody you're feeling you can put on the most amazing show of happiness. My mother is an expert on that," Diana was quoted as saying.
Shand Kydd is survived by her son Earl Spencer and daughters Lady Jane Fellowes and Lady Sarah McQuorquodale. Her stepson, Adam Shand Kydd, 49, died from an overdose of sleeping pills in Cambodia in April.
By Jane Wardell