Mann, who had battled prostate cancer since 1997, died late Tuesday, according to a friend, Sy Johnson. A funeral home in Santa Fe said it was making arrangements with Mann's family.
Mann had moved to Santa Fe in the late 1980s after spending most of his life in his native New York City.
Mann was known for performing different musical styles and creatively combining them. Always seeking out new rhythms and harmonies, he toured the world, spending time in Africa, Brazil and Japan.
Family of Mann, formed in 1973, played world music before it was called that. Mann's best-selling "Memphis Underground" was a founding recording of fusion.
He continued to work diligently on his music at a time most people consider retirement.
"I'm playing better than I've ever played," Mann said in a 1995 Associated Press interview. "I'm practicing. I always thought I could get by just with my natural instincts. As far as I'm concerned, almost everything I've done in the past has been on the surface or just a hair below. Now I'm getting serious."
When he left Atlantic Records in 1979 he started producing his own records, and later he launched his own label, Kokopelli. In all, he made more than 100 albums as leader.
Touring, he said, was "a killer, the hours and food. I always thought if you made good records your records could do the traveling for you."
Album titles reflect Mann's versatility: "At the Village Gate" (1962); "African Suite" (1959); "Brasil, Bossa Nova & Blues" (1962); "Latin Mann" 1965; "Memphis Two Step" (1971); and "Eastern European Roots" (2000).
"As much as I love music, I never really thought it was my life. I thought it was the vehicle I used to express my life," he said.
Born Herbert Solomon in Brooklyn in 1930, he started his career when he was 15, playing in groups at Catskill Mountain summer resorts. He studied saxophone but preferred flute. In the 1950s, after three years in the Army playing with the Army Band in Trieste, Italy, Mann toured France and Scandinavia.
He credited visits to Africa and Brazil in the early 1960s with changing his musical outlook.
"When I came back (from Africa), I hired (Babatunde) Olatunji, a Nigerian drummer living here, and we started doing music based on African motifs," he told the AP.
As for the Brazil tour, he said, "Revelation doesn't touch it. Up to that point, the ethnic music I had heard had 14 drums playing different parts but the melodies were very simple. Then I saw the 'Black Orpheus' movie and heard multiple rhythm parts along with the most beautiful melodies in the world.
He returned and recorded with Brazilian musicians, including Antonio Carlos Jobim and a 19-year-old Sergio Mendes.
He was 70 when he put out "Eastern European Roots."
"I've played Cuban music, but I'm not Cuban," he told the Rocky Mountain News. "I've played Brazilian music, but I'm not Brazilian. I've played jazz, but I'm not African-American. What I am is an Eastern European Jew. I love all the music I've played, but I wanted something that is mine."