Brown, who had been battling lung cancer and heart disease, had been in ill health for the past year and lost his home in Slidell, Louisiana, to Hurricane Katrina.
"He was completely devastated," said Rick Cady, Brown's booking agent. "I'm sure he was heartbroken, both literally and figuratively. He evacuated successfully before the hurricane hit, but I'm sure it weighed heavily on his soul."
Brown's career first took off in the 1940s with blues hits "Okie Dokie Stomp" and "Ain't That Dandy," but he bristled when he was labeled a bluesman.
In the second half of his career, he became known as a musical jack-of-all-trades who played a half-dozen instruments and culled from jazz, country, Texas blues, and the zydeco and Cajun music of his native Louisiana.
By the end of his career, Brown had more than 30 recordings and won a Grammy award in 1982.
"I'm so unorthodox, a lot of people can't handle it," Brown said in a 2001 interview.
Brown recorded with Eric Clapton, Ry Cooder, Bonnie Raitt and others, but he took a dim view of most musicians - and blues guitarists in particular. He called B.B. King one-dimensional. He dismissed his famous Texas blues contemporaries Albert Collins and Johnny Copeland as clones of T-Bone Walker, whom many consider the father of modern Texas blues.
"All those guys always tried to sound like T-Bone," Brown said.
Brown's versatility came partly from a childhood spent in the musical mishmash of southwestern Louisiana and southeastern Texas. He was born in Vinton, La., and grew up in Orange, Texas.
Brown often said he learned to love music from his father, a railroad worker who sang and played fiddle in a Cajun band. Brown, who was dismissive of most of his contemporary blues players, named his father as his greatest musical influence.
"If I can make my guitar sound like his fiddle, then I know I've got it right," Brown said.
Cady said Brown was quick-witted, "what some would call a 'codger."'
Brown started playing fiddle by age 5. At 10, he taught himself an odd guitar picking style he used all his life, dragging his long, bony fingers over the strings.
In his teens, Brown toured as a drummer with swing bands and was nicknamed "Gatemouth" for his deep voice. After a brief stint in the Army, he returned in 1945 to Texas, where he was inspired by blues guitarist T-Bone Walker.
Brown's career took off in 1947 when Walker became ill and had to leave the stage at a Houston nightclub. The club owner invited Brown to sing, but Brown grabbed Walker's guitar and thrilled the crowd by tearing through "Gatemouth Boogie" - a song he claimed to have made up on the spot.
He made dozens of recordings in the 1940s and '50s, including many regional hits - "Okie Dokie Stomp," "Boogie Rambler," and "Dirty Work at the Crossroads."
But he became frustrated by the limitations of the blues and began carving a new career by recording albums that featured jazz and country songs mixed in with the blues numbers.
"He is one of the most underrated guitarists, musicians and arrangers I've ever met, an absolute prodigy," said Colin Walters, who is working on Brown's biography. "He is truly one of the most gifted musicians out there.
"He never wanted to be called a bluesman, but I used to tell him that though he may not like the blues, he does the blues better than anyone," added Walters. "He inherited the legacy of great bluesmen like Muddy Waters and John Lee Hooker, but he took what they did and made it better."
Brown - who performed in cowboy boots, cowboy hat and Western-style shirts - lived in Nashville in the early 1960s, hosting an R&B television show and recording country singles.
In 1979, he and country guitarist Roy Clark recorded "Makin' Music," an album that included blues and country songs and a cover of the Billy Strayhorn-Duke Ellington classic "Take the A-Train."
Survivors include three daughters and a son.