Davis, 57, died at her home in Gloucester.
Davis was found to have breast cancer in 2005 and had a recurrence this year. Her health took a turn for the worse during the past week, her office said.
She became Virginia's first woman elected to Congress in 2000, and she was a member of the House Armed Services Committee and the Foreign Affairs Committee.
"She inspired Virginians as she battled breast cancer while continuing to serve the commonwealth," Gov. Timothy M. Kaine, a Democrat, said in a statement.
Her first piece of legislation, passed by the House in 2001, increased the life insurance benefit paid to survivors of military members killed on duty.
Before Congress, Davis served four years in the Virginia House of Delegates.
A conservative who came from modest means, Davis was known for her unquenchable inquisitiveness and how quickly and deeply she learned about any legislative issue.
She attended Hampton Roads Business College; she got her real estate license in 1984 and her real estate broker's license four years later. In 1990, she opened Jo Ann Davis Realty.
She underwent chemotherapy treatments and a mastectomy when her cancer was first diagnosed in 2005. When the cancer returned, she underwent chemotherapy again and often monitored hearings from home.
Survivors include her husband, Chuck, two sons and a granddaughter.