Mr. Bush said Johanns was "an experienced public servant from America's agricultural heartland" with a long record of being "a faithful friend to America's farmers."
Johanns, 54, would succeed Ann M. Veneman, who recently announced her resignation despite saying earlier that she wanted to stay.
CBS News Correspondent Peter Maer reports Johanns is known as a strong advocate for ethanol production. He led the Governors' Association committees on power and bio-tech efforts.
Johanns served in county and city government before becoming mayor of Lincoln, Neb., in 1991.
He won the governor's office in 1998 and in 2002 became the first Republican to win re-election in more than four decades.
Johanns had been considered a possible challenger to Democratic Sen. Ben Nelson in 2006.
Veneman, a peach farmer's daughter who became the first woman to the head the Agriculture Department, presided during a period of unprecedented wariness about the safety of the nation's food supply. Weeks after taking office in 2001, an outbreak of foot-and-mouth disease in Europe prompted Veneman to increase inspections and testing to prevent its arrival in the United States. After the Sept. 11 attacks that year, concern grew that terrorists might seek to contaminate the nation's food supply.
Then came the discovery last year of the first case of mad cow disease in the United States. Veneman quickly upgraded the country's defenses, banning high-risk meat products and meat from cows that could not stand or walk on their own, testing more cattle and promising to speed a nationwide animal tracking system. Farm interests for the most part praised Veneman's response, while consumer advocacy groups said Veneman and her agency had not done enough.
So far, seven of Mr. Bush's 15-member Cabinet have said they won't be part of the second term. The latest departure was Homeland Security Secretary Tom Ridge, who announced his resignation on Tuesday.
Other Bush Cabinet figures who are leaving include Attorney General John Ashcroft, Commerce Secretary Donald Evans, Education Secretary Rod Paige, Secretary of State Colin Powell and Energy Secretary Spencer Abraham.
Mr. Bush has chosen national security adviser Condoleezza Rice for the State Department, White House counsel Alberto Gonzales for the Justice Department and Carlos Gutierrez for Commerce.
More resignations are expected, and administration officials say Health and Human Services Secretary Tommy Thompson appears to be next.