Mr. Bush said Johanns "brought focus and energy to the department" but was "serious about going home and possibly serving the nation in a different capacity."
Mr. Bush announced Johanns' departure at a Rose Garden ceremony. He said Deputy Agriculture Secretary Charles Conner would take over as acting secretary.
Several state officials said that Johanns is expected to seek the Senate seat being vacated at the end of next year by Sen. Chuck Hagel, R-Neb.
A Johanns entry into the Senate sweepstakes would be welcomed by Republicans who lost control of the chamber in the 2006 elections and who also are seeing some key incumbents like Hagel and Sen. John Warner of Virginia opt against running for re-election.
"He would make an outstanding member of the United States Senate and there is no doubt in my mind," Mr. Bush said of Johanns.
Mr. Bush earlier this week proposed filling another Cabinet vacancy the position of attorney general - with retired federal judge Michael Mukasey, who was nominated to succeed Alberto Gonzales.
Inside the White House, Mr. Bush has seen the departures recently of longtime advisers Karl Rove and Dan Bartlett. Press secretary Tony Snow's last day was Friday.
Mr. Bush said he wanted to wish Johanns a "warm farewell," saying he was "not only a decent person, an honest person, but a person who gets things done."
Johanns said that his decision to "leave this post has not been an easy one."
"I grew up with farmers and ranchers as my childhood heroes and my mentors. Representing them in Washington has been a great privilege," he said.
Mr. Bush said that Johanns had come to him and asked for advice about what course to take. "And my answer to Mike was ... 'I support you and I encourage you to follow your heart,'" Mr. Bush said.
Johanns drew immediate criticism for leaving his post before Congress finished work on a new farm bill, a goal he had said he wanted to reach by the end of the year. The politically popular legislation gives billions in aid to farmers and pays for nutrition programs. The current version expires Sept. 30.
Former Nebraska congressman Hal Daub, already a candidate for the Senate seat, said it's unfortunate Johanns decided to quit when he "could have been instrumental in securing the passage of a strong farm bill, legislation that is so desperately needed by our state's farmers, ranchers and agribusiness owners."
And the state Democratic party sent out a news release criticizing Johanns' early departure. "Unfortunately, the secretary has turned his back on farmers and ranchers repeatedly, most recently walking away from a critical farm bill," the release said.
Republican officials in Nebraska said Johanns would announce his Senate bid soon. Vicki Powell, Johanns' campaign manager for his two gubernatorial races, said she expects an announcement "in the near future."