"My legacy can't be compromised or diminished in any way by what we're going through now. This too shall pass," said Conyers on a local Michigan radio station Tuesday morning.
He added, "I want you to know that my legacy will continue through my children. I have a great family here and especially in my oldest boy, John Conyers III who incidentally I endorsed to replace me in my seat in Congress."
Shortly after the announcement, Rep. Sheila Jackson Lee, D-Texas, took to the House floor to read a statement from Conyers. She said he asked her to read his statement announcing his decision and that he's notified House Speaker Paul Ryan, Rep. Nancy Pelosi and Michigan Governor Rick Snyder of his plans to step down.
"Given the totality of the circumstance of not being afforded the right of due process in conjunction with current health conditions, and to preserve my legacy and good name, I am retiring. I hope my retirement will be viewed in the larger perspective of my record of service as I enter a new chapter," the statement from Conyers read.
While Conyers maintains he would be retiring immediately, CBS News' Steve Chaggaris reports that Conyers' resignation from the House will trigger a special election to replace him to be called by the governor, per Michigan law. That will take place as soon as the Michigan governor's office receives his letter of resignation.
According to Speaker Ryan's office, nothing has been received notifying Ryan of his resignation. His resignation will not be effective until the office receives the original letter announcing the resignation and a copy of his signed letter to Governor Snyder.
The news comes after Conyers' great-nephew, 29-year old Michigan state Senator Ian Conyers, told CBS News' Jonathan Blakely he would be running for the elder Conyers' seat.
According to a phone interview with The New York Times, the state senator was adamant that the elder Conyers was not resigning, but retiring instead. The 88-year-old Democrat is the longest-serving current House member.
"His doctor advised him that the rigor of another campaign would be too much for him just in terms of his health," Ian Conyers told the Times.
He added that he believed his great-uncle's constituents in Michigan would continue to support him and the rest of the family "as we continue to fight, as we have for leading up to a century."
The move comes amid numerous calls,, for Conyers to step down in light of allegations of sexual misconduct and harassment against him.
BuzzFeed was the first news outlet to report last week about a settlement Conyers made with a former staffer who had accused him of sexual misconduct. Conyers eventually acknowledged the settlement, but denied wrongdoing. Another former staff member accused Conyers of sexual misconduct and filed a lawsuit, but dropped the suit when the court refused to seal the case.
Speaking on the Mildred Gaddis radio show, Conyers said he's taking the allegations "in stride."
"This goes with the issue of politics. The game of politics which we're in, we take what happens, we deal with it, we pass on and move on forward as we keep going trying to make as much as we can of this tremendous opportunity that's been given to me for so long," said Conyers on Tuesday.
Ian Conyers did not go into detail with the Times on the allegations against his family member, but said he would stand with him.
"I stand with my uncle in terms of his belief of no specific wrongdoing," he said. "However, those things need to have their day in court."
The elder Conyers, however, continued to deny the allegations leveled against him.
"Whatever they are they are not accurate, they are not true, and I think that they are something that I can't explain where they came from," said Rep. Conyers.
Following news of the allegations,for a stress-related illness, according to his lawyer Arnold Reed.
The revelations against Conyers came amid a flurry of allegations of sexual misconduct against several of his colleagues in both the House and Senate, including Reps. Joe Barton, R-Texas, Blake Farenthold, R-Texas, and Sen. Al Franken, D-Minnesota.
When asked if Congress should disclose the details of settlements made by other lawmakers in cases of sexual misconduct or harassment, Conyers replied with "a strong, unequivocal yes."
"I think there should be a complete disclosure and revealing to all of the citizens of the country of what your federal legislators are doing or not doing and any cost that may have been incurred as a result of that," added Conyers.
Conyers has alreadyon the House Judiciary Committee late last month. He will now be leaving the seat he's held since 1965.
CBS News' Rebecca Kaplan and Catherine Reynolds contributed to this report.