The release aims to strengthen the rule of Palestinian Prime Minister Mahmoud Abbas, who has vowed to carry out the U.S.-backed "road map" peace plan.
A top Palestinian official, however, expressed disappointment at the limited release. Radical groups, whose members are not slated for release, immediately condemned it.
The United States and Israel are strongly backing Abbas, also known as Abu Mazen, as an alternative to Yasser Arafat and as a Palestinian leader they believe can crack down on radical groups and help bring peace.
"All professional assessments which were submitted to Prime Minister Sharon indicated that releasing prisoners is likely to significantly contribute to the strengthening of Palestinian Prime Minister Abu Mazen, with whom we are negotiating," Israel's Cabinet said in a statement agreeing to the release.
"A strong Palestinian administration will be able to act more effectively against terror, and therefore, Israel has a strong interest."
Sunday's Cabinet decision excludes members of militant groups like Hamas and Islamic Jihad that oppose peace with Israel and excludes Palestinians who carried out or planned violent attacks against Israelis. Hamas, Islamic Jihad and Arafat's Fatah movement are observing a cease-fire of at least three months.
It was unclear how many prisoners would be released, but Sharon met earlier Sunday with Avi Dichter, head of the Shin Bet security service, who presented him with a list of several hundred prisoners slated for release. Israel's Channel 2 TV said 400 would be released.
In Israel, the arrests of militants has been credited with helping to reduce terror attacks, and the Cabinet vote was controversial. The ministers first deadlocked 10-10 and only confirmed the measure 13-9 after the Cabinet agreed to set up a committee to monitor Palestinian compliance with the peace plan, Tourism Minister Benny Elon said.
"The release would be carried out in tightly supervised, small and measured doses in relation to proven Palestinian actions in the security sphere; in other words, if terror continues and there is no genuine Palestinian effort to fight and eliminate it, the releases will stop," Sharon said after the Cabinet meeting.
Defense Minister Shaul Mofaz said the releases will not begin immediately.
"I think that it will take some time before the committee convenes and makes decisions," Mofaz said.
Palestinian security chief Mohammed Dahlan said he was disappointed that the releases will be so limited.
"We have discussed with the Israelis our disappointment after Israel's selective policy of releasing prisoners," Dahlan said after meeting with Mofaz.
Most of the prisoners who will be released have been detained without trial, have only a few months to serve, are either under 18 or over 60, or are women, an Israeli official said.
Palestinian Information Minister Nabil Amr said the Palestinian Authority will "continue working for the release of all prisoners regardless of their political affiliations."
In Gaza City, some 1,200 people marched, carrying pictures of family members in Israeli jails and chanting, "No peace without the release of all detainees!"
Nafez Azzam, a spokesman for Islamic Jihad, warned that all prisoners must be freed in order for the week-old cease-fire to hold.
"Holding the Palestinian detainees behind bars will leave no hope for the continuation of any peace efforts," Azzam said.
The Al Aqsa Martyrs' Brigades, linked to Yasser Arafat's Fatah movement, issued a statement late Sunday demanding that all prisoners be freed and threatening that otherwise, "We are ready to carry out the most powerful and dangerous military attacks inside Israel and in the settlements."
Dahlan and Mofaz also discussed Israeli plans to withdraw from additional Palestinian cities.
Israel last week withdrew troops from Bethlehem and parts of Gaza and has promised further pullbacks in line with the "road map."
"We will continue to transfer additional towns, step by step, according to the ability of the Palestinians and according to the results of the security activities they have taken in the areas under their responsibility," Mofaz said.
"In the time that has passed since the Palestinians accepted responsibility for two areas ... there is some reduction in warnings (of terror attacks) and some reduction in incitement, but there is a long way to go toward fulfilling all the commitments."
Palestinian leaders have promised to pursue militants who violate the cease-fire but Abbas has said his police will not forcibly disarm them for fear of sparking a civil war.