The rally puts further pressure on Olmert to resign after an official inquiry described his handling of last summer's Lebanon war as a failure, reports CBS News correspondent Robert Berger.
"We think that Olmert can no longer disregard the will of the public," a student among the 100,000 demonstrators said.
Not so, say Olmert aides.
"We have already known what the public thought for a long time because of the polls," a source close to Olmert told the Jerusalem Post. "It doesn't matter how many people came to the square, because decisions are made in the Knesset and not in demonstrations, because we are not a banana republic."
In other developments:
Edan Mehallel rode to Tel Aviv from the northern port of Haifa, where he lived through Hezbollah rockets last summer, to join a mass call for the resignation of Prime Minister Ehud Olmert because of his role in the failures of the Lebanon war.
Mehallel, 16, understood the significance. "The more people there are, the more influence the demonstration will have," he said.
Olmert remained defiant, hoping to beat back a rising wave of calls to step down.
Olmert's spokeswoman, Miri Eisin, reiterated on Friday his position that "the best way to contend with the situation ... is to have a stable government, not a transition period, and to immediately ... fix the mistakes and face the challenges."
Thursday's crowd was made up of a cross-section of Israelis — moderates and hard-liners, secular and religious, young and old, a rare mix symbolizing the widespread dissatisfaction with Olmert. But it remained to be seen whether the outpouring of anger would be enough to oust the prime minister.
"Failures, Go Home!" read the banner behind the podium, referring to Olmert and his defense minister, Amir Peretz. Organizers decided not to allow politicians to address the crowd, to give the gathering a grass-roots nature, said Uzi Dayan, a retired general and a main speaker.
Also addressing the gathering was Moshe Muskal, 50, from the central Israeli town of Mazkeret Batya. His son, Rafnael, was killed during the war. "I am glad that the public is not passive or despairing," he told The Associated Press after he spoke. He said the soldiers "fulfilled their mission fully. Our mission is to make our country a little bit better."
Meanwhile, Secretary of State Condoleezza Rice was expected to return to the region to discuss the deadlines with both sides, but the Palestinian negotiator, Saeb Erekat, said no date has been set.
Erekat said the Palestinians welcome the document and would study it carefully. "I believe that this is the right approach," Erekat said. "This is transferring words to deeds."
In Washington, the State Department confirmed that American officials distributed a proposed list of steps for both sides and a spokesman said "the idea would be to do these in fairly quick order."
The document, published Friday in the Haaretz newspaper, is in line with a more hands-on approach to peacemaking by the U.S. in recent weeks. The deadlines are not binding.
Israel also welcomes the plan, but in the Middle East, implementation never comes easy, reports Berger.
An official in Olmert's office said some of the ideas contained in the document were already at various stages of implementation, citing relaxed restrictions at the Karni cargo crossing between Israel and the Gaza Strip. "There are a few Israel will not be able to address at present because of security concerns," the official added, without elaborating.
The official spoke on condition of anonymity because discussions of the document have been confidential.
The deadlines range from May 1 to August 1. Haaretz said high-level talks on the document were put off over Israel's domestic political crisis.
Past protests in the Tel Aviv square have started political earthquakes. A demonstration there after Israel's disastrous 1973 war led to the resignation of legendary Prime Minister Golda Meir and Defense Minister Moshe Dayan.
In 1982, hundreds of thousands marched to the square to protest Israel's involvement in the massacre of Palestinian refugees in Beirut by a Christian militia, a step toward the resignation of then-Defense Minister Ariel Sharon and the eventual retirement of Prime Minister Menachem Begin.
In 1995, after a peace rally, Prime Minister Yitzhak Rabin was assassinated in the square by an Israeli opponent of his policy of compromise for peace with the Palestinians. The square was renamed for the fallen leader.
Israel went to war against Hezbollah guerrillas in Lebanon on July 12 after guerrillas crossed into Israel, killed three soldiers and captured two others.
For many Israelis, the 34-day war was a failure because it didn't achieve the two main goals Olmert set — returning the soldiers and crushing Hezbollah, which fired nearly 4,000 rockets at northern Israel.