Sharon, 77, suffered a massive stroke with widespread bleeding in his brain Wednesday, and Friday's surgery was his second in two days.
Despite the improvement, Sharon is in critical but stable condition, said Hadassah Hospital director Dr. Shlomo Mor-Yosef. He said Sharon was returned to intensive care after the surgery and a brain scan.
Sharon had been rushed to the operating room Friday morning, after a brain scan indicated rising cranial pressure and bleeding.
"During surgery, the increased intracranial pressure was released, part of the blood clots that remained after the first operation were drained," he said. "At the end of the operation, there is no active bleeding and the intracranial pressure has returned to normal."
"I can say that in comparison to previous CT scans ... there is a significant improvement in the way the brain exam looks to Hadassah's experts," Mor-Yosef said.
The fact that Sharon went back into surgery is seen as a very bad sign, and is underscored by the news that several top aides left meetings to come to the hospital, reports CBS News correspondent Allen Pizzey.
Shimon Peres, Israel's elder statesman and a Sharon ally, said he was "very worried."
Sharon's sons, Omri and Gilad, were camped out in a room next door to their father's at the neurological intensive care unit.
Sharon's sudden, grave illness left his ambitious peace agenda in doubt and stunned Israelis, who were grappling with the likelihood that the man who dominated politics in the regions for decades would never return to power.
"It's a nation in mourning. The radio is playing sad songs, television is non-stop coverage," reports 60 Minutes correspondent Bob Simon. "It's mourning and it's shock, and it's very comparable to my mind to the death of [former Israeli Prime Minister Yitzhak] Rabin 10 years ago.
At Jerusalem's Western Wall, one of the holiest sites in Judaism, Orthodox Jews were praying for Sharon's recovery.
Rabbi Shmuel Rabinovitch of the Western Wall said he had received dozens of e-mails praying for Sharon's health that he printed out and stuck in the cracks of the holy site. Callers from as far away as Venezuela and the United States asked for advice in praying for Sharon, he said.
"Even if he had some things that he did that you're not happy with or disagree with, that can't change ... how he helped build the country and everything he did for the country," one Israeli told CBS News' Larry Miller.
The surgery Friday followed a seven-hour operation Sharon underwent Thursday morning after he suffered a massive brain hemorrhage. Doctors had put him in a medically induced coma to give his body time to heal, but most outside experts said his chances for recovery were slim.