Moments of silence were observed at 8:46 and 9:03 a.m., the times two planes slammed into the trade center towers on Sept. 11, 2001, and at 9:59 and 10:29, when the two towers collapsed.
Many of the family members cried quietly and embraced each other.
"Every day is hard, but this day is a little bit harder," said Nancy Brandemarti, who was attending the ground zero remembrance for the first time. "This day is just a day to think about him."
At Arlington National Cemetery, there was a moment of silence at 9:37, the time that another hijacked plane hit the Pentagon, where 184 people died. In Pennsylvania, more than 1,500 people gathered in the field where the fourth plane went down, killing the 40 passengers and crew aboard Flight 93.
The president and first lady also observed a moment of silence on the White House's South Lawn as the nation began a day of remembrance.
At the trade center site in lower Manhattan, parents and grandparents stood in pairs at two podiums, reading the list of the 2,749 people killed there - a recitation that lasted more than three hours.
They read slowly and precisely, and added poignant dedications when they reached the names of their own loved ones. "And our loving son, Paul Robert Eckna, our tower of strength - we love and miss you," said Carol Eckna.
Mayor Michael Bloomberg, Gov. George Pataki, and former Mayor Rudolph Giuliani each delivered readings. Pataki's quoted former President Dwight D. Eisenhower, describing losses of World War II: "There's no tragedy in life like the death of a child. Things never get back to the way they were."
As the reading of names began, other family members of victims descended the ramp into the pit of the trade center site and laid flowers in two small reflecting pools meant to evoke the footprints of the twin trade center towers. Others scrawled messages on the edges of the pools.
The area, seven stories below street level, is considered sacred ground by many. It was there that rescue workers combed the debris with rakes, painstakingly searching for the tiniest fragments of human remains.
Last year, children recited the names of the dead, and on the first anniversary, it was dignitaries, community leaders and some relatives of victims. This year, parents and grandparents had the honor.
Pat Hawley, 44, was among the family members at the site Saturday morning. Hawley, of Charlotte, N.C., said he comes to the ground zero ceremony every year to remember his older sister, Karen Sue Juday.
"It seems like it gets harder every year, because it's that much more time since I've been able to talk to my sister and be with her," he said.
After the fourth moment of silence, Rosanne Cash sang a muted rendition of the traditional Irish ballad "Danny Boy."
At Arlington National Cemetery, Defense Secretary Donald Rumsfeld joined relatives of those killed in the Pentagon near a large granite memorial marker that bears the names of each victim.
"The lives that were lost on September 11th have meaning. They live on as a testament to a country that is courageous, that is determined, to a people that are resilient despite great loss, and to a cause that continues until that mission is accomplished and beyond," Rumsfeld said.
Family members laid flags at the marker and ran their fingers across the names inscribed.
Both President Bush and Democratic presidential nominee John Kerry spoke on the anniversary of the attacks. Mr. Bush used the occasion to warn of continued danger and pledge victory in the war on terror.
"We will not relent until the terrorists who plot murder against our people are found and dealt with," he said in a rare live weekly radio address from the Oval Office.
Kerry praised the spirit of the country in the days and months after the attacks. "We are one America in our prayers for those who were taken from us on September 11 and for their families," he said. "And we are one America in our unbending determination to defend our country to find and get the terrorists before they get us."
Kerry spoke to a memorial service at the Boston Opera House, and was the speaker in this week's Democratic radio address.
The two hijacked planes that crashed at the World Trade Center originated at Boston's Logan Airport. Victims of the attacks on the World Trade Center and the Pentagon included nearly 200 people who either lived in Massachusetts or had ties to the state.
In a field in western Pennsylvania where Flight 93 went down, volunteers rang two large bells as the names of each of the plane's 40 passengers and crew were read.
"No words, no memorials, can ever replace all that you have lost," Homeland Security Secretary Tom Ridge said to the victims' families at the ceremony.
An official helping to plan a memorial for the site said the Americans aboard the flight changed the course of history through their selfless acts of courage.
Flight 93 was en route to San Francisco from Newark, N.J., when it was hijacked and ultimately crashed in a reclaimed strip mine in Somerset County, killing all 40 passengers and crew.
The official 9/11 Commission report, released in July, said the hijackers crashed the plane as passengers tried to take control of the cockpit.
"It is sometimes said that this is the forgotten site of Sept. 11, but I think this site will never be forgotten," said Pamela Tokar-Ickes, a Sommerset County Commissioner who serves on the Flight-93 Advisory Commission. "...(While) the sheer magnitude of the World Trade Center and the Pentagon have drawn much attention, this site has captured the hearts of the people."
Thousands of people a week visit the windy hilltop overlooking the plane's crash site, she said.
"It is where we come to try to make sense out of the incomprehensible," Tokar-Ickes said.
In Boston, a small plane pulled an American flag behind it as about 150 family members of Sept. 11 victims paused for a moment of silence. Air Force Col. Jim Ogonowski, whose brother John, piloted one of the hijacked planes, urged grieving families to remember more than the horrors of that day.
"I ask you to remember what followed the attacks," he said. "Strangers helping strangers, neighbors helping neighbors, acts of good will everywhere. That's the part of September 11th the terrorist don't want to remember."
Nationwide, communities will observe Sept. 11 in their own ways, with services at local firehouses, memorial dedications, bell-ringing events and flag ceremonies.
U.S. Army soldiers in Iraq held small ceremonies to coincide with the moment the first jetliner slammed into the World Trade Center.
Three years after the terror attacks, work still continues to identify the 20,000 pieces of human remains recovered from the trade center site. The medical examiner's office has identified about 1,570 victims, or just 60 percent. They do not expect to match the remains of every victim because some remains were too badly damaged in the fiery collapse.
Meanwhile, much has changed at the 16-acre site where the 110-story towers once stood. A 20-ton granite cornerstone was laid on July 4 for the 1,776-foot-tall Freedom Tower, the skyscraper expected to be completed by 2009.
A few blocks from the site, friends and families of the 73 people who died at Windows on the World, the restaurant at the top of the trade center, tossed roses into the Hudson River for each of the victims.
"We used to be a family and more than a family. We used to say, `God bless Windows on the World,"' said Fekkek Mamdouh, who was a waiter there.
At sundown, light beams that evoke the twin towers were to be projected upward from a lot near the site, to remain on through the night. The memorial lights were first seen on March 11, 2002, with a plan to light them each year for the anniversary.
Victims were to be honored at several other events in New York City on Saturday, including a Mass at St. Patrick's Cathedral for fallen firefighters and the dedication of a memorial to Staten
Island victims at the ferry terminal across the harbor from the trade center site.
Saturday also marked the beginning of a competition to design a permanent national memorial to honor Flight 93's passengers and crew. The memorial site encompasses 2,200 acres, including the spot where the plane crashed.