The hurtling rock passed about 26,500 miles above the southern Atlantic Ocean at 5:08p.m. EST Thursday.
Paul Chodas, of the near-Earth object office at NASA's Jet Propulsion Laboratory, told CBS Radio News the asteroid was moving so fast, "it travels the distance from the Earth to the Moon in about 15 hours."
It was the closest recorded encounter between Earth and an asteroid, said Steven Chesley, an astronomer also at NASA's Jet Propulsion Laboratory who works on a program looking for such objects.
Such encounters, however, are actually believed to occur at the rate of one every two years and have simply not been detected, he said.
"There certainly have been closer encounters that we didn't know about," he added.
Astronomers were continuing to observe the asteroid, 2004 FH, which was expected to be beyond the moon's orbit by early Friday.
It won't come fairly close to Earth again until 2044, when it will be within 930,000 miles.
Chesley said there was a lingering chance, of the order of one in a million, that it could hit sometime in the future, but that possibility is expected to be eliminated as its orbit is further refined.
The asteroid was close enough to Earth on Thursday to be visible through binoculars from vantage points in the southern hemisphere, Asia and Europe, Chesley said.
If it had hit Earth it likely would have broken up in the atmosphere. Its shock wave could have been strong enough to break windows on the ground, but nothing like the disastrous climate-changing effects that could result from the impact of an asteroid more than a half-mile in diameter, he said.
Astronomers had to scramble to observe 2004 FH because it was only discovered late Monday during a survey by two telescopes in New Mexico.