The man, Inman Morales, 35, was pronounced dead at a hospital after his nearly 10-foot fall Wednesday. Police said he suffered serious head trauma when he hit the sidewalk.
"He wasn't hurting anybody, witness Racquel McDonald told CBS station WCBS in New York. "They could have just grabbed him and bring him down, but they Tased him instead and he fell to his death and it was real wrong."
Officers had radioed for an inflatable bag as the incident unfolded, but it had not yet arrived at the scene when Morales fell, WCBS reports.
"He was there cracking jokes, posing for the cops and everything. The man wasn't a threat to nobody but himself," witness Kirk Giddings told WCBS. They had about eight cops that could break the fall. They just moved back."
The lieutenant who directed the use of the stun gun was stripped of his gun and badge, and the officer who shocked Morales was placed on desk duty as the investigation continues, WCBS reported. Their names were not released.
The death of the man, who witnesses and neighbors said had become distraught and had threatened to kill himself earlier in the day, brought renewed focus to the use of Tasers by the police.
It also raised questions over why Morales was shocked with the stun gun when there was no inflatable bag placed on the sidewalk to catch him if he fell.
"They didn't try to brace his fall. They did nothing. I've seen a lot of things in my time. But what they did was wrong," said neighbor Kirk Giddens, 39, in Thursday editions of the Daily News.
In a video posted on the Web site of the New York Post, Morales can be seen clambering along a building's fire escape until he reaches a ledge and begins swinging a large fluorescent light bulb at officers below. One of the officers raises a Taser at Morales, who freezes and topples over headfirst.
Police spokesman Paul J. Browne told The New York Times that Morales' death was under investigation. He said that it was unclear whether an inflatable bag had been requested or whether it had not yet arrived at the scene.
Officers are allowed to use Tasers if they believe psychologically distressed people are a danger to themselves or to others.
Thousands of city police sergeants began carrying Tasers on their belts this year. The pistol-shaped weapons fire barbs up to 35 feet and deliver 50,000-volt shocks to immobilize people.