Omar Edwards, 25, an African American only two years on the job, was gunned down within blocks of the Harlem police station where he worked, according to Police Commissioner Raymond Kelly.
Edwards had just finished his shift when he headed to his car and saw that the driver's-side window had been smashed and a man was going through the vehicle, Kelly said.
Edwards struggled with the man, who got away from him by slipping out of his sweater, Kelly said. Edwards chased the man up two streets with his gun drawn, he said.
A sergeant and two plainclothes officers in an unmarked police car saw the pursuit and made a U-turn to follow the men, Kelly said. One of the officers jumped out of the car and fired six times, hitting Edwards twice; once in the arm and once in the chest, he said.
Kelly said Edwards did not fire his weapon. He died at the Harlem Hospital Center about an hour after the shooting.
It was unclear whether the officers identified themselves. The name of the officer who fired the shots has not been released, but Kelly said he had worked at the department for four years.
"While we don't know all the details of what happened tonight, this is a tragedy. Rest assured that we will find out exactly what happened here, and we will learn from it so it doesn't happen again," Mayor Michael Bloomberg said at a news conference from the hospital.
Kelly said Edwards had worked in the housing bureau. He was recently married and had two young children.
In the past three years in the New York City area, there have been two other cases of off-duty policemen being shot and killed by other officers.
In 2008, a black, off-duty Mount Vernon police officer was killed by a Westchester County policeman while holding a gun on an assault suspect in suburban White Plains. A grand jury found the victim had failed to identify himself as an officer.
In 2006, a New York City police officer, Eric Hernandez, was shot and killed by an on-duty patrolman who was responding to an attack at a White Castle in the Bronx.