The first attempt a few hours earlier by the Army Corps of Engineers failed when a rope tied around the whale's tail loosened and the mammal plunged to the depths of the harbor.
The Army Corps sent in a work boat, Hayward, with a 20-ton capacity crane to retrieve the carcass, which had been left overnight tied to a pier.
Two officers from the NYPD scuba team then arrived and successfully retrieved the whale, despite "zero visibility" underwater.
The whale retrieved Thursday was a baby, only 12 feet long, and had bloody marks on its belly.
It died early Wednesday evening. The end was witnessed by spectators who had been drawn to the dock area in Gowanus Bay by news accounts.
Tim Lafontaine, of the Army Corps, said it could have taken two days for the carcass, about 3,500 to 5,000 pounds, to surface on its own. If it was not retrieved, it could have posed navigational hazards.
Once divers wrapped a rope around the whale's tail, it was hooked to the Hayward's derrick and laid on its deck. The boat then headed for an Army Corps of Engineers dock in New Jersey, where marine experts were to perform a necropsy later Thursday to determine the cause of death.
Animal activists said the minke whale, about a year old, had been too young to survive on its own.
Kim Durham, a rescue specialist at the Long Island-based Riverhead Foundation for Research and Preservation, who had monitored the animal's activities said the situation took a turn for the worse early Wednesday afternoon when the whale's swimming patterns changed.
Durham said a colleague, marine biologist Robert DiGiovanni, was observing the animal when "it suddenly began heavy splashing, hit the dock and then just went quiet."
The whale was first spotted on Tuesday in Gowanus Bay, a small estuary off industrial south Brooklyn that is the outlet from the Gowanus canal, a narrow 1.2-mile waterway once lined with pollution-generating coal yards, scrap yards and small industries.
The canal has improved in recent years due to environmental cleanup efforts. After a huge underwater fan, designed to keep the water flowing, was reactivated, crabs and other marine creatures began turning up.
The estuary is lined with docks, storage warehouses and a large fuel oil depot.
Minke whales are a subspecies of baleen whales, common in northern Atlantic waters, and feed on plankton and krill. They are not known for singing like their cousins the humpback whales. Underwater listening devices picked up only a few "grunts" in the Gowanus waters, Durham said.