Police confiscated the 16th-century painting - known as "The Taking of Christ," or "The Kiss of Judas" - and arrested the three Ukrainians and a Russian on Friday, Germany's Federal Criminal Police Office said.
They are believed to be members of an international gang of art thieves, and 20 other suspected members of it were arrested in Ukraine, the police said.
The painting, worth several million dollars, was stolen from the Museum of Western and Eastern Art in Odessa, Ukraine, in July 2008 by thieves who entered the museum at night and cut the painting out of its frame.
Anke Spriestersbach, a police spokeswoman, declined to release any information about the potential buyer, saying the investigation is still under way.
The arrests in Berlin were made in cooperation with Germany's special GSG 9 forces and Ukrainian police.
Caravaggio, a Baroque master from Italy, was known for his dramatic use of light, novel perspective and the use of ordinary people in religious and mythological scenes.