The names of the officials will not be released because they work undercover.
The two CIA officials were in the highest echelon of clandestine service, the directorate of operations, The New York Times reported in Thursday's editions.
The federal official confirmed the departures, on condition of anonymity.
A former intelligence official tells the Times they are leaving because they did not feel comfortable with the new management.
The retirements come just 10 days after two other top officials - Deputy Director for Operations Stephen Kappes and his immediate deputy, Michael Sulick - told colleagues they were leaving the CIA.
It was unclear whether Kappes and Sulick resigned voluntarily or were asked to step down.
Earlier this month, the agency's No. 2 official, John McLaughlin, retired, citing personal reasons.
The CIA has been heavily criticized for prewar intelligence lapses in Iraq and the Sept. 11, 2001, attacks. And former officials have described intense friction within the agency with Goss now in charge.
Some said there were concerns that more officers at the CIA's counterterrorist center and elsewhere may be asked to resign or told that they no longer have a future at the agency.
Goss, a former CIA officer and congressman who headed the House Intelligence Committee, made waves with the clandestine service even before President Bush nominated him to head the CIA.
Officials as senior as former CIA Director George Tenet fumed over assertions by Goss' committee this summer that the clandestine service "needs fixing" and that the agency could become a "stilted bureaucracy incapable of even the slightest bit of success."