In a speech before naval officers and contractors, Gates did not say he was planning to cut any programs or its budget.
But he did say the military must rethink whether it can afford such a massive naval fleet at a time when the Army and Marine Corps need more money to take care of troops and their families.
"Do we really need 11 carrier strike groups for another 30 years when no other country has more than one?" Gates asked.
He noted that the Navy's most expensive resources are not on the front lines when it comes to countering many modern threats, such as piracy.
"As we learned last year, you don't necessarily need a billion-dollar guided missile destroyer to chase down and deal with a bunch of teenage pirates wielding AK-47s" and rocket-propelled grenades, Gates said.
John Pike, director of the GlobalSecurity.org defense website based said this is the first time, as far as he knows, that Gates has addressed the cost of maintaining the roughly 300-ship Navy.
Pike said that Gates, by raising the issue of the size and composition of the fleet with the Navy league, had entered "the Lion's den" - a reference to the pushback he's likely to get from Congress. Many lawmakers protect the Navy shipbuilding industry because it means jobs in their districts.
Gates said the military still has a way to go to develop capabilities useful in places such as Afghanistan, where small insurgent groups are the primary threat, and Haiti, where the military is aiding humanitarian workers.
This year, the Defense Department requested nearly $190 billion to buy and develop weapons, but only 10 percent of that is dedicated toward counterinsurgency, humanitarian and similar missions.
"This approach ignores the fact that we face diverse adversaries with finite resources that consequently force them to come at the U.S. in unconventional and innovative ways," he said.