In a speech Wednesday at Duke University, Gates said this disconnect has imposed a heavy burden on a small segment of society and wildly driven up the costs of maintaining an all-volunteer force.
Because fewer Americans see military service as their duty, troops today face repeated combat tours and long separations from family. The 2.4 million people serving in the armed forces today represent less than 1 percent of the country's total population.
To attract and retain recruits, the Defense Department finds itself spending more money, including handsome recruiting and retention bonuses and education benefits. The money spent on personnel and benefits has nearly doubled since the 2001 invasion of Afghanistan, from $90 billion to $170 billion.
"That is our sacred obligation," Gates told the audience of compensating troops. "But given the enormous fiscal pressures facing the country," the nation must devise "an equitable and sustainable system of military pay and benefits that reflects the realities of this century."
Gates, who, has been using academic-style speeches to outline what he believes to be the nation's toughest challenges that lie ahead when it comes to defense.
Earlier this year, Gates asked whether troops were training for the right kinds of missions and called into question the utility of D-Day style amphibious landings handled historically by the Marine Corps. He has also embarked on a cost-cutting initiative to prepare for what he says are leaner days ahead for the department.
As is the case in most of these speeches, Gates on Wednesday tried to raise awareness about a long-term problem rather than solve it. He offered no solution to what he described as a growing divide between Americans in uniform and those who aren't.
"Whatever their fond sentiments for men and women in uniform, for most Americans the war remains an abstraction - a distant and unpleasant series of news items that does not affect them personally," Gates said.
Even after the Sept. 11 terrorist attacks, for most Americans "service in the military - no matter how laudable - has become something for other people to do," he added.
More on Gates
Gates gave his speech in front of some 1,200 faculty and students at Duke, considered one of the nation's top universities.
Like most elite colleges, only a small fraction of Duke students consider military service. With 34 of its 6,400 undergraduates enrolled in its Reserve Officers' Training Corps, an officer commissioning program known as ROTC, Duke is actually considered among the more military-friendly elite colleges.
Yale, for example, has only four of its 5,200 students enrolled in ROTC, whereas Harvard doesn't allow ROTC or military recruiters on campus.
Without calling out any one particular university, Gates said he was disappointed in institutions that "used to send hundreds of graduates into the armed forces, but now struggle to commission a handful of officers every year."
The wars in Iraq and Afghanistan are considered the first large-scale, protracted conflicts since the Revolutionary War fought entirely with volunteers. Most military officials agree that this isn't a bad thing. Today's U.S. military forces are considered more professional and better educated than their predecessors.
More enlisted troops hold a high school diploma, or its equivalent, than their civilian peers. Two-thirds of new recruits come from neighborhoods that are at or above the median household income.
But the military isn't representative of the country as a whole. Recruits are most likely to serve only if they grow up around others who do so. The military also draws heavily from rural areas, particularly in the South and the mountain West.
The trend is reinforced by the location of military bases, which tend to be in rural areas and the South where land is cheapest, rather than close to the big cities and the Northeast and West.
Today, most soldiers who are not deployed are stationed in Texas, Washington, Georgia, Kentucky and North Carolina. Many military facilities in the Northeast and along the West coast, meanwhile, have been shut down for environmental and budgetary reasons.
Whereas Alabama hosts 10 ROTC programs, the city of Los Angeles - with twice the population - hosts only four.
"There is a risk over time of developing a cadre of military leaders that politically, culturally and geographically have less and less in common with the people they have sworn to defend," Gates said.
The premise underlying an all-volunteer force also has changed. Initiated in 1973, the concept was that such a force would fight in short, conventional conflicts like the 1991 Gulf War, or defend the U.S. and its allies against Soviet aggression.
But after almost a decade of warfare since the Sept. 11, 2001 attacks, troops who have escaped combat unscathed still faced repeated deployments with long separations from their families. In Iraq at one point, some combat tours stretched to 18 months. More than 1 million soldiers and Marines have been deployed there during the course of the conflict.
The consequences of long deployments in combat zones have been real. Suicide figures have increased, while the divorce rate among enlisted soldiers has nearly doubled.
"No matter how patriotic, how devoted they are, at some point they will want to have the semblance of a normal life - getting married, starting a family, going to college or grad school, seeing their children grow up - that they have justly earned," Gates said.
Without offering specifics, Gates said a system must be created that is generous enough to recruit and retain people without causing the Defense Department to sink under the weight of personnel costs.