Doctors are starting to sound the alarm over what many see as a disturbing trend.
As CBS News correspondent Jan Crawford reports, an unjustified fear of concussions -- in part because of hype in the media -- is helping cause a sharp decline in kids' participation in sports.
And that, some doctors say, is much more harmful to kids long-term than a concussion.
A few years ago, Bret Strachan, director of day-to-day-operations at Blessed Sacrament Football, had a waiting list of players hoping to join his program. Now he's barely filling some of his rosters.
"Our numbers are down, some of our competitors are dwindling, so the league was reduced to seven teams," Strachan said. "Now we're down to five teams."
It's all happened in just in the last four years, Strachan said.
And football isn't the only sport taking a hit.
From 2008 to 2012, participation in soccer and baseball is down 7 percent, and for basketball, 8 percent.
Experts say one of the main reasons for the decline is the fear of concussions, fueled by media coverage of high-profile lawsuits by players taking repeated hits throughout their careers in the NFL.
But now a growing number of doctors are saying we've gone too far. Medical science doesn't justify parents' fears about concussions in youth sports.
"There's really no good evidence to suggest that a child, after a single concussion, is at any significant risk for long term effects," NYU Langone Medical Center's Director of Neurophysiology Dr. William Barr said.
In fact, doctors say the risk to kids from inactivity is greater than the danger of harm from concussions.
"If somebody says 'I like playing soccer, but my mother and father are worried that I am going to get a concussion so therefore I'm going to chose not to play soccer,' -- that is a tragedy," Dr. Marc Difazio, child neurologist at the Children's National Medical Center said.
He and other doctors are emphasizing there's no definitive evidence that a concussion causes long-term damage. And there's no scientific evidence supporting some current recommendations for treating concussions -- such as extended periods of rest, often for weeks.
While doctors recommend immediately removing kids from play if they've sustained a concussion, studies indicate long periods of rest, which are increasingly recommended, can actually mimic and prolong concussion symptoms.
"It's not that we don't have data to support it," Dr. Difazio said, "it's that there is some data that suggests it may be counterproductive and potentially harmful."
Harmful because it's proven that kids need exercise -- not only to head off obesity and heart disease, but for brain development and learning.
"What has happened now is that it's gone so far that parents are afraid to have their children participate in these sports," Dr. Barr said.
That's a fear Theresa and Thomas Urban know well. Their 16-year-old daughter Maddie was diagnosed with a concussion after a game.
"At first the doctor said two weeks off," Theresa explained. "No TV, no cell phone, you need to shut down. No stimulation, just rest, rest, rest."
But two weeks of forced rest became a year, and her symptoms persisted.
"A lot of it was like, feeling empty, and I was dizzy a lot, I didn't remember a lot of things," Maddie said. "A lot of it was just always being tired."
"It's horrible to watch your daughter go through that," Theresa said. "You don't want them to hurt. You don't want them to be lonely. You don't want them to be sad."
Today Maddie is back on the soccer field, playing with a different coach and team after a doctor at Children's Hospital of Philadelphia said to heal, she needed to play.
"His recommendation was get Maddie out there," Theresa said. "She needs to be with the girls. She needs to, like they say, take the bubble wrap off, mom. She needs to get back to her life."
Maddie's parents are suing her old coach, saying he kept her in the game and made her injury so severe.
They are hoping to raise awareness about proper concussion management -- and that's exactly what these doctors say has to happen.