Veteran Suicides: How We Got The Numbers
Since CBS News first aired our veteran suicide report on November 13, we have received several questions from viewers wanting more information. In order to provide as much information as possible and to fully answer all questions, this is a summary of the methodology and results of the data we presented.
When CBS News began looking into veteran suicide, we found that no federal organization or agency tracks the number of veteran suicides nationally. To our knowledge, no one is keeping count. We wanted to know how many veterans are committing suicide nationwide and how the rate of suicide for veterans compares to non-veterans.
Looking for Data
CBS News first approached the Department of Defense (DoD), the Department of Veterans Affairs (VA), the National Center for Health Statistics (NCHS) and the Centers for Disease Control and Prevention (CDC) in June and July asking for suicide data for those who have served in the military.
We were told the DoD only tracks suicides committed by military personnel who are on active duty. The VA said they had no suicide data on file at the time. The NCHS (which is the federal organization responsible for maintaining most of the nation's health statistics and is a center within the CDC) does not monitor the military status of those who have committed suicide. Finally, the CDC's Injury Center (also known as the National Center for Injury Prevention and Control) started a program in 2003 called the National Violent Death Reporting System which has collected some veteran suicide data from state death records but does not currently have nationwide counts. The CDC told CBS News that the information they have about veteran suicides "cannot be generalized to draw conclusions about the entire country."
Finding the Numbers
CBS News learned that state vital statistics departments keep count of suicides based on information obtained from death records which can be separated by military status.
Therefore, over the course of five months, we contacted all 50 states asking them to provide us with their suicide data. Most of the states were able to tally the counts from their databases while some states had to go through death records counting suicides by hand.
Some states charged us a fee for the time and labor involved in gathering the data. In total, CBS News paid about $3,000 in processing fees to the states.
Privacy Issues
Obtaining suicide data from the states involved more than just a basic public records request.
Initially, several states refused to provide the data to CBS News for privacy concerns. Here's why: the suicide numbers in some categories are small enough that individuals could be identified, violating state privacy policies. For example, one state could have two non-white females between the ages of 30-34 committing suicide in 2004 who had served in the armed forces. Because of the small number in that category, those individuals could be identified and the cause of their death would then be made public.
Therefore, in order to get the data, CBS News had to give assurance to the states that we would keep the raw data confidential. Some states insisted upon written agreements to this effect. The data, however, can be obtained upon request from the files of each individual state.
Gathering Data - Stage 1
CBS News collected the data in two stages over the course of several months.
First, we asked all 50 states to provide us with veteran and non-veteran suicide counts from 1995 to 2005 broken down by year, age, race, gender and manner of suicide. (Note: The most recent year most states have suicide data available is for the year 2005.)
Forty-five states responded to this initial request which resulted in a massive amount of varied raw information.
Five states did not provide us with raw data and total counts for suicides because they said it was not available. Those states were Georgia, Kentucky, Nebraska (only had 2004 data), Nevada and North Carolina.
In looking at the 2005 suicide counts, among 45 states that shared data, there were a total of 6256 suicides by those who served in the military for both men and women of all ages and races.
Consulting Experts
While gathering the data, CBS News consulted with several leading epidemiologists and biostatisticians from across the country including experts at the NCHS, CDC, Harvard, the University of Illinois and Columbia University to name a few.
We learned that in order to present suicide data accurately and appropriately it needs to be analyzed by adjusting the rates of suicide for age and gender as described below.
What are Adjusted Rates?
A "rate of suicide" is a type of analysis that shows how many suicides occurred per 100,000 people in a given population.
There are two basic types of suicide rates: "crude" and "adjusted." Crude is a basic calculation that does not take into consideration the make-up of the population. Adjusted rates are a more sophisticated statistical calculation (beyond simple mathematics) that take into consideration variables unique to a particular population like age and gender. The veteran population, for example, is mostly made up of older males, so the data had to be statistically adjusted state by state in order to accurately compare with the non-veteran populations.
Important Note: All of the rates of suicide that CBS News presented are adjusted. The overall rates are adjusted for age and gender in both the veteran and non-veteran populations. The male and female rates are age adjusted. And, the age breakdowns are adjusted for gender.
Gathering Data - Stage 2
CBS News decided to do a more in-depth analysis of the years 2004 and 2005 because those are the two most recent years most states had data available.
In order to do a rate of suicide analysis, we went back to the states and asked them to provide us with the 'resident' suicide counts for the years 2004 and 2005 in a uniform 'cross classified' format. ('Cross classified' means data is formatted in such a way that variables like age, gender can be matched by one another.)
Important note: A 'resident' suicide is one committed by a person who is a resident of the state as opposed to an 'occurrence' which is a suicide that occurs in the state by a non-resident.
Forty states provided us with 'resident' 'cross classified' data for the years 2004 and 2005, which was used to statistically calculate rates of suicide for the entire country. The states that did not provide us with cross classified data for the years 2004 and 2005 (because it was not available or they refused for privacy reasons) were: Colorado, Georgia, Kentucky, Louisiana, Mississippi, Nevada, North Carolina, North Dakota and Rhode Island. Nebraska could only provide 2004 data; and, the state of California only had 2005 data available.
Finding an Expert
In order to calculate rates of suicide, we needed to find a reputable independent biostatistician. We consulted with several experts from across the country and approached the University of Georgia's Epidemiology and Biostatistics Department.
The acting head of the department, Prof. Steve Rathbun, agreed to do the analysis for CBS News. He did this as a public service entirely free of charge. We choose Prof. Rathbun because of his expertise in statistics and because he had no ties with any federal agency that could pose a conflict of interest in looking at military suicide data. For example, he had no paid relationship with the DoD or the VA.
CBS News did not pay Prof. Rathbun, the University of Georgia or any of its employees for the analysis.
To avoid bias, a protocol for statistical data analysis was specified to Prof. Rathbun prior to obtaining the data. This protocol included specific questions posed by CBS News like "what is the rate of suicide for veterans versus non-veterans," specification of the statistical model that was to be fit to the data, and the methods for computing age and gender adjusted suicide rates. All subsequent analyses were conducted according to this pre-specified protocol.
Population Sources
The population data that was used to calculate rates of suicide came from the following sources:
Veteran population numbers were obtained by CBS News from the Department of Veterans Affairs. The non-veteran population numbers came from subtracting the veteran population from the general population numbers the U.S. Census Bureau provided to CBS News.
Important Note: Suicide death rates that are publically available (by going to the CDC's Wisquar's website) are for the general U.S. population. The general population includes both veterans and non-veterans together. The rates CBS News presented will look nothing like those for the general population because we compared "veterans" to "non-veterans." Comparing veteran suicides to the general population is misleading and an inaccurate analysis because, the general population includes veterans. Furthermore, the CDC general population suicide rates are age-adjusted only. The CBS analysis is adjusted for both age and gender.
Assuring Accuracy
CBS News took extreme care in collecting the data and took all measures possible to make sure the results would be accurate.
We asked each state to provide us with veteran and non-veteran suicide data. The military status of an individual who dies is typically filled out on death certificates by coroners or funeral directors. While most states assured us that the veteran counts they provided us did not include suicides by active-duty service members, we were concerned that some states many have inadvertently included such suicides in the data they provided us. So, CBS News asked the Department of Defense to provide us with the total number of active duty suicides that took place in the United States in 2004 and 2005 (including those on reserve and in the National Guard) broken down by age and gender. In an effort to prevent any over count, we then analyzed the data both with and without subtracting the DoD active-duty suicides from the state veteran counts. The results are reflected as a range in the data we reported.
Results 2004
Overall Rates
Veterans: 17.5 to 21.8 per 100,000
Non-Veterans: 9.4 per 100,000
Male Rates
Veterans: 30.6 to 38.3 per 100,000
Non-Veterans: 18.3 per 100,000
Female Rates
Veterans: 10.0 to 12.5 per 100,000
Non-Veterans: 4.8 per 100,000
Results 2005
Overall Rates
Veterans: 18.7 to 20.8 per 100,000
Non-Veterans: 8.9 per 100,000
Male Rates
Veterans: 31.5 to 35.3 per 100,000
Non-Veterans: 17.6 per 100,000
Female Rates
Veterans: 11.1 to 12.3 per 100,000
Non-Veterans: 4.5 per 100,000