Credant Uses Funny Numbers in PR Campaign
Statistics are a dangerous tool in the hands of the unwary or uncaring. If you don't know how they work, you can be conned, and if you don't care about how you use them, you can fool at least some of the people some of the time. In high tech, the people fooled are often in the press, a fact PR people well know. The latest use hitting my desk is by Eskenzi PR & Marketing for security product vendor Credant Technologies.
A press release sent by Eskenzi for Credant discusses a survey purporting to show that "IT Security professionals admit that they are suffering from password fatigue when it comes to using their mobile devices, which leaves their data exposed to personal and corporate identity theft if these devices were to fall into the wrong hands." The release claims that it surveyed 227 "IT professionals with the majority drawn from companies that employee more than 1,000 people." It then claims that 35 percent don't use passwords on their mobile phones, though they know they should, and call it "alarming" that "the very people who are responsible for IT security are not much better at protecting the information on their business phones than most of their co-workers." There's only one problem: they data they have simply does not mathematically support such claims.
It's not that the number surveyed is too low. Going through the mechanics with a sample size calculator, you can see that even if there were a pool of 5 million IT people in companies of that size, a sample of 267 would give you a confidence interval (that "margin of error" phrase you see in the news) of 6 percent points with a confidence interval of 95 percent. That would mean if you repeated the same survey with different groups 100 times, 95 times you'd get the same results within that range of 6 percentage points.
However, that requires a number of assumptions, the main one being that the people are randomly drawn from the entire population you're studying. But in this case, the study was "carried out amongst 227 IT professionals at Infosecurity Europe 2009."
- That makes it a self-selected study, because the group probably isn't representative of all IT people.
- There's no telling how many people were approached before they got 227, which becomes another factor in the accuracy of a study.
- There is also no indication of the wording of the questions and the order in which they appeared. The interviewers might have stacked the deck to lead people to the response they wanted.
Image via stock.xchng user kikashi, site standard license.