Read "Numerati" for a Penetrating Look at Data Mining

The "Numerati" are an evolving class of quant-humping, algorithm experts who will be playing an enormous role in shaping our society, our economy and our lives. They are the types who founded Google and Yahoo but they are going beyond simple searching to manipulating and massaging the tremendous mass of data that we generate from Web clicks and cell phones.

Stephen Baker has written an engrossing, elegant little book (Houghton Mifflin) about the entire genre of data mining mathematicians who are at the controls of this revolution. It's been a while since I've read a business book this good, but I must disclose that Steve has been a colleague of mine off and on for 20 years. I have always admired his writing and analytic talent and his way of explaining things in ways that are both warm-hearted and wry.

Who are the Numerati? There's Samer Takriti, the Syrian-born math Ph.D. who works for IBM and is an expert at stochastic analysis, or trying to tie predictions to random seemingly events. M.I.T.-trained Frenchman Pierre Haren is a whiz at arranging that airplane passengers from mainland China and Taiwan don't bump into each other at Singapore's airport. And, there's Rayid Ghani, a Pakistani whose expertise is studying shopping behavior by examining lots and lots of receipts. These are just a few.

Non-techies such as myself can learn a lot from Steve's book. For instance, bargain-clipping shoppers who roam from store to store snapping up specials are called "barnacles" by data miners because they are useless drains on grocery chains which count their very slender profit margins in tens of a percentage point. In the political realm, there are "Right Clickers," who are conservatives who are so savvy with computers they instinctively click on the right side of a mouse and are prime candidates for Web-based fund raising.

And, if you have an elderly parent as I do, you might be interested to know that data miners are considering putting in linoleum kitchen floors filled with sensors that can reveal tell-tale signs of problems such as weight gain. If the gain is sudden, it can mean that the elderly one is retaining lung fluid because of heart malfunctions. Or, erratic patterns can signal the onset of Alzheimer's.
Steve visits the National Security Agency which has drawn criticism for collecting billions of data bits from e-mails and cell phone calls after the 9/11 terrorist attacks. The problem, an NSA mathematician explains, is that while they can handle the massive data, it is really hard to match a name with a face, which is the sine qua non of snaring terrorists.

Some of the material is familiar as are some of the fears. Bosses can hover over us with an electronic clipboard threatening our personal privacy. Every single movement of our life can be tracked thanks to grocery loyalty cards and traffic toll payment devices.

There has been plenty written about the Data Big Brother controlling us. To Steve's credit, he doesn't fall victim to hyperventilating paranoia. He addresses the good and the bad that can come with the Numerati's growing ability to watch us and predict our next moves.

My only criticism is that the book is a little too short. I wanted Steve to draw even more detailed sketches of the individual Numerati. All in all, though, his book is excellent.

(Image by Claudecf via Flickr, CC 2.0)