Shortly after lunch on Tuesday, James Cabe and Derek Manky, of cybersecurity company Fortinet, got a call from CenturyLink, an Internet service provider. Amazon had reported an attempted hack, and CenturyLink traced an unusual amount of web traffic aimed at the site back to the Mandalay Bay hotel in Las Vegas, which was hosting the Black Hat conference, an annual congregation of security professionals and professional hackers.
Fortinet was hired to protect Black Hat's networks. The team flipped on its real-time firewall monitoring technology and within 30 seconds was able to spot the source: a classroom on the hotel's third floor. The instructor there was teaching a course on advanced web penetration attacks.
The lesson had gotten a little out of hand. Rather than shut down the class, Fortinet sought out Bart, a Black Hat engineer and a "mountain of a man" who stands over six feet tall.
"We sent Bart down there to say, 'Guys what are you doing?'" Cabe told CBS News. The traffic eased.
A couple days earlier, network power mysteriously went out then came back up.
But such is to be expected when you put several thousand hackers into one hotel on the Vegas strip.
And why on Earth would you ever want that many hackers under one roof showing off how to take down corporate networks and consumer products? In a word: education. Call it hacking for the greater good. Or an institutionalized version of keeping your friends close and your enemies closer.
"As with any profession, individuals who are driven to grow professionally get together to collaborate and learn from each other," said Ted Harrington, an executive partner at Independent Security Evaluators, a consultancy that helps companies find and fix potential hacking risks in their products.
This year, he and his collaborators launched IoT Village at DEF CON, the world's longest-running hacker conference. The less corporate, more hostile-hacking-and-mohawks gathering started Thursday, following directly on Black Hat's heels, just down the strip. The "village," a sort of conference within a conference, challenges hackers to exploit weaknesses in connected products within the "Internet of Things (IoT)" -- turning on televisions, opening door locks, hijacking baby monitors, blood pressure monitors, children's toys and more.
"The stakes are further elevated in the security community because security researchers are often motivated by a desire to defend, to find issues before the bad guys do, so that harm can be avoided," Harrington said, adding that hacker conventions provide a platform for security researchers to publish their findings "in a way that is meaningful to the many businesses, governments and consumers that are impacted by that research."
DEF CON has been running for 23 years. Last year, 16,000 people attended. Black Hat's sixteenth year saw 11,000 researchers, tech execs and general code-play enthusiasts taking part in seminars, workshops and dramatic look-what-I-hacked revelations.
"Conferences like this started with mostly outliers attending -- that is hackers, not feds -- but as network technology and the threat level started to get very real, two things happened: More outliers and edge cases began attending and more representatives of very serious security establishment did as well," said cyberwarfare advisor David Gewirtz.
"That created something wonderful: People who should talk but otherwise would never meet were in the same place, learning and talking about the same things."
"You have to let the badness occur"
As with any conference (especially one in Sin City), there's bound to be "a bit of away-from-home craziness," Gewirtz added.
When it's a conference of hackers, shenanigans take the form of breaking into mobile devices, skimming credit cards and, oh yeah, bombarding Amazon. If it's online, it's fair game. Veterans leave their laptops at home. Some bring burner phones.
In the seminal 80s graphic novel "Watchmen," about a group of vigilante crime fighters, there's a lingering question: Who watches the Watchmen? At Black Hat you might wonder: Who's hacking the hackers?
The answer is Fortinet.
"We're trying to think like Black Hat attackers," said Manky, who spent the week of the conference, which wrapped up Thursday, camped with Cabe and 30 other engineers in a dark room lit only by the glow of laptops and network maps, watching for suspicious activity.
"You have to let the badness occur," said Cabe. "We're here to keep it in the bounds of the playground."
Fortinet's August 6 "Bad Guy Report" counted 43 types of viruses and spyware being circulated around the grounds. Malicious activity peaked in the late afternoon and dropped to zero by the time the dinners and parties started.
Most of what goes on at Black Hat is "ethical hacking." Researchers figure out how to compromise a piece of code and show off their discoveries -- but typically not before alerting the companies affected and giving them ample time to protect themselves and their customers from attack.
Such was the case with the IoT Village participants, when Chris Valasek and Charlie Miller took over a Jeep Cherokee via its infotainment system (an exploit that led to the recall of 1.4 million vehicles), and when teams identified multiple weaknesses in Google's Android mobile operating system.
Still, sometimes hackers will be hackers. Before Marc Rogers and Kevin Mahaffey presented their hack of a Tesla Model S at DEF CON they warned:
Note - only one of the 6 vulnerabilities we will discuss and release has been fixed. Disclaimer: With great access comes great responsibility - In other words we are not responsible for any Tesla Model S bricked by over enthusiastic attendees of this talk :)
Caveat Tesla emptor.