Google blames software update for lost email
Google said late Monday that a storage software update was responsible for causing some Gmail users to lose access to their e-mail data and said full functionality would be restored soon.
Some Gmail users complained yesterday of suddenly and mysteriously losing e-mails, contacts, and folders. Google originally said 0.29 percent of the user base was affected by the issue but has since revised that figure to less than 0.02 percent, or about 40,000 of the service's 200 million accounts.
In an update on the situation this evening, Ben Treynor, Google VP of engineering and site reliability czar, apologized for the inconvenience and said the company expects to have the lost data restored soon.
"The good news is that email was never lost and we've restored access for many of those affected," Treynor wrote in a company blog. "Though it may take longer than we originally expected, we're making good progress and things should be back to normal for everyone soon."
Treynor explained that, even though Google keeps multiple copies of the data in multiple data centers, sometimes "software bugs can affect several copies of the data." As a safety measure, the company backs up data on offline tapes, which makes restoring the data more time consuming than transferring requests between data centers, Treynor said.
When Google realized the software update was responsible for introducing the bug, it stopped the deployment and reverted to the old software version, Treynor said.
The issue came to Google's attention yesterday when users started lighting up the company's support forums with complaints of lost e-mails. "I have lost ALL on my emails/folders etc. from gmail. Why would this happen? How can I restore everything?" wrote user bkishan wrote in the forum.
Treynor also cautioned that e-mail sent to affected users between 6 p.m. PT yesterday and 2 p.m. today likely went undelivered but that the message sender should have received a notification that the e-mail was undelivered.
Affected users can follow Google's efforts to restore their data on the Apps Status Dashboard.