March 1, 2009
Bye, Tech: Dealing With Data Rot
As Storage Media And Software Applications Advance Or Die Out, Years Of Precious Memories Are Threatened
-
-
Audio recordings of 40-year-old interviews with rock stars were transferred to new tape to preserve them, but the newer tapes are disintegrating. (CBS)
-
Employees at the Library of Congress scan its collections to help ensure they are available for generations to come. (CBS)
-
Did you have precious data from the 1980s stored on an 8-inch floppy disk? Good luck retrieving it now. (AP)
-
-
Play CBS Video Video Keeping Up With Data Rot Computer formats come and go leaving some users with data no longer compatible with software or hardware. As David Pogue reports, this is called data rot.
-
Photo Essay Preserving Old Movies To save classic films, archivists put the chill on Dorothy, Lassie and Scarlett O'Hara.
-
Interactive PC Perils Facts on viruses and other computer menaces, security tips and a timeline of virus attacks.
Lydia Robertson is a filmmaker. But you've probably never seen her first movie, the one she made in high school.
"It's called 'The Chicken Lady,' a horror-comedy," she said. "It's the most ridiculous film ever made. It might be one of the worst films ever made! But we learned a lot, and that was the point."
Trouble is, she hasn't seen it, either.
"By 1986, when I was graduating from New York University film school, you couldn't get these machines to play it back on. So I had no way to play it. I've not seen it since it was made."
Her tapes are fine, but the machines that can play them don't exist anymore. In other words, her movie is a victim of data rot.
Data rot affects computers, too. Over the years, both the hardware and software programs become obsolete and are abandoned. Just ask biotech worker Bill LaVia, who can no longer open his slideshow presentations from ten years ago.
"The program is Aldus Persuasion, and it was slideshow presentation program. They basically went out of business, because PowerPoint took over that marketplace." PowerPoint can’t open his Persuasion files.
"My data is unreadable," he said.
In fact, so many computer formats have come and gone, they could fill a museum. And they do: The Computer History Museum in Silicon Valley.
Dag Spicer, the museum's senior curator, said, "There's a Pez museum, a Barbie museum, a mustard museum - why wouldn't you preserve the computer, which is maybe the most influential piece of technology in the last 100 years?
Spicer showed us the world's first hard disk, invented in 1956 in San Jose. It weighs 1,000 pounds and holds five megabytes. Compared to a modern iPod which holds 10,000 songs, the 1956 hard drive would hold one song.
(Computer History Museum)
"And it would be a real pain to carry around," Pogue said of the half-ton accessory.
"Yeah, you need big batteries!"
At a computer museum, you can't miss the point that, sooner or later, every recording format eventually gets left in the dust.
In fact, these days, new formats are coming faster and faster, and each one expires faster than what came before it.
"There's a consensus that as the ability to store more and more data [increases], the data itself has become less and less reliable," said Don Mennerich, an archivist at the New York Public Library.
He's been working on preserving some historic 1968 audio interviews with rock stars: Eric Clapton, Jimi Hendrix, Frank Zappa, The Jefferson Airplane, The Grateful Dead, Brian Wilson, Carl Wilson, Mike Love of The Beach Boys, Phil Everly.
In 1996 the library tried to rescue those 30-year-old recordings by transferring them to fresh, brand-new tapes. But Mannerich says now, less than 12 years later, "those tapes are already significantly degraded.
"The tape actually, physically will shed against the head of the tape player, until it's unplayable."
"The original tapes are holding up better than the 1996 transfer?" Pogue asked.
"Yeah."
But not everyone is unhappy about data rot. An entire industry has sprouted up to help you recover your old recordings and data. You can buy do-it-yourself gadgets that convert your old audiocassettes, records and slides into digital formats.
For audio and video recordings, there are professional converting companies like VidiPax in New York City.
"We restore, and then we make something else out of it,' said Sam Verga, the director of marketing.
(CBS)
"CBS News doesn't have the machine they need to play their own tapes?" Pogue asked.
"That's correct," Verga said.
"And you have one?"
"That's correct. We actually have eight!" Verga laughed.
Half the challenge is just keeping all those antique machines running. Verga showed Pogue a wire recorder dating back to the 1950s.
"It looks like monofilament fishing line," Verga said, "and that's actually embedded with an audio signal."
The sound that come forth was John F. Kennedy's inauguration speech.
"So are you telling me you actually get customers now and then sending you spools of wire and saying, 'Turn this into a CD?'" Pogue asked.
"Absolutely."
Of course, if you're worried about your old recordings, look at the bright side: some people have much bigger collections to worry about … like the Library of Congress.
Laura Campbell is the chief information officer of the Library of Congress, which houses the world's largest collection of movies and recorded sound, as well as 56 million manuscripts, six million maps, and a million and a half rare books.
Since the early '90s, Campbell has overseen a multimillion-dollar effort to digitize the Library's most precious holdings, both to preserve them and to make them available to the public on the Internet.
"Well, really, what we're trying to do is to save our collective memory, to be able to pass on human record of our time," she said.
And so, every day, in back rooms in Washington, D.C., library employees are painstakingly digitizing the library's 16 million photos, prints, and posters … one scan at a time.
Their boss is Helena Zinkham, acting chief of prints and photographs.
(Library of Congress)
"Actually, this is about a guy named 'Abram Lincoln,'" Pogue notes.
"True. They ran out of a little bit of space on that line in the flag, and so they left out the H."
So, working to create digital copies of artifacts that are 100, even 200 years old, how do they choose an electronic format that can be guaranteed to still be viewable 200 years from now?
"It's an open standard, a very plain type of digital file called TIFF," Zinkham said. "And we have actually had to migrate these files already."
There's that word again: Migration. The more you talk to data-rot experts, the more often you hear it. And the more you realize that converting your old recordings to digital formats is only the beginning. Preserving those files is a job that will last the rest of your life, and beyond.
"A hard drive lasts about five years," Spicer said. "The low range of CDs' and DVDs' longevity is five years. So the basic lesson is: Look after your own data and make sure that you take steps to keep it moving onto new formats about once every ten years."
In the digital age, migration is the only chance we have of preserving our recordings and our files. And again, preserving recordings is a lifelong job. But you can do it. Just follow these three easy steps.
1. You convert whatever you can afford to digital.
2. Store your tapes and films in a cool, dry place.
3. And above all, remain vigilant. As you now know, every ten years or so, you're going to have to transfer all your important memories to whatever format is current at the time, because there never has been, and there never will be, a recording format that lasts forever.
Just ask Sam Verga.
"Is there any permanent, forever storage media today?"
"Only the brain, the human brain," Verga said. "And that lasts for about 75 years."
For more info:
© MMIX, CBS Interactive Inc. All Rights Reserved. This material may not be published, broadcast, rewritten, or redistributed.
Michelle Obama tells how her role as the First Lady has changed her perspective.
One of my greatest pleasures is looking at the old photos that my dad took of me and my siblings (paper versions that still hold up). Realizing that my kids may not have that opportunity- unless I get on the ball and back them up every 5-10 years- makes me sad.
When you transfer digitally captured material from hard drive to CD or DVD and establish your migration process of constant transfer from one digital medium to the next, you might consider that your information is now ?archived?. In fact, nothing could be further from the truth. While digital media excel at capturing and manipulating images, they lack the chemical and mechanical stability to provide true archiving.
Now consider film. Chemically, it's been scientifically proven to have a useful lifespan of hundreds of years. Mechanically, it can?t fail because there?s nothing to fail. As a resolution-independent medium, film allows you to easily access the image at any time, regardless of how computer technology evolves in the future. As long as you have a source of light, you?re in business.
Why not transfer those valuable digital recordings to archival-quality 35-mm film? Never have to worry about migration again? ever. And know that if and when you need those recordings they will be there? hundreds of years from now.
DIGINEG Ltd. is a milestone in digital image preservation. This unique service uses high-tech film recorders to transfer digitally acquired or digitally born still image or video material to archival-quality 35-mm film. The film recording technology utilized by DIGINEG® was awarded a Scientific and Engineering Award, for scientific and technical achievement, by the Academy of Motion Picture Arts and Sciences at the March Academy Awards presentation.
http://www.digineg.com/
WHAT HAPPENED TO BEN STEIN????
My hero of the social conciousness got cut off!!!
Someone sleeping at the control panel?
Someone not like his views?
I was watching it, his thing started out talking about the economy with his 'economist hat' on and then blip....goes back to the segment on the kid who bakes cookies....
Very unprofessional since it is a taped show and there should be no mess ups...
You know, Sunday morning viewers deserve to see that one aired in it's entirety again.
Actually, they left out the "HA". In your article maybe the "A" in "HA" was a victim of data rot?
Tip: Computer Files
Establish a backup system so your computer files, including personal photographs and other media, are copied on a regular basis to CDs, DVDs or, better yet, to an external hard drive. Make more than one copy of your digital files and store the copies in different physical locations.
At Risk: Digital Photos
Family memories and special events that future generations would value are increasingly documented as digital photographs. But 10 years from now current memory sticks and cameras will most likely be obsolete, trapping the images in unusable or unsupported storage media.
----
the standard .gif .jpg. .tiff .bmp etc have been around for many years, they are standard image formats on all computers and readable by all, readable in email software, web browsers and image viewers, they aren't going to go away and neither is .txt or .html
Take a digital photo or audio file and keep converting and saving it to other formats, and watch how some of the data gets lost- you look at the file size for a jpg or something, some of the "unnecessary colors" and the like are deleted to make the file smaller too.
DVD's and CD's are buggy, they dont always work, Ive seen plenty of both that work just fine on one computer but cant be read on another one. Better bet is buy 2 or 3 USB or firewire external drives and duplicate all you want to save to all 2 or 3, the article says drives last 5 years but that is *IN DAILY USE* with the platter spinning at 3500 RPM or faster. If the drives are stored in the closet and added to once a month or just kept as archives they will last many many years. If one does go bad you have 2 others with the same data on them.
Emulation is a way to support access to old programs via old software running within new environments. It still needs all the 1s and 0s to be there (so you need to keep copying the files from one device to the next), but it solves the issue of programs that no-one can run anymore on their current computers. There are smart people around the world working on creating emulators to ensure future access to old digital documents. See here for a detailed examination of current projects working on emulation: http://www.spellboundblog.com/2007/12/25/digital-preservation-via-emulation-dioscuri-and-the-prevention-of-digital-black-holes/
The other part of the puzzle not mentioned here is the idea of context. If you have 100,000 digital records which were born digital (ie, created within a computer - not scanned or otherwise digitized), how do you know what the information within them means even if you CAN read them? Imagine a large database, the data within which was added via a now defunct program. Or a massive spreadsheet with cryptic column names and undocumented formulas that would only make sense to the document's creator. No easy answer here - just another challenge to consider. If you are interested in how these issues might impact archivists and journalists alike, take a look at http://www.spellboundblog.com/2007/02/17/understanding-born-digital-records-journalists-and-archivists-with-parallel-challenges/
I would add one more step to those listed at the end of the article: Document what you are keeping in the most basic text files that are migrated along with your precious digital documents. You might even keep (gasp) paper copies of file inventories and documentation.
P.S. I love your show.
It appears as though this administration needs to be held accountable for what it says; primarily because it claims such perfection and honesty. Last Sunday Meet the Press shown a news clip of President Obama saying "HE WOULD NOT RUN FOR PRESIDENT." People can change their minds, but do not be a hypocrite and say one thing and do another without being held accountable. Mrs. Robsinson said "my children say they are going to do something and just do it." Yet, Michele Obama said last week while meeting with chldren at a school in Washington, D.C.; "I hate when people ask children, what do you want to be when you grow up?" She stated her age and claimed, "I still don't know what I want to be when I grow up." Sounds like she and her mother need to compare scripts.
When so many people are trusting this Administration to help in a time of economic and job crisis; we need complete honesty. We need to hold each other accountable for our words and actions. The world needs transparency to continue to evolve on a positive path with high standards and ethics. This shift will only happen if we all take responsibility for honesty and integrity.
President Obama shunned Hilary Clinton in full view of the public in the Chambers while they were both running for President at that time. Joe Biden's wife said her husband was offered both jobs; V.P. and Sec. of State. I think the world would agree that Hilary deserved the V.P. position based on voters choices in the primary. Obama constantly says "the people voted." Where and why was that basis for decision making omitted when he chose his V.P. ?
These aren't complaints or just looking for discrepancies. The old school of politics and all the problems it has created needs to be challenged in order for a real transformation and change to occur. I agree people want change; but it also requires the Press to be clear and responsible and fearless to hold our elected officals accountable.
Thank you. I don't pretend to be a perfect human being; But I also have not chosen to be in politics and pretend to honestly represent voters.
With sincere concern, the intention to lift our goals and improve our ethics, and therefore, our present circumstances.
Janice Bakow
- by HoweHowse March 1, 2009 10:03 AM EST
- depressing subject matter to say the least, especially when you have a large collection of media. Reading more about it in the forms at cnet.com are not comforting either. Hopefully one day someone will find the right solution so we don't lose everything
- Reply to this comment
See all 13 Comments