All the data in the world

There was the Stone Age, then the Bronze Age and then the Iron Age. If there is a name for the period of time we are in now, it must be the Information Age. I guess it really started with Gutenberg about 600 years ago, when, almost overnight, there was the possibility for cheap widespread dissemination of information. For many years, information solely meant printed matter, but, gradually at first, electronic information began to appear. Initially this was audio recordings and later video. And then digital computers came along and everything changed.

Access to more and more data is surely a mixed blessing. Although I can find or check on any piece information in an instant, keeping track of all that stuff is becoming harder and harder …

For me personally, the information revolution over last 30 years or so has been a mixed bag. On the positive side, I used to have a ridiculous number of books, which needed storing, organizing, dusting and moving from time to time. There are two reasons why I have a lot fewer nowadays. Firstly, most reference books are obsolete because I can get the information that I need more easily and quickly from the Internet. Only a few reference books, like dictionaries seem worth hanging on to. For all other reading matter, I am a total convert to e-books – my Kindle follows me everywhere. I guess my collection of historic software and electronics books may stick around. Another interest of mine is photography and I am totally happy to have said goodbye to film – sorting and storing slides and negatives was such a pain. Although I take far more pictures now, like all users of digital cameras, I do have a cataloging system that means that I can find them. I am even getting better at using the “digital photographer’s best friend”: the DELETE key.

The move to digital storage of information has been very rapid. Some recent research at the University of Southern California has looked at how things have progressed in recent years. Between 2000 and 2007, we went from 75% of information being stored in analogue form, like video cassettes, to 94% being stored on digital media. The time-frame that they investigated actually started in 1986, which coincidentally was when I first used an IBM PC. It had a 10Mb hard drive, which seemed quite adequate. I recently bought a 1Tb NAS drive [for almost no money] – 100,000X bigger. I have a feeling that I will have no trouble filling it in due course.

I think that Dr. Martin Hilbert, who led the University of Southern California study, summed up the impact and contemporary significance of information rather well: “Basically what you can do with information is transmit it through space, and we call that communication. You can transmit it through time; we call that storage. Or you can transform it, manipulate it, change the meaning of it, and we call that computation.”

I got to wondering how much data there is in the world. Hilbert’s team’s research gave me an answer: in 2007 it was 295 exabytes [1000 gigbytes = 1 terabyte; 1000 terabytes = 1 petabyte; 1000 petabytes = 1 exabyte; and, BTW, 1000 exabytes = 1 zettabyte]. They describe what this might look like as a pile of CDs or a books spread over the Earth’s surface. I would be interested to hear how you might visualize such an amount of data by comment or email.

Leave a Reply

This article first appeared on the Siemens Digital Industries Software blog at https://blogs.sw.siemens.com/embedded-software/2011/09/29/all-the-data-in-the-world/