Sunday, October 15, 2006

Storing Your Digital Images

If you value your digital images, you should have a proper backup system in place. In this article, we will look at two storage methods and some backup tips so that you can enjoy your images not only in the short term, but also much further into the future.

Magnetic Storage - Hard disks

The building blocks of digital images are "bits", which can either be "zero" or "one". Magnetic storage devices such as hard disks distinguish a "one" from a "zero" by changing the magnetic properties of the disk in that location. The great thing about hard disks is that their capacities are constantly increasing while prices are constantly dropping. Two hundred gigabyte (1) hard disks (3.5" IDE 7200rpm with 8 MB cache) currently retail under US$100. Such hard disks can hold about 70,000 six megapixel JPEG or 23,000 six megapixel uncompressed RAW images. That's about 700 JPEG or 230 RAW images per dollar.

Portable housings exist with built-in international 100-240V power supply, USB 2.0 (Hi-speed), and FireWire IEEE1394 connections, costing between US$50 to US$100. To avoid overheating, get a case with a large diameter cooling fan below the hard disk. Note that the market is gradually moving towards SATA (Serial ATA) drives and away from the older and slower IDE drives. However, the choice of external housings for SATA drives is still limited.

If you store images on your computer, it is recommended to store them on a dedicated partition (or if you have multiple hard disks, then ideally on a different physical drive) than your operating system (e.g. use C:\ for the operating system and software, and D:\ or E:\ for your images). This ensures that if your operating system crashes and you need to reinstall it, your images will be preserved.

(1) Usually "200" stands for 200 billion bytes, which is equivalent to 186 gigabytes.

Optical Storage - CDs and DVDs

Optical storage media such as CDs and DVDs are polymer disks which contain the "ones" and "zeros" as "pits" and "lands" that vary the strength of the drive's laser beam as it passes through the polymer and gets bounced back to the receiver via the "mirror", which is at the back of the printed top surface.

CDs

Writable CDs (CD-Rs) come in 650 and 700 MB versions. An average CD-R costs 25 cents and can hold about 250 six megapixel JPEG or 80 six megapixel uncompressed RAW images, so about 1,000 JPEG or 320 RAW images per dollar.

DVDs

Single layer DVDs (2) currently have a maximum capacity of 4.38 GB (3), about 6 times more than a CD. DVD drives can read CDs but not the other way around. A key benefit of CDs is that they are very universal. DVDs come in DVD-R, DVD+R, DVD-RW, and DVD+RW formats. Not all DVD drives recognize all formats but nowadays most DVD burners support at least -R and +R. It is not recommended to use -RW or +RW for long term archiving purposes as they are rewritable (and pricier). The -R and +R read-only formats prevent accidental overwriting.

(2)
Double layer DVDs allow for 8.5 Billion Bytes but are less compatible.
(3) 4.38 GigaByte or 4.7 Billion Bytes (which is what is usually printed on the packaging).

Burning Tips

If you burn for example 300 MB of images onto a blank 700 MB CD-R and "close" the disk, you will not be able to add information in the future. However, if you select the "multi-session" option in your CD-writer software, you can add additional information in subsequent sessions in the future until the CD is full. Note that this is not without risks. Sometimes adding a new session can render previous sessions inaccessible. This is especially true if previous sessions were created using another CD-writer and/or burning software. One of many reasons to have more than one backup copy.

Most burning software packages have a "verification" option. This will considerably lengthen the burning session but is safer because the software will verify that the data on your hard disk corresponds to that on the CD/DVD. Burning errors are not uncommon, especially if you have been using other applications during the burning process.

Note that the "X" read and write speed specifications of DVD-writers are lower than those on CD-writers. However, for a CD-writer, 1X stands for 150 KB of data per second, while for a DVD-writer it is 1,385 KB/s or 9.2 times more. So an 8X DVD-writer will write as much data per second as a 74X CD-writer (8 x 1,385 KB/s = 74 x 150 KB/s).

Data Stability

Just like magnets become weaker over time, the magnetic properties of a hard disk will diminish in the very long term and can be affected by environmental factors such as strong magnetic fields. The materials which are used to make CDs and DVDs decay over time and the problem is that even minor changes in the data can make the whole disk unreadable. Different brands and different grades of optical media advertise different life spans. It also depends on how they are stored, how well you take care of them, how often they are read, etc.

But regardless of the above lifetime issues, even a simple scratch can render your CD or DVD unreadable. Hard disks are high precision mechanical devices spinning at high speeds, typically 7,200 rpm. So there is always the possibility of failure due to a shock or a power surge. But even without a physical failure, it is possible that you suddenly lose the content of your whole hard disk, e.g. due to corruption of the file structure. So it is important to have multiple backups because data recovery is tricky, as explained below.

Data Recovery

Once a CD or DVD is damaged or corrupted, it is very unlikely that you will be able to recover anything.

Chances of data recovery from hard disks are usually very good, but by no means guaranteed. Also, data recovery can be time consuming and expensive. So if you store your images on a hard disk, you should have at least one extra copy on another independent hard disk or on a CD/DVD. By "independent" hard disk I mean an external hard disk which is only connected to your computer when you do the backup. Two internal hard disks provide insufficient protection because both can be affected in case of damage due to lightning or loss of data due to a virus attack. Also avoid storing images in the root directory of a partition as that significantly lowers chances of data recovery via software.

Data Removal

Sometimes you may have the opposite problem: getting rid of your images permanently, e.g. destroy old backups or cleanup up your hard disk before you sell your computer. CDs or DVDs are easy to destroy, but securely erasing data from your hard disk is not as straightforward as it seems. There are plenty of affordable recovery programs which can recover data from a formatted hard disk. Formatting the hard disk, then copying dummy data to the hard disk until full capacity, followed by a format will prevent software based data recovery and should be sufficient for most of us (4).

(4)
Advanced note: (expensive) hardware based recovery techniques used in forensics and intelligence agencies can reconstruct the overwritten data based on physical differences between areas which have been "zero" for a long time and areas which have been "one" for a long time and which were only recently changed into a "zero" via overwriting and erasing. More sophisticated erasing programs with multiple and random write-and-erase cycles will make even hardware based recovery impossible.

Long Term Storage: "Migrate, Consolidate, and Refresh"

If we think in terms of decades instead of years, certain media will become useless in terms of capacity, or incompatible, or both. A typical example is the floppy disk which can barely store a single 2 megapixel JPEG image and few computers still come with a floppy drive.

In the nineties, I used 80 MB "magneto-optical" disks (5). My magneto-optical drive only had drivers up to Windows 98, so I recently migrated these onto my hard disk via an older computer which still had a parallel port and Windows 98.

(5) They have characteristics of optical and magnetic storage and are very reliable. Currently they come in higher capacities but are less frequently used, rather expensive, and require a dedicated reader/writer.

Some of the CDs burned with older burners are no longer recognized by newer drives.

So to avoid compatibility issues, it is advisable to migrate your data to newer media. Since capacities of magnetic and optical storage are constantly increasing, you can at the same time consolidate your data. For instance, 500 floppies can be consolidated into a single CD, 58 magneto-optical 80 MB disks can be consolidated into a single DVD, 275 CDs can fit on a single 200 GB hard disk, etc. So once or twice a decade you will have to migrate and consolidate your old and small capacity media to the new and larger magnetic or optical media that become available. This has the additional benefit of "refreshing" your data to overcome the earlier mentioned issue of long term data stability.

Storage Cost

Medium CD DVD 200 GB Hard disk
Type Optical Optical Hard disk - Magnetic
Cost (US$) (6) 0.25 0.75 100
Capacity (GigaByte) 0.68 4.4 186
Number of six megapixel JPEG images (7) 250 1,500 70,000
Number of six megapixel uncompressed RAW images (8) 80 500 23,000
Number of six megapixel JPEG images per US$ (7) 1,000 1,500 700
Number of six megapixel uncompressed RAW images per US$ (8) 320 660 230

(6) Estimation only. Prices vary a lot depending on where you buy, the quantity, the quality, the barnd, and the speed rating, and in case of DVD whether it is DVD+R or DVD-R.
(7) Estimated, depending on the compression and content of the image.
(8) Estimated, depending on the content of the image and the camera. Compressed RAW will allow for about 60 to 70% more images.

Since US$1 allows you to store hundreds of images, deciding whether or not you want to keep an image is not worth your time. So I just back up and keep everything I shoot. Then in a later stage I then create a second series with the best images. Image management will be discussed in a future article.

Magnetic Versus Optical Storage

. Magnetic Optical
Cost Low, but higher than optical Low
Compatibility High High for CDs, lower for DVDs
Data Stability Relatively stable, but less than optical Stable, but lifetime depends on quality
Accidental Deletion of data High (9) Low as it is read-only (10)
Environmental Resistance Sensitive to shocks and power surges Sensitive to scratches
Data Recovery Excellent but can cost time and money Virtually impossible
Backup Convenience Very convenient Requires burning
Reorganizing data Easy Requires re-burning
Traveling Convenience One 200 GB disk is relatively compact 275 CDs or even 45 DVDs are bulky.
(9) Can be reduced by setting the file attributes to "Read-Only".
(10) As explained earlier, risks are higher for multi-session burning.

So if you make at least one magnetic and at least one optical backup, you are combining the benefits of both media.

Backup Tips

1. Always maintain at least two independent copies of your images, for instance:
- one magnetic and one optical (recommended)
- two magnetic (e.g. internal and external (disconnected) hard disk)
- two optical

2. To have even more peace of mind, consider:
- two independent magnetic backups and one optical backup, or
- one magnetic and two optical backups

3. As a protection against unfortunate incidents such as fire, tornados, floods, etc., store one of your backups in a different location.

4. Be careful with multi-session CDs or DVDs and make sure you verify the data.

5. When buying a new system, "migrate and consolidate" your data to new and larger capacity media. This will at the same time "refresh" your data.

This article is written by Vincent Bockaert, author of
The 123 of digital imaging Interactive learning Suite.
Click here to visit 123di.com

No comments: