Photographer's Guide to the Digital Lifecycle: Understanding Storage and Server Solutions
- Examining Storage Devices
- Exploring Storage Systems
- Real-world Scenarios
This chapter is from the book
This chapter is from the book
This chapter is from the book
)
On the surface, building a server and storage infrastructure might seem daunting. But with the right approach and a solid understanding of the choices available to you, you’ll have the confidence you need to utilize these essential tools in your digital workflow.
Storage simply refers to a place for keeping your photos, videos, and images, and it can take many forms. A server is a centralized device that manages user access, allows files to be made available to users, and utilizes some form of storage. A server can also provide additional centralized services such as digital asset management and backup, but those are discussions for other chapters. You can have storage without a server, but for the most part, you need to have storage as part of a server environment.
In this chapter, I’ll discuss the various storage options, what a server does and doesn’t do, and a number of other topics revolving around storage and servers. At the end of this chapter you’ll find recommendations for choosing an appropriate system for your digital assets.
Examining Storage Devices
The history of computers is littered with the remains of storage devices tossed to the side of the information superhighway as new technologies came along. When choosing a storage device, it’s best to keep this in mind. How many of you have data stored on a Zip or Jaz disk, or possibly even a SyQuest disk? Do you have a drive that can read that disk? Did you set aside that format when a bigger, faster, and cheaper device came along? Of course you did: That is the great benefit of technology. Migrating data from one storage format to another keeps the data viable and available.
Because specific storage recommendations change frequently, I’ll provide a broad overview of the current and acceptable storage devices. I’ll also provide the overlying reasons and techniques needed for understanding how to choose storage methods that will remain constant for years to come. You’ll need to understand the distinctions between devices to make the best choices for your situation.
)
)
Let’s begin by looking at the current storage offerings—solid state, spinning disk, and tape—and then look at larger systems that leverage the abilities of these devices into complete storage systems.
Solid State Storage
The two main types of solid state storage are memory cards and solid state drives (SSD). Memory cards are typically used to temporarily store data, such as photos in digital cameras. Thumb drives are also a type of memory card, but they use a USB connection to plug into a computer. SSDs are designed as replacements for regular hard drives that use spinning disks.
Solid state storage is the current trend in compact storage devices and for high-performance storage. Examples include the various memory sticks used in cameras, such as CompactFlash, SD, xD, and MMC. In the video world Panasonic’s P2 is an example of a specialized solid state storage device.
The advantage of solid state storage is its speed, nonvolatile nature, and lack of moving parts. If you think of RAM as solid state storage, you would be right, but it shows volatility, which means it loses its information when you remove power from it. RAM works fine in a computer to provide quick, temporary data storage during usage. But because data stored in RAM is lost every time you shut off the machine, it is considered unacceptable as a solid state storage device.
)
)
Physically rugged, nonvolatile solid state storage is best for storing data in cameras without having to worry about the safety of your files. Think of it this way: If you took a standard hard drive out of your computer and dropped it down a flight of steps, how likely is it that it would still work properly? But if you try the same thing with a thumb drive or SD card, it would work just fine. I’ve been known to leave a thumb drive in my pants pockets more than once when doing a load of laundry, and in every case the thumb drive worked fine afterwards, not to mention it’s squeaky clean! Not that laundering your memory devices is recommended, but it shows their properties well.
Solid state storage advantages:
- Physically rugged
- Relatively speedy
- Nonvolatile (doesn’t require constant power)
- Low power usage
Solid state storage disadvantages:
- Costly on a per gigabyte basis
- Limited life span
Disk Drive Storage
Standard hard drives have been around since the mid 1950s and have been getting smaller with larger capacities every year. To give you an idea how of far they’ve come, the first 1 GB hard drive was introduced around 1980 and was the size of a large kitchen appliance—hardly portable, as you can imagine. Now, drives will fit in your hand and have capacities that are measured in terabytes. Progress is good in the computer world.
Being able to store huge amounts of digital data in such small spaces comes with some issues. Inside a hard drive are two moving parts and a bunch of electronics to control them. The disks, also known as platters, spin at speeds ranging from 4200 RPM to 15,000 RPM. The disks are connected to a spindle that has bearings and a motor. A magnetic head that reads from and writes data to the magnetized disk surface hovers less than a tenth of a millimeter above the spinning disk surface.
)
Due to such high speeds and small distances between the head and disk, any contaminants that find their way into the drive enclosure will damage both the disk surface and head, leading to a loss of precious data. The disk can also be damaged by a bump or bang, bringing the head into contact with the spinning disk. These are called head crashes and must be avoided. Obviously, a spinning disk is not appropriate for environments susceptible to motion or contaminants.
But what about laptop computers? Technology built into some hard drives protects against drops. When triggered, the heads back into the parked location protecting against a head strike against the disk. This is done so quickly that the heads will be parked within a few inches of the fall. Cool stuff to be sure, but it is best not to rely on that technology. Make sure that you treat any device with a standard, spinning-disk hard drive with care.
Disk drive advantages:
- Large storage capacity
- Inexpensive
- Can be high performance
Disk drive disadvantages:
- Not rugged
- Limited life span
- High power usage
- Heat production
Disk drive connections
Over the years, hard drive connections have changed considerably. Here I’ll provide a brief overview of past and present connections. Why do I discuss the history of drive connections? Because the manufacturers keep changing the interfaces: The drive you have sitting on your desk now may not work in your next computer. Also, you might find that you’ll need to go back to older devices to recover data, so it helps to know what the different connection types are.
Back in the mid 1990s when I started using computers in a digital photography workflow, the drive interface widely used was SCSI (Small Computer Serial Interface). It uses a daisy-chain topography where one device is connected to the next. Each device in the chain is called a node. It had its own level of inelegance because it required you to set ID numbers for each node so they wouldn’t conflict with other devices on the SCSI chain. Some computers came with SCSI built in, whereas others relied on an accessory card that would plug into an expansion slot. The number of SCSI devices you could connect together on the chain was limited. Occasionally, the SCSI chain would stop working for no apparent reason. Consequently, you would need to play “SCSI roulette” where you would change device ID numbers to try to get the SCSI chain working again.
)
)
)
SCSI was largely replaced in desktop computers by Parallel ATA (Parallel Advanced Technology Attachment or PATA) during the mid 1990s. PATA is also referred to as IDE (Integrated Drive Electronics). This made life easier because you only had to define a primary and secondary device via jumpers on the drives or device, like a CD reader. Each connection (or interface) had a limit of two devices, but most computers had multiple interfaces. Some will argue that SCSI hard drives are superior to PATA drives, and in the past that was true. SCSI drives were typically built for server and storage device use that required high speed and reliability but at a high cost. PATA devices typically were used in low-cost desktop computers where reliability and performance weren’t as much of a concern. But as time went on, the speed and reliability of PATA increased.
It was a PATA world until SATA (Serial ATA) hit the market in the early to mid-2000s. SATA is the current standard. The advantage of SATA is a simplified and reduced size connector scheme as compared to that of PATA. SATA also offers higher performance. The connectors lend themselves to use in a mounting configuration where you can slide the drive in and out of the computer or storage device without having to unplug a separate connector.
)
)
Another drive connection type is called SAS (Serial Attached SCSI). You might come across it in very high-performance systems or high-end servers. One of the features of SAS is that it is custom made for moving data from place to place safely as it leverages the advantage of the SCSI protocol and wraps it in a user friendly manner. Its connectors are identical to SATA, but only systems designed to work with SAS drives will recognize them.
Tape Storage
The death of tape as a storage medium has been reported for years, but tapes refuse to fade away. This is for good reason, because tape solves some issues that other storage formats have. Tape has a long life, which is beneficial for long-term data storage such as archiving—the storing of data for future use or reference. This capacity is helpful when backing up large amounts of data, and tape is relatively inexpensive for its capacity. The downside is that other than backup and archiving, tape is unsuitable for most other storage requirements because it does not provide direct access to your photos or films.
Other storage devices, such as hard drives or solid state devices have random access ability, meaning that you can request data at any point in the storage device. The read head in a disk drive can access your files immediately by swinging over the area where the data is stored; in a solid state device, the read head can go directly to the data. However, if you need files on tape and they are in the middle of the tape, the tape has to forward to that area, and that takes time. In addition, if there is an error while reading a file, the tape has to back up and try reading the file again.
)
Another problem with tape is that you need software to format and write to or read your files from the tape. This may lead to issues in the future if you don’t have the software available to retrieve the data or even the proper tape drive.
Tape advantages:
- Inexpensive for its capacity
- Long life
- Fairly rugged (the cartridges)
Tape disadvantages:
- Requires a tape drive
- Needs software to read from and write to the tape
- Complex implementation
- No random access to the data