Configuring Servers for Streaming, Part Two

Server Storage Pointers

The idea that streaming media can consume a considerable amount of storage is an understatement. The old law of storage utilization that says "users will consume 110% of the amount of available storage" could probably be updated to: "streaming media will consume 1000% of available storage."

Many organizations will store files in several formats for streaming. Some will store content in one master file format, often MPEG-2. A general rule of thumb is that MPEG-2 compressed video data takes about 1 GB of storage per hour of video content.

To estimate the file size of a Windows Media .asf file, Microsoft suggests you use the following calculation: (X Kbps * S seconds) / 8192 = Y MB
X is the encoded bit rate in kilobits per second (Kbps) and S is the length of the stream in seconds. Y is the approximate total file size in megabytes (MB).

Here is a table of asf file storage requirements, courtesy of Microsoft:

Aggregate bit rate	Minutes of content	Approximate file size
22 Kbps	30	4.8 MB
37 Kbps	30	8.2 MB
50 Kbps	30	11 MB
100 Kbps	30	22 MB
300 Kbps	30	67 MB
1 Mbps	30	220 MB

Storage calculations become much more complicated when calculating Microsoft’s Multiple Bit Rate (MBR) and RealSystems’ SureStream features for encoding and storing files at several target rates. Microsoft suggests you factor in the aggregate bandwidth of each target rate plus approximately two-thirds of the lowest selected bit rate to arrive at storage requirements. PlayStream has a handy calculator for figuring out storage for multiple bit rate files. Go to www.streamingcalculator.com/

Tip: Avoid using the Network File System (NFS) as either a client or a server. Because so many Internet companies have standardized on Solaris (and other Unix flavors), it may be tempting to utilize NFS. Don’t. It adds overhead and latency to the network and server, resulting in poorer performance.

Tip: De-fragment regularly. With today’s large capacity disk drives (e.g. 180GB and up) it is now practical to use single disks to hold streaming media data. If you do, you must ensure that your disk is de-fragmented regularly. A fragmented disk distributes data blocks on different part of the disk platter, forcing data-reads to search all over the disk for the next data block. A de-fragmented disk stores media in a contiguous pattern, ensuring fast disk-reads from one data block to the next. This results in a nice, smooth stream of data.

Tip: Look for high-rotation speed disks – e.g. 10,000 to 15,000 rpm – to pull bits off the disk as fast as possible.

Tip: Direct server attached SCSI arrays are the fastest means of moving disk data to your streaming media server. However, if you routinely share large amounts of data over a network, consider a storage area network (SAN). Because SANs can be complicated, select from among vendors that specialize in high performance storage systems and rich media data transfer. These companies include Compaq, EMC, Network Appliance and Rorke Data.

Most streaming media storage subsystems will utilize RAID technology. The following is a discussion of general RAID principles and various RAID levels as applied to streaming media.

When it comes to streaming, RAID is king for two reasons:
Disk striping, which improves I/O performance by balancing I/O load across the disks and uses all of the read/write heads at once during an I/O operation.
Multiple disks, which provide fault tolerance (typically through associated data parity logging, allowing the recovery of user data should a disk fail).

More About RAID Striping
RAID arrays stripe data across the disks (a.k.a. chunks and elements) within the array, making several disks look like just one disk to the server (it’s magic!). Let’s look at two examples:

Imagine you have one stream of media that is four data blocks long and you have a RAID array comprised of four disks. In this example, striping the data would write one block on each of those four disks. When you want to read that four-block stream of data, each of the read/write heads of each of the four disks reads its one block and shoves the block, in the correct order, down the I/O bus. It’s fast.

Now imagine that those four blocks are four different streams of data (e.g. four different movies). If four users want four separate streams at once, each disk drive in the RAID array can read the requested data and deliver it to the server for each user request. Here, the RAID array is acting like four separate disk drives.

When writing those data blocks, which disk gets which block? RAID array write operations are controlled through a round-robin algorithm. When an actual data stream consisting of thousands of blocks of data is written to disk stripes, how many blocks actually get written to each stripe is defined by the size of the stripe (chunk size).

Tip: A single stream of media achieves optimum performance from small chunk sizes in RAID arrays. This is because the data stream can be read by all drives in a parallel operation utilizing all the drives at once. If multiple users are requesting many different pieces of content all at once from the streaming server, it is best to have each content data stream on its own small-sized stripe-set. Four different movies should be placed on four different RAID devices, each with a small stripe size. Large arrays of disks can be broken up into many logical RAID arrays.

Next Page: More About RAID Parity >>