|Page (1) of 4 - 08/16/05||email article||print page|
Shared File Systems: Foundations for Digital Post ProductionComparative Review of ADIC StorNext, Avid Unity, and SGI CXFS
Recent advances in technology now allow the intermediate step of film production cutting and splicing negatives, adding optical effects, and printing distribution copies to be performed digitally, hence, the term Digital Intermediate (DI). As an alternate to the film lab, DI is not only cleaner, quicker and more flexible, but is also more practical and convenient.
The output from DI is expected to match, or supersede, the quality of a film intermediary. The main task of a DI infrastructure is to move digital film images between various pieces of equipment in a DI facility. As high resolution image files predominate, film sequences require extremely large amounts of data, from 200 to 1200 MB for every second (24 film frames per second).
Storage Area Networks (SAN) with dedicated Fibre Channel networking are the primary method for providing high-performance shared storage in DI environments. SANs provide applications with direct access to files and provide faster access to large files. A shared file system is a critical component of a DI SAN infrastructure. Shared file systems are cross- platform software packages that support distributed clients and applications running on different operating systems (e.g., Mac OS, Windows, UNIX, etc.) to access and share the same storage. Shared file systems also provide a single, centralized point of access control for managing DI files and databases, which can help lower total costs by simplifying administration.
The three shared files systems most widely used in post-production facilities today are ADIC StorNext, Avid Unity, and SGI CXFS.
The StorNext file system leads in multi-platform support for a wide range of available client platforms. The StorNext file system and its predecessor, the CentraVision file system, have been shipping for over six years, so maturity is another favorable factor. Integration with ADICs back-up and data management tools is another strength.
The main weakness in the StorNext solution is that it is a software-only point product that can be complex to integrate into existing DI client and storage infrastructures. In comparison, other shared file system offerings come as integrated components of full-function hardware/software solutions with less complexity and risk.
The main strengths of the Avid Unity file system are that it is proven, provides real-time frame-level collaboration, and is developed as part of an end-to-end solution (editor applications, media management and other tools), coming as a fully integrated component of the Avid Unity MediaNetwork shared network storage solution. This integration reduces, even almost eliminates, the complexity and risk associated with deploying a media-shared storage environment.
The limitations of Avid Unity MediaNetwork are in performance scalability. Aggregate bandwidth limits, a 20TB native capacity limit, and lack of full-band support for 2K/4K resolutions do not allow using the Unity MediaNetwork in higher-end DI facilities at this time. In addition, third-party storage systems are not supported in an MediaNetwork infrastructure.
The key strength of SGI CXFS is that it is the center-piece of a complete DI-focused solution that includes networking, storage systems, and integration services. A major weakness in CXFS is that third-party storage systems are not supported as part of the CXFS infrastructure.
While shared file systems do a good job at providing the facilities for robust sharing of DI content, a number of infrastructure challenges remain in the way of high-performance and reliable delivery of DI data which we anticipate to be the focus of the next generation of DI-focused storage and networking infrastructure.
The fundamental problem with existing storage architectures now deployed in DI environments is that the storage and delivery of digital video and film images are tightly coupled. To deliver 1.2 GB /s, every segment of the data path, from the storage through the data link, to the end workstation adapter, and finally to the application receiving buffers, must meet the necessary quality of delivery requirement at the same 1.2 GB/s throughput.
Obviously the weakest link in the data path determines overall system performance. In most cases, the storage system is the weakest link. One reason is that storage systems today are all based on conventional disk drives whose I/O performance is closely related to the rotational speed of the disk platter. In addition, disk drive based storage systems often suffer severe performance degradation when multiple read/write requests are applied to data blocks concurrently, resulting in rapid thrashing of the disk read/write actuators.
A new class of media switch products is emerging which complements shared file systems by enabling high-quality DI media delivery in a shared storage network environment. Media switches can use a mix of capabilities for optimal media delivery performance including decoupling data storage from data I/O, reducing or eliminating data fragmentation from data storage, and data caching.
Digital Intermediate (DI): A New Intermediate Production Step
One of the three major steps in film production is changing. The three traditional steps include:
? Image capture and ingest
? Intermediate (accepting shot material, production of finished ?film deliverables)
? Mastering for distribution, projection, and transmission
Traditionally, a film lab has performed the intermediate step cutting and splicing negatives, adding optical effects, and printing distribution copies. However, recent advances in technology now allow this intermediate step to be performed digitally, hence, the term Digital Intermediate (DI). As an alternate to the film lab, DI is not only cleaner, quicker and more flexible, but is also more practical and convenient.
DI Infrastructure: The Basis for Workflow Efficiency
The output from DI is expected to match, or supersede, the quality of a film intermediary. DI work is performed at High Definition (HD), 2K, and 4K resolutions. From a business perspective, image size costs money. An uncompressed HD image requires about 8 MB of data, while a 2K image requires about 12 MB of data per 10-bit log RGB frame. A 4K image requires about 48 MB of data, quadrupling storage and networking bandwidth requirements.
The main task of a DI infrastructure is to move digital film images between various pieces of equipment in a DI facility. As high resolution image files predominate, film sequences require extremely large amounts of data, from 200 to 1200 MB for every second (24 film frames). A DI facility is typically forced to use several types of data networking technology, applied to different areas, to achieve an efficient workflow and avoid bottlenecks. To maintain this performance level, in addition to sophisticated networking technology, applications and storage systems must continuously handle data at the required rate and handle the demands on the network by other users. Therefore, choosing the correct infrastructure hardware and software components, and using networking technology advantageously, is imperative.
Storage Area Networks (SAN) with dedicated Fibre Channel networking are the primary method for providing high-performance shared storage in DI environments. SANs provide applications with direct access to files and provide faster access to large files. A shared file system is a critical component of a DI SAN infrastructure. Shared file systems are cross- platform software packages that support clients and applications on different operating systems (e.g., Mac® OS, Windows®, UNIX, etc.) to access and share the same storage.
Shared file systems also provide a single, centralized point of control for managing DI files and databases, which can help lower total costs by simplifying administration. Shared file systems typically allow administrators to manage volumes, content replication, and point-in-time copies from the network. This capability provides a single point of control and management across multiple storage subsystems.
Shared file systems can accommodate both SAN and Gigabit Ethernet-based Network Attached Storage (NAS) clients side-by-side to offer a wide scope for sharing and transferring content. Although NAS does not perform as well as SAN, it is easier to scale and manage, and is often used for lower resolution projects.
Shared file systems require metadata servers to support real-time demands of media applications. In large concurrent post-production facilities, thousands of file requests for video and audio files come from each application. In DI applications, requests could number as many as 24 file requests per second per user. Metadata servers and the networks that support shared file systems must be able to sustain these access demands. Out-of-band metadata networks can provide a significant advantage over in- band servers that share the same network link as the media content because metadata and content are not sharing the same bandwidth.
In a hardware-based RAID, as the number of concurrent users increases, the stripe group must be increased to met the total bandwidth demand and not drop frames. High resolution files require significant increases in bandwidth for each additional user forcing RAID expansion. As stripe groups increase, it becomes increasingly difficult to maintain data synchronization, calculate parity, drive ports, and maintain data integrity.
When concurrent high-resolution content users must rely on large file-based RAIDs and large network switches, performance is difficult to maintain and infrastructure problems arise. Spindle contention becomes an issue when multiple users request the same content within a stripe group; available bandwidth is reduced, variable latencies are created and the file system cannot deliver frame content accurately. If a RAID storage system becomes more than 50% full, content data fragments over time, storage performance drops, and users lose bandwidth. These infrastructure issues must be resolved before users can take full advantage of shared file systems in a high resolution digital environment.
Three shared files systems are the most widely used in post-production facilities today.
? ADIC StorNext File System
? Avid Unity File System
? SGI CXFS File System
Related Keywords:white paper, industry analyst, Saqib Jang, file systems, digital post production, ADIC StorNext, Avid Unity, SGI CXFS, strengths, inefficiencies, strategy, digital intermediate, DI, posproduction infrastructure