Building a data fabric for movie magic to maximize performance and reduce costs
Dave Frederick
If you’ve watched a classic movie lately there’s no doubt you’ve noticed that certain aspects of the viewing experience are different from today’s new releases. It’s not just the clothes and the hairstyles that date a film. It is also the visual effects and image quality. We could rabbit hole on the evolution of the art and science in movie making. But for this discussion, film quality comes down to one important factor: data. Lots more data. And data creates challenges for the technical folks whose names appear at the bottom of the credit roll, long after most of us have left the theater.
The media and entertainment (M&E) industry is no stranger to coping with technical challenges and exploiting opportunities to increase efficiency. The confluence of digital transmission, high-quality image compression, and more vivid displays allow movie buffs to enjoy a truly stunning viewing experience. Likewise, the developments in faster computing, graphics processing, high-speed networking, and virtually unlimited storage capacity have enabled content creators to fill the millions of digital screen pixels with more and more compelling material.
All this progress is not free. Unlike most IT storage environments where data files are comparatively small and easy to move around, M&E infrastructure needs to support the storage of and access to potentially petabytes of content across hundreds of creatives. These media production files are far too large to be copied from system to system as you would with a spreadsheet or document. This is why M&E storage systems are built so that artists can work on the content in place, and that requires a great deal of speed.
Is flash the answer?
From the introduction of solid-state storage devices and arrays, the media industry saw an opportunity to achieve levels of performance that previously required a roomful of spinning disks, but at the capacities required for their workloads. Flash-based systems capable of being a single, massive repository were simply too costly to consider. And using a smaller, high performing all-flash system in front of a larger, lower cost and lower performing system can create data management complexity, require expensive add-on software, and reduce access for artists.The NetApp approach
The benefit of an all-flash environment is the performance and access users get from it—especially in situations where files can be quite large like in M&E. In practice, designers and editors are not accessing everything that is stored in a repository. In fact they may only need access to a small subset of work-in-progress content. As long as they can find and access the files they need—through their regular applications—and still get flash performance, there’s no reason why “cold” data can’t be moved to a more cost-efficient tier of storage designed specifically to be a forever repository, namely an object storage system like NetApp StorageGRID.The combination of StorageGRID object storage with a NetApp AFF storage and connected by ONTAP with FabricPool creates an automatic flash-forward, self-managing tiering environment. The NetApp approach is unlike other hierarchical storage schemes that rely on stub files to reference relocated files and require entire files to be restored before they become usable. FabricPool marks blocks within files as cold when they go unread or unchanged for a policy-determined period of time. The user can browse for and access files normally from their application and the file is delivered from flash as it is accessed, even though the file still resides in StorageGRID.
This approach is perfect for media and entertainment applications where it is far more common to read files during the workflow than to overwrite them. Also, since not all M&E applications can directly access object storage systems, ONTAP and FabricPool allow these applications to access files in StorageGRID with no modifications or changes to the program. System administrators don’t need to manage the movement of data as it is accessed, and users don’t need to involve administrators to pull content from the object storage or make additional copies to convert the files back from objects.
The NetApp data fabric for media
With NetApp ONTAP, FabricPool, and StorageGRID, media and entertainment producers can benefit from a lower cost storage environment that seamlessly and automatically combines the performance of an all-flash system with the long-term durability and scalability of an object storage system. This is truly a case of the two best tools for their respective jobs working together seamlessly. To learn more about these technologies, visit the StorageGRID and ONTAP pages on NetApp.com. Or to find out the seven reasons why NetApp is best for Flash.
Dave Frederick
Dave Frederick, media solutions strategist for NetApp, is a 30-year veteran of the digital media technology industry, starting with digital audio and electronic music, digital video and 3D animation, and moving on to broadcast and post-production infrastructure. Dave’s approach is that although technology is important, it must never come between artists and their tools.
