Data Deduplication in Depth

An Industry Insight by Janae Lee of Quantum Corporation

Data deduplication is currently one of the hottest topics in data protection. BM asks Janae Lee, VP of Corporate and Product Marketing at Quantum Corporation, about the key business benefits of data deduplication.

Data deduplication is all about efficiency, it allows customers to dramatically reduce – by 90% or more – the storage and network bandwidth needed to take backups, store them on disk, and move the data offsite. Data deduplication makes disk-based backup and WAN-based replication (for disaster recovery) affordable. The result is faster backups, more data available for fast, reliable restore, lower administrative costs and more secure disaster recovery options.

Whenever a new data storage technology comes along companies considering deduplication are wary of losing data that’s falsely deemed duplicative. For example, when RAID was new people would ask: “You’re writing my data in stripes across different disks? Are you sure I’ll be able to read it?” Any new data protection technology that doesn’t improve on data integrity doesn’t get adopted, and deduplication is no exception. Data in any commercial quality deduplication system has improvements in data integrity when compared to standard data stored in a RAID system. The foundational patent on variable-length block data deduplication was issued in 1999 (Quantum holds that patent), solutions have been in the market for several years and there are thousands of end users applying the technology, if an end user is concerned, they should ask their prospective vendors how they protect data integrity.

Data deduplication also makes it practical to use standard WANs – even lower bandwidth WANs – to copy backup data between sites. That is a really profound and far-reaching change. Today, most end users have isolated disk or tape backup systems in many locations. DR protection is provided by periodically using ‘tapes in trucks’ – creating removable media and moving it off site. This creates administration, logistics and data security issues particularly for smaller sites with limited IT staff. Deduplicated replication means users can keep local copies for restores and create daily remote copies for DR without having to move anything. So they get more reliable protection with lower administrative hassle and cost.

Key developments

Deduplication is in the process of extending its range and integrating with other data protection technologies. Most deduplication systems were initially designed to suit either midrange or enterprise environments – we are now seeing the first products that use the same technology for both large and small systems, providing end-to-end solutions. This trend will accelerate. This is also true for integration with tape, which is important because users will continue to leverage the economic, power and cooling advantages of removable media for medium to long-term retention, reserving disk for data that needs frequent restores. We have deduplication systems that can automatically create tape today, but there is literally only one backup package with hooks to manage that process. Soon, we expect to see many more applications integrate tape creation and replication management. Likewise we will see more software tools offered to let users manage all those connected resources from a single interface.

As VP of Corporate and Product Marketing at Quantum Corporation, Janae Lee oversees all outbound customer and partner relationship marketing, product line management across Quantum’s disk, software and tape portfolios and corporate marketing functions.