The Growing Market for Email Archiving

IDC’s Laura DuBois explains how archiving can help protect and manage information assets to alleviate pressures brought on by regulation, litigation, IT limitations and overall business requirements.

Today’s corporations are focused on managing their valuable information assets. This includes protecting information from violation or compromise, and ensuring it is retained according to regulatory requirements and is readily available in the event of audits, legal discovery or for general business use. Firms cannot tolerate risk and financial impact from the compromise of sensitive corporate information or the lack of its accessibility.

As more firms communicate and conduct business via electronic means, the volume of e-mail grows exponentially, placing pressures on the IT organization to manage e-mail to optimize performance, availability and capacity. Increasingly, valuable information assets include content that must be managed to mitigate regulatory and legal risk, avoid compromise to sensitive corporate information and reduce CAPEX and OPEX costs associated with growing volumes. E-mail archiving solutions address these business and technology challenges.

E-mail archiving applications provide an automated and efficient way of storing, indexing and retrieving individual e-mail messages and file attachments in real-time by individual users, IT staff and other authorized parties from both inside and outside the firm. E-mail archiving differs from backup and retrieval solutions in that the latter are designed to provide regularly scheduled copies of an e-mail server or disk to enable e-mail to be brought back up and running after the server or disk fails, and therefore do not maintain copies of e-mails exchanged between backups, retain copies of e-mails deleted by users after the backup is replaced with a newer one, or move e-mail content off primary e-mail servers to more efficient storage systems. Today, e-mail archiving can be delivered as on-premises software solution, sometimes pre-installed on an appliance, or as a hosted service.

E-mail archiving in the context of electronic discovery

Firms in particularly litigious industries such as financial services, healthcare, life sciences, construction, retail and manufacturing often face high numbers of legal matters, and need to retain, preserve and access relevant information contained in bodies of e-mail messages. The discovery of electronic information in e-mail during corporate litigation is increasing, impacted by amendments to the US Federal Rules of Civil Procedure (FRCP) in December 2006. In 2000 alone, Wal-Mart was sued 4851 times – or nearly once every two hours, every day of the year, and it is not uncommon for large institutions to face numerous legal matters at any one time.

Firms have a duty to preserve electronic information for the purposes of discovery before litigation starts, at the point when litigation can be reasonably anticipated. Prior case laws, such as Zubalake versus UBS Warburg LLC, in combination with amendments to the US Federal Rules of Civil Procedure, are applying more clarification to what is considered reasonable and what constitutes anticipated. Once litigation is reasonably anticipated, firms have a ‘duty to preserve’ electronic records, and current records retention and rotation policies must be halted. State and federal court rulings have become more stringent regarding corporate information spoliation, which can be called if proved that documents were not preserved once litigation was anticipated.

During the discovery phase, the challenge of collecting the data begins to mount as IT must locate and/or restore data in response to discovery. The collection effort can be very complex and time-consuming and must be done in a manner that preserves the chain of custody. The time and cost of retrieving e-mails can mean significant time and cost to the IT department as well as lost productivity. However, just producing the information is only a piece of the cost.

Even more costly than physical retrieval is the legal review and production process; courts have fined companies millions of dollars for not producing information in a timely manner. As a result, electronic discovery continues to be the primary business driver among large enterprises in investing in their next generation of e-mail archiving applications. E-mail archiving solutions aid in indexing content and allowing for search and retrieval of messages and attachments based on user identified keywords and can provide a consistent means to support preservation. In the largest firms, e-mail archiving applications are being evaluated and deployed as part of a much broader information risk management and governance focus.

Supporting compliance with records retention requirements

Longstanding federal, state and local regulations stipulate how companies manage and retain content found in official business records; and regulatory bodies such as the SEC, DHHS, EPA, FDA and FAA are more strictly monitoring and enforcing these regulations. These regulatory mandates subject companies to requirements that specific business content is captured and retained in a way that ensures information integrity, security and accessibility.

Legislation is most commonly concerned with the content and not the format in which it is stored. It is the firm’s responsibility to identify the file formats, applications or medium that contain this regulated content and ensure that the corresponding electronic records, which may include e-mail messages, are managed and retained according to requirements. In regulated industries such as the financial services securities broker/dealer environments, firms are required to preserve client communications, including those conducted over e-mail, for prescribed periods of time. E-mail archiving solutions can support and enforce these record retention policies.

Regulations may stipulate that related content contained in electronic records be retained for specified time periods or outline rules to secure the privacy, security and lack of compromise to sensitive information. Central to satisfying regulatory compliance requirements is the fast provisioning of electronic records in response to a regulatory audit, typically within a 24-48 hour time period. As a result, electronic records subject to regulatory compliance should be indexed for easy search, retrieval and production. Equally important as retention is the proper disposition and destruction of electronic records, absent any legal requirements to preserve, once they no longer have any regulatory or business value. E-mail archiving supports records retention requirements and also enables content indexing for easy search and retrieval of relevant messages and attachments based on audit requests.

User mailbox management via e-mail archiving

Outside of the legal and regulatory pressures on managing valuable information within e-mail messages, firms must balance the growing volume of e-mail against usability, access and performance. Archiving for mailbox management is the use of an e-mail archive solution to reduce the operational expenses associated with operating an e-mail system. Business users keep an increasingly large set of e-mail in their primary mailbox and its associated folders. This causes frequent upgrades to the e-mail system in the form of larger quantities of expensive, high availability storage, additional e-mail servers, and e-mail server licenses.

A very effective way to manage mailbox size is by regularly archiving older or infrequently accessed data. Archiving saves space by moving messages from the e-mail server into an archive repository. Archiving for mailbox management allows for restrictive quotas on e-mail age or mailbox size on the e-mail server. However, rather than deleting these e-mails, the e-mail archiving product simply archives the e-mail but leaves a link or stub in place. This saves the storage but provides the user with exactly the same look and feel within the mail client. Users get the benefits of an unlimited mailbox, while e-mail server operational costs plummet.

E-mail archiving for mailbox management allows for an infinite user mailbox and provides seamless, policy-based migration out of the primary mail server to an archive server. Access to archived e-mail from the user inbox is maintained via stubs that give users transparent access to archived e-mails but also improve Exchange server performance and optimize storage by placing primary Exchange server data on tier one storage, while archive e-mails can be placed on tiered archive storage. Other benefits in using an e-mail archiving solution for mailbox management include mobile user support, which allows users to take their full mailbox with them even when the server based mail store has been archived; and end-user based restore of deleted messages from the archive without administrator intervention.

E-mail archiving helps satisfy today’s business requirements

E-mail is a core component of business communications today, used to conduct transactions, to negotiate terms, to satisfy client or employee requests and the like. With the pervasive nature of e-mail in today’s corporate environment, firms need to ensure that adequate controls are in place to protect, retain and preserve relevant e-mail content in the event of audits, legal discovery or for general business use. Moreover, with the volume of e-mail increasingly annually, firms need to find a way to manage growth with relatively flat budgets and limited IT administration.

Today’s e-mail archiving applications provide an automated and efficient way of storing, indexing and retrieving individual e-mail messages and file attachments by individual users, IT staff and other authorized parties for the purposes of legal, regulatory or business need.

About the author

Laura DuBois serves as Research Director of Storage Software research. The Storage Software program covers storage resource management, replication, and backup & archive software. Ms. DuBois is responsible for market research, analysis and client relationships for IDC's worldwide Storage Software program with a focus on the storage resource management segment.

An expanding market

A $477 million dollar market in 2006, the e-mail archiving applications market realized 45 percent growth year over year, fueled by regulatory requirements around record retention, litigation and electronic discovery overall mailbox management. Recent and future market growth is expected to come from:

1. Serving both business and IT user needs for e-mail archiving, at once satisfying use cases for electronic discovery and record retention as well as mailbox management for storage, performance, and backup optimization
2. Support for archiving of systems in adjacent unstructured content markets such as file system and content management application and end-point file archiving and expansion to other messaging platforms such as IBM Lotus Notes
3. Expansion of archiving products based on a unicode architecture to support localization of e-mail archiving products to local markets, languages and management policies
4. Packaging of e-mail suites and solutions that satisfy needs for e-mail business continuity, security, disaster recovery and archiving
5. Offering integrated, turnkey, and cost-effective archiving solutions, including hosted capabilities for small and midsize firms that have long-term archive and retention requirements.