A measure of success

Carole Stern Switzer, Executive Vice President and General Counsel of the Open Compliance and Ethics Group, offers her tips for measuring and evaluating your compliance and ethics program.

Measure what is measurable and make measurable what is not so. Galileo’s advice for understanding the world is nearly 500 years old, but it could have been said today by any business leader seeking to evaluate an entity’s successes and failures.

Following a wave of corporate scandals, and passage of laws such as Sarbanes-Oxley that impose greater responsibility for ensuring legal compliance on senior executives, many companies are formalizing and expanding processes to drive compliance and detect incidents of non-compliance with a wide array of regulatory requirements.

Yet, how do we know if the compliance and ethics program is working? How do know if it is using corporate resources wisely? How do we know if it designed well and operating according to the design?

Measurement is the first step to understanding whether the compliance and ethics program is effective, and whether it is enhancing or hindering achievement of the organization’s overall business goals. There are numerous benefits and challenges to measuring the performance of a program. Ideally, measurement will help management demonstrate the results of program activities, show how these results support organizational objectives, identify what works and what doesn't, justify capital allocation, promote accountability, motivate and provide tangible feedback to employees and enhance the ability to communicate with stakeholders.

The Open Compliance & Ethics Group (OCEG) has recently released a new Measurement and Metrics Guide (the OCEG MMG) to help management devise and maintain a high-performing compliance and ethics program – one that meets the challenges of constant change, increasing complexity, rapidly evolving threats and the need to continuously improve the results.

Avoid the obstacles to success

Despite the value of measuring program performance, there are roadblocks. The OCEG MMG presents a number of examples, including the following:

• Ineffective planning can lead to unintended consequences. Just counting something does not provide meaningful data for benchmarking. The metrics collected must be planned with an eye toward their end use. Very often, an individual metric does not provide insight until it is combined with or contrasted with another data point. By planning the evaluations in advance, managers can more effectively determine what metrics to track.
• Data is not readily available or is incompatible. In some companies, individual departments or business units maintain their own isolated databases using information technology systems different from those in other parts of the company. Several functional areas of the company may track the same data but express it differently. To gain a total picture of organizational performance, “organizational silos” must be broken down and data from separate IT systems must be normalized and merged.
• Perception may be reality. Some program metrics are based on surveys of employee perceptions. Sometimes the perception is the ‘fact’ that management is looking to measure, even if the perceptions are false. For example, if employees perceive there is some type of misconduct going on in the organization, (even if that is factually untrue), the perception is there and must be addressed. Management must be careful to separate perception from reality, and to determine if the perception is real. If so, the underlying issue must be resolved before perception will change.
• Proving a negative is difficult. By definition, a key outcome of the compliance and ethics program is the deterrence or prevention of noncompliance. Deterrence measurement requires consideration of what would happen in the absence of the program. It is often difficult to isolate the impact of an individual program element on behavior that may be affected by multiple factors.

Take a principled approach

Given these challenges, a solid measurement system and approach should be implemented that embodies these principles:

• Contribute to business objectives: Program metrics should help management understand how the program contributes to overall enterprise objectives. If the program does not support the business goals, it will not be given adequate resources (see sidebar “Mapping programs to Strategy”).
• Focus on Outcome: While process and activity metrics are important, the outcomes are the ultimate goal – never lose sight of this.
• Use a simple measurement system: The measurement system and approach should be simple, and cost-effective to ensure sustainability. Management should look for opportunities to gather data from existing systems rather than creating whole new systems. If it costs more (both in time and capital) than it is worth, the measurement program will fail.
• Obtain commitment and establish accountability
  Senior management and the board of directors should commit to a measurement approach and ensure that a high-level executive is charged with overall accountability. This should include a commitment to the longevity of the program as it will take a few years to realize the full potential of a measurement program.
• Make it positive not punitive
  The measurement system and approach should be a positive contributor to help improve performance. It should not be used for punitive purposes.
• Establish SMART indicators
  Key metrics and indicators should be Specific (and simple), Measurable, Actionable, Relevant and Timely.
• Balance leading and lagging indicators: Lagging indicators show how the company has already done (revenue growth in the past quarter; number of workplace accidents in the last year). Leading indicators are those that may predict future performance. Examples are on-time delivery rate, which can lead to higher customer satisfaction ratings and, in turn, more sales to existing customers.
• Have a short and long-term view. Indicators should provide visibility into both short-term and long-term objectives. Overemphasis on short-term objectives can stifle a company's long-term growth, by short-changing new product development. Emphasis on short-term financial results, such as quarterly profits, can lead to reduction in spending on research for new product development, or purchasing cheaper components to raise profit margins, leading to lower product quality, more product returns, complaints from customers, and loss of business.
• Focus on internal trends before external benchmarks. Program metrics and measurement should help management understand internal trends. Once internal trends are understood, the use of external benchmarks will be more meaningful.
• Review the system over time: Performance measurement system should be reviewed and improved on an ongoing basis. It is only by gaining experience measuring performance that the organization can really refine and improve the system.

Measure what matters

Simply counting things that are countable usually does not provide useful information. Management must carefully assess the goals of the compliance and ethics program and determine what metrics and measurements will enhance understanding of the program’s operation. What do we need to know to determine if our program is operating according to its design, or even if the design is addressing our objectives? What do we need to assess to know if we have applied limited resources well? What must we evaluate to enable us to improve the program? These are only some of the questions that must be asked.

The OCEG MMG offers tools to assist in answering these and other questions about program measurement and evaluation. The MMG helps an organization understand the issues and processes involved to evaluate and report on the PERFORMANCE of a compliance and ethics program. PERFORMANCE is a carefully selected word that encompasses a number of dimensions. In particular, the MMG focuses on how an organization can go beyond legal "effectiveness" of a program and look at the degree to which a program is helping an organization achieve its enterprise objectives. In this context, PERFORMANCE captures:

• Program outcomes and whether it contributes to business objectives
• Program effectiveness (both design effectiveness and operating effectiveness)
• Program efficiency (both financial and human capital)
• Program responsiveness (speed, cycle time, and flexibility)

Over 300 candidate metrics are provided in a companion spreadsheet that can be adapted for use by any organization.

Involve all key stakeholders

By nature, people support that in which they are involved and have a personal stake. The engaged involvement of key stakeholders is critical to the success of any effort to establish a comprehensive measurement program to evaluate compliance and ethics initiatives.

By working together, compliance and ethics officers, executive management, and the board can help ensure the performance reporting of the compliance and ethics program contributes to the improvement of the organization’s governance practices and the accomplishment of its business objectives.