Friday, February 15, 2008

OAG Audit Criteria for Performance Information

The Office of the Auditor General (OAG) assesses a number of annual reports every year.

Here are the criteria it uses to do so:


Fairness Criterion

Relevant

The performance information reports in context, tangible, and important accomplishments against objectives and costs.
  • Program context includes

    • the mandate, strategic outcomes and objectives, which are linked to government priorities

    • program structures

    • key horizontal initiatives and partners used to deliver the results, and

    • a discussion of the external environment and the key risks faced

  • A description of intended users

  • Reported results are focused on outcomes with related program activity types and outputs identified. (Results Chain)

  • New initatives for improving outcomes are described

  • The measures used to support the results chain are valid and complete

  • The public performance report should link financial and non-financial information to show how resources and strategies influence results, including relating costs to outputs and outcomes. Where possible, cost information includes both direct costs and allocated indirect costs of programs and services to present 'full cost' data.

Meaningful

The performance information describes expectations and provides benchmarks against which performance is compared.
  • Expectations are set out which are

    • clear, concrete, and measurable, identifying the amount and direction of the change, the target groups, and timeframes,

    • focused on outcomes with relevant activities and outputs identified, and

    • consistent with the strategic outcomes.

  • Comparisons are provided between reported accomplishments (actuals) and the expected performance with a realistic interpretation of the gap between the two

  • Comparisons are provided with relevant benchmarks, such as similar activities, programs or organizations, or trends over time, and their significant explained.

  • Key lessons learned about past performance and any resulting actions are discussed.

Attributable

The performance information demonstrates why the program made a difference.
  • The contribution that has been made by the program to the reported results is demonstrated, including evidence regarding attribution, where available (such as evaluations).

  • The contribution of key partners and other external factors is discussed.

Balanced

A representative and clear picture of performance is presented, which does not mislead the reader.
  • The explanation of variances would include both positive and negative aspects of performance, as well as major challenges identified for programs and services, in order to provide a complete picture of performance

  • Significant unintended impacts are reported

  • There is coverage of all key objectives and only performance information that is significant to the organization is reported.

  • There are no distortions of information through presentation, or through omission of information or context.

  • The emphasis on information presented is proportional to its importance and materiality.

Reliability Criterion

Reliable

The performance information adequately reflects the facts
  • Valid and consistent/comparable measures of performance are used.

  • Performance information is based on data that can be readily replicated to produce similar results.

  • The data that supports the performance measures are complete

  • The basis for confidence in the reliability of the information (including third party data) being reported is disclosed, including
    • methods of data collection

    • explanations for any limitations

    • sources of information are reported

source: http://www.oag-bvg.gc.ca/internet/English/oag-bvg_e_10217.html