Public Release of Three Evaluation Reports

Published on April 9 2014

Evaluation of Canadian Blood Services Grant and Contribution Programs

Health Canada supports two programs of the Canadian Blood Services (CBS) through a grant and a contribution: the Organ and Tissue Donation and Transplantation (OTDT) Program and the Blood Research and Development (R&D) Program. The overall goal is to support the development of a national organ and tissue donation and transplantation system that will improve and extend the quality of the lives of Canadians, while respecting the federal role and interest in organ and tissue donation and transplantation.

To evaluate both the OTDT and R&D components, Science-Metrix used multiple methods including a document review, two surveys, and interviews with both internal groups, and external groups. The evaluation concluded that:

  • Both components continue to address a demonstrable need although the nature of Health Canada's role for the OTDT Program was subject to conflicting expectations and different views with respect to the appropriate level of direct involvement and provision of “national leadership”. On the other hand, closer involvement and knowledge sharing of Health Canada with CBS' R&D Program was generally desired.

  • In terms of performance, both components have produced expected outputs and are making progress toward their expected immediate and intermediate outcomes.

  • Both components demonstrated economy and efficiency in their operations and management. Alternative funding models have been identified that may help CBS explore additional funding sources to support its R&D activities and meet growing expectations of its stakeholders.

On the basis of these conclusions, the evaluation made two recommendations to help Health Canada define its future policy direction with respect to these CBS components.

Evaluation of the Forest Ecosystems Science and Application (FESA) Program Sub-Activity

Under the responsibility of the Canadian Forest Service (CFS), FESA was established with the objective to increase scientific knowledge on forest ecosystems and support stakeholders in their sustainable forest management (SFM) policies and practices. FESA conducts research, national assessments and monitoring to develop, synthesize and integrate scientific knowledge. This knowledge is used by the appropriate government jurisdictions, industry, and other stakeholders to develop forest management practices and policies, and by NRCan and other federal government departments, to meet international reporting obligations, from Canada’s negotiating positions on international environmental issues related to forests, and counter misconceptions of Canada’s forest practices. This Program Sub-Activity consisted of the following six project areas:

  1. Forest Biodiversity

  2. Forest Carbon Research, Reporting and Policy Advice

  3. Assessing and Understanding Ecosystem Productivity and Dynamics in Support of Sustainable Forest Management

  4. Global Leadership in Development of the International Model Forest Network (IMFN)

  5. National Forest Information and Assessment (NFIA)

  6. Land Reclamation

Science-Metrix was mandated to conduct this evaluation on behalf of NRCan, in accordance with the Federal Accountability Act, the 2009 Treasury Board Policy on Evaluation and NRCan’s Strategic Evaluation Division’s evaluation policies. The primary objectives of the evaluation were to assess issues relating to the relevance and performance of the FESA Program Sub-Activity and provide recommendations as necessary.

The evaluation drew on various sources of data to ensure that the combined lines of evidence resulted in an in-depth and comprehensive analysis, including 1) a document file and data review; 2) a review of CFS ProMIS planning database; 3) stakeholder interviews and focus groups; 4) an online survey of project area component leads; and 5) bibliometrics and webmetrics.

The evaluation found that FESA remains highly relevant, addressing several ongoing needs related to each project area. FESA provides the unique and long-standing expertise and specialization in forest ecosystem science that are particularly relevant to address environmental issues and to support forestry sector competitiveness. Furthermore, the global objectives of FESA were found to be consistent with federal government roles and responsibilities and priorities, as well as NRCan strategic outcomes throughout the evaluation period. However, efforts should be made by CFS to more clearly define its niche with respect to the Land Reclamation project area better align the IMFN and Biodiversity project areas to FESA.

The FESA project areas have been generally successful in producing high quality outputs, which have been accessed and used by a wide range of target audiences. The work of CFS is regarded as credible and science-based such that provinces/territories often use FESA information or expertise to fill their own capacity gaps. FESA has also realized immediate outcomes and contributed to some intermediate level outcomes. Importantly, during the evaluation period, CFS has increasingly oriented its science towards policy priorities within available resources.

The evaluation made six recommendations. The first two recommendations pertain to the need to further examine the relevance of FESA’s role in the International Model Forest Network and Land Reclamation, and the risks associated with the reallocation and/or reduction of biodiversity expertise for CFS and the federal government. Other recommendations revolve around the need to better interact, communicate and disseminate the science produced with stakeholders and improve the tracking and reporting of performance and financial data to ensure that reliable information is used to manage FESA activities.

Evaluation of the Pulp and Paper Green Transformation Program

The Pulp and Paper Green Transformation Program (PPGTP) was implemented by the Canadian Forest Service (CFS) within Natural Resources Canada (NRCan). Its objective was to improve the environmental performance of Canada’s pulp and paper industry, contributing to environmental and hence commercial sustainability. The Program assisted Canadian pulp and paper companies in making capital investments that would improve facility environmental performance, by providing a $0.16 per litre credit for black liquor produced at mills between January 1, 2009, and December 31, 2009, or to a cap of $1.0 billion in total credits issued, whichever came first.  

Four methods were used to collect and analyze evidence for this evaluation: 1) document file and data review (approximately 500 documents); 2) stakeholder interviews (36); 3) an online survey of funded project participants (N = 115); and 4) three case studies on Program planning, management and delivery.

The evaluation concluded that the PPGTP was relevant as it addressed a demonstrable need for the Canadian public and the pulp and paper industry. Additionally, the PPGTP objectives were aligned with government priorities as well as with the departmental strategic objective of Environmental Responsibility. There was a clear role for the federal government in this type of programming. The evaluation also concluded that the intended outcomes of the PPGTP were achieved, with good progress towards the longer-term environmental sustainability of the pulp and paper industry. Positive commercial impacts were also achieved. The PPGTP also clearly resulted in enhanced capacity-building and relationship-building for both internal and external stakeholders.

Based on these conclusions, five recommendations were made to help NRCan build on the linkages established through the program, and leverage the resulting best practices and lessons learned.

Read the reports: