Multi-Year Evaluation of the Electricity and Natural Gas Energy Efficiency Program Portfolio

Efficiency Manitoba was created as an energy efficiency agency in 2020 to administer and manage electric and natural gas energy efficiency programs in the province. Such programs used to be administered by Manitoba Hydro, the province’s electricity and natural gas provider. 

Efficiency Manitoba hired Econoler to evaluate its entire portfolio of electricity and natural gas energy efficiency programs for the 2020-2025 period. The portfolio comprises 40 programs and enabling strategies aimed at providing residential, commercial, industrial, agricultural, low-income and Indigenous customers with financial incentives, technical support, tools, and knowledge to implement energy efficiency projects to reduce their electricity and natural gas consumption. 

Before conducting the 2020-2025 evaluations, Econoler developed an Evaluation, Measurement and Verification (EM&V) Framework and Plan to provide a common understanding of EM&V best practices and lay the groundwork for a transparent and rigorous evaluation process. Considering the size of the Efficiency Manitoba program portfolio, this Framework and Plan also included guidelines to determine the types of evaluations that should be conducted for each program, as well as when and why. These guidelines were based on considerations such as program lifecycles, the amount of expected savings, the level of uncertainty around savings assumptions, and the presence of any known issues.

In addition, different savings evaluation approaches were developed, one of which was a savings verification that involved ensuring savings had been correctly and consistently compiled and calculated by Efficiency Manitoba by applying all the necessary parameters. This approach ensured some level of savings validation without conducting a full-scale impact evaluation that would involve revising and updating savings parameters. Econoler also conducted process and market evaluations. The process, market, and impact evaluations involved data-collection activities such as surveys and in-depth interviews with key stakeholders such as program staff, participants, contractors, and distributors to assess program experiences, effectiveness, market conditions as well as site visits and project reviews to verify project-specific conditions and savings assumptions. Each evaluation year included cost-effectiveness assessments to determine whether each program and the program portfolio as a whole were cost-effective. In addition, Econoler conducted assessments of the impacts of changes to codes and standards on savings in the province. 

In addition to these evaluation services, Econoler also supported Efficiency Manitoba in the design and implementation of programs by informing program staff about industry best practices and conducting targeted research on programs in other North American jurisdictions.

Through these evaluations and support services, Econoler provided Efficiency Manitoba with accountability and greater understanding of program impacts on the various sectors they serve as well as assistance in the implementation of their program portfolio comprised of many new programs.