Program Evaluation: Large Commercial, Industrial, and Agriculture Sector

Challenge: The California Public Utilities Commission (Commission) is committed to ensuring the long-term success of California’s dynamic energy efficiency portfolios, consisting of integrated programs with clearly defined ex ante energy savings measurements, accompanied by ex post evaluated measure energy savings resulting from the various program initiatives undertaken by Energy Efficiency Program Administrators (PAs) in California, including the Investor-Owned Utilities (IOUs). The main drivers of the scope and timing of work required for energy savings Estimation, Measurement and Verification (EM&V) activities include state energy efficiency strategic plans, legislative mandates and Commission decisions and legislative mandates. Moreover, the Commission continues to strive to improve the accuracy, speed and cost-effectiveness of the ratepayer funded program evaluation process. For this contract, Tierra was a core member of the broader / cohesive SBW Consulting team selected by the Commission to provide savings EM&V, Program Evaluation, Market Assessment, Policy Planning Support, and Financial and Management Audit Support for the Group D Sectors: Large Commercial, Industrial, and Agriculture Sector Programs, and Customized Project Reviews (CPR) and Strategic Energy Management (SEM).

Approach: Tierra’s role across the breadth of Group D evaluation responsibilities focused on the application of gross-, best practice EM&V data collection and analysis methodologies to facilitate the calculation / reporting of statistically significant Gross Realization Rates (GRRs) across measure categories, programs, participants, etc. Tierra’s data collection efforts were adapted to the unique considerations of the COVID-19 pandemic which restricted on-site EM&V prospects, available data sources, and recruitment potential. Similarly, Tierra calibrated the gross-EM&V analysis methods to reflect the scope of supporting project / measure performance data, along with the regulatory reporting requirements. Examples of Tierra’s gross-EM&V responsibilities, included:

• Integrating the unique combination of available data sources to comply with industry best practices in evaluating sampled custom projects / custom measures, including AMI data, Energy Management System (EMS) trend data, participant interviews, on-site inspection findings and building attributes, secondary data sources (e.g., Dodge New Construction, etc.), etc.
• Reviewing, verifying, and updating custom Savings By Design (SBD) EnergyPro models to accurately reflect current, ex post operating conditions consistent with International Performance Measurement and Verification Protocol (IPMVP) Option D.
• Developing data collection protocols and priorities for virtual site audits, an adaptive framework for procuring evaluation-supporting data while simultaneously complying with regional health and safety guidelines in light of the COVID-19 pandemic.