Research evaluation requires rigor and experience
PA's experienced evaluation research team provides:
- design and implementation of research program evaluation
- market assessment and tracking
- policy and planning support
- benchmarking
- design and implementation of measurement and verification systems
- support for global climate change mitigation activities.
We are particularly experienced in the energy industry, where we have conducted hundreds of evaluations of energy efficiency programs, demand management programs and load-shifting programs over the last 15 years.
Program design and implementation
PA's energy-efficiency experts work closely with the Market Analytics team to conduct research to characterize the supply chain and the market for energy efficiency products and services. With a firm understanding of the market dynamics and key influencers, the program design team uses a systematic approach to develop the program theory that includes program logic models to establish key activities by stakeholder and expected outcomes. Based on the program logic models, the program administration and implementation process is clearly defined and documented through program descriptions, procedures manuals, and strategic marketing plans. Working with our evaluation experts, the program procedures and tracking mechanisms are established to meet specific program objectives while facilitating ongoing program monitoring and impact measurement.
Process evaluation
We provide the full complement of process evaluation services to program sponsors and implementers. We collect data from all stakeholders involved in a program to assess whether program operations are congruent with plans, whether goals and objectives are being met cost-effectively and whether lost opportunities are being minimized. We consolidate the data we gather to make recommendations for program changes that will improve operations and increase program success.
Impact evaluation
Our evaluation team has extensive experience in a range of impact evaluation methods. The key goal of an impact evaluation is to calculate the ultimate effects of a market intervention, for example, an energy efficiency program. For energy programs, we have expertise in reviewing engineering calculations of energy savings, validating program-tracking databases to support savings calculations, and implementing econometric analyses of energy and other data to estimate program impacts. We also have expertise in calculating the environmental, economic and reliability benefits of programs.
Market assessment
Our data collection and analysis capabilities help PA's clients study the operation and dynamics of markets. We identify and quantify barriers to markets, characterize existing and potential market actors and estimate the potential for specific products and services. Our market assessment information can be used to support program and evaluation design recommendations and establish baselines against which change can be measured.
Market tracking
Our staff is experienced in monitoring markets to look for changes in indicators of successful programs. We design and implement data collection and analysis methods to track market changes over time, look back in time to estimate changes when no baseline is available and forecast future market conditions. We have particular expertise in quantifying changes in complex market indicators that require measurement from a variety of perspectives.
Benchmarking
For program designers and individual companies, our evaluation staff offers experience in designing and implementing benchmarking services, which provide feedback on performance relative to the market. We design benchmarking methodologies; specify and create databases to contain benchmarking data; design and implement the data collection methods; and conduct the analyses for benchmarking reports.
Measurement and verification systems (M&V)
Our evaluation staff has extensive experience in designing, reviewing and implementing M&V applications, particularly for energy projects. We design M&V parameters, specify and create databases to contain and analyze M&V data, design and implement field and survey data collection methods, and design and implement data analysis and reporting symptoms using the M&V data.
Support for global climate change mitigation activities
Our evaluation support for global climate change mitigation activities has included estimation of grid-specific emissions factors for quantifying the avoided power plant emissions attributable to energy efficiency program impacts. We also apply methodologies of the International Performance Measurement and Verification Protocols (IPMVP) to provide third-party independent documentation of energy impacts as a basis for tradable emission reduction credits. We are currently assisting a US state's public benefits program in its entry into emissions trading markets.
