Calibrated models

Calibrated models may be developed post-occupancy to align the theoretical (design phase) model with the actual building performance to:

• More reliably test ECMs
• Determine if the building is performing as expected
• Identify additional opportunities for operational improvements

This table outlines the difference between an early design-phase model, final design, and a calibrated model (completed post-occupancy)

MODEL INPUTS	DESIGN MODEL	AS BUILT MODEL (Calibration Baseline)	CALIBRATED MODEL (Actual)
Weather File	TMY	Actual Meteorological Year (AMY)	Actual Meteorological Year (AMY)
Occupancy	ASHRAE or typical occupancy	Actual occupancy	Actual occupancy
Schedules	ASHRAE or typical schedules	Actual schedules	Actual schedules
Envelope	Design assemblies and thermal properties	Design assemblies and thermal properties	Constructed assemblies and thermal properties
Lighting	Design wattage by lighting zone	Design wattage by lighting zone	Installed wattage by lighting zone
Mechanical Equipment	Design systems and control settings	Design systems and control settings	Installed systems and control settings

Example: The simulation may predict lower lighting energy consumption than the actual performance of the building because perhaps the lighting controls are not operating appropriately

Performance can vary between theoretical design models and actual building performance.

For example:

• Incorrect lighting power density (W/ft2) may cause a consistent annual consumption increase or decrease and would impact HVAC performance
• Faulty enthalpy-based economizer would result in an increase in HVAC energy consumption and ventilation air volume
• Reduced U-factor in overall envelope leads to reduced HVAC loads (once they’ve hit a point of diminishing return in a moderate climate)
• Inappropriate lighting controls can cause lower lighting energy consumption than observed in operation

Calibration guidelines

There are several calibration guidelines that are widely used.

Each of these guidelines provides frameworks and standards for ensuring reliable, verifiable energy savings, with IPMVP and FEMP focused more on performance contracts and ASHRAE Guideline 14 offering detailed statistical calibration methods.

This table summarizes a few of the key differences between each guideline.

	IPMVP	FEMP M&V Guidelines	ASHRAE Guideline 14
Primary Focus	Framework for defining, calculating, and reporting energy savings with multiple options (A, B, C, D).	Guidance for performance contracts and federal projects, emphasizing cost-effectiveness.	Detailed technical calculations and validation of energy savings.
Audience	Broad	Federal agencies, ESCOs, and contractors.	Engineers and professionals performing rigorous energy analysis
Level of Detail	General framework with flexibility	Moderate	High
Statistical Metrics	Uses general methods for error analysis	Simple methods to ensure compliance with federal requirements	Requires use of CVRMSE, NMBE, and other metrics for validating results
Energy Model Use	Allows the use of energy models for M&V under Option D (calibrated simulation)	Encourages energy modeling when appropriate for federal projects but less detailed	Focuses extensively on calibrated simulation techniques and provides validation criteria

International Performance Measurement & Verification Protocol (IPMVP)

Purpose: IPMVP provides a framework for verifying energy savings and quantifying the impact of energy efficiency measures (EEMs).

Approach: IPMVP outlines four M&V options (A, B, C, and D), each suited to different types of energy-saving projects:

• Option A: Retrofit Isolation (Key Parameter Measurement) – measures key parameters, estimates others.
• Option B: Retrofit Isolation (All Parameter Measurement) – measures all parameters affected by the project.
• Option C: Whole Facility – uses utility data to measure savings across the entire facility.
• Option D: Calibrated Simulation – applies calibrated energy modeling to estimate savings.

Application: Used in projects where it’s important to document and verify energy savings for performance contracts, retrofits, and energy conservation measures.

FEMP M&V Guidelines

Purpose: Developed by the U.S. Department of Energy’s Federal Energy Management Program (FEMP), these guidelines are designed specifically for federal projects to verify energy savings and support energy performance contracting.

Approach: FEMP M&V Guidelines include options that align closely with IPMVP options but provide more specific guidance for federal facilities and performance contracting:

• Emphasizes a standardized approach to data collection, baseline adjustment, and savings verification.
• Provides detailed instructions on M&V planning, baseline creation, and post-installation verification.

Application: Primarily used for federal energy projects, with emphasis on energy performance contracts (EPCs) and ensuring compliance with government regulations.

ASHRAE Guideline 14

Purpose: ASHRAE Guideline 14 provides comprehensive methods and standards for measuring energy savings and calibrating energy models, ensuring accuracy and consistency in M&V.

Approach: It defines statistical criteria for calibration accuracy, including:

• Normalized Mean Bias Error (NMBE) and Coefficient of Variation of the Root Mean Square Error (CVRMSE) as measures of model accuracy against measured data.
• Detailed procedures for establishing baseline conditions, adjusting for weather and occupancy, and assessing model reliability.

Application: Widely used in building energy modeling and M&V, particularly when high-accuracy model calibration is required for retrofit projects, savings verification, or energy audits.

Uncertainty criteria

All models are wrong to some extent.

ASHRAE Guideline 14 calibration guidance targets are shown in the table.

The CV(RMSE) is a commonly used statistic for model calibration. It compares the deviation between the measured and modeled data, normalized by the mean of the measured data. It helps indicate how well the model fits the actual measured data. The time-step refers to the interval at which data (such as energy use) is collected and compared (e.g., hourly, daily, monthly).

Mean bias error (MBE) is another statistical metric used in energy modeling to quantify the average difference between simulated or predicted values and actual measured data. It indicates whether, on average, a model tends to overpredict or underpredict compared to the real data, helping assess the model's accuracy and calibration.

Note, it is more difficult to calibrate to smaller timesteps (hourly) than larger ones (monthly or annual) due to unpredictable anomalies in occupant behavior or weather variations. Therefore, the tolerances for deviation are greater for the smaller timesteps.

Additional methods and techniques

METHOD	DESCRIPTION	PRIMARY USES
Bayesian – statistical approach	Uses prior data and probability distributions to estimate likely ranges for model parameters	Model calibration Uncertainty quantification
Pattern-based approach	Identifies patterns in historical or measured data to forecast future conditions	Predictive modeling Fault detection
Multi-objective optimization	Considers multiple objectives (like cost, energy savings, etc.) to find the optimal solutions	System optimization Design decision-making
Data disaggregation	Breaks down total consumption into end-uses to separate data, not predict parameter ranges	End-use analysis Detailed model calibration

Bayesian calibration is a statistical approach that uses prior information (from previous model iterations) and updates this information based on new data (measured data). It employs likelihood functions to assess how well the model fits the measured data and to predict the most likely range of values for model parameters. Bayesian methods are iterative, meaning they continuously refine their predictions based on prior runs and the likelihood of various outcomes.

A pattern-based approach involves identifying patterns in the data or system behavior and using those patterns to inform the model calibration. While this can be a useful method, it is typically not focused on prior iterations or likelihood functions. Instead, it relies on recognizing recurring trends or patterns in the data.

Multi-objective optimization involves optimizing several different objectives simultaneously, such as minimizing energy use while maximizing occupant comfort. While this approach is commonly used in energy modeling, it does not necessarily rely on prior iterations or likelihood functions derived from measured data. It focuses on finding the best compromise among multiple objectives.

Data disaggregation is the process of breaking down aggregate data into more detailed components. For example, energy consumption could be disaggregated into individual end uses (lighting, HVAC, etc.). While useful for understanding detailed energy use, this approach is not based on prior iterations or likelihood functions. It's more about separating data than predicting parameter ranges.