Accuracy and Uncertainty

Building energy simulation programs are quite accurate when we provide accurate inputs. That can be a challenge for many cases, especially for early design models when many assumptions are necessary.

Often, during the design process, the focus of BEM analysis is comparing alternative design options. The relative performance of different designs is less sensitive to those assumptions, assuming that assumptions are consistent between the alternative models.

However, if the goal is to calculate an estimate of absolute energy consumption, then precise inputs are very important, and if the inputs align with real-world design and operation, then models can produce very accurate results.

Sources of uncertainty

Potential sources of uncertainty in BEM models. (Source: IBPSA-USA BEM Workshop)

There are several potential sources of uncertainty in energy simulation results.

One source is uncertainty of the model inputs, and a second is the simulation engine itself and the calculation methods it uses to estimate thermal loads and equipment performance.

In practice, model input uncertainty dominates when it comes to the impact on accuracy.

Model input uncertainty

Input uncertainty can be due to inputs that cannot be known with certainty in advance, such as actual weather and occupant behavior.

Another type of uncertainty is for inputs that could theoretically be known, but for which exact information is not available in normal practice. This type of uncertainty is common especially when modeling existing buildings and examples of these inputs include the building geometry, envelope material thermal properties, and cooling equipment efficiencies.

Strategies to address input uncertainty

One of our challenges as energy modelers is to take reasonable measures to reduce uncertainty in our inputs and then try to understand the level of uncertainty that remains.

Strategies include:

• Information gathering
  • Prioritize important inputs
  • Review inputs with designers
  • Identify likely range of input parameters
  • Characterize input uncertainties
• Analysis approaches
  • Sensitivity analysis—vary uncertain inputs and see how much impact they have on results.
  • Monte Carlo analysis—While it’s not always practical, you might be able to perform an analysis such as a Monte Carlo analysis where you identify probability distributions for inputs and run lots of simulations to develop a probability distribution for the results.
• Presentation of results - be sure to address uncertainty when presenting results
  • Show ranges
  • Identify uncertain inputs

Software uncertainty

Validation approaches for software simulation engines. (Source: IBPSA-USA BEM Workshop)

There are three different types of tests performed on building energy simulation software.

Empirical validation compares real data from a building or test cell to simulation results. This is ideally what we want to know – how closely our model results match the real world. Unfortunately, these are very difficult studies, for a number of reasons:

• Need very accurate measurements of actual conditions and actual loads.
• Need very accurate specifications for the structure and systems to be modeled.

Due to the difficulty, they are usually quite limited, looking at simple structures and systems and for a limited range of conditions. However, there have been a number of empirical validation studies that cover a number of different simulation programs. These studies show that the accuracy load calculations in simulation software with very accurate inputs is within the range of +/-5%. And there are continuing studies that will improve our understanding.

The analytical verification and comparative testing approaches are more practical, and these are the approaches currently used in ASHRAE Standard 140. These testing approaches are discussed in more detail on the page ASHRAE Standard 140.

Model calibration

Improving the accuracy of existing building models with model calibration. (Source: IBPSA-USA BEM Workshop)

In practice, models of existing buildings can be calibrated to provide within +/- 10% of actual energy consumption, with modest effort.

Read more about model calibration.