ASHRAE Standard 140

Validation of software simulation engines with ASHRAE Standard 140. (Source: IBPSA-USA BEM Workshop)

ASHRAE Standard 140 aims to establish a consistent and reliable method for evaluating the capabilities and limitations of building energy modeling software. The standard provides a structured approach to assess the accuracy and reliability of predictions made by these programs through a series of tests, ensuring that they meet industry-accepted performance criteria.

The tests are software neutral - they are applicable to any BEM software program regardless of its underlying algorithms or code.

Audiences for the standard include:

• Building designers and engineers: To make informed decisions about choosing tools to evaluate energy-related aspects of building design.
• Software developers: To improve the accuracy and capabilities of their BPS tools.
• Policymakers: To require reliable energy analysis tools for the use in evaluating different building strategies.

Scope

The standard primarily focuses on evaluating the predictive accuracy of BEM programs by comparing their results with reference data. The scope includes:

1. Testing Procedures: The standard prescribes a set of procedures for conducting simulations and collecting data. It defines the input conditions, simulation steps, and output data requirements for the testing process.
2. Metrics and Criteria: The standard establishes metrics and criteria for assessing the accuracy of BEA program predictions. This includes criteria for comparing simulated results with measured data and determining the acceptability of the software's performance.
3. Reference Buildings and Systems: The standard provides reference building and system configurations to use in the evaluation process. These reference cases represent common building types and HVAC systems to ensure a comprehensive assessment of software capabilities.

Contents

Example of ASHRAE 140 HVAC test case results. (Source: ASHRAE Standard 140-2020)

ASHRAE 140 thermal fabric test case example. (Source: ASHRAE Standard 140-2020)

The standard is structured around several key components, including:

1. Test Cases: The standard includes a series of test cases that cover different building types, climates, and HVAC systems. These test cases serve as a benchmark for evaluating software performance.
2. Metrics and Verification Procedures: The standard defines metrics such as annual energy use, peak demand, and temperature predictions. It also outlines verification procedures to ensure consistency and reliability in the evaluation process.
3. Simulation Protocols: The standard outlines the simulation protocols to be followed during the evaluation. This includes specifying the simulation settings, environmental conditions, and other parameters to ensure a standardized testing approach.

HVAC test cases

HVAC tests look at outputs such as cooling and heating coil loads, cooling and heating energy consumption, and HVAC component energy consumption.

Thermal fabric test cases

There are also comparative tests in Standard 140. These consist of several thermal fabric tests using simple single-zone models.

Example may include starting with a simple model and then adding shading or changing window orientation. There are also tests with high and low mass constructions.

Performing the tests

Software developers model each of the test case buildings in their software. Then they compare results to other software programs, and the standard includes results for 6 programs that can be used for comparison. These are shown in the example chart images, which plot the annual cooling and heating load results.

There’s currently no pass/fail criteria in the Standard (as of the 2020 version), because we can’t say which results are the absolute truth. However, in future versions of ASHRAE 140, pass/fail criteria will be introduced using statistical confidence approaches.

Prior to the pass/fail criteria, the standard can still help to identify software that produces significantly different results, which can help in finding bugs or help us to understand potential uncertainty.

Benefits

Some key benefits of BEM tools performing the ASHRAE 140 tests include:

• Increased confidence in BEM results: Standardized testing reduces uncertainty and leads to more reliable predictions of building performance.
• Improved software development: By highlighting issues, the standard guides developers towards building better BEM tools.
• More effective energy-saving strategies: Accurate simulations inform building design decisions that optimize energy efficiency.
• Harmonized data and methodology: Enables standardized comparisons between different BEM programs and facilitates collaboration.

Limitations

• Not a comprehensive assessment of every software feature or capability.
• Requires careful interpretation of results and consideration of additional factors beyond standardized tests.
• Does not guarantee perfect predictions, as real-world buildings can introduce unpredictable variables.

Additional resources

• ASHRAE Standard 140 resource files
• Department of Energy: ASHRAE Standard 140 Maintenance and Development