Blog

Verification and Validation in Computer-Aided Engineering — Explained

Validation & Verification

Computer-aided engineering (CAE) is one of the most powerful tools of the recent era, being used in almost every field to provide efficient, cost-effective results. Conducting physical experiments for each design version is time and cost intensive. Instead, design engineers perform numerical studies to reduce the number of prototypes, saving a significant amount of effort, money, and time. Still, no matter how well the simulation software they use performs, it is unlikely that the numerical analysis will provide 100% accuracy.

For a reliable numerical analysis, verification and validation assessments — which are abbreviated as V&V — are the proposed models that have been implemented in the various fields such as engineering, medicine, informatics, and economics. Even though verification and validation are generally specified with similar expressions, they are distinct, investigating the accuracy and credibility of a numerical study from different perspectives.

Verification

  • Investigates if the computational model accurately represent the underlying mathematical model and if the resulting code can be properly used for an analysis [1]
  • Asks the question: “Are we building the system right?” [2]
  • Focuses on solving the equations right [3]
  • Examines numerical uncertainty

Validation

  • Determines if the computational simulation agrees with physical reality [1]
  • Asks the question: “Are we building the right system?” [2]
  • Focuses on solving the right equations (ie. getting the physics right) [3]
  • Examines the difference between numerical simulation and experimental results

Standards in Verification and Validation

With the increasing use of computer-aided engineering software during the past 50 years, a standard regarding Verification & Validation became a necessity. The first V&V guideline was issued by the American Nuclear Society in 1987. Further development of V&V guidelines were conducted by other professional organizations to provide industry-specific standards. In 1998, the American Institute of Aeronautics and Astronautics (AIAA) released its first modern standards document. Afterward, the American Society of Mechanics Engineers (ASME) V&V Standards Committee issued documents related to fluid dynamics and solid mechanics, including:

  • Standard for Verification and Validation in Computational Fluid Dynamics and Heat Transfer, ASME V&V-20 (2009).
  • An Illustration of the Concepts of Verification and Validation in Computational Solid Mechanics, ASME V&V-10.1 (2012).

Let’s use mathematical commands for an easily-expressed example of Verification & Validation. Imagine a study of the product of 3 and 1. To obtaining the result of 3, either multiplication or division can be applied. When division is applied to solve the problem, it gives an accurate result for the product of 3 and 1, but when the same logic is applied to the product of 4 and 2 to check for consistency, the division command severely fails (Figure 1).

Fig1

Figure 1: Examination of the verification and validation process according to the example of fundamental math commands.

Aside from basic mathematical commands, both CFD and FEA encompass non-linear equations and complicated simulation steps to describe physical systems. Hence, V&V assessments should be carried out separately and incrementally to ensure reliable processes. In order to avoid a false conclusion about the validity of the model, it is plausible that a verification assessment needs to be performed before a validation assessment. If the verification assessment has ambiguities which result in errors, the validity of the model fails simultaneously [5]. If the verification process succeeds, the validation process must be confirmed with experimental studies since it is required that the comparison studies should rely upon real-time data.

Verification is the process that investigates the solution of the equations with regards to the verification of code (programming) and the verification of the calculation (mathematical models and numerical methods). The fundamental duties of the process consist of finding programming errors and examining mathematical calculations in order to solve the equations right. The steps of verification assessment [4] include:

  • Error detection in the underlying computer programming code
  • Iterative convergence
  • Grid convergence
  • Temporal convergence analysis for transient cases
  • Comparison of results with a reliable solution

Validation assessment examines the relationship between the equations and physical experiments in order to solve the right equations. An experimental setup such as a benchmark is used to conduct studies to investigate the physics. During experimental studies, some level of error occurs and thus it is essential to ensure that experimental results are reproducible. The main steps of validation process [4] can be concluded as follows:

  • Comparison between the CFD or FEA results and the experimental data
  • Investigation of numerical model uncertainties (for example, the turbulence model)

The complete process of analysis based on V&V assessments is shown in Figure 2:

Figure 2

Figure 2: V&V assessment through whole analysis according to ASME V&V-20 [4]

Up until now, it is obvious that V&V assessments should be implemented in all studies encompassing numerical analysis. Due to varying error rates throughout both the Verification & Validation processes, 100% of accuracy is almost impossible. Hence, for V&V assessments, it is essential to specify the level of acceptable error regarding the model and solution.

If you are interested in learning more about Verification and Validation, check out this list of public validations performed with SimScale.

Figure 3

Figure 3: An example of comparison between experimental data and SimScale simulation.


References:

[1] Overview of CFD Verification and Validation, NASA, 2008.

[2] Barry Boehm, Software Engineering Economics, 1981.

[3] Patrick J. Roache, Verification and Validation in Computational Science and Engineering, 1998.

[4] Standard for Verification and Validation in Computational Fluid Dynamics and Heat Transfer, ASME, V&V-20, 2009.

[5] Guide for Verification & Validation in Computational Solid Mechanics, ASME, V&V10, 2006.

 

SimScale is the world's first cloud-based simulation platform, enabling you to perform CFD, FEA, or thermal analyses. Sign up for the 14-day free trial and join the community of 70 000 engineers and designers. No payment data required.