Some fine points of determining conformity to specification

In the latest Data Points column, Peter Fortini discusses the absolute and rounding methods when comparing a test result to a specification limit.
Peter Fortini

Q. If there is no information on how results should be interpreted when comparing a test result to a specification limit, should we use the absolute or the rounding method?

A. The standard practice for using significant digits in test data to determine conformance with specifications (E29) describes two methods to compare results to specification: the absolute method and the rounding method. This question comes up regularly in queries directed to E29. Unfortunately, a definite answer, absolute or rounding, cannot be given, though some general guidance might apply.

The principal concept contained in E29 is the rounding method of interpreting specifications. This is surely not original to E29 but was made into a standard in recognition of existing practice in industry. With the rounding method, the test result is rounded to the place of decimals given in specification limits. Then the test passes if that rounded value is within the limits. Specification limits are effectively extended by half the rounding interval. So the rounding rule is really just a stratagem to take a test result value that is very close to the limit, and move it into the passing region.

As an example, consider elongation at break in a tensile test having a minimum specification of 2%. A test result of 1.8% does not meet the specification if the absolute method is used. It does meet the specification if the rounding method is used with a rounding interval of 1% or 0.5%. Some tensile test standards specify rounding; others leave it unstated. If the standard specifies the rounding method, then the 1.8% result meets the specification and may be reported either as 1.8% (preferred) or as the rounded value 2% or 2.0%. If the standard specifies the absolute method, then the 1.8% result does not meet the specification. If the standard leaves unstated whether rounding or absolute applies, then the supplier might simply report 2% as the result, attempting to avoid the issue. Suppliers often try to protect themselves by only reporting properties rounded to the exact digits of the specification. Then the rounding or absolute method makes no difference, as only the rounded value is seen.

Historically, the need for formal specifications in industry started in the 19th century with size dimensions so that parts could be made interchangeable. Specifications for material properties came later. Organizations like ASTM are part of their development. Whether a test result meets specification limits should be a completely straightforward comparison. However, the cost of wasted material is considerable at the margins, so a supplier may want material to be accepted while a receiver may want to enforce the rules. Both supplier and receiver also want to avoid supply disruption.

E29 itself states that it only applies when the specification or another applicable standard cites E29, and it states whether the absolute or rounding method applies, and if rounding, the rounding interval. This is usually placed in a footnote to the specification table but may also be written into the test method itself.

When there is no information on whether to use the rounding or absolute method, the next source of guidance is what is customary in the field. Typically, mechanical size dimensions are interpreted absolute, chemical composition and material properties like strength and hardness with rounding. For mechanical dimensions in the U.S., dimensioning and tolerancing (ASME Y14.5) removes ambiguity by specifying the absolute method. Chemical composition and material properties such as strength, modulus, and more have been inclusive of rounding. In the U.S., compendia such as the United States Pharmacopoeia specify the rounding method.

Disagreement between supplier and receiver is possible when the interpretation of limits is not stated and when supplier and receiver differ in how they are to be interpreted.

The statistician’s interest is that the report of test results should preserve the information in the test data itself. Guidance in E29 is that test results should generally be reported to within half the precision standard deviation in order to preserve the information. This is independent of any comparison to specification limits.

How can we remove or reduce ambiguity between the rounding and absolute methods? Ultimately, this must be done in the specification itself. It is removed when that requires rounding or absolute comparison. Another way to reduce ambiguity in a specification is to add another digit to the specification value. It cuts the extra room with the rounding method by a factor of ten.

Uncertainty Adjustment of Test Results

A recent revision (2022) of E29 started when a different, though perhaps related, question was asked: Under what conditions can you use uncertainty (or precision of measurement) to adjust a test result to fall within a specification limit, when the result deviates by less than uncertainty from the limit?

In order to be applied directly to test results for product acceptance, specifications should incorporate an allowance for measurement error over what is functionally required. However, this is not always the case. It may be found, when interlaboratory trials are finally undertaken, that test variability is too high.

On its face, the idea of adjusting a reported value on this basis should be entirely unacceptable. The initial drafts of the revision did in fact state that no such uncertainty adjustments could be made. During the balloting process, it was pointed out that in some fields, such a treatment of uncertainty is actually practiced. Therefore, the final statement is more nuanced, stating that adjustment for uncertainty is only to be allowed if permitted by the specification or agreed to by the supplier and receiver. In addition, the original value, before any adjustment, as well as the method used for adjustment, must be reported.

Ultimately, the goal of any rules around the reporting of data and their comparison to specifications is transparency of communication. ■

Peter Fortini is the chair of the committee on quality and statistics (E11). He is an ASTM International fellow and Award of Merit honoree.

John Carson, Ph.D., is senior statistician for Neptune and Co. and the Data Points column coordinator. Carson is a member of the committees on quality and statistics (D11), petroleum products, liquid fuels, and lubricants (D02), air quality (D22), and environmental assessment, risk management, and corrective action (E50).

Industry Sectors

Issue Month
Issue Year