Interested in learning more about rounding fractions? In the September/October 2024 installment of Data Points, statistics expert Katie Daisey, Ph.D., explains some of the most common methodologies and how ASTM International approaches this topic.

Q. When converting a fraction to a decimal, is there a standard practice for rounding?

A. There are two topics that need to be addressed when answering this question: rounding and significant digits. Let’s first consider the topic of rounding. In general, the standard practice for using significant digits in test data to determine conformance with specifications (ASTM E29) recommends the use of “bankers rounding,” or the rounding of a number with the last digit followed by 0 to the nearest even number. For example, 5/16 or 0.31250 would round to 0.0312.

There are alternative rounding rules that exist but are not included in E29. One example is rounding upwards (i.e., 0.313). Another example is notating the final digit in subscript (i.e., 0.3125). Note that these rounding examples retain three significant digits. This may not be the case for all examples, as the number of significant digits to retain may be dependent upon the specifics of the particular use case.

FOR YOU: Inside ASTM's Laboratory Services and Certification Programs

When it comes to significant digits, the standard practice for conversion from fractional to decimal notation depends on the context in which the resulting decimal form is being used. Three common use cases are covered in E29: use as a specification limit, use as a test result, and use as an observed value. Each case addresses the significant digits to retain and the rounding method to be used.

For use as a specification limit, E29 4.3 advises to notate absolute or rounded method for specification limits. For the absolute method, 5/16 would be an exact limit and no rounding would take place. So, the reported value would be 0.3125. Under the rounding method, the number of significant digits and rounding method would be agreed upon by the committee (in a standard) or by parties (in a private agreement).

For use as a test result, the number of significant figures should be informed by the precision of the method expressed as the repeatability standard deviation. According to E29 7.4, the “rounding interval” for a test result should be between 0.05 and 0.5 times the repeatability standard deviation. For example, if 5/16 is the test result, and the repeatability standard deviation is 0.03, then 5/16 should be rounded to 0.31 (since 0.01 is between 0.015 and 0.0015) and reported as 0.31 ± 0.03.

READ MORE: Global Supply Chain Standards, from Ownership to Container Availability

For use as an observed value, the number of significant figures used in 5/16 depends directly on the markings on the measuring instrument. Per E29 8.2, “When recording direct measurements, as in reading marks on a buret, ruler, or dial, all digits known exactly, plus one digit which may be uncertain due to estimation, should be recorded.” For example, say a ruler was used and the ruler is marked down to thirty seconds of an inch (i.e., 1/32”).  Suppose the observed reading is 10/32”, which reduces to 5/16”. Further suppose that the uncertainty due to estimation is 1/128”, which is 1/4 of the 1/32” marking. This is less than the additional estimation digit allowed by E29 but is reasonable. In fractional notation, this would be noted as 5/16” ± 1/128” and converted to decimal as 0.3125” ± 0.0078125”. The estimation uncertainty is then rounded to 0.008”, indicating that three digits should be retained for a final result of 0.312” ± 0.008”.

Table 1 provides an overall summary of the guidelines in E29.

It is important to note that E29 guides users to retain all digits during conversions and address significant digits and rounding at the end. For example, if 5/16 is represented inches and the need is to convert inches to meters, 5/16 should be kept as 0.3125 with the conversion resulting being 0.0079375 m. After the conversion is completed, the number of significant digits and rounding method can be determined based on use case. If rounding is performed prior to the completion of a conversion process, the rounding error is propagated through the conversion as a percent error.

NIST Special Publications 811 and 1038 give additional information in sections B7 and 4.4.1, respectively.¹

References

¹      National Institute of Standards and Technology. “Guide for the Use of the International System of Units (SI).” NIST Special Publication 811. U.S. Department of Commerce, 2008; National Institute of Standards and Technology. “The International System of Units (SI) – Conversion Factors for General Use.” NIST Special Publication 1038. U.S. Department of Commerce, 2006.

Katie Daisey, Ph.D., is a scientist at Arkema Inc., supporting R&D and manufacturing in the areas of statistics, chemometrics, and digital transformation. Daisey currently serves as vice chair of the committee on quality and statistics (E11).

John Carson, Ph.D., is senior statistician for Neptune and Co. and coordinator of Data Points. He is a member of the committees on quality and statistics (E11), petroleum products, liquid fuels, and lubricants (D02), air quality (D22), and more.