Return to Statistics Topics

 

Accuracy and Scientific Notation

The decimal place accuracy of a number is the number of digits to the right of the decimal point. The decimal point is a period written between the digits of a number. If there is no decimal point, it is understood to be after the last digit on the right and there is no place (zero place) accuracy.

The significant digits of a number are those digits that are most accurate. If a number has no place accuracy and there is no string of zeroes ending the number on the right, all the digits are significant. If a number has no place accuracy and there is a string of zeroes ending the number on the right, the significant digits are those digits to the left of the string of zeroes. If a number has a decimal point, the significant digits are the digits starting from the first non-zero number on the left to the last digit written at the right end. In either case the number of significant digits is just the count of these digits.

Decimal notation is the regular written format for a number. Scientific notation of a number just writes the significant digits followed by an appropriate power of ten.

The most common form of scientific notation inserts a decimal point after the first significant digit, follows the significant digits with times, "x", and then 10 to a power. If the original number is at least one, the power is the number of digits between the decimal point and the first number on the left. If the number is less than one, the power is the negative of the number of digits to the right of the decimal point up to and including the first non-zero number.

Calculators and computer software sometimes write scientific notation with the significant digits followed by the letter "E" and then the power of 10, without writing the base. A decimal point is usually inserted after the first significant digit.

Examples

The number 403,000 has no decimal places. The three significant digits are 4, 0, 3. In scientific notation, it would be written 4.03 x 105. A calculator might write this as 4.03E5.

The number 0.01390 has five decimal place accuracy. The four significant digits are 1, 3, 9, 0. In scientific notation, it would be written as 1.3890 x 10-2. A software program might write this as 1.390E-2.

The number 10.42 has two decimal place accuracy. The four significant digits are 1, 0, 4, 2. In scientific notation, it would be written 1.042 x 101. A calculator might write this as 1.042E1.

Exercises: Answers can be found by clicking on "Answers" below.

In problems (1) through (8), state the number of decimal places, state the number of significant digits, and write the number in scientific notation.

1) 54,200 2) 178,460,000 3) 0.0002314 4) 0.00980
5) 132.502 6) 37.41 7) 0.3473 8) 1445.3

In problems (9) through (11), round the number down to the stated number of decimal places.

9) 4.89723 three places 10) 245.1281 two places 11) 83.123 one place

In problems (12) through (14), round the number down to the stated number of significant digits.

12) 0.43247 three digits 13) 112.15 two digits 14) 27.3451 four digits

In problems (15) through (22), convert the number from scientific notation to decimal notation.

15) 5.395 x 105 16) 4.39 x 106 17) 4.35 x 10-3 18) 6.288 x 10-4
19) 5.7789E4 20) 3.122E5 21) 8.324E-3 22) 7.69E-5

Answers

Accuracy in this Class

I am very particular about the accuracy of answers in this course. When computing statistics, you are only allowed one more decimal place or one more significant digit than is used in the most accurate of the original data values. If a set of data is accurate to at most two decimal places, than any statistics you compute from this set of data can have at most three-decimal place accuracy. I would let four-decimal place accuracy pass, but any more than that will cost you points.

In terms of significant digits, if the data has at most four significant digits, then a statistic computed from this data can have at most five significant digits. You may use any value from a table as is. When using the P-value approach in hypothesis testing, you just need enough accuracy to decide whether or not the P-value is less than the significance level. If you cannot tell which is smaller, use more significant digits in your P-value.

Return to Statistics Topics