LibreOfficeの計算精度

固有の精度の問題

LibreOffice Calcは、他の多くの表計算ソフトウェアと同様に、ハードウェアで利用可能な浮動小数点演算機能を使用します。現在のほとんどのハードウェアは、IEEE 754規格で定義された精度制限付きのバイナリ浮動小数点演算を使用しているため、多くの小数(0.1のような単純なものを含む)は、LibreOffice Calc(内部で64ビット倍精度数値を使用)では正確に表現できません。

Calculations with those numbers necessarily results in rounding errors, and those accumulate with every calculation.

This is not a bug, but is expected and currently unavoidable without using complex calculations in software, which would incur inappropriate performance penalties, and thus is out of question. Users need to account for that, and use rounding and comparisons with machine epsilon (or unit roundoff) as necessary.

An example with numbers:

A

1

31000.99

2

32000.12

3

=A1-A2


This will result in -999.129999999997 in A3, instead of expected -999.13 (you might need to increase shown decimal places in cell format to see this).

An example with dates and times:

Due to the specific of time representation in Calc, this also applies to all calculations involving times. For example, the cells A1 and A2 below show the date and time data as entered (in ISO 8601 format):

A

1

2020-04-13 12:18:00

2

2020-04-13 12:08:00

3

=A1-A2


Cell A3 will show 00:10:00 if the default formatting [HH]:MM:SS is applied to the cell. However, cell A3 will show 00:09:59.999999 instead of expected 00:10:00.000000 if formatted using [HH]:MM:SS.000000 format string. This happens despite only whole numbers of hours and minutes were used, because internally, any time is a fraction of a day, 12:00 (noon) being represented as 0.5.

The data in A1 is represented internally as 43934.5125, and in A2 as 43934.5055555555591126903891563 (which is not exact representation of the entered datetime, which would be 43934.505555555555555555...).

Their subtraction results in 0.00694444443287037, a value slightly less than expected 0.00694444444444..., which is 10 minutes.