IEEE Floating Point Standard: Representation, Operations, and Puzzles - Prof. Jian-Guo Liu, Study notes of Mathematics

Download IEEE Floating Point Standard: Representation, Operations, and Puzzles - Prof. Jian-Guo Liu and more Study notes Mathematics in PDF only on Docsity! Page 1 Floating Point Sept 5, 2002 Topics
IEEE Floating Point Standard
Rounding
Floating Point Operations
Mathematical properties class04.ppt 15-213 “The course that gives CMU its Zip!” – 2 – 15-213, F’02 Floating Point Puzzles
For each of the following C expressions, either: Argue that it is true for all argument values Explain why not true • x == (int)(float) x • x == (int)(double) x • f == (float)(double) f • d == (float) d • f == -(-f); • 2/3 == 2/3.0 • d < 0.0
((d*2) < 0.0) • d > f
-f > -d • d * d >= 0.0 • (d+f)-d == f int x = …; float f = …; double d = …; Assume neither d nor f is NaN – 3 – 15-213, F’02 IEEE Floating Point IEEE Standard 754
Established in 1985 as uniform standard for floating point arithmetic  Before that, many idiosyncratic formats
Supported by all major CPUs Driven by Numerical Concerns
Nice standards for rounding, overflow, underflow
Hard to make go fast  Numerical analysts predominated over hardware types in defining standard – 4 – 15-213, F’02 Fractional Binary Numbers Representation
Bits to right of “binary point” represent fractional powers of 2
Represents rational number: bi bi–1 b2 b1 b0 b–1 b–2 b–3 b–j• • •• • • . 1 2 4 2i–1 2i • • • • • • 1/2 1/4 1/8 2–j bk ⋅2 k k =− j i  Page 2 – 5 – 15-213, F’02 Frac. Binary Number Examples Value Representation 5-3/4 101.112 2-7/8 10.1112 63/64 0.1111112 Observations
Divide by 2 by shifting right
Multiply by 2 by shifting left
Numbers of form 0.111111…2 just below 1.0 1/2 + 1/4 + 1/8 + … + 1/2i + … → 1.0 Use notation 1.0 – ε – 6 – 15-213, F’02 Representable Numbers Limitation
Can only exactly represent numbers of the form x/2k
Other numbers have repeating bit representations Value Representation 1/3 0.0101010101[01]…2 1/5 0.001100110011[0011]…2 1/10 0.0001100110011[0011]…2 – 7 – 15-213, F’02 Numerical Form
–1s M 2E Sign bit s determines whether number is negative or positive Significand M normally a fractional value in range [1.0,2.0). Exponent E weights value by power of two Encoding
MSB is sign bit
exp field encodes E
frac field encodes M Floating Point Representation s exp frac – 8 – 15-213, F’02 Encoding
MSB is sign bit
exp field encodes E
frac field encodes M Sizes
Single precision: 8 exp bits, 23 frac bits 32 bits total
Double precision: 11 exp bits, 52 frac bits 64 bits total
Extended precision: 15 exp bits, 63 frac bits Only found in Intel-compatible machines Stored in 80 bits » 1 bit wasted Floating Point Precisions s exp frac Page 5 – 17 – 15-213, F’02 Distribution of Values 6-bit IEEE-like format
e = 3 exponent bits
f = 2 fraction bits
Bias is 3 Notice how the distribution gets denser toward zero. -15 -10 -5 0 5 10 15 Denormalized Normalized Infinity – 18 – 15-213, F’02 Distribution of Values (close-up view) 6-bit IEEE-like format
e = 3 exponent bits
f = 2 fraction bits
Bias is 3 -1 -0.5 0 0.5 1 Denormalized Normalized Infinity – 19 – 15-213, F’02 Interesting Numbers Description exp frac Numeric Value Zero 00…00 00…00 0.0 Smallest Pos. Denorm. 00…00 00…01 2– {23,52} X 2– {126,1022}
Single ≈ 1.4 X 10–45
Double ≈ 4.9 X 10–324 Largest Denormalized 00…00 11…11 (1.0 – ε) X 2– {126,1022}
Single ≈ 1.18 X 10–38
Double ≈ 2.2 X 10–308 Smallest Pos. Normalized 00…01 00…00 1.0 X 2– {126,1022}
Just larger than largest denormalized One 01…11 00…00 1.0 Largest Normalized 11…10 11…11 (2.0 – ε) X 2{127,1023}
Single ≈ 3.4 X 1038
Double ≈ 1.8 X 10308 – 20 – 15-213, F’02 Special Properties of Encoding FP Zero Same as Integer Zero
All bits = 0 Can (Almost) Use Unsigned Integer Comparison
Must first compare sign bits
Must consider -0 = 0
NaNs problematic  Will be greater than any other values  What should comparison yield?
Otherwise OK  Denorm vs. normalized  Normalized vs. infinity Page 6 – 21 – 15-213, F’02 Floating Point Operations Conceptual View
First compute exact result
Make it fit into desired precision Possibly overflow if exponent too large Possibly round to fit into frac Rounding Modes (illustrate with $ rounding) $1.40 $1.60 $1.50 $2.50 –$1.50
Zero $1 $1 $1 $2 –$1
Round down (-∞) $1 $1 $1 $2 –$2
Round up (+∞) $2 $2 $2 $3 –$1
Nearest Even (default) $1 $2 $2 $2 –$2 Note: 1. Round down: rounded result is close to but no greater than true result. 2. Round up: rounded result is close to but no less than true result. – 22 – 15-213, F’02 Closer Look at Round-To-Even Default Rounding Mode
Hard to get any other kind without dropping into assembly
All others are statistically biased Sum of set of positive numbers will consistently be over- or under- estimated Applying to Other Decimal Places / Bit Positions
When exactly halfway between two possible values Round so that least significant digit is even
E.g., round to nearest hundredth 1.2349999 1.23 (Less than half way) 1.2350001 1.24 (Greater than half way) 1.2350000 1.24 (Half way—round up) 1.2450000 1.24 (Half way—round down) – 23 – 15-213, F’02 Rounding Binary Numbers Binary Fractional Numbers
“Even” when least significant bit is 0
Half way when bits to right of rounding position = 100…2 Examples
Round to nearest 1/4 (2 bits right of binary point) Value Binary Rounded Action Rounded Value 2 3/32 10.000112 10.002 (<1/2—down) 2 2 3/16 10.001102 10.012 (>1/2—up) 2 1/4 2 7/8 10.111002 11.002 (1/2—up) 3 2 5/8 10.101002 10.102 (1/2—down) 2 1/2 – 24 – 15-213, F’02 FP Multiplication Operands (–1)s1 M1 2E1 * (–1)s2 M2 2E2 Exact Result (–1)s M 2E
Sign s: s1 ^ s2
Significand M: M1 * M2
Exponent E: E1 + E2 Fixing
If M
2, shift M right, increment E
If E out of range, overflow
Round M to fit frac precision Implementation
Biggest chore is multiplying significands Page 7 – 25 – 15-213, F’02 FP Addition Operands (–1)s1 M1 2E1 (–1)s2 M2 2E2
Assume E1 > E2 Exact Result (–1)s M 2E
Sign s, significand M:  Result of signed align & add
Exponent E: E1 Fixing
If M
2, shift M right, increment E
if M < 1, shift M left k positions, decrement E by k
Overflow if E out of range
Round M to fit frac precision (–1)s1 M1 (–1)s2 M2 E1–E2 + (–1)s M – 26 – 15-213, F’02 Mathematical Properties of FP Add Compare to those of Abelian Group
Closed under addition? YES But may generate infinity or NaN
Commutative? YES
Associative? NO Overflow and inexactness of rounding
0 is additive identity? YES
Every element has additive inverse ALMOST Except for infinities & NaNs Monotonicity
a
b
a+c
b+c? ALMOST Except for infinities & NaNs – 27 – 15-213, F’02 Math. Properties of FP Mult Compare to Commutative Ring
Closed under multiplication? YES But may generate infinity or NaN
Multiplication Commutative? YES
Multiplication is Associative? NO Possibility of overflow, inexactness of rounding
1 is multiplicative identity? YES
Multiplication distributes over addition? NO Possibility of overflow, inexactness of rounding Monotonicity
a
b & c
0
a *c
b *c? ALMOST Except for infinities & NaNs – 28 – 15-213, F’02 Floating Point in C C Guarantees Two Levels float single precision double double precision Conversions
Casting between int, float, and double changes numeric values
Double or float to int  Truncates fractional part  Like rounding toward zero  Not defined when out of range » Generally saturates to TMin or TMax
int to double  Exact conversion, as long as int has
53 bit word size
int to float  Will round according to rounding mode