------- Comment #100 from pinskia at gcc dot gnu dot org 2007-12-29 19:46 -------
*** Bug 34616 has been marked as a duplicate of this bug. ***


--

pinskia at gcc dot gnu dot org changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |ismail at pardus dot org dot
| |tr


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=323

------- You are receiving this mail because: -------
You are on the CC list for the bug, or are watching someone who is.


--
To UNSUBSCRIBE, email to debian-gcc-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

------- Comment #109 from pepalogik at seznam dot cz 2008-05-20 16:59 -------
I also encountered such problems and was going to report it as a bug in GCC...
But in the GCC bug (not) reporting guide, there is fortunately a link to this
page and here (comment #96) is a link to David Monniaux's paper about
floating-point computations. This explains it closely but it is maybe too long.
I have almost read it and hope I have understood it properly. So I'll give a
brief explanation (for those who don't know it yet) of the reasons of such a
strange behaviour. Then I'll assess where the bug actually is (in GCC or CPU).
Then I'll write the solution (!) and finally a few recommendations to the GCC
team.

EXPLANATION
The x87 FPU was originally designed in (or before) 1980. I think that's why it
is quite simple: it has only one unit for all FP data types. Of course, the
precision must be of the widest type, which is the 80-bit long double.
Consider you have a program, where all the FP variables are of the type double.
You perform some FP operations and one of them is e.g. 1e-300/1e300, which
results in 1e-600. Despite this value cannot be held by a "double", it is
stored in an 80-bit FPU register as the result. Consider you use the variable
"x" to hold that result. If the program has been compiled with optimization,
the value need not be stored in RAM. So, say, it is still in the register.
Consider you need x to be nonzero, so you perform the test x != 0. Since 1e-600
is not zero, the test yields true. While you perform some other computations,
the value is moved to RAM and converted to 0 because x is of type "double". Now
you want to use your certainly nonzero x... Hard luck :-(
Note that if the result doesn't have its corresponding variable and you perform
the test directly on an expression, the problem can come to light even without
optimization.
It could seem that performing all FP operations in extended precision can bring
benefits only. But it introduces a serious pitfall: moving a value may change
the value!!!

WHERE'S THE BUG
This is really not a GCC bug. The bug is actually in the x87 FPU because it
doesn't obey the IEEE standard.

SOLUTION
The x87 FPU is still present in contemporary processors (including AMD) due to
compatibility. I think most of PC software still uses it. But new processors
have also another FPU, called SSE, and this do obey the IEEE. GCC in 32-bit
mode compiles for x87 by default but it is able to compile for the SSE, too. So
the solution is to add these options to the compilation command:
-march=* -msse -mfpmath=sse
Yes, this definitely resolves the problem - but not for all processors. The *
can be one of the following: pentium3, pentium3m, pentium-m, pentium4,
pentium4m, prescott, nocona, athlon-4, athlon-xp, athlon-mp, k8, opteron,
athlon64, athlon-fx and c3-2 (I'm unsure about athlon and athlon-tbird). Beside
-msse, you can also add some of -mmmx, -msse2, -msse3 and -m3dnow, if the CPU
supports them (see GCC doc or CPU doc).
If you wish to compile for processors which don't have SSE, you have a few
possibilities:
(1) A very simple solution: Use long double everywhere. (But be careful when
transfering binary data in long double format between computers because this
format is not standardized and so the concrete bit representations vary between
different CPU architectures.)
(2) A partial but simple solution: Do comparisons on volatile variables only.
(3) A similar solution: Try to implement a "discard_extended_precision"
function suggested by Egon in comment #88.
(4) A complex solution: Before doing any mathematical operation or comparison,
put the operands into variables and put also the result to a variable (i.e.
don't use complex expressions). For example, instead of { c = 2*(a+b); } ,
write { double s = a+b; c = 2*s; } . I'm unsure about arrays but I think they
should be OK. When you have modified your code in this manner, then compile it
either without optimization or, when using optimization, use -ffloat-store. In
order to avoid double rounding (i.e. rounding twice), it is also good to
decrease the FPU precision by changing its control word in the beginning of
your program (see comment #60). Then you should also apply -frounding-math.
(5) A radical solution: Find a job/hobby where computers are not used at all.

RECOMMENDATIONS
I think this problem is really serious and general. Therefore, programmers
should be warned soon enough.
This recommendation should be addressed especially to authors of programming
coursebooks. But I think there could also be a paragraph about it in the GCC
documentation (I haven't read it wholly but it doesn't seem there's any warning
against x87). And, of course, there should be a warning in the bug reporting
guide (http://gcc.gnu.org/bugs.html). It's fine there's a link to this page
(Bug 323) but the example with (int)(a/b) is insufficient. It only demonstrates
that real numbers are often not represented exactly in the computer. It doesn't
demonstrate the x87 pitfall. Hence there should be an example such as the
initial code of this "GCC bug 323 report". Because when one sees the example
with (int)(a/b), he can say "It's trivial" and not click the link (as I did the
first time).

EPILOGUE
I hope my effort of writing this "comment #109" will be helpful for many
people.
If you want more info, read the David Monniaux's work or something else about
FPUs.
Thanks to David Monniaux.


--

pepalogik at seznam dot cz changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |pepalogik at seznam dot cz


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=323

------- You are receiving this mail because: -------
You are on the CC list for the bug, or are watching someone who is.


--
To UNSUBSCRIBE, email to debian-gcc-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

------- Comment #110 from pepalogik at seznam dot cz 2008-06-12 14:14 -------
I used an old version of GCC documentation so I omitted some new processors
with SSE: core2, k8-sse3, opteron-sse3, athlon64-sse3, amdfam10 and barcelona.
I think you can use -march=pentium3 for all Intel's CPUs (of course, starting
with P3). I'm unsure about AMD. (Maybe you know it better.)


--


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=323

------- You are receiving this mail because: -------
You are on the CC list for the bug, or are watching someone who is.


--
To UNSUBSCRIBE, email to debian-gcc-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

--

rohit dot x dot tripathi at jpmchase dot com changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |rohit dot x dot tripathi at
| |jpmchase dot com


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=323

------- You are receiving this mail because: -------
You are on the CC list for the bug, or are watching someone who is.


--
To UNSUBSCRIBE, email to debian-gcc-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

------- Comment #111 from vincent at vinc17 dot org 2008-06-20 16:09 -------
(In reply to comment #109)
> WHERE'S THE BUG
> This is really not a GCC bug. The bug is actually in the x87 FPU because it
> doesn't obey the IEEE standard.

Concerning the standards: The x87 FPU does obey the IEEE754-1985 standard,
which *allows* extended precision, and double precision is *available*. In
fact, one could say that GCC even obeys the IEEE standard (which doesn't define
bindings: the definition of "destination" page 4 of the IEEE754-1985 standard
is rather vague and lets the language to define it exactly), but it doesn't
obey the ISO C99 standard on some point.

Concerning the x87 FPU: One can say however that the x87 is a badly designed
because it is not possible to statically specify the precision. Nevertheless
the OS/language implementations should take care of this problem.

Note: the solution chosen by some OS'es (*BSD, MS-Windows...) is to configure
the processor to the IEEE double precision by default (thus "long double" is
also in double precision, but this is OK as far as the C language is concerned,
there's still a problem with "float", but in practice, nobody cares AFAIK).

> If you wish to compile for processors which don't have SSE, you have a few
> possibilities:
> (1) A very simple solution: Use long double everywhere.

This avoids the bug, but this is not possible for software that requires double
precision exactly, e.g. XML tools that use XPath. See other examples here:

http://www.vinc17.org/research/extended.en.html

Also this makes maintenance of software more difficult because long double can
be much slower on some platforms, which support this type in software to
provide more precision (e.g. PowerPC Linux and Mac OS X implement a
double-double arithmetic, Solaris and HPUX implement quadruple precision).

> (But be careful when transfering binary data in long double format between
> computers because this format is not standardized and so the concrete bit
> representations vary between different CPU architectures.)

Well, this is not specific to long double anyway: there exist 3 possible
endianess for the double format (x86, PowerPC, ARM).

> (2) A partial but simple solution: Do comparisons on volatile variables only.

Yes (but this is also a problem concerning the maintenance of portable
programs).

> (4) A complex solution: [...]

Yes, this is the workaround I use in practice.

> RECOMMENDATIONS
> I think this problem is really serious and general. Therefore, programmers
> should be warned soon enough.

Yes, but note that this is not the only problem with compilers. See e.g.

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=36578

for a bug related to casts to long double on x86_64 and ia64. This one is now
tested by: http://www.vinc17.org/software/tst-ieee754.c (which has also tested
bug 323 for a long time).


--

vincent at vinc17 dot org changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |vincent at vinc17 dot org


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=323

------- You are receiving this mail because: -------
You are on the CC list for the bug, or are watching someone who is.


--
To UNSUBSCRIBE, email to debian-gcc-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

------- Comment #112 from pepalogik at seznam dot cz 2008-06-21 22:38 -------
(In reply to comment #111)
> Concerning the standards: The x87 FPU does obey the IEEE754-1985 standard,
> which *allows* extended precision, and double precision is *available*.

It's true that double *precision* is available on x87. But not the *IEEE-754
"double precision" type*. Beside the precision of mantissa, this includes also
the range of exponent. On the x87, it is possible to set the precision of
mantissa but not the range of exponent. That's why I believe it doesn't obey
the IEEE. (I haven't ever seen the IEEE-754 standard but I base on the work of
David Monniaux.)

> Note: the solution chosen by some OS'es (*BSD, MS-Windows...) is to configure
> the processor to the IEEE double precision by default (thus "long double" is
> also in double precision, but this is OK as far as the C language is concerned,
> there's still a problem with "float", but in practice, nobody cares AFAIK).

Do you mean that on Windows, long double has (by default) no more precision
than double? I don't think so (it's confirmed by my experience). According to
the paper of David Monniaux, only FreeBSD 4 sets double precision by default
(but I know almost nothing about BSD).

> > (1) A very simple solution: Use long double everywhere.
> This avoids the bug, but this is not possible for software that requires double
> precision exactly, e.g. XML tools that use XPath.

Yes, of course. I don't say this can be used everywhere.

> > (But be careful when transfering binary data in long double format between
> > computers because this format is not standardized and so the concrete bit
> > representations vary between different CPU architectures.)
> Well, this is not specific to long double anyway: there exist 3 possible
> endianess for the double format (x86, PowerPC, ARM).

OK but David Monniaux mentions portability issues just in the case of long
double, so the differences are probably more frequent in this case (maybe even
within the x86 architecture).

> Yes, but note that this is not the only problem with compilers. See e.g.
> http://gcc.gnu.org/bugzilla/show_bug.cgi?id=36578

Thanks for info.


--


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=323

------- You are receiving this mail because: -------
You are on the CC list for the bug, or are watching someone who is.


--
To UNSUBSCRIBE, email to debian-gcc-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

------- Comment #113 from vincent at vinc17 dot org 2008-06-22 00:52 -------
(In reply to comment #112)
> It's true that double *precision* is available on x87. But not the *IEEE-754
> "double precision" type*.

It is available when storing a result to memory.

> Beside the precision of mantissa, this includes also the range of exponent.
> On the x87, it is possible to set the precision of mantissa but not the range
> of exponent.

The IEEE754-1985 allows this. Section 4.3: "Normally, a result is rounded to
the precision of its destination. However, some systems deliver results only to
double or extended destinations. On such a system the user, which may be a
high-level language compiler, shall be able to specify that a result be rounded
instead to single precision, though it may be stored in the double or extended
format with its wider exponent range. [...]"

> That's why I believe it doesn't obey the IEEE. (I haven't ever seen the
> IEEE-754 standard but I base on the work of David Monniaux.)

See above. Also beware of subtilities in the wording used by David Monniaux.
FYI, the IEEE754-1985 standard (with minor corrections) is available from the
following page:
http://www.validlab.com/754R/
(look at the end). AFAIK, the IEEE754-1985 standard was designed from the x87
implementation, so it would have been very surprising that x87 didn't conform
to IEEE754-1985.

> Do you mean that on Windows, long double has (by default) no more precision
> than double? I don't think so (it's confirmed by my experience).

I don't remember my original reference, but here's a new one:
http://msdn.microsoft.com/en-us/library/aa289157(vs.71).aspx
In fact, this depends on the architecture. I quote: "x86. Intermediate
expressions are computed at the default 53-bit precision with an extended range
provided by a 16-bit exponent. When these 53:16 values are "spilled" to memory
(as can happen during a function call), the extended exponent range will be
narrowed to 11-bits. That is, spilled values are cast to the standard double
precision format with only an 11-bit exponent.
A user may switch to extended 64-bit precision for intermediate rounding by
altering the floating-point control word using _controlfp and by enabling FPU
environment access (see The fpenv_access Pragma). However, when extended
precision register-values are spilled to memory, the intermediate results will
still be rounded to double precision.
This particular semantic is subject to change."

Note that the behavior has changed in some version of Windows (it was using the
extended precision, then it switched to double precision for x86). Now, this
may also depend on the compiler.

> According to the paper of David Monniaux, only FreeBSD 4 sets double
> precision by default (but I know almost nothing about BSD).

I've noted that amongst the BSD's, NetBSD does this too (I don't remember if
I've tried or got it from some document, and this might also depend on the
NetBSD version and/or the processor).


--


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=323

------- You are receiving this mail because: -------
You are on the CC list for the bug, or are watching someone who is.


--
To UNSUBSCRIBE, email to debian-gcc-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

------- Comment #114 from pepalogik at seznam dot cz 2008-06-22 16:59 -------
(In reply to comment #113)
> It is available when storing a result to memory.

Yes, but this requires quite a complicated workaround (solution (4) in my
comment #109). So you could say that the IEEE754 double precision type is
available even on a processor without any FPU because this can be emulated
using integers.
Moreover, if we assess things pedantically, the workaround (4) still doesn't
fully obey the IEEE single/double precision type(s), because there remains the
problem of double rounding of denormals.

> The IEEE754-1985 allows this. Section 4.3: "Normally, a result is rounded to
> the precision of its destination. However, some systems deliver results only to
> double or extended destinations. On such a system the user, which may be a
> high-level language compiler, shall be able to specify that a result be rounded
> instead to single precision, though it may be stored in the double or extended
> format with its wider exponent range. [...]"
> [...]
> AFAIK, the IEEE754-1985 standard was designed from the x87
> implementation, so it would have been very surprising that x87 didn't conform
> to IEEE754-1985.

So it seems I was wrong but the IEEE754-1985 standard is also quite "wrong".

> > Do you mean that on Windows, long double has (by default) no more precision
> > than double? I don't think so (it's confirmed by my experience).
> I don't remember my original reference, but here's a new one:
> http://msdn.microsoft.com/en-us/library/aa289157(vs.71).aspx
> In fact, this depends on the architecture. I quote: "x86. Intermediate
> expressions are computed at the default 53-bit precision with an extended range
> [...]"

I quote, too:
"Applies To
Microsoft® Visual C++®"


--


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=323

------- You are receiving this mail because: -------
You are on the CC list for the bug, or are watching someone who is.


--
To UNSUBSCRIBE, email to debian-gcc-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

------- Comment #115 from pepalogik at seznam dot cz 2008-06-22 17:28 -------
That ® should be (R).


--


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=323

------- You are receiving this mail because: -------
You are on the CC list for the bug, or are watching someone who is.


--
To UNSUBSCRIBE, email to debian-gcc-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

------- Comment #116 from vincent at vinc17 dot org 2008-06-22 21:14 -------
(In reply to comment #114)
> Yes, but this requires quite a complicated workaround (solution (4) in my
> comment #109).

The problem is on the compiler side, which could store every result of a cast
or an assignment to memory (this is inefficient, but that's what you get with
the x87, and the ISO C language could be blamed too for *requiring* something
like that instead of being more flexible).

> So you could say that the IEEE754 double precision type is available even on
> a processor without any FPU because this can be emulated using integers.

Yes, but a conforming implementation would be the processor + a library, not
just the processor with its instruction set.

> Moreover, if we assess things pedantically, the workaround (4) still doesn't
> fully obey the IEEE single/double precision type(s), because there remains the
> problem of double rounding of denormals.

As I said, in this particular case (underflow/overflow), double rounding is
allowed by the IEEE standard. It may not be allowed by some languages (e.g.
XPath, and Java in some mode) for good or bad reasons, but this is another
problem.

> I quote, too:
> "Applies To
> Microsoft® Visual C++®"

Now I assume that it follows the MS-Windows API (though nothing is certain with
Microsoft). And the other compilers under MS-Windows could (or should) do the
same thing.


--


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=323

------- You are receiving this mail because: -------
You are on the CC list for the bug, or are watching someone who is.


--
To UNSUBSCRIBE, email to debian-gcc-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org