Comments on: Reminder: Floats are not precise

By: Jose

Jose — Wed, 17 Jun 2009 20:52:17 +0000

actionscript 3.0

if you trace (0.1+0.1+0.1);

//output// = 0.30000000000000004
and
0.30000000000000004 is diferent to 3/10, as3.0 is right!:S

By: .: WEREMSOFT :. » El diario del domingo - 17 de mayo del 2009

.: WEREMSOFT :. » El diario del domingo - 17 de mayo del 2009 — Sun, 17 May 2009 19:51:35 +0000

[...] Un tipo muy muy groso explica con ejemplos el error matematico de los floats en flash, java y ruby LINK [...]

By: Marc

Marc — Fri, 15 May 2009 12:04:41 +0000

Thanks Kathryn, I completely missed that. It’s fixed now.

By: kathryn

kathryn — Fri, 15 May 2009 00:38:42 +0000

this post is a good reminder about float accuracy. but i noticed that you started your actionscript with a different value (.1) than the other languages (0). so at the end of the AS loop, you’d expect the value to be 1.1, not 1. it would be absurd for actionscript to be as far off the mark as nearly a whole decimal place after just ten additions.

By: Marc

Marc — Thu, 14 May 2009 12:53:03 +0000

I haven’t run across one yet, but I’m sure somebody somewhere did a BigDecimal implementation.

And hell, if not, someone should

But take note, these classes are always orders of magnitude slower than integers or floats. If you can, picking a larger order of magnitude for your unit of measure and storing it as an int will be much easier and faster.

By: jim

jim — Thu, 14 May 2009 10:17:17 +0000

do you know of any as3 class that handles precise numbers?

By: Alex

Alex — Wed, 13 May 2009 20:33:50 +0000

This reminds me of a (possibly anecdotal, I can’t recall where I read it) aspect of an old Cray design: To save the delay of a single NAND gate, the last few bits of a floating point operation weren’t guaranteed to be accurate. The point being that if you wanted that accuracy you should be doing a different operation in the first place.

By: Marc

Marc — Wed, 13 May 2009 16:52:31 +0000

Ah yes, I didn’t mean to imply whole classes of applications should or should not use floating math. Whenever it matters to be exact like health, score, etc. those are all big candidates to stay in the integer realm.

By: Iain

Iain — Wed, 13 May 2009 16:05:28 +0000

Even in games you can still get in trouble with this for variables like health. Best to use really big integers, with a max of e.g. 10,000