> Believe I read in 50g material that the calculator always does things to > 12 place accuracy but only displays as whatever the operator configures > the display (it is their preferance). Actually a large number of operations take place internally at 15 digits - especially matrix, trigonometric and exponential operations. Also, on some operations the final resulting answer is determined by the current display setting. I think integration is the only one like this, but I could be wrong. If you set display to FIX 3, your integer is within 1 digit of the third decimal place. Set it to 9 and the answer will take longer to compute. TW
Message
Re: Numeric Display Modes
2009-07-24 by Tim Wessman
Attachments
- No local attachments were found for this message.