Believe I read in 50g material that the calculator always does things to 12 place accuracy but only displays as whatever the operator configures the display (it is their preferance). rickwendel763242 wrote: > > > Appendix C in the 50G users manual explains what the reasoning is > behind this. From that you may be able to extrapolate something. > Approximate mode doesn't allow maximum precision but you have to have > it on if you want to use engineering format. I work with nanoseconds > with 3 significant digits and I've never been beat in accuracy yet. > I'd be interested to know just how accurate this is, in engineering > format at 10-12. > >
Message
Re: [50g] Re: Numeric Display Modes
2009-07-23 by aa0p
Attachments
- No local attachments were found for this message.