Improved method of rounding off (truncation of least-significant bits) in integer processing of data devised. Provides for reduction, to extremely low value, of numerical bias otherwise generated by accumulation of truncation errors from many arithmetic operations. Devised for use in integer signal processing, in which rescaling and truncation usually performed to reduce number of bits, which typically builds up in sequence of operations. Essence of method to alternate direction of roundoff (plus, then minus) on alternate occurrences of truncated values contributing to bias.
Reducing Truncation Error In Integer Processing
NASA Tech Briefs ; 19 , 2
1995-02-01
Sonstige
Keine Angabe
Englisch
Reducing Truncation Error in Integer Processing
Online Contents | 1995
REDUCING TRUNCATION ERROR ON UNSTRUCTURED MESHES BY VERTEX MOVEMENT
British Library Conference Proceedings | 2012
|On Residual Estimate of Local Truncation Error
AIAA | 2023
|On Residual Estimate of Local Truncation Error
TIBKAT | 2023
|