fixed point tests: some high-precision tests are failing

Testing slightly-out-of-range values is failing (i.e. the value seems to be
wrongfully accepted by the parameter-framework) starting from high precisions,
e.g. q0.16.

The cause is:

1) When converting strings to fixed-points, the parameter-framework first
parses the input string as double. Doubles have 52 fraction bits and there for
15 decimal-digits-precision (52 * log10(2) == 15).

2) When displaying Qn.m numbers, we decided to display all representable digits
(i.e. m digits) instead of only displaying significant digits because that
would cause unpredictable behaviour. See
d299108157ee4d0eadb7683b2fa6a6635bc63d95 and
59cc1e33810c55e6fa1e3bd320e1cf29e24d23be for more information.

3) However, the precision for a Qn.m fixed-point number isn't m but
m * log10(2). This is guaranteed to be less than the precision of a double but
the tests mentioned at the beginning were written as if the precision was m,
which may be larger than the precision of a double. This is what caused the
test failures.

The tests are adapted to try and set a value of which the last significant
digit is one-off too far outside of the allowed range.

For lower-precision number, other less-far-off values could be correctly
rejected by the parameter-framework but that's only because of the underlying
implementation (using doubles as intermediate values) and it seems hardly
relevant to check.

Signed-off-by: David Wagner <david.wagner@intel.com>

1 files changed, 2 insertions, 1 deletions

diff --git a/test/test-fixed-point-parameter/Main.py b/test/test-fixed-point-parameter/Main.py
index 94d0753..318b2d1 100755
--- a/test/test-fixed-point-parameter/Main.py
+++ b/test/test-fixed-point-parameter/Main.py
@@ -32,6 +32,7 @@ import PyPfw
 
 import logging
 from decimal import Decimal
+from math import log10
 
 class PfwLogger(PyPfw.ILogger):
     def __init__(self):
@@ -85,7 +86,7 @@ class FixedPointTester():
         # bigValue is to be sure a value far out of range is refused
         bigValue = (2 * self._quantum)
         # little is to be sure a value just out of range is refused
-        littleValue  = 10 ** -fractional
+        littleValue = 10 ** -(int(fractional * log10(2)))
         self._shouldBreak = [
                 Decimal(self._lowerAllowedBound) - Decimal(bigValue),
                 Decimal(self._upperAllowedBound) + Decimal(bigValue),