Rounding error with Double type in GPScript

Scripting with Gig Performer
1

I have two instances of GP, each with a Global Rackspace Script that checks the current value of a widget mapped to the volume of a Gain Control block. The script displays a message if the widget’s value does not correspond to 0 dB.

Interestingly, on the “Synth” GP instance the widget value for 0 dB is 70.7, and and the “Drums” instance it’s 70.8. On both, when the widget’s value does correspond to 0 dB and I test for a return value of 0 from v = GetWidgetValue(), if v != 0 fails: v is not equal to 0. So instead I have a workaround that checks for a very small difference between v and 0:

v = GetWidgetValue(SynthVolStereo)
d = v-0.7070000
If d<0 or d>0.0000001  /* account for GP rounding error; "!=" doesn't work! */
Then
    Print("STARTUP: Synth Vol Stereo was not .707")
    Print("         it was " + v)
    OpenLogWindow()
End

If I replace the If test with If v != 0, and the widget corresponds to 0 dB, I get this:

image

Using the If d<0 or d>0.0000001 workaround, I get these results when the widget is one tick off from 0 dB (-1 dB and 1 dB, respectively):

image

image

Here are the corresponding cases in the “Drums” instance:

v = GetWidgetValue(DrumsVol)
d = v-0.7080000
If d<0 or d>0.0000001  /* account for GP rounding error; "!=" doesn't work! */
Then
    Print("STARTUP: Drums Volume was not .708")
    Print("         it was " + v)
    OpenLogWindow()
End

If I replace the If test with if v != 0, and the widget corresponds to 0 dB, I get this:

image

Using the workaround, I get these results from -1 dB and 1 dB:

image

image

P.S. I’m perfectly happy with my workaround, just thought you might want to know about this rounding error. And I know floating point values are hard to compare when they’re very small, so maybe this is just the way it is.

2

You can also work around it with this:

d = Round(v*1000 - 708)
If  d != 0 
Then....
1 Like
3

Well, kinda impossible really. Generally people use a function where you pass in the two values to be compared and the function subtracts one from the other (using absolute values) and checks that the difference is less than some defined epsilon value.

1 Like