[Tutor] Associate decimal values with an integer range

[Steven D'Aprano]

On 30/04/13 00:28, sparkle Plenty wrote:
> Hi, I need some hints on where to start solving this problem.
> I have to interpret a one-byte floating point number as a range between 0
> and 240.  This is for a gui that controls an amplifier.  I am getting a
> value such as 0.8367 from a fader control, and my Python is interpreting it
> as either a 0 or 1.

You're going to need to be a bit more specific than that. In what way are you getting this value? What is happening that you think that Python is interpreting it as 0 or 1?

> I need to define this number to Python as an integer
> value between 0 and 240.


There are at least two ways of interpreting the numbers you quote, and both are equally plausible (or implausible, if you prefer).

A single byte can take 256 different values, starting with 0x00 and going to 0xFF in hexadecimal. If 0x00 -> 0.0, and 0xFF -> 240.0, then the smallest non-zero value is 0x01 -> 240/255 = 0.9411 (approx). So 0.8367 is an impossible value.

On the other hand, if we allow that 0x00 -> 0.0, and 0xFF -> 1.0, then 0.8367 is still an impossible value. The closest values would be either 0.8352 (0xD5) or 0.8392 (0xD6), both values also being approximate.

I'm going to guess that the second case is more likely, and if that's wrong, please say so.

My guess is that your fader control generates a number between 0.0 and 1.0 as a single byte. That means that there are 256 possible bytes, giving 256 possible floats:

0x00 -> 0/255 = 0.0
0x01 -> 1/255 = 0.00392156862745098
0x02 -> 2/255 = 0.00784313725490196
...
0xFE -> 254/255 = 0.996078431372549
0xFF -> 255/255 = 1.0


Whatever you use to get the value into Python, it comes in as a *byte*, not a float, so you see it as an integer between 0 and 255 inclusive. You want to scale that into an integer value between 0 and 240. So you convert the byte into a mini-float, then the mini-float into an int. Mathematically, you would use:

byte/255 * 240

But if you are using Python 2, you need to convince Python to use *floating point division* rather than integer division. (An unfortunate historical mistake, corrected in Python 3.)

So write this:

byte/255.0*240  # make sure you use 255.0 and not just 255

The result you get will not be an exact integer value, so you have to drop the fractional part some how. You have two choices, either truncating, or rounding. I think rounding is better in this situation, so do this:

int(round(byte/255.0*240))


which should give you a nice exact integer between 0 and 240 inclusive for every byte between 0 and 255.



Does this help?




-- 
Steven