Getting rid of bitwise operators in Python 3?

[John Machin]

On 22/09/2007 1:44 PM, Carl Banks wrote:
> Anyone with me here?  (I know the deadline for P3 PEPs has passed; this 
> is just talk.)
> 
> Not many people are bit-fiddling these days.  One of the main uses of bit 
> fields is flags, but that's not often done in Python because of keyword 
> arguments and dicts, which are lot more versatile.  Another major use, 
> talking to hardware, is not something oft done in Python either.

You need to get out more often :-)

Python's bitwise operators are useful for and *used* for:

(1) packing and unpacking arcane structs used in communication protocols 
and in proprietary file formats

(2) trial implementations of pseudocode from academic papers, before 
transliterating to pyrex or C

E.g. main loop for a bit-parallel O(N) algorithm for a Levenshtein distance:
       for c in t:
          pm_new = pm[ord(c)]
          d0 = (((pm_new & vp) + vp) ^ vp) | pm_new | vn
          hp = vn | ~(d0 | vp)
          hn = d0 & vp
          if hp & mask:
             x += 1
          if hn & mask:
             x -= 1
          hp = (hp << 1) | 1
          vp = (hn << 1) | ~(d0 | hp)
          vn = d0 & hp
       return x

> 
> It seems like this occasional usage wouldn't justify having built-in 
> operators on its own.  And with the int/long unification, it makes a bit 
> less sense to think of integers as a bit field.

"to think of integers as a bit field" makes no sense to me. Possibly you 
meant "to think of an integer as a collection of bit fields" -- but this 
is putting the cart before the horse. Simply: if the data is a 
collection of bit fields, and the data is held in an int or a long or 
the unification thereof, then you need bitwise operators to extract/pack 
the bit fields.


>  Python has these 
> operators because of its heritage, but Python continues to move away from 
> the bad habits of its ancestors (integer division and so on), and I 
> wonder if this isn't another one.
> 
> Of course I'm not suggesting to get rid of bitwise operations altogether; 
> just make them builtin functions: "x & 1" becomes "bitwise_and(x,1)" and 
> so on.

Function call overhead?? No thanks. E.g. from real working code:

                 r = Rowinfo()
                 # Using upkbits() is far too slow on a file
                 # with 30 sheets each with 10K rows :-(
                 #    upkbits(r, bits2, (
                 #        ( 0, 0x00000007, 'outline_level'),
                 #        ( 4, 0x00000010, 'outline_group_starts_ends'),
                 #        ( 5, 0x00000020, 'hidden'),
                 #        ( 6, 0x00000040, 'height_mismatch'),
                 #        ( 7, 0x00000080, 'has_default_xf_index'),
                 #        (16, 0x0FFF0000, 'xf_index'),
                 #        (28, 0x10000000, 'additional_space_above'),
                 #        (29, 0x20000000, 'additional_space_below'),
                 #        ))
                 # So:
                 r.outline_level = bits2 & 7
                 r.outline_group_starts_ends = (bits2 >> 4) & 1
                 r.hidden = (bits2 >> 5) & 1
                 r.height_mismatch = (bits2 >> 6) & 1
                 r.has_default_xf_index = (bits2 >> 7) & 1
                 r.xf_index = (bits2 >> 16) & 0xfff
                 r.additional_space_above = (bits2 >> 28) & 1
                 r.additional_space_below = (bits2 >> 29) & 1


> Is it worth it to make such a change?  It would remove a lot of operators 
> (11 by my count), vastly simplifying syntax,  Which, IMHO, is no small 
> thing.  New numerical types would have fewer operations to support.

Specious argument. A new numerical type doesn't have to support bitwise 
operations now, if they make no sense for that type.

> And 
> let's face it: unlike arithmetic opertaions, there's not a lot of 
> different meanings for bit operations. And it would also, um, make new 
> special characters available *cough*.
> 
> Obviously, how widespread their usage is would matter.  But keep in mind 
> it would also be easy to convert the code automatically, because the 
> Python parser could reliably find all bitwise operations reliably.  (The 
> problem would be types that overloaded them to be something other than 
> bitwise operations: I'm looking at you, set.  That could very well be a 
> deal breaker.  I've never approved of things with completely different 
> meanings being deliberately overloaded to have the same spelling; this is 
> one reason why.)

Too late. That instance of Pandora's box had its lid nailed open ab initio.

> 
> If anyone says, "But that takes away an easy test for oddness (x&1)!", 
> or, "But you can multiply powers of two using left shift!  Isn't that 
> cool?", I'm not buying it.  Those are gimmicks.  Arithmetic operations 
> should be done with arithmetic operators.  The bitwise operators make the 
> code much less readable, especially to people unfamiliar with this usage.

Agreed, but this is beside the point. People will write obfuscated code 
  using any and every operator or built-in that you give them.

In case you hadn't guessed by now: -1

Cheers,
John