Hello, Is there any way to have fast dispatch based on the type of a variable? I'm talking about code of the form: t = type(var) if t is int: i(v) elif t is long: l(v) elif t is float: f(v) elif t is str: s(v) elif t is unicode: u(v) ... I have tried these ideas: - Having the types as keys in a dict and the functions as lambdas. - Creating a list from min(type_hashes) to max(type_hashes) (with lambdas as list values) and indexing in it with hash(var_type) - min(type_hashes) But both were slower than the multiple ifs. The ideal case would be to have an optimization like C/C++ compilers do to switch statements, where they would create a binary search over the multiple cases like below: Assuming that: hash(int) < hash(long) < hash(float) .... t=hash(type(var)) if t < hash(float): if t < hash(long): i(v) else: l(v) else: if t< hash(unicode): s(v) else: u(v) The problem in Python is that the order of type_hashes is not constant. So it is not possible to create the binary search code. Kind regards, l.

On 15/7/2014 8:28 μμ, Armin Rigo wrote:

Hi,

On 15 July 2014 18:37, Eleytherios Stamatogiannakis wrote:

...

t = type(var) if t is int: i(v) elif t is long: l(v) elif t is float: f(v) elif t is str: s(v) elif t is unicode: u(v) ...

This should already give you the fastest possible execution on PyPy, because the first type inspection should promote the type in the JIT. All subsequent "if" checks are constant-folded. However, to be sure, you need to check with jitviewer.

Note however that if all paths are eventually compiled by the JIT, the promotion will have a number of different cases, and searching through them is again done by linear search for now. This can be regarded as a bug waiting for improvement.

Above code gets hit millions of times with different variable types. So in our case all paths are compiled and we are linear. Another idea that i have is the following. At startup i could sort all the hash(types), create (in a string) a python method that does binary sorting and eval it. Would the JIT be able to handle eval gymnastics like that? Thank you. l.

On 16/07/14 17:31, Armin Rigo wrote:

Hi,

On 15 July 2014 19:11, Elefterios Stamatogiannakis wrote:

...

Above code gets hit millions of times with different variable types. So in our case all paths are compiled and we are linear.

Another idea that i have is the following. At startup i could sort all the hash(types), create (in a string) a python method that does binary sorting and eval it. Would the JIT be able to handle eval gymnastics like that?

You need to try. There are far too many variations to be able to give a clear yes/no answer. For example, linear searches through only 6 items is incredibly fast anyway.

But here's what I *think* should occur with your piece of code (untested!):

t = type(x)

Here, in this line, in order to get the application-level type, we need to know the exact RPython class of x. This is because the type is not always written explicitly: a Python-level int object, for example, is in RPython an instance of the class W_IntObject. We know that all instances of W_IntObject have the Python type 'int'; it doesn't need to be written explicitly as a field every time. So at the line above, there is promotion of the RPython class of x. Right now this is done with linear searching through all cases seen so far.

Could this be made faster with binary search or jump tables, like what C++ compilers use to optimize switches? I also noticed that "if" ladders checking multiple "isinstance" happen a lot in Python's standard library. Maybe an optimization like that would generally improve the speed of PyPy ?

If there are 5-6 different cases it's fine. (Note that RPython class != Python class in general, but for built-in types like int, str, etc. there is a one-to-one correspondence.)

So at the line above, assuming that x is an instance of a built-in type, we end up with t being a constant already (a different one in each of the 5-6 paths).

if t is int: ... elif t is long: ...

In all the rest of the function, the "if... elif..." are constant-folded away. You don't gain anything by doing more complicated logic with t.

A bientôt,

Armin.

On Tue, Jul 15, 2014 at 12:50 PM, Yichao Yu wrote:

On Tue, Jul 15, 2014 at 5:05 PM, anatoly techtonik wrote:

...

Hi,

Is it possible at all to define a class in Python that can read name of variable it is assigned to on init?

...

...

...

MyObject = SomeClass() print(MyObject) 'MyObject'

I thing in general a normal object in Python does not have a name and there's nothing special about the name of the variable it is assigned to first. To see why is is not going to work, what do you expect you print function to do if the object is created like

some_function(SomeClass())

I don't need this case, so I can ignore it. But reliably detecting this to distinguish from other situations would be nice.

or some_other_object.some_attribute = SomeClass()

Detect that name is an attribute, handle distinctly if needed, for my purpose print 'some_other_object.some_attribute'

or some_variable = another_variable = SomeClass()

Print closest assigned variable name, i.e. 'another_variable'

or some_variable = (SomeClass(),)

There is no direct assignment. Don't need this.

or even SomeClass() # not assigning to anything etc....

Good to detect. Don't need. Actually all cases that I don't need are the same case on this one - there is no direct assignment to variable.

I guess it would be better if you can describe what you really want to do.

I described. Or you need use case or user story? I think I want to link object instances to variable names without to print those names (if possible) in __repr__ and debug messages to save time on troubleshooting.

I guess this feature is mainly useful for debugging since it is really hard to do it consistantly in python. And IMHO, for debuging, it might be more useful to record the position (file + line num etc) where the object is created, which has less ambiguity and can probably already be done by tracing back the call stack in the constructor (until the most derived constructor or even just record the whole call stack). For not writing the name of a named tuple twice, won't it be a better idea to just use it as a base class, i.e.: class TupleName(named_tuple_base('a', 'b', 'c', 'd')): pass or maybe even using the syntax and trick of the new Enum class introduced in python 3.4(? or 3.3?) class TupleName(TupleBase): a = 1 b = 1 IMHO, this fits the python syntax better (if the current one is not good enough :) ) Yichao Yu On Wed, Jul 16, 2014 at 10:03 AM, Steven D'Aprano wrote:

On Tue, Jul 15, 2014 at 5:05 PM, anatoly techtonik wrote:

...

Is it possible at all to define a class in Python that can read name of variable it is assigned to on init?

...

...

...

MyObject = SomeClass() print(MyObject) 'MyObject'

This feature would be useful for things like namedtuple, where we currently have to write the name twice:

record = namedtuple('record', 'a b c d')

But I'm not sure why Anatoly is asking here. It would be a change in semantics of Python, and while I suppose it's possible for PyPy to lead the way with a semantic change for Python 3.5 or higher, or even an implementation-specific feature that other Python's don't offer, I would expect that normally this idea should go through CPython first.

-- Steven _______________________________________________ pypy-dev mailing list pypy-dev@python.org https://mail.python.org/mailman/listinfo/pypy-dev