[Tutor] __new__ and __init__

[Steven D'Aprano]

On 02/08/12 00:28, rail shafigulin wrote:
> Hello everyone.
>
> I'm trying to understand how to use the two methods. I know that __new__ is
> used to create an object, while __init__ to initialize. But I'm not sure
> what happens when I create an object.
>
> I found the following code in the book (Python 3 Object Oriented
> Programming by Dusty Phillips)

That's a rather complex example to be teaching the basics.

Here's a simple toy class:


class MyClass(object):
     def __new__(cls, x, y, z):
         print("constructing instance with arguments:")
         print("    ", cls, x, y, z)
         instance = super(MyClass, cls).__new__(cls, x, y, z)
         return instance
     def __init__(self, a, b, c):
         print("initializing instance with arguments:")
         print("    ", self, a, b, c)
         self.a = a
         self.b = b
         self.c = c


If you copy and paste that into the Python interactive interpreter, and then 
test it, you should see something like this:

py> inst = MyClass(23, 42, 'spam')
constructing instance with arguments:
     <class '__main__.MyClass'> 23 42 spam
initializing instance with arguments:
     <__main__.MyClass object at 0xb7c85a6c> 23 42 spam



So what's going on?

When you call MyClass(...), this creates a new instance of the class. Python 
first calls the constructor, __new__, to actually build an instance in memory.

Notice how the first argument to __new__ is not "self", as methods usually 
have, but "cls" instead? That's because no instance exists yet, so "self" 
can't exist! Instead, the first argument to the method is not the instance, 
but the class. The other arguments, x y and z, are whatever arguments you give 
when you call MyClass(...).

The __new__ method then builds a MyClass instance. How does it know how to 
build an instance? Because MyClass inherits from object, the built-in special 
class. Everything[1] inherits from object, and object's __new__ knows how to 
build instances.

MyClass.__new__ uses super() to delegate the work of actually building an 
instance to the superclass (hence the name!), object. The call to super() is a 
bit tricky, mostly because __new__ is special, so for now just take this as a 
magic incantation:

     super(MyClass, cls).__new__(cls, x, y, z)

You have to write it like that in Python 2; in Python 3, there is a magic 
shortcut that you can use:

     super().__new__(cls, x, y, z)

Once the instance is created, your __new__ method can change it, add 
attributes, turn it inside out and paint it pink if you like. But normally you 
don't do that from the constructor, you leave that to the initializer, 
__init__. So the constructor, __new__, will normally just return the instance, 
and then Python will automatically call __init__.

Note: __new__ doesn't have to return the instance. It can return anything you 
like. But if it is *not* an instance of the class, Python will not call __init__.

Unlike __new__, __init__ is just an ordinary method that sees the instance as 
the first argument, "self".

Notice that I have deliberately called the arguments by different names in 
__new__ and __init__? x, y, z versus a, b, c. I've done that to make a point, 
but normally you would use the same argument names, for clarity and simplicity.

Inside __init__, you do whatever you need do to the instance -- paint it pink, 
attach attributes, whatever. And then you're done. Unlike __new__, which has 
to return something (normally the instance), __init__ doesn't have to return 
anything. It can if you like, but there is no point: Python will just ignore it.

So when should you use __new__ and when should you use __init__?

The short answer is: you nearly always should use __init__ and not __new__.

The longer answer is: if you are subclassing built-in types like int, float, 
str or tuple, then you have to use __new__. There's usually not much point to 
trying to change the value in __init__, because it's too late. Here's an 
example: suppose you want a subclass of int that is always positive or zero. 
This won't work:


py> class PositiveInt(int):  # WRONG!
...     def __init__(self, arg):
...         if arg < 0:
...             self = -arg
...
py> n = PositiveInt(-1)
py> n
-1


Trying to change the instance after it is created doesn't do anything. All you 
do is change the object referred to by the name "self". You have to change the 
value *before* the instance is created, because ints are immutable and can't 
be changed once they exist:


py> class PositiveInt(int):  # RIGHT!
...     def __new__(cls, arg):
...         if arg < 0:
...             arg = -arg
...         return super().__new__(cls, arg)
...
py> n = PositiveInt(-1)
py> n
1


[1] This is true in Python 3. It isn't *quite* true in Python 2. If you care 
about the difference, ask about "classic classes" and "new-style classes".



To answer your questions:


> I assume when I call CarModel(<parameters>) __new__is being called first
> and then __init__ after it.

Correct.

> 1) Does it mean that __new__ and __init__ must have the same parameters? In
> this particular case __new__ and __init__ both have model_name and if I
> understand correctly when __new__ is called the rest of the parameters
> (air, tilt, cruise_control, etc) are absorbed by the *args argument. Please
> correct me if I am wrong.

Correct.

Apart from the special first argument (cls and self), yes, __new__ and 
__init__ will receive the same arguments. That doesn't mean that you have to 
give those arguments the same names, although you usually should.



> 2) What happens if I don't use the same parameters, say in the case of
> __init__ I will remove model_name, will I still be able to call dx =
> CarModel("Fix DX")

If your parameters are different, Python will blindly line up the arguments 
supplied with the given parameters. If something doesn't match, Python will 
raise a TypeError.


-- 
Steven