confused about __new__

[Ian Kelly]

On Mon, Dec 26, 2011 at 10:48 PM, Fredrik Tolf <fredrik at dolda2000.com> wrote:
> I'm also a bit confused about __new__. I'd very much appreciate it if
> someone could explain the following aspects of it:
>
>  * The manual (<http://docs.python.org/reference/datamodel.html>) says
>   that __new__ is "a static method (special-cased so you need not declare
>   it as such)". What does "special-cased" mean? Apparently, for
>   instance, in OP's case,  Python did not automatically detect that it
>   should not be bound as a method.

It apparently has to do with the class creation.  For the
special-casing to happen, the __new__ method has to be present when
the class is created.  If it is, then it automatically gets wrapped in
a staticmethod.  In the OP's case, he was adding the __new__ method
after class creation, so the wrapping did not happen automatically.
Compare:

>>> def my_new(cls): return object.__new__(cls)
...
>>> class Foo(object):
...     __new__ = my_new
...
>>> class Bar(object): pass
...
>>> Bar.__new__ = my_new
>>> Foo.__dict__['__new__']
<staticmethod object at 0x0237D6F0>
>>> Bar.__dict__['__new__']
<function my_new at 0x02381430>

>  * Is there any part of the manual that explains, holistically, the
>   greater context of object instantiation into which __new__ fits? I can
>   only find small parts of it spread around the documentation for __new__
>   and __init__, but no complete explanation of it. There are several
>   things I'm wondering about, like what it means to call a type object at
>   all;

I don't know of anything that organizes it that way specifically, but
I believe the Data Model reference pretty much covers what you're
looking for.  From the type hierarchy, under "Callable Types":
http://docs.python.org/reference/datamodel.html#the-standard-type-hierarchy
"""
Class Types
    Class types, or “new-style classes,” are callable. These objects
normally act as factories for new instances of themselves, but
variations are possible for class types that override __new__(). The
arguments of the call are passed to __new__() and, in the typical
case, to __init__() to initialize the new instance.
"""

AFAIK, that's pretty much it.  When a type is called, __new__ is
called to create the new instance, and then __init__ is called to
initialize it (if __new__ returned an instance of the type).

> how methods, properties and the like are bound;

When they're accessed, using the descriptor protocol, not as part of
the instantiation process.  See:
http://docs.python.org/reference/datamodel.html#invoking-descriptors

> how pickle can
>   instantiate a class without calling __init__;

By calling the __new__ method directly instead of calling the type:
http://docs.python.org/library/pickle.html#object.__getnewargs__

> when and whether __dict__
>   is created and a couple of other things.

Under the hood as part of the object creation process, unless the
class uses __slots__.