[Tutor] I can't understand where python class methods come from

[Ben Finney]

voger <vogernewsletters at yahoo.gr> writes:

> I have a really hard time understanding where methods are defined in
> python classes.

Reading the rest of your message, I think you're saying that you can't
locate *which* class defines a method

> […] With python everything looks like magic.

It can seem that way. Fortunately, most of the magic needs to be
explicitly declared in the code, so it's a matter of knowing what to
look for.

In this message, I'll guide you in when you should expect magic, and
where to look for it.


> Methods and properties appearing out of nowhere and somehow easing the
> task at hand. I am not complaining for their existence but I really
> can't understand how and what is defined so I can use it.

This is a valid complaint, and certainly some libraries can get rather
too clever with the “magic behaviour”, leading to difficulty in
understanding what is happening well enough to debug problems.

> Enough ranting and let me give an example of what I mean.
>
> Let's have a look at the pygeocoder library located here:
> http://code.xster.net/pygeocoder/src/8888c863f907f8a706545fa9b27959595f45f8c5/pygeolib.py?at=default

The important part of that file is the ‘GeocoderResult’ class
definition. It defines a lot of magic behaviour.

In general, the magic often happens with “dunder”-named attributes; that
is, attributes named with a leading and traling double-underscore.

An attribute named “__foo__” is a strong indication that it will be
treated specially and be invoked by the Python interpreter, *without*
the programmer needing to invoke the attribute by name.

Here is the class definition showing just the dunder-named attributes::

class GeocoderResult(collections.Iterator):

    …

    def __init__(self, data):
        …

    def __len__(self):
        …

    def __iter__(self):
        …

    def __getitem__(self, key):
        …

    def __unicode__(self):
        …

    if sys.version_info[0] >= 3:  # Python 3
        def __str__(self):
            return self.__unicode__()

        def __next__(self):
            return self.return_next()
    else:  # Python 2
        def __str__(self):
            return self.__unicode__().encode('utf8')

        def next(self):
            return self.return_next()

    …

    def __getattr__(self, name):
        …


So already you should be on high alert: there is a large amount of
“magic” in this class! Seemlingly-simple uses of this class and its
instances will result in invocations of those custom methods, even
without seeing the names used explicitly.

> Some properties I can see them defined but some others like
> sublocality or administrative_area_level_1 I don't see them defined
> anywhere.

The specific magic happening here is that, when you access an attribute
on an object, Python will allow interception of that request by a custom
‘__getattr__’ method. As you can see, the ‘GeocoderResult’ class defines
that method for its instances, so you will need to look there for custom
behaviour when accessing attributes on instances of this class.

<URL:http://docs.python.org/3/reference/datamodel.html#object.__getattr__>

Similarly, the class defines ‘__getitem__’ to intercept ‘foo[bar]’, it
defines ‘__len__’ to intercept ‘len(foo)’, it defines ‘__iter__’ to
intercept attempts to iterate over an instance; and so on. Much magic!

Personally, I'm suspicious whether this class really needs to be so
awfully clever; it is certainly making the behaviour less intuitive, as
demonstrated in your case. It smells of a poor design.

> I hope I didn't bore you with this long text but this is really
> confusing for me and I hope someone can help me understand these
> things.

One good decision made by the Python core developers early on was a
convention of naming these special, magical-behaviour “hook” attributes
in a way that they are clearly special and signal to the reader that
Python will be using them implicitly.

So the general advice is to understand that any “dunder”-named attribute
needs to be researched, because it can be invoked *without* you
necessarily being aware of that.

Hope that helps!

-- 
 \            “Choose mnemonic identifiers. If you can't remember what |
  `\                mnemonic means, you've got a problem.” —Larry Wall |
_o__)                                                                  |
Ben Finney