[Tutor] Understanding (Complex) Modules

Wayne Watson sierra_mtnview at sbcglobal.net
Fri Mar 19 15:46:19 CET 2010

(I meant to post this to both the list and Steven back a week or so  
ago. I missed the list, so am posting it here now. Unfortunately, I'm 
again in a position where I cannot respond for several days, but I will 
be back early next week to examine some posts that followed this.)

Thanks. I'm on the road for a few days and will be able to read this 
more carefully when I get back. I had a feeling from my first attempts 
of looking at MPL and other heavy-duty libs that it was going to require 
work to dig out what I needed from the module itself. In my personal 
view, the usage and learning documents are fairly limiting. Perhaps on 
in MPL, which is less traditional than numpy and the others, which are 
founded on common and historic implementations of math and science from 
older languages.

From: Steven D'Aprano <steve at pearwood.info>
To: tutor at python.org
Sent: Thu, March 4, 2010 6:24:15 PM
Subject: Re: [Tutor] Understanding (Complex) Modules

On Thu, 4 Mar 2010 12:24:35 pm Wayne Watson wrote:
 > First a little preamble before my questions.
 > Most of my work in Python has required modifying a program that uses
 > modules that were imported by the original program. I've made some
 > use of modules on a command line like math, and have used the idea of
 > a qualifier.  On occasion, I've used examples from matplotlib that
 > required from matplotlib.image import AxesImage. Further, I've
 > created some simple classes, and produced  objects with them. No use
 > of inheritance though.  So far so good.  In a few places, it is said
 > modules are objects. I'm slightly puzzled by that, but in some way it
 > seems reasonable from the standpoint of period notation. So far I
 > have not created a module.

Yes you have, you just don't know it.

In simple language, a file with Python-usable code in it is a module.
There are some technicalities and complexities, but in basic terms,
that's all it is.

Technically, "module" refers only to the object which exists inside the
Python environment after you call "import thing". The import statement
does a whole lot of tricks, but in a nutshell it:

* finds a file containing code called 'thing'
* loads it into memory, executing code as needed
* creates a 'module' object to store the code and data in the file
* and makes it available to your code

Python can create module objects from:

compiled Windows DLLs .dll
compiled Linux object files .so
Python source code .py
Python byte code .pyc and .pyo (and .pyw on Windows)
Zip files containing any of the above

and probably other things as well, but they're the main ones.

So any Python file you create (any .py file) is a module, or at least it
would be a module if you import it.

Ignoring all the various compiled files (.dll, .so, etc) what happens
when you run a .py file from the command line (or from IDLE or another
IDE). E.g. you type something like:

python.exe myscript.py [enter]

Python reads the file myscript.py and executes it, but no module object
is created. It is possible that a module object *is* created, for
internal use, then thrown away when the script is finished. But your
script doesn't see the module object.

However, if you enter the Python interactive environment, and do this:

 >>> import myscript  # no .py

Python loads the file myscript.py into a module object, executing any
code in it, and makes it available as a module.

 > Here comes the questions. Recently I began to use matplotlib, scipy,
 > and pylab, mostly matplotlib. I've ground out a few useful pieces of
 > code, but I'm fairly baffled by the imports required to get at
 > various elements, of say, matplotlib (MPL).  Some of the MPL examples
 > use of imports make sense, but how the writer pulled in the necessary
 > elements is not.  How does one go about understanding the
 > capabilities of such modules?

Time, experience, and a lot of hard work. Welcome to programming!

If the documentation is good, then read the documentation.

If the documentation is lacking, or bad, or even wrong, then read the
source code, or search the Internet for a tutorial, or buy a book about
it (the numpy people, for example, sell books about numpy).

Python makes experimentation easy: there are a lot of powerful
introspection facilities in Python. The interactive interpreter is your
best friend. You will live with it, eat with it, sleep with it, take it
on double-dates, and throw yourself on live grenades to protect it.
Whenever I'm programming, I almost always have three or five
interactive sessions open for experimentation.

The dir() and help() functions are also good friends. In an interactive

import math
dir(math)  # prints a list of names in the math module

Don't feel that you have to understand the entire module before you use
it. Many (alas, not all) modules have a reasonably gentle learning
curve: you can start using math.sin without needing to know what
math.sinh is for.

Google and Wikipedia are also your friends, although not your *best*
friends. (Don't necessarily believe *everything* you read on the
Internet.) Don't forget other search engines apart from Google: they're
good, but not perfect.

 > MPL seems to have a lot of lower level
 > components. Some of them are laid out over numerous pages as in the
 > form of a, say, English language, description. How does one decipher
 > this stuff.  For example, open the module in an editor and start
 > looking at the organization? I thinkthe so called MPL guide ins 900
 > pages long. Even the numpy guide (reference?), I believe borders on
 > 1000 pages. There must be some way to untangle these complex modules,
 > I would think. Some of the tutorials seem nothing more than a
 > template to follow for a particular problem. So far, looking at the
 > plentiful number of examples of MPL, and probably some of the other
 > modules mentioned above have not provided a lot of insight.

Big, complex modules tend to have steep learning curves. There's no
magic path to learning how to use a big complex module any more than
there is a magic path to learning how to be a brain surgeon, or a car

 >  Is there some relationship between modules and objects that I'm not
 > seeing that could be of value?

Modules are themselves objects. Everything in Python is an object:
strings, ints, floats, lists, tuples, everything.

Modules are compound objects, in that they contain other objects
accessible by name:


means "look up the name 'sin' in the math module, and return whatever
you find", which in this case is a function object.

And that's pretty much it.

Steven D'Aprano
Tutor maillist  -  Tutor at python.org

More information about the Tutor mailing list